Skip to content

[HOLD] Bump protobufjs to v8 and drop proto3-json-serializer #863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mjameswh wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[HOLD] Bump protobufjs to v8 and drop proto3-json-serializer #863

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2faf0c1
Don't assume default-valued proto fields survive a round trip
mjameswh Aug 10, 2026
4235442
Bump protobufjs to v8 and drop proto3-json-serializer
mjameswh Aug 10, 2026
036113d
Trim down comments
mjameswh Aug 10, 2026
6fe87ed
Drop the Buffer/Uint8Array injection workarounds
mjameswh Aug 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions features/data_converter/binary_protobuf/feature.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,17 @@ import * as assert from 'assert';
import { Feature } from '@temporalio/harness';
import * as proto from '@temporalio/proto';

// Inject Buffer and Uint8Array from the node context to the workflow context to workaround SDK bug
// TODO(antlai-temporal) Remove when SDK bug is fixed
const g = globalThis as any;
g.Uint8Array = g.constructor.constructor('return globalThis.Uint8Array')();

const expectedResult = proto.temporal.api.common.v1.DataBlob.create({
encodingType: proto.temporal.api.enums.v1.EncodingType.ENCODING_TYPE_UNSPECIFIED,
data: new Uint8Array([0xde, 0xad, 0xbe, 0xef]),
});

// Do an encode/decode roundtrip to make sure our test expectations match exactly
// what protobufjs 8 will produce (e.g. default values are omitted, etc.)
const expectedResultOnWire = proto.temporal.api.common.v1.DataBlob.decode(
proto.temporal.api.common.v1.DataBlob.encode(expectedResult).finish(),
);

// An "echo" workflow
export async function workflow(
res: proto.temporal.api.common.v1.DataBlob,
Expand All @@ -30,7 +31,7 @@ export const feature = new Feature({
async checkResult(runner, handle) {
// verify client result is DataBlob `0xdeadbeef`
const result = await handle.result();
assert.deepEqual(result, expectedResult);
assert.deepEqual(result, expectedResultOnWire);

// get result payload of WorkflowExecutionCompleted event from workflow history
const payload = await runner.getWorkflowResultPayload(handle);
Expand All @@ -45,7 +46,7 @@ export const feature = new Feature({
assert.ok(payload.data);
const resultInHistory = proto.temporal.api.common.v1.DataBlob.decode(payload.data);

assert.deepEqual(resultInHistory, expectedResult);
assert.deepEqual(resultInHistory, expectedResultOnWire);

// get argument payload of WorkflowExecutionStarted event from workflow history
const payloadArg = await runner.getWorkflowArgumentPayload(handle);
Expand Down
21 changes: 11 additions & 10 deletions features/data_converter/json_protobuf/feature.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import * as assert from 'assert';
import { fromProto3JSON } from 'proto3-json-serializer';
import * as protojson from 'protobufjs/ext/protojson';
import { Feature } from '@temporalio/harness';
import * as proto from '@temporalio/proto';
import { patchProtobufRoot } from '@temporalio/proto/lib/patch-protobuf-root';
Expand All @@ -9,16 +9,17 @@ import { decode } from '@temporalio/common/lib/encoding';
const patched = patchProtobufRoot(proto) as any;
const dataBlobType = patched.lookupType('temporal.api.common.v1.DataBlob');

// Inject Buffer and Uint8Array from the node context to the workflow context to workaround SDK bug
// TODO(antlai-temporal) Remove workaround when SDK bug is fixed
const g = globalThis as any;
g.Buffer = g.constructor.constructor('return globalThis.Buffer')();

const expectedResult = proto.temporal.api.common.v1.DataBlob.create({
encodingType: proto.temporal.api.enums.v1.EncodingType.ENCODING_TYPE_UNSPECIFIED,
data: new Uint8Array([0xde, 0xad, 0xbe, 0xef]),
});

// Do an encode/decode roundtrip to make sure our test expectations match exactly
// what protobufjs 8 will produce (e.g. default values are omitted, etc.)
const expectedResultOnWire = proto.temporal.api.common.v1.DataBlob.decode(
proto.temporal.api.common.v1.DataBlob.encode(expectedResult).finish(),
);

// An "echo" workflow
export async function workflow(
res: proto.temporal.api.common.v1.DataBlob,
Expand All @@ -37,7 +38,7 @@ export const feature = new Feature({
async checkResult(runner, handle) {
// verify client result is DataBlob `0xdeadbeef`
const result = await handle.result();
assert.deepEqual(result, expectedResult);
assert.deepEqual(result, expectedResultOnWire);

// get result payload of WorkflowExecutionCompleted event from workflow history
const payload = await runner.getWorkflowResultPayload(handle);
Expand All @@ -50,9 +51,9 @@ export const feature = new Feature({
assert.equal(Buffer.from(payload.metadata.messageType).toString(), 'temporal.api.common.v1.DataBlob');

assert.ok(payload.data);
const resultInHistory = fromProto3JSON(dataBlobType, JSON.parse(decode(payload.data)));
const resultInHistory = protojson.fromJson(dataBlobType, JSON.parse(decode(payload.data)));
assert.ok(resultInHistory);
assert.deepEqual(resultInHistory, expectedResult);
assert.deepEqual(resultInHistory, expectedResultOnWire);

// get argument payload of WorkflowExecutionStarted event from workflow history
const payloadArg = await runner.getWorkflowArgumentPayload(handle);
Expand All @@ -65,7 +66,7 @@ export const feature = new Feature({
assert.equal(Buffer.from(payloadArg.metadata.messageType).toString(), 'temporal.api.common.v1.DataBlob');

assert.ok(payloadArg.data);
const resultArgInHistory = fromProto3JSON(dataBlobType, JSON.parse(decode(payloadArg.data)));
const resultArgInHistory = protojson.fromJson(dataBlobType, JSON.parse(decode(payloadArg.data)));
assert.ok(resultArgInHistory);
assert.deepEqual(resultInHistory, resultArgInHistory);
},
Expand Down
58 changes: 28 additions & 30 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 1 addition & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@
"commander": "^8.3.0",
"ms": "^3.0.0-canary.1",
"nexus-rpc": "^0.0.1",
"proto3-json-serializer": "^1.1.1",
"protobufjs": "7.5.1"
"protobufjs": "^8.7.1"
},
"devDependencies": {
"@tsconfig/node24": "^24.0.4",
Expand Down
7 changes: 1 addition & 6 deletions sdkbuild/typescript.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,18 +141,13 @@ func BuildTypeScriptProgram(ctx context.Context, options BuildTypeScriptProgramO
"commander": "^8.3.0",
"ms": "^3.0.0-canary.1",
"nexus-rpc": "^0.0.1",
"proto3-json-serializer": "^1.1.1"
"protobufjs": "^8.7.1"
},
"devDependencies": {
"@tsconfig/node24": "^24.0.4",
"@types/node": "^24.1.0",
"tsconfig-paths": "^3.12.0",
"typescript": "^5.9.3"
},
"pnpm": {
"overrides": {
"protobufjs": "7.5.1"
}
}
}`
if err := os.WriteFile(filepath.Join(dir, "package.json"), []byte(packageJSON), 0644); err != nil {
Expand Down
Loading