feat: support multi-line string literals in the Pure protocol - #5404
Draft
rafaelbey wants to merge 1 commit into
Draft
feat: support multi-line string literals in the Pure protocol#5404rafaelbey wants to merge 1 commit into
rafaelbey wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: 3b1b8fc The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
rafaelbey
marked this pull request as draft
August 3, 2026 19:23
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5404 +/- ##
==========================================
+ Coverage 50.36% 50.40% +0.03%
==========================================
Files 2779 2779
Lines 556495 556581 +86
Branches 19971 25608 +5637
==========================================
+ Hits 280276 280522 +246
+ Misses 275672 274816 -856
- Partials 547 1243 +696
🚀 New features to boost your workflow:
|
Engine is adding multi-line ('''...''') string literals to the Pure
grammar and records them in the protocol in two places:
- CString gains an optional `multiLine` flag, omitted from the wire
when false (finos/legend-engine#4998)
- TaggedValue's `value` becomes a CString, so it is sent as a plain
string unless the value was authored multi-line, in which case it
is an object carrying the flag (finos/legend-engine#5008)
Both changes are backward compatible on engine's side. Studio, however,
drops unknown keys on deserialization, so without this it would silently
rewrite a multi-line documentation as an escaped single-line literal on
the next save.
Tagged values keep a flat `value: string` with a sibling `multiLine`
flag rather than mirroring engine's CString typing - the union is
handled entirely in `V1_taggedValueModelSchema`, which leaves every
`taggedValue.value` consumer untouched. The flag is hashed on both the
metamodel and the protocol side so that toggling it registers as a
change; note it has to be stringified first, as `hash.js` digests any
boolean to the hash of the empty string. It is left out of
`V1_CString`'s hash to keep that compatible with
`PrimitiveInstanceValue`, whose metamodel has no counterpart.
rafaelbey
force-pushed
the
port-multiline-string-protocol
branch
from
August 3, 2026 19:59
dd38bd2 to
3b1b8fc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Blocked on finos/legend-engine#5008. Kept as a draft until that lands.
The
TaggedValuehalf of this change is written against that PR's hand-rolled Jackson serializer, whose wire shape can still move in review. Before marking this ready, re-readTaggedValue.javaon the merged commit and diff it againstV1_taggedValueModelSchema.The
CStringhalf depends only on finos/legend-engine#4998, which is already merged, and could be split out and landed independently if that is preferred.Engine is adding multi-line (
'''...''') string literals to the Pure grammar, and records them in the protocol in two places:CStringgains an optionalboolean multiLine, annotated@JsonInclude(NON_DEFAULT)so it is omitted from the wire whenfalse.TaggedValue.valuebecomes aCString. A hand-rolled Jackson serializer keeps the wire shape a plain string unless the value was authored multi-line, in which case it is{"_type":"string","multiLine":true,"value":"..."}. Both shapes deserialize.Engine is backward compatible in both directions. Studio is not forward compatible — nothing here knew about the flag, and
serializrdrops keys that are not in the schema. So a{_type, multiLine, value}tagged value would deserialize intoV1_TaggedValue.valueas an object (which then flows intohashArray,.split(','),.trim(),<textarea value=…>), and even in the benign single-line case Studio would silently stripmultiLineon the entity roundtrip — rewriting a user's'''...'''documentation as an escaped single-line literal on the next save.Approach
Tagged values keep a flat
value: stringwith a siblingmultiLineflag on both the metamodel and the protocol class, rather than mirroring engine'svalue: CStringtyping. Engine changed the field type because Java has nowhere else to hang the flag; here that would ripple through ~28taggedValue.valueread sites acrosslegend-graph,legend-query-builder,legend-application-studio,legend-application-queryand three DSL extensions. The union instead lives entirely inV1_taggedValueModelSchema, which is where engine put it too, and which all ~25 consumers already share.Hashing:
multiLineis hashed onTaggedValueandV1_TaggedValue(kept in lockstep) — this hash drives change detection, so without it toggling a value to multi-line would not register as a change and could not be saved.multiLineis not hashed onV1_CString, to keep that hash compatible withPrimitiveInstanceValue.hashCode— the metamodel has noCStringcounterpart to carry the flag, and diverging would produce phantom changes.Also covered: the parallel raw value specification hierarchy (
V1_RawPrimitiveInstanceValue), which carries its primitive type as a_typefield rather than by subclass, so it never surfaces in agrep V1_CString. It is reachable from relational milestoninginfinityDateand frombuildRawValueSpecification.Testing
V1_TaggedValueBackwardCompatibleSerialization.test.ts— a direct port of engine'sTestTaggedValueCompatibility.java: legacy plain string, plain string containing\n(must stay single-line), object form with and without the flag, single-line serializing back to a bare string, multi-line serializing to an object, and both shapes round-tripping.V1_ValueSpecificationBackwardCompatibleSerialization.test.ts—multiLineround-trips, is omitted whenfalse, and survives the legacyvalues: [...]shim.V1_RawValueSpecificationSerialization.test.ts— same for the raw hierarchy.TEST_DATA__ClassWithMultiLineTaggedValuein the entity roundtrip — asserts entity-in === entity-out, and viaTEST__checkGraphHashUnchangedalso pins protocol hashes to metamodel hashes.coretest group is green. Thedata-cubeandengine-roundtripgroups were not evaluated locally (they need a running engine).Out of scope
Authoring support: there is no multi-line toggle in
TaggedValueEditor, and the Monaco Pure tokenizer does not yet highlight'''...'''blocks. This PR only guarantees Studio carries the flag through faithfully.