Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/mdast-www-dot-escape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@lexical/mdast': patch
---

[fix] With `MdastGfmExtension` or `MdastAutolinkLiteralExtension`, export escapes the `.` of `www.` in text whatever follows it, also where an inline element with no export rule, such as a `MarkNode`, splits the `www`, so the characters after it are read back as typed

Text that such an element splits is escaped as one text in general, so `1` before a mark of `. x` is written as `1\. x` instead of starting an ordered list.

Verify: export snapshots of text with `www.` before the marker of a format or a link, before a non-ASCII character beside formatting, or before punctuation that the export escapes, such as `www\.**x**` (formerly `www.**x**`), `www\.世**!**` (formerly `www.世**!**`, which was read back as a link to `http://www.&#x4E16`, then `;` and `!`) or `www\.\*x` (formerly `www.\*x`). Plain `www.example.com` was written as `www\.example.com` already.
61 changes: 60 additions & 1 deletion packages/lexical-mdast/src/MdastExport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -825,11 +825,12 @@ export function getExportedLengths(values: string[]): number[] {
* again when its version changes, and vendor the fix if no release has it.
*/
function toMarkdownWithStandIns(
tree: Root,
exported: Root,
extensions: ToMarkdownExtension[],
isCjkFriendly: boolean,
measure?: (values: string[]) => string[],
): string {
const tree = joinAdjacentText(exported);
const used = new Set<number>();
const counts = countStoodIn(tree, used);
if (counts.size > 0) {
Expand Down Expand Up @@ -888,6 +889,64 @@ function toMarkdownWithStandIns(
return measuring(measure, () => toMarkdown(tree, {extensions}));
}

/**
* `node` with each run of adjacent text siblings joined into one text, as an
* inline element with no export rule of its own (a MarkNode) leaves its text
* beside the text around it. mdast-util-to-markdown gives a text only the
* last character of the one before it to decide its escapes, so a `www.`
* split over two texts would go out unescaped. A text with anything but its
* value, such as `data` or a `position` that an export rule set for a handler
* to read, is kept apart. The tree itself is not changed.
*/
function joinAdjacentText<T extends Nodes>(node: T): T {
if (!('children' in node)) {
return node;
}
const source = node.children as Nodes[];
// Copied on the first change only, so a tree with nothing to join is kept.
let children: Nodes[] | null = null;
for (let i = 0; i < source.length; i++) {
const child = source[i];
const last = children ? children[children.length - 1] : source[i - 1];
if (i > 0 && isPlainText(child) && isPlainText(last)) {
if (children === null) {
children = source.slice(0, i);
}
children[children.length - 1] = {
type: 'text',
value: last.value + child.value,
};
} else {
const joined = joinAdjacentText(child);
if (children === null && joined !== child) {
children = source.slice(0, i);
}
if (children !== null) {
children.push(joined);
}
}
}
return children === null ? node : {...node, children};
}

/**
* Whether `node` is a text with nothing but its value: no `data`, no
* `position`, nor anything else set.
*/
function isPlainText(node: Nodes | undefined): node is Text {
return (
node !== undefined &&
node.type === 'text' &&
// Symbol keys too, which Object.entries leaves out.
Reflect.ownKeys(node).every(
key =>
key === 'type' ||
key === 'value' ||
(node as unknown as Record<PropertyKey, unknown>)[key] === undefined,
)
);
}

/** Runs `serialize` with `measure` as how {@link getExportedLengths} measures. */
function measuring(
measure: ((values: string[]) => string[]) | undefined,
Expand Down
21 changes: 20 additions & 1 deletion packages/lexical-mdast/src/MdastExtension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -500,7 +500,26 @@ export const MdastAutolinkLiteralExtension = defineExtension({
// literal, and gfmAutolinkLiteralFromMarkdown finds literals in the
// decoded text, where a reference is its character.
micromarkExtensions: [referenceAgnostic(gfmAutolinkLiteral)],
toMarkdownExtensions: [/* @__PURE__ */ gfmAutolinkLiteralToMarkdown()],
toMarkdownExtensions: [
/* @__PURE__ */ gfmAutolinkLiteralToMarkdown(),
// Its rule escapes the `.` of `www.` only before an ASCII letter or
// digit, `_`, `-` or `.` as typed, but micromark starts a literal
// before anything else the export writes there too, such as a
// character reference, an escape or the marker of a format, and
// reads it as part of the link. This rule has no `after`, because
// mdast-util-to-markdown skips a rule that has an `after` where a
// rule with no condition escapes the next character, as one does `*`.
{
unsafe: [
{
before: '[Ww]{3}',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional, non-blocking: before: '[Ww]{3}' needs all three ws in the string safe() checks, but mdast-util-to-markdown gives a text node only the last character of its previous sibling as before. A www. split across sibling text nodes, such as one where an inline element with no export rule (a MarkNode) starts inside www, still goes out unescaped: see ww + mark(w.世) + bold ! exports as see www.&#x4E16;**!** and reads back as a link to http://www.&#x4E16, then ;. main does the same, so this is not a regression; joining adjacent text siblings before serializing, or a line in this comment about the limit, would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Joined adjacent text siblings before serializing in f4e59ed, so the rule sees the whole www. That also stops 1 + mark(. x) from starting an ordered list.

character: '.',
inConstruct: 'phrasing',
notInConstruct: ['autolink', 'link', 'image', 'label'],
},
],
},
],
}),
],
name: '@lexical/mdast/AutolinkLiteral',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -772,6 +772,9 @@ describe('@lexical/mdast import/export', () => {
'visit www.example.com now',
// A CommonMark autolink stays wrapped even with the literal grammar on.
'wrapped <https://lexical.dev> stays',
// The `.` of `www.` is escaped in text only.
'[x](http://www.x.com "www.t")',
'```www.x\ncode\n```',
]) {
expect(roundTrip(markdown)).toBe(markdown);
}
Expand Down
Loading
Loading