Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/mdast-www-dot-escape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@lexical/mdast': patch
---

[fix] With `MdastAutolinkLiteralExtension`, export escapes the `.` of `www.` in text whatever follows it, so the characters after it are read back as typed: `www.世` before bold `!` was written as `www.世**!**` and read back as a link to `http://www.&#x4E16`, then `;` and `!`
Comment thread
rootnode-bot[bot] marked this conversation as resolved.
Resolved

Verify: export snapshots of text with `www.` before the marker of a format or a link, before a non-ASCII character beside formatting, or before punctuation that the export escapes, such as `www\.**x**` (formerly `www.**x**`), `www\.世**!**` (formerly `www.世**!**`) or `www\.\*x` (formerly `www.\*x`). Plain `www.example.com` was written as `www\.example.com` already.
21 changes: 20 additions & 1 deletion packages/lexical-mdast/src/MdastExtension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -500,7 +500,26 @@ export const MdastAutolinkLiteralExtension = defineExtension({
// literal, and gfmAutolinkLiteralFromMarkdown finds literals in the
// decoded text, where a reference is its character.
micromarkExtensions: [referenceAgnostic(gfmAutolinkLiteral)],
toMarkdownExtensions: [/* @__PURE__ */ gfmAutolinkLiteralToMarkdown()],
toMarkdownExtensions: [
/* @__PURE__ */ gfmAutolinkLiteralToMarkdown(),
// Its rule escapes the `.` of `www.` only before an ASCII letter or
// digit, `_`, `-` or `.` as typed, but micromark starts a literal
// before anything else the export writes there too, such as a
// character reference, an escape or the marker of a format, and
// reads it as part of the link. This rule has no `after`, because
// mdast-util-to-markdown skips a rule that has an `after` where a
// rule with no condition escapes the next character, as one does `*`.
{
unsafe: [
{
before: '[Ww]{3}',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional, non-blocking: before: '[Ww]{3}' needs all three ws in the string safe() checks, but mdast-util-to-markdown gives a text node only the last character of its previous sibling as before. A www. split across sibling text nodes, such as one where an inline element with no export rule (a MarkNode) starts inside www, still goes out unescaped: see ww + mark(w.世) + bold ! exports as see www.世**!** and reads back as a link to http://www.&#x4E16, then ;. main does the same, so this is not a regression; joining adjacent text siblings before serializing, or a line in this comment about the limit, would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Joined adjacent text siblings before serializing in f4e59ed, so the rule sees the whole www. That also stops 1 + mark(. x) from starting an ordered list.

character: '.',
inConstruct: 'phrasing',
notInConstruct: ['autolink', 'link', 'image', 'label'],
},
],
},
],
}),
],
name: '@lexical/mdast/AutolinkLiteral',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -772,6 +772,9 @@ describe('@lexical/mdast import/export', () => {
'visit www.example.com now',
// A CommonMark autolink stays wrapped even with the literal grammar on.
'wrapped <https://lexical.dev> stays',
// The `.` of `www.` is escaped in text only.
'[x](http://www.x.com "www.t")',
'```www.x\ncode\n```',
]) {
expect(roundTrip(markdown)).toBe(markdown);
}
Expand Down
115 changes: 114 additions & 1 deletion packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import type {Paragraph} from 'mdast';
import type {Extension as MicromarkExtension} from 'micromark-util-types';

import {buildEditorFromExtensions} from '@lexical/extension';
import {$createLinkNode} from '@lexical/link';
import {$createLinkNode, $isLinkNode} from '@lexical/link';
import {$createListItemNode, $createListNode, $isListNode} from '@lexical/list';
import {
$createParagraphNode,
Expand Down Expand Up @@ -119,6 +119,23 @@ function importRuns(editor: LexicalEditor, markdown: string): Runs {
return editor.read('latest', $getRuns);
}

/**
* `runs` with each run joined to the run just before it when the two have
* the same format.
*/
function mergeRuns(runs: Runs): Runs {
const merged: Runs = [];
for (const [text, format] of runs) {
const last = merged[merged.length - 1];
if (last && last[1] === format) {
last[0] += text;
} else {
merged.push([text, format]);
}
}
return merged;
}

describe('Mdast punctuation modes', () => {
// runs, export for strict readers, export for CJK-friendly readers.
it.each<[runs: Runs, strict: string, cjkFriendly: string]>([
Expand Down Expand Up @@ -907,6 +924,102 @@ describe('Mdast punctuation modes', () => {
).toEqual(runs);
});

// runs, export for strict readers, export for CJK-friendly readers.
it.each<[runs: Runs, strict: string, cjkFriendly: string]>([
[
[
['www.世', 0],
['!', bold],
],
'www\\.&#x4E16;**!**',
'www\\.世**!**',
],
[
[
['www.世', 0],
['。', bold],
],
'www\\.&#x4E16;**。**',
'www\\.世**。**',
],
// Only the `.` of `www.` is escaped.
[
[
['aww. www.世', 0],
['!', bold],
],
'aww. www\\.&#x4E16;**!**',
'aww. www\\.世**!**',
],
[
[
['WWW.é', 0],
['!', bold],
],
'WWW\\.&#xE9;**!**',
'WWW\\.&#xE9;**!**',
],
[
[
['www.', 0],
['x', bold],
],
'www\\.**x**',
'www\\.**x**',
],
[
[
['www.', bold],
['x', 0],
],
'**www\\.**&#x78;',
'**www\\.**&#x78;',
],
[[['www.*x', 0]], 'www\\.\\*x', 'www\\.\\*x'],
[[['www.&#x4E16; now', 0]], 'www\\.\\&#x4E16; now', 'www\\.\\&#x4E16; now'],
Comment thread
rootnode-bot[bot] marked this conversation as resolved.
Resolved
])('keeps the characters after `www.` in %j', (runs, strict, cjkFriendly) => {
// micromark starts a literal on `www.` before a reference, an escape
// or a marker, which then reads as part of the link.
for (const [mode, markdown] of [
['strict-commonmark', strict],
['cjk-friendly', cjkFriendly],
] as const) {
const [exported, imported] = roundTrip(runs, mode, mode);
expect(exported).toBe(markdown);
// The import still links the decoded text, which may split a run.
expect(mergeRuns(imported)).toEqual(runs);
}
});

it('keeps a link after `www.`', () => {
const editor = createEditor();
editor.update(
() => {
$getRoot()
.clear()
.append(
$createParagraphNode().append(
$createTextNode('www.'),
$createLinkNode('u').append($createTextNode('x')),
),
);
},
{discrete: true},
);
const markdown = editor.read('latest', () => $convertToMarkdownString());
expect(markdown).toBe('www\\.[x](u)');
editor.update(() => $convertFromMarkdownString(markdown), {discrete: true});
editor.read('latest', () => {
const x = $getRoot().getAllTextNodes().at(-1);
assert(x !== undefined);
const link = x.getParent();
assert($isLinkNode(link));
expect($getRoot().getTextContent()).toBe('www.x');
expect(x.getTextContent()).toBe('x');
expect(link.getURL()).toBe('u');
});
});

it.each<[string, boolean | undefined, string, string]>([
['in a paragraph', undefined, '世', '[世**。**](u)'],
['in a checked item', true, '世', '- [x] [世**。**](u)'],
Expand Down
Loading