Repository navigation
Conversation
…er it import as typed ## Description With `MdastAutolinkLiteralExtension` (part of `MdastGfmExtension`), export escapes the `.` of `www.` only before an ASCII letter or digit, `_`, `-` or `.` as typed. What the export writes after it can still be something else: a reference when the character is non-ASCII and beside formatting, an escape when it is punctuation, or the marker of a format that starts or ends there, or of a link. micromark starts a literal on `www.` before any of these and reads it as part of the link, so the characters change on import. `www.世` before bold `!` is written as `www.世**!**` and read back as a link to `http://www.世`, then `;` and `!`; before bold `。` the link takes the `**` and the bold is lost; `www._x` is read back as a link to `http://www.\_x`; `www.` before bold `x` is written as `www.**x**` and read back as a link to `http://www.**x`, then `**`. GitHub renders `www.世**!**` the same way. This adds an unsafe rule to the extension that escapes the `.` after `www` (any case) in text, whatever follows it. The rule has no `after`, because mdast-util-to-markdown skips a rule that has an `after` where a rule with no condition escapes the next character, as one does `*`. The import may still link the decoded text, but its characters are the ones typed. Plain `www.example.com` was written as `www\.example.com` already. The escape is also written where it is not needed, such as `see www\. then`, which imports as before. ## Changeset `.changeset/mdast-www-dot-escape.md` (`@lexical/mdast` patch). ## Test plan ### Before `TZ=UTC pnpm vitest run --project unit packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts` with the new tests and the source of `main`: ``` FAIL |unit| packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts > Mdast punctuation modes > keeps the characters after `www.` in [["www.世",0],["!",1]] AssertionError: expected 'www.世**!**' to be 'www\.世**!**' // Object.is equality ... Tests 9 failed | 101 passed (110) ``` Importing that export on `main` (both punctuation modes) gives `LINK(http://www.世)["www.世",0] [";",0] ["!",1]`. ### After ``` $ TZ=UTC pnpm vitest run --project unit packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts packages/lexical-mdast/src/__tests__/unit/MdastImportExport.test.ts Tests 229 passed (229) $ TZ=UTC pnpm run test-unit Test Files 1021 passed | 6 skipped (1027) Tests 40248 passed | 423 expected fail | 174 skipped (40845) $ VITEST_BROWSER=chromium,firefox,webkit TZ=UTC pnpm vitest run --project browser dev-examples/mdast-editor/src/__tests__/browser Test Files 12 passed (12) Tests 90 passed (90) ``` These exited 0 with this output, apart from a Browserslist data-age warning from eslint and build: ``` $ tsc $ tsc -p ./tsconfig.test.json $ prettier --list-different . $ pnpm exec eslint packages/lexical-mdast/src/MdastExtension.ts packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts packages/lexical-mdast/src/__tests__/unit/MdastImportExport.test.ts $ node scripts/build.mjs $ node ./scripts/changesets/check.mjs Changeset check passed: 192 pending changeset(s) valid. ``` The browser tests above are the ones that use `@lexical/mdast`. No e2e test was run: the playground does not use `@lexical/mdast`.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
AI review status for this pull request.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
There was a problem hiding this comment.
This looks good, and thanks for the thorough reproduction. Escaping the . of www. in phrasing text, outside links and labels, matches how micromark's autolink-literal grammar works: it starts a literal after www. whatever character follows, so no after condition could list every case. Link titles and fence info strings stay unescaped, and the new cases fail against main and pass here.
Two optional, non-blocking suggestions:
.changeset/mdast-www-dot-escape.md:5: most editors get this extension throughMdastGfmExtension, so naming that bundle in the summary would help changelog readers recognise the change. The worked example could also move into theVerify:paragraph, so the summary stays one short line (see.changeset/README.md).packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts, the table ending at line 979: the old export also loseswww.before inline code (see www.`x` nowimports as a link that swallows the code span). A row like[['see www.', 0], ['x', code]]would pin that marker too.
…angeset summary ## Description Follow-up to the review: the export on `main` also loses `www.` before inline code (``see www.`x` now`` imports as a link to ``http://www.`x` `` and the code format is lost). This adds that case to the `www.` test table, and shortens the changeset summary to one line that names `MdastGfmExtension`, with the example moved into `Verify:`. No source change. ## Changeset `.changeset/mdast-www-dot-escape.md` (summary and `Verify:` reworded). ## Test plan ### Before `TZ=UTC pnpm vitest run --project unit packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts` with the source of `main`: ``` AssertionError: expected 'see www.`x` now' to be 'see www\.`x` now' // Object.is equality ... Tests 10 failed | 101 passed (111) ``` ### After ``` $ TZ=UTC pnpm vitest run --project unit packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts packages/lexical-mdast/src/__tests__/unit/MdastImportExport.test.ts Tests 230 passed (230) $ TZ=UTC pnpm run test-unit Test Files 1021 passed | 6 skipped (1027) Tests 40249 passed | 423 expected fail | 174 skipped (40846) $ VITEST_BROWSER=chromium,firefox,webkit TZ=UTC pnpm vitest run --project browser dev-examples/mdast-editor/src/__tests__/browser Test Files 12 passed (12) Tests 90 passed (90) $ node ./scripts/changesets/check.mjs Changeset check passed: 192 pending changeset(s) valid. ``` These exited 0 with this output, apart from a Browserslist data-age warning from eslint: ``` $ tsc -p ./tsconfig.test.json $ prettier --list-different . $ pnpm exec eslint packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts ```
There was a problem hiding this comment.
This looks good. Escaping the . of www. in phrasing text, outside links, images and labels, matches how micromark's GFM autolink-literal grammar behaves: it starts a literal at www. and reads through whatever the export writes next, so no after condition could list every case. Link titles and fence info strings are untouched, and the new cases fail against main and pass here. Thanks for adding the inline-code row and moving the worked example into Verify:.
One optional, non-blocking note:
packages/lexical-mdast/src/MdastExtension.ts:515:before: '[Ww]{3}'needs all threews in the stringsafe()checks, butmdast-util-to-markdowngives a text node only the last character of its previous sibling asbefore. Awww.split across sibling text nodes therefore still goes out unescaped. That happens when an inline element with no export rule of its own, such as aMarkNode, starts or ends insidewww. For example,see ww+ mark(w.世) + bold!exports assee www.世**!**and reads back as a link tohttp://www.世, then;.maindoes the same, so this is not a regression. A follow-up could join adjacent text siblings before serializing, or the comment above the rule could note the limit.
| { | ||
| unsafe: [ | ||
| { | ||
| before: '[Ww]{3}', |
There was a problem hiding this comment.
Optional, non-blocking: before: '[Ww]{3}' needs all three ws in the string safe() checks, but mdast-util-to-markdown gives a text node only the last character of its previous sibling as before. A www. split across sibling text nodes, such as one where an inline element with no export rule (a MarkNode) starts inside www, still goes out unescaped: see ww + mark(w.世) + bold ! exports as see www.世**!** and reads back as a link to http://www.世, then ;. main does the same, so this is not a regression; joining adjacent text siblings before serializing, or a line in this comment about the limit, would cover it.
There was a problem hiding this comment.
Joined adjacent text siblings before serializing in f4e59ed, so the rule sees the whole www. That also stops 1 + mark(. x) from starting an ordered list.
…ent splits ## Description Follow-up to the review: an inline element with no export rule of its own, such as a `MarkNode`, exports its children in place, beside the text around it, as separate text nodes. `mdast-util-to-markdown` gives a text only the last character of the one before it to decide its escapes, so the `www` rule does not see a `www` split over them: `see ww` + mark(`w.世`) + bold `!` was still written as `see www.世**!**` and read back as a link to `http://www.世`, then `;` and `!`. `main` does the same. The export now joins each run of adjacent text siblings into one text before serializing, so escapes are decided on the whole text: the rule sees the whole `www`, and `1` before a mark of `. x`, which was written as `1. x` and read back as an ordered list, is written as `1\. x`. A text with anything but its value, such as `data` or a `position` that an export rule set for a handler to read, is kept apart. ## Changeset `.changeset/mdast-www-dot-escape.md` (summary now names this case). ## Test plan ### Before `TZ=UTC pnpm vitest run --project unit packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts` with the new tests and the previous `MdastExport.ts`: ``` FAIL |unit| packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts > Mdast punctuation modes > keeps the characters after a `www.` a mark splits (strict-commonmark) AssertionError: expected 'see www.世**!**' to be 'see www\.世**!**' // Object.is equality FAIL |unit| packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.ts > Mdast punctuation modes > escapes text a mark splits as one text (cjk-friendly) AssertionError: expected '1. x' to be '1\. x' // Object.is equality Tests 7 failed | 116 passed (123) ``` ### After ``` $ TZ=UTC pnpm vitest run --project unit packages/lexical-mdast Tests 552 passed (552) $ TZ=UTC pnpm run test-unit Test Files 1021 passed | 6 skipped (1027) Tests 40261 passed | 423 expected fail | 174 skipped (40858) ``` These exited 0 with this output, apart from a Browserslist data-age warning from eslint and build: ``` $ tsc $ tsc -p ./tsconfig.test.json $ node ./scripts/check-flow-types.mjs Running Flow... Found 0 errors $ prettier --list-different . $ pnpm exec eslint packages/lexical-mdast/src $ node scripts/build.mjs $ node ./scripts/changesets/check.mjs Changeset check passed: 192 pending changeset(s) valid. ```
Description
With
MdastAutolinkLiteralExtension(part ofMdastGfmExtension), export escapes the.ofwww.only before an ASCII letter or digit,_,-or.as typed. What the export writes after it can still be something else: a reference when the character is non-ASCII and beside formatting, an escape when it is punctuation, or the marker of a format that starts or ends there, or of a link. micromark starts a literal onwww.before any of these and reads it as part of the link, so the characters change on import.www.世before bold!is written aswww.世**!**and read back as a link tohttp://www.世, then;and!; before bold。the link takes the**and the bold is lost;www._xis read back as a link tohttp://www.\_x;www.before boldxis written aswww.**x**and read back as a link tohttp://www.**x, then**. GitHub renderswww.世**!**the same way.This adds an unsafe rule to the extension that escapes the
.afterwww(any case) in text, whatever follows it. The rule has noafter, because mdast-util-to-markdown skips a rule that has anafterwhere a rule with no condition escapes the next character, as one does*. The import may still link the decoded text, but its characters are the ones typed. Plainwww.example.comwas written aswww\.example.comalready. The export also joins adjacent text siblings before serializing, so awwwthat an inline element with no export rule (aMarkNode) splits is escaped as one text, and so is a1before a mark of. x, which would otherwise start an ordered list; a text with anything but its value, such asdataor apositionan export rule set, is kept apart. The escape is also written where it is not needed, such assee www\. then, which imports as before.Changeset
.changeset/mdast-www-dot-escape.md(@lexical/mdastpatch).Test plan
Before
TZ=UTC pnpm vitest run --project unit packages/lexical-mdast/src/__tests__/unit/MdastPunctuation.test.tswith the new tests and the source ofmain:Importing that export on
main(both punctuation modes) givesLINK(http://www.世)["www.世",0] [";",0] ["!",1].After
These exited 0 with this output, apart from a Browserslist data-age warning from eslint and build:
The browser tests above are the ones that use
@lexical/mdast. No e2e test was run: the playground does not use@lexical/mdast.