Repository navigation
[lexical-markdown] Bug Fix: Read hard line breaks from the Markdown, not from the imported text - #9434
[lexical-markdown] Bug Fix: Read hard line breaks from the Markdown, not from the imported text#9434x-labb wants to merge 3 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/yjs
commit: |
|
Hola |
There was a problem hiding this comment.
Thanks for the fix: it looks good. Reading the hard break from each line's Markdown matches CommonMark 0.31.2 (§6.7 hard line breaks, §2.4 backslash escapes, and §6.1, where backslash escapes do not work in code spans), and the new tests pin the escaped-backslash and inline-code cases in both directions. Both suggestions below are optional and non-blocking.
- Guard the backslash strip (
packages/lexical-markdown/src/MarkdownTransformers.ts, line 404). When the line's Markdown ends in an unescaped\, the last character of the line's last TextNode is removed without checking that it is a backslash. A customTextMatchTransformerwhose import pattern consumes a line-final\into a TextNode (for example/#(\w+)\\$/replaced by a TextNode#foo) then importsx #foo\+ newline +bas#foand a hard break, losing a letter. None of the built-in transformers do this, butif (marker === '\\' && lastText.endsWith('\\'))keeps the previous text check for that case. - A few more test cases (
packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts): inline code at the end of a quote or list line (> `a\`/> b,- `a `/b) andshouldMergeAdjacentLines: truewitha\\\/balso change with this fix and would be cheap to pin.
…not from the imported text A hard line break is a backslash, or two or more spaces, at the end of a line outside code. Import decided this from the text of the previous line's last node, after that line's escapes and code spans had already been read: - an escaped backslash (`a\\`) at the end of a line became a hard break, and the literal backslash was lost - `a\\\` (an escaped backslash, then a hard break) ended with two backslashes once its escapes were read, so it was not a hard break and an extra backslash stayed in the text - inline code ending in `\` or two spaces (`` `a\` ``, `` `a ` ``) gained a hard break and lost those characters Every save added or removed characters, and in code it changed the code. This change reads whether each imported line ends in a hard break from the line's Markdown, with the existing `parseMarkdownHardLineBreak()`, before its inline content is imported, and records it against the last node the line produced. When the next line is appended, in a paragraph, a quote or a list item, `$createMarkdownLineBreakNode()` uses that record. Nodes that were not made from a Markdown line fall back to the current text-based check, and the line-merging path in `normalizeMarkdown()`, which already reads Markdown, is unchanged. The fix follows the one suggested in the issue. Closes meta#9408 `pnpm exec vitest --project unit --no-watch packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts` ``` × reads an escaped backslash at the end of a line as a literal backslash × reads an escaped backslash followed by a hard break × keeps backslashes and spaces at the end of inline code × round trips these lines unchanged AssertionError: expected 'a<hard \>b' to be 'a\<soft>b' // Object.is equality AssertionError: expected 'a\\<soft>b' to be 'a\<hard \>b' // Object.is equality AssertionError: expected 'code(a)<hard \>b' to be 'code(a\)<soft>b' // Object.is equality AssertionError: expected 'a\\nb' to be 'a\\\nb' // Object.is equality Tests 4 failed | 2 passed (6) ``` `pnpm exec vitest --project unit --no-watch packages/lexical-markdown packages/lexical-mdast packages/lexical-playground/__tests__/unit` ``` Test Files 45 passed (45) Tests 1181 passed (1181) ``` `tsc`, `tsc -p tsconfig.test.json`, ESLint and Prettier pass on the changed files. Only unit tests (Node) were run. The `shouldPreserveNewLines` and `shouldMergeAdjacentLines` options are covered only by the existing tests.
…cover more cases
## Description
When a line's Markdown ends in an unescaped `\`, the importer removes the last
character of the line's last text node. A custom `TextMatchTransformer` whose
import pattern consumes that line-final `\` into a text node leaves a node that
no longer ends with it, so the unconditional strip dropped a letter: `x #foo\`
followed by a line `b` imported as `#fo` and a hard break.
The strip now happens only when the text still ends with a backslash. When it
does not, the existing text-based check decides, as it did before this change.
This also adds the tests the review suggested: inline code ending in `\` or two
spaces at the end of a quote or list line, the hard break with
`shouldMergeAdjacentLines`, and the custom text match transformer above. A
changeset with a `Verify:` paragraph is added, as the repo now asks for fixes
that change output.
## Test plan
### Before
Without the `lastText.endsWith('\')` check:
`pnpm exec vitest --project unit --no-watch packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts`
```
× keeps the text of a node that a text match transformer took the backslash into
AssertionError: expected 'x #fo<hard \>b' to be 'x #foo<soft>b' // Object.is equality
Tests 1 failed | 8 passed (9)
```
### After
`pnpm exec vitest --project unit --no-watch packages/lexical-markdown packages/lexical-mdast packages/lexical-playground/__tests__/unit`
```
Test Files 57 passed (57)
Tests 1843 passed (1843)
```
`tsc`, `tsc -p tsconfig.test.json`, ESLint, Prettier and
`pnpm run check-changesets` pass. Only unit tests (Node) were run.
1fffe68 to
bd8b3d7
Compare
There was a problem hiding this comment.
Thanks for the update: this looks good. Reading the hard break from each line's Markdown matches CommonMark 0.31.2 (§6.7 hard line breaks, §2.4 backslash escapes, and §6.1, where backslash escapes do not work in code spans). The lastText.endsWith('\\') guard in packages/lexical-markdown/src/MarkdownTransformers.ts closes the custom text-match case raised in the earlier inline comment, and the new tests now cover inline code in quotes and list items and shouldMergeAdjacentLines. Both suggestions below are optional and non-blocking.
- Refresh the Test plan output. The
### Beforeand### Afterblocks in the description come from the first commit (Tests 4 failed | 2 passed (6)), butHardLineBreakFromMarkdown.test.tsnow has 9 tests and the branch sits on a newermain. AGENTS.md ("Commits and Pull Requests") asks the Test plan to carry the real output of the command you ran, so rerunning it without and with the fix and pasting both results would bring it up to date. - Pin the character-reference case (
packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts). The fix also changesa\followed byb: it used to import asaplus a hard break, losing the backslash, and now imports asa\plus a soft break, which is what CommonMark §2.5 asks for (character references cannot stand in for structural characters). One assertion would keep it that way.
…lash stays a literal backslash ## Description Adds a unit test that `a&meta#92;` followed by `b` imports as `a\` and a soft line break. The character reference is a literal backslash (CommonMark 0.31.2, section 2.5), not an escape and not a hard break. Before the hard-break fix it became a hard break and lost the backslash. ## Test plan ### Before Source files from the base `main`: `pnpm exec vitest --project unit --no-watch packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts` ``` × reads a character reference for a backslash as a literal backslash AssertionError: expected 'a<hard \>b' to be 'a\<soft>b' // Object.is equality Tests 7 failed | 3 passed (10) ``` ### After ``` Tests 10 passed (10) ``` `pnpm exec vitest --project unit --no-watch packages/lexical-markdown packages/lexical-mdast packages/lexical-playground/__tests__/unit`: 57 test files, 1844 tests passed. `tsc -p tsconfig.test.json`, Prettier and `pnpm run check-changesets` pass. Only unit tests (Node) were run.
There was a problem hiding this comment.
Thanks for the update: this looks good. Reading each line's hard break from its Markdown matches CommonMark 0.31.2 (§6.7 hard line breaks, §2.4 backslash escapes, §2.5 character references, and §6.1, where backslash escapes do not work in code spans). The lastText.endsWith('\\') guard in packages/lexical-markdown/src/MarkdownTransformers.ts covers the custom text-match case from the earlier inline comment, and the new \ test pins the character-reference case. Both suggestions below are optional and non-blocking.
- Pin a line that is joined before the next one arrives (
packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts). Every escaped-backslash case is two lines, so the line whose end is recorded is always the block's first line and is never moved before it is read. Ina⏎b\\⏎c, line 2's record has to survive thesplicein$importBlocks()(MarkdownImport.ts) before line 3 reads it. It does today; one assertion would keep it that way:expect(importFirstBlock('a\nb\\\\\nc')).toBe('a<soft>b\\<soft>c');
- Refresh the Test plan output. The
### Beforeblock showsTests 4 failed | 2 passed (6)and### Aftershows 1181 tests, both from the first commit; the file now has 10 tests, 7 of which fail without the fix. AGENTS.md ("Commits and Pull Requests") asks the Test plan to carry the real output of the command you ran, so rerunning it without and with the fix and pasting both results would bring it up to date.
| expect(importFirstBlock('**a\\\\**\nb')).toBe('a\\<soft>b'); | ||
| expect(importFirstBlock('> a\\\\\n> b')).toBe('a\\<soft>b'); | ||
| // The continuation line keeps its indent, as it already did. | ||
| expect(importFirstBlock('- a\\\\\n b')).toBe('a\\<soft> b'); |
There was a problem hiding this comment.
Optional, non-blocking: each escaped-backslash case here is two lines, so the line whose end is recorded is never moved before its record is read. In a three-line paragraph, line 2's record has to survive the splice in $importBlocks() first; it does today, and one assertion would keep it that way:
| expect(importFirstBlock('- a\\\\\n b')).toBe('a\\<soft> b'); | |
| expect(importFirstBlock('- a\\\\\n b')).toBe('a\\<soft> b'); | |
| // A joined line keeps its record until the next line reads it. | |
| expect(importFirstBlock('a\nb\\\\\nc')).toBe('a<soft>b\\<soft>c'); |
Description
A hard line break is a backslash, or two or more spaces, at the end of a line outside code. Import decided this from the text of the previous line's last node, after that line's escapes and code spans had already been read:
a\\) at the end of a line became a hard break, and the literal backslash was losta\\\(an escaped backslash, then a hard break) ended with two backslashes once its escapes were read, so it was not a hard break and an extra backslash stayed in the text\or two spaces (`a\`,`a `) gained a hard break and lost those charactersEvery save added or removed characters, and in code it changed the code.
This change reads whether each imported line ends in a hard break from the line's Markdown, with the existing
parseMarkdownHardLineBreak(), before its inline content is imported, and records it against the last node the line produced. When the next line is appended, in a paragraph, a quote or a list item,$createMarkdownLineBreakNode()uses that record. Nodes that were not made from a Markdown line fall back to the current text-based check, and the line-merging path innormalizeMarkdown(), which already reads Markdown, is unchanged. The fix follows the one suggested in the issue.AI disclosure: this change, its tests and this description were prepared with the help of an AI coding assistant.
Closes #9408
Test plan
Before
pnpm exec vitest --project unit --no-watch packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.tsAfter
pnpm exec vitest --project unit --no-watch packages/lexical-markdown packages/lexical-mdast packages/lexical-playground/__tests__/unittsc,tsc -p tsconfig.test.json, ESLint and Prettier pass on the changed files. Only unit tests (Node) were run. TheshouldPreserveNewLinesandshouldMergeAdjacentLinesoptions are covered only by the existing tests.