Repository navigation
[lexical-markdown] Bug Fix: Read hard line breaks from the Markdown, not from the imported text #9434
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
x-labb
wants to merge
3
commits into
meta:main
Choose a base branch
from
x-labb:fix-9408-hard-break-from-markdown
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+246
−0
Open
[lexical-markdown] Bug Fix: Read hard line breaks from the Markdown, not from the imported text #9434
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| '@lexical/markdown': patch | ||
| --- | ||
|
|
||
| [fix] Read a hard line break at the end of an imported line from its Markdown, so an escaped backslash and inline code ending in `\` or two spaces no longer gain a hard break, and `a\\\` keeps its hard break | ||
|
|
||
| Verify: import `a\\` followed by `b` and `` `a\` `` followed by `b`, which keep their backslash and stay soft line breaks, then export them and import the result again. `a\\\` followed by `b` keeps its hard break. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
175 changes: 175 additions & 0 deletions
175
packages/lexical-markdown/src/__tests__/unit/HardLineBreakFromMarkdown.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,175 @@ | ||
| /** | ||
| * Copyright (c) Meta Platforms, Inc. and affiliates. | ||
| * | ||
| * This source code is licensed under the MIT license found in the | ||
| * LICENSE file in the root directory of this source tree. | ||
| * | ||
| */ | ||
|
|
||
| // @vitest-environment node | ||
|
|
||
| import {buildEditorFromExtensions} from '@lexical/extension'; | ||
| import { | ||
| $convertFromMarkdownString, | ||
| $convertToMarkdownString, | ||
| type Transformer, | ||
| TRANSFORMERS, | ||
| } from '@lexical/markdown'; | ||
| import { | ||
| $getRoot, | ||
| $getState, | ||
| $isElementNode, | ||
| $isLineBreakNode, | ||
| $isTextNode, | ||
| type LexicalNode, | ||
| } from 'lexical'; | ||
| import {describe, expect, it} from 'vitest'; | ||
|
|
||
| import { | ||
| hardLineBreakState, | ||
| type TextMatchTransformer, | ||
| } from '../../MarkdownTransformers'; | ||
| import {MarkdownTestExtension} from '../utils'; | ||
|
|
||
| // Describes the inline content of the first block: text as it is, code as | ||
| // `code(...)`, and each line break as `<soft>` or `<hard \>` / `<hard >`. | ||
| function $describeNode(node: LexicalNode): string { | ||
| if ($isLineBreakNode(node)) { | ||
| const marker = $getState(node, hardLineBreakState); | ||
| return marker ? `<hard ${marker}>` : '<soft>'; | ||
| } | ||
| if ($isTextNode(node)) { | ||
| const text = node.getTextContent(); | ||
| return node.hasFormat('code') ? `code(${text})` : text; | ||
| } | ||
| if ($isElementNode(node)) { | ||
| return node.getChildren().map($describeNode).join(''); | ||
| } | ||
| return node.getTextContent(); | ||
| } | ||
|
|
||
| function importFirstBlock( | ||
| markdown: string, | ||
| shouldMergeAdjacentLines = false, | ||
| transformers: Transformer[] = TRANSFORMERS, | ||
| ): string { | ||
| using editor = buildEditorFromExtensions([MarkdownTestExtension]); | ||
| let result = ''; | ||
| editor.update( | ||
| () => { | ||
| $convertFromMarkdownString( | ||
| markdown, | ||
| transformers, | ||
| undefined, | ||
| false, | ||
| shouldMergeAdjacentLines, | ||
| ); | ||
| const first = $getRoot().getFirstChild(); | ||
| result = first ? $describeNode(first) : ''; | ||
| }, | ||
| {discrete: true}, | ||
| ); | ||
| return result; | ||
| } | ||
|
|
||
| function roundTrip(markdown: string): string { | ||
| using editor = buildEditorFromExtensions([MarkdownTestExtension]); | ||
| editor.update( | ||
| () => { | ||
| $convertFromMarkdownString(markdown, TRANSFORMERS); | ||
| }, | ||
| {discrete: true}, | ||
| ); | ||
| return editor.read(() => $convertToMarkdownString(TRANSFORMERS)); | ||
| } | ||
|
|
||
| // In single-quoted TS strings, '\\' is one backslash. | ||
| // https://spec.commonmark.org/0.31.2/#hard-line-breaks | ||
| describe('hard line breaks are read from the Markdown (#9408)', () => { | ||
| it('reads an escaped backslash at the end of a line as a literal backslash', () => { | ||
| expect(importFirstBlock('a\\\\\nb')).toBe('a\\<soft>b'); | ||
| expect(importFirstBlock('_a\\\\_\nb')).toBe('a\\<soft>b'); | ||
| expect(importFirstBlock('**a\\\\**\nb')).toBe('a\\<soft>b'); | ||
| expect(importFirstBlock('> a\\\\\n> b')).toBe('a\\<soft>b'); | ||
| // The continuation line keeps its indent, as it already did. | ||
| expect(importFirstBlock('- a\\\\\n b')).toBe('a\\<soft> b'); | ||
| }); | ||
|
|
||
| it('reads an escaped backslash followed by a hard break', () => { | ||
| expect(importFirstBlock('a\\\\\\\nb')).toBe('a\\<hard \\>b'); | ||
| }); | ||
|
|
||
| it('keeps backslashes and spaces at the end of inline code', () => { | ||
| expect(importFirstBlock('`a\\`\nb')).toBe('code(a\\)<soft>b'); | ||
| expect(importFirstBlock('`a `\nb')).toBe('code(a )<soft>b'); | ||
| }); | ||
|
|
||
| it('round trips these lines unchanged', () => { | ||
| for (const markdown of [ | ||
| 'a\\\\\nb', | ||
| 'a\\\\\\\nb', | ||
| '`a\\`\nb', | ||
| '`a `\nb', | ||
| '**a\\\\**\nb', | ||
| '> a\\\\\n> b', | ||
| '- a\\\\\n b', | ||
| ]) { | ||
| expect(roundTrip(markdown)).toBe(markdown); | ||
| } | ||
| // Italic is exported with `*`. | ||
| expect(roundTrip('_a\\\\_\nb')).toBe('*a\\\\*\nb'); | ||
| }); | ||
|
|
||
| it('still reads real hard breaks', () => { | ||
| expect(importFirstBlock('a\\\nb')).toBe('a<hard \\>b'); | ||
| expect(importFirstBlock('a \nb')).toBe('a<hard >b'); | ||
| expect(importFirstBlock('a\\\\\\\\\nb')).toBe('a\\\\<soft>b'); | ||
| expect(importFirstBlock('**a**\\\nb')).toBe('a<hard \\>b'); | ||
| expect(importFirstBlock('> a\\\n> b')).toBe('a<hard \\>b'); | ||
| expect(importFirstBlock('a\\\nb\\\nc')).toBe('a<hard \\>b<hard \\>c'); | ||
| expect(importFirstBlock('[a](https://lexical.dev)\\\nc')).toBe( | ||
| 'a<hard \\>c', | ||
| ); | ||
| for (const markdown of ['a\\\nb', 'a \nb', 'a\\\\\\\\\nb', '**a**\\\nb']) { | ||
| expect(roundTrip(markdown)).toBe(markdown); | ||
| } | ||
| }); | ||
|
|
||
| it('keeps backslashes and spaces at the end of inline code in a quote or list', () => { | ||
| expect(importFirstBlock('> `a\\`\n> b')).toBe('code(a\\)<soft>b'); | ||
| expect(importFirstBlock('- `a `\n b')).toBe('code(a )<soft> b'); | ||
| expect(roundTrip('> `a\\`\n> b')).toBe('> `a\\`\n> b'); | ||
| expect(roundTrip('- `a `\n b')).toBe('- `a `\n b'); | ||
| }); | ||
|
|
||
| it('reads the hard break the same way when adjacent lines are merged', () => { | ||
| expect(importFirstBlock('a\\\\\\\nb', true)).toBe('a\\<hard \\>b'); | ||
| expect(importFirstBlock('a\\\\\nb', true)).toBe('a\\ b'); | ||
| }); | ||
|
|
||
| it('keeps the text of a node that a text match transformer took the backslash into', () => { | ||
| // The trailing `\` is part of the match, so the text of the line's last | ||
| // node no longer ends with it. Stripping a character anyway would turn | ||
| // `#foo` into `#fo`, so the text-based check decides, as it did before. | ||
| const HASHTAG_BACKSLASH: TextMatchTransformer = { | ||
| dependencies: [], | ||
| importRegExp: /#(\w+)\\$/, | ||
| regExp: /#(\w+)\\$/, | ||
| replace: (textNode, match) => { | ||
| textNode.setTextContent('#' + match[1]); | ||
| }, | ||
| type: 'text-match', | ||
| }; | ||
| expect( | ||
| importFirstBlock('x #foo\\\nb', false, [ | ||
| HASHTAG_BACKSLASH, | ||
| ...TRANSFORMERS, | ||
| ]), | ||
| ).toBe('x #foo<soft>b'); | ||
| }); | ||
|
|
||
| it('keeps a backslash at the end of a block as text', () => { | ||
| expect(importFirstBlock('a\\')).toBe('a\\'); | ||
| expect(importFirstBlock('# h\\')).toBe('h\\'); | ||
| }); | ||
| }); | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Optional, non-blocking: each escaped-backslash case here is two lines, so the line whose end is recorded is never moved before its record is read. In a three-line paragraph, line 2's record has to survive the
splicein$importBlocks()first; it does today, and one assertion would keep it that way: