diff --git a/.changeset/escape-markdown-text.md b/.changeset/escape-markdown-text.md new file mode 100644 index 0000000000..69c4d0da89 --- /dev/null +++ b/.changeset/escape-markdown-text.md @@ -0,0 +1,28 @@ +--- +'@portabletext/markdown': patch +--- + +fix: escape markdown syntax in plain span text during serialization + +Literal markdown punctuation in span text now survives the round trip: serialization escapes it, so it parses back as the same literal text instead of turning into markup. + +```ts +// span text markdown (before) markdown (now) re-parses as +'*bar*' // *bar* \*bar\* the text `*bar*` (was: an `em` span reading `bar`) +'# heading' // # heading \# heading the text `# heading` (was: an `h1`) +'[x]: y' // [x]: y \[x]: y the text `[x]: y` (was: nothing, consumed as a link reference) +``` + +Escaping accounts for the block context a span renders into (a heading, a blockquote, a list item, a table cell) and for hazards that only appear once adjacent spans or marks are joined, such as a link pattern or an ordered-list marker split across spans. Text with the `code` decorator is never backslash-escaped; instead its backtick delimiters widen past any backtick run in the content (with space padding when the content starts or ends with a backtick), so the content survives verbatim: + +```ts +// span text with the `code` decorator markdown (before) markdown (now) +'a`b' // `a`b` (broken) ``a`b`` +'`a' // ``a` (broken) `` `a `` +``` + +Markdown output for text containing such punctuation gains backslash escapes it didn't have before. + +Consumer mark and block renderers now receive pre-escaped `children`. A renderer that needs the original, unescaped text (the `code` decorator's own default renderer does this) reads it from the `text` argument instead. + +Two exceptions. A bare URL, email, or `www.` address in plain text is never escaped: it keeps its text and typically gains only a `link` mark on the next parse, since linkifying such text is expected parser behavior, not something to suppress. Adjacent inline constructs can still claim marks of their own (an entity reference, a backtick, or a mark boundary sitting inside what would otherwise be that URL or email means the parser wouldn't have linkified it either, so normal escaping applies there instead). Leading or trailing whitespace that CommonMark's own block parsing trims is unaffected by this change, same as before it. diff --git a/apps/docs/src/content/docs/conversion/markdown-to-portable-text.mdx b/apps/docs/src/content/docs/conversion/markdown-to-portable-text.mdx index c0bef8616d..1b22e72508 100644 --- a/apps/docs/src/content/docs/conversion/markdown-to-portable-text.mdx +++ b/apps/docs/src/content/docs/conversion/markdown-to-portable-text.mdx @@ -104,7 +104,7 @@ This package uses markdown-it as its Markdown parser. Remark and unified plugins Converting Markdown to Portable Text and back isn't a lossless mirror. Five things to expect: - Translation normalizes rather than preserves: a first Markdown → Portable Text → Markdown pass rewrites Markdown to one canonical spelling. Autolinks and reference links become inline links, indented code becomes fenced code, and `` comes back as `[https://portabletext.org](https://portabletext.org)`. -- The normalized form is a fixpoint for the constructs in the table above: parsing it and serializing again reproduces it byte-for-byte. The exception is plain text containing literal Markdown punctuation: serialization doesn't yet escape it, so `\*bar\*` comes back as `*bar*`, which a second parse reads as emphasis. +- The normalized form is a fixpoint for the constructs in the table above, and for plain text: parsing it and serializing again reproduces it byte-for-byte, because literal Markdown punctuation in plain text is backslash-escaped on the way out (`*bar*` comes back as `\*bar\*`, which a second parse reads as the literal text). Three exceptions: a bare URL, email, or `www.` address stays byte-identical but gains a `link` mark on the next parse; a hard break inside a heading forces a structural split into a second block on reparse, since an ATX heading is single-line (the text itself still survives, split across the two blocks); and leading or trailing whitespace that CommonMark's own block parsing trims isn't part of the fixpoint claim. - Unrecognized constructs degrade, they don't fail. A mark, list, or task checkbox whose type isn't in the schema keeps its text and drops the formatting: an undeclared `strong` decorator turns `**bar**` into a plain span reading `bar`. - Portable Text structures with no Markdown form degrade predictably going back out. GFM has one header row, so extra header rows flatten into the body; deep or level-skipping lists collapse to relative nesting; unknown object types render as a fenced JSON block. - Keys and span boundaries aren't identity: every parse regenerates block and span keys, and adjacent spans with identical marks merge. diff --git a/packages/markdown/README.md b/packages/markdown/README.md index 820837520e..7261bd91ed 100644 --- a/packages/markdown/README.md +++ b/packages/markdown/README.md @@ -88,12 +88,14 @@ const markdown = portableTextToMarkdown([ [ref link][id] -> [ref link](https://example.com "title") ``` -2. The normalized Markdown is a fixpoint for the constructs in the [Supported features](#supported-features) table above: parsing it and serializing again reproduces it byte-for-byte, pinned by a full-document round-trip test. This doesn't yet extend to plain text that happens to contain literal Markdown punctuation: serialization doesn't escape it, so a second parse reads it back as markup instead of literal text. +2. The normalized Markdown is a fixpoint for the constructs in the [Supported features](#supported-features) table above, and for plain text: parsing it and serializing again reproduces it byte-for-byte, pinned by a full-document round-trip test. Literal Markdown punctuation in plain text is backslash-escaped on serialization, so a second parse reads back the same characters instead of markup. ``` - \*bar\* -> *bar* (serialized unescaped; a second parse reads this as emphasis) + *bar* -> \*bar\* (escaped on serialization; a second parse reads back the literal text) ``` + Three exceptions. A bare URL, email, or `www.` address is never escaped: text identity holds, but it gains a `link` mark on the next parse (autolinking is a parser feature, not a round-trip bug). A hard break inside a heading forces a structural split into a second block on reparse, since an ATX heading is single-line; the text itself still survives, split across the two blocks. And leading or trailing whitespace that CommonMark's own block parsing trims isn't part of the fixpoint claim. + 3. MD→PT survival is schema-driven. Constructs whose type the schema doesn't declare degrade predictably: they keep their content and drop the structure that named them. Marks drop formatting but keep the text (`**bar**` with no `strong` decorator in the schema becomes a plain span reading `bar`); tables flatten their cell content into top-level blocks; images fall back to their Markdown source as plain text; task-list checkboxes strip to plain list items. 4. PT structures with no Markdown form degrade predictably on PT→MD. GFM tables have one header row, so header rows beyond the first flatten into the body. Deep or level-skipping lists collapse to relative nesting. A list's first item renders at the top level whatever its `level`, and each deeper jump between items indents one step, however many levels it skips. Multi-block table cells join their blocks with spaces. Unknown object types render as a fenced JSON block; unknown marks pass their text through unformatted. diff --git a/packages/markdown/package.json b/packages/markdown/package.json index 09303ee97f..376cbd4e41 100644 --- a/packages/markdown/package.json +++ b/packages/markdown/package.json @@ -52,6 +52,7 @@ "@mdit/plugin-alert": "^0.23.2", "@portabletext/schema": "workspace:^", "@portabletext/toolkit": "^6.0.0", + "linkify-it": "^5.0.2", "markdown-it": "^14.3.0" }, "devDependencies": { @@ -59,6 +60,7 @@ "@portabletext/types": "^4.0.2", "@sanity/pkg-utils": "catalog:tooling", "@sanity/tsconfig": "catalog:tooling", + "@types/linkify-it": "^5.0.0", "@types/markdown-it": "^14.1.2", "typescript": "catalog:tooling", "vite": "catalog:tooling", diff --git a/packages/markdown/src/escape.test.ts b/packages/markdown/src/escape.test.ts index 0b23f4e409..b027d083b5 100644 --- a/packages/markdown/src/escape.test.ts +++ b/packages/markdown/src/escape.test.ts @@ -77,6 +77,16 @@ describe(escapeTableCell.name, () => { expect(escapeTableCell('a \\| b')).toBe('a \\| b') }) + test('escapes a pipe behind an even number of backslashes', () => { + // Two backslashes cancel out to a literal backslash, leaving the pipe + // live, the shape leaf escaping produces from a literal `\|` in a span. + expect(escapeTableCell('a\\\\| b')).toBe('a\\\\\\| b') + }) + + test('leaves a pipe behind an odd number of backslashes intact', () => { + expect(escapeTableCell('a\\\\\\| b')).toBe('a\\\\\\| b') + }) + test('leaves backslashes alone', () => { expect(escapeTableCell('a\\b')).toBe('a\\b') }) diff --git a/packages/markdown/src/escape.ts b/packages/markdown/src/escape.ts index 5222352a26..f30ec963eb 100644 --- a/packages/markdown/src/escape.ts +++ b/packages/markdown/src/escape.ts @@ -23,16 +23,21 @@ export function escapeImageAndLinkTitle(text: string): string { * Escapes characters that have special meaning at the row level of a GFM * table cell. * - * A literal `|` ends the cell, so unescaped pipes are replaced with `\|`. - * Newlines end the row, so they are replaced with `
` to keep the - * visible line break inside the cell. Already-escaped pipes (`\|`) are - * left intact so that escapes introduced by mark renderers survive the - * pass. + * A literal `|` ends the cell, so a pipe preceded by an even number of + * backslashes (including zero) gets one more: paired backslashes cancel + * out to a literal backslash and leave the pipe live, so parity, not mere + * presence, decides whether it is already escaped. Newlines end the row, + * so they are replaced with `
` to keep the visible line break inside + * the cell. * - * Backslashes are intentionally not escaped here so that other escapes - * already in the rendered cell (such as `\[` and `\]` in link text) are - * not double-escaped. + * Backslashes themselves are left alone here; only the parity check reads + * them, so escapes already in the rendered cell (such as `\[` and `\]` in + * link text) survive the pass untouched. */ export function escapeTableCell(text: string): string { - return text.replace(/(?') + return text + .replace(/(\\*)\|/g, (match, backslashes: string) => + backslashes.length % 2 === 0 ? `${backslashes}\\|` : match, + ) + .replace(/\n/g, '
') } diff --git a/packages/markdown/src/example-document.advanced.out.md b/packages/markdown/src/example-document.advanced.out.md index b14e7c58bb..21131a45c9 100644 --- a/packages/markdown/src/example-document.advanced.out.md +++ b/packages/markdown/src/example-document.advanced.out.md @@ -90,7 +90,7 @@ Here's a link to [a website](http://foo.bar), to a [local
doc](local-doc.html), and to a [section heading in the current
doc](#an-h2-header). Here's a footnote [^1]. -[^1]: Footnote text goes here. +\[^1]: Footnote text goes here. Tables can look like this: @@ -155,4 +155,4 @@ math should get its own line and be put in in double-dollarsigns: $$I = \int \rho R^{2} dV$$ And note that you can backslash-escape any punctuation characters
-which you wish to be displayed literally, ex.: `foo`, *bar*, etc. +which you wish to be displayed literally, ex.: \`foo\`, \*bar\*, etc. diff --git a/packages/markdown/src/example-document.out.md b/packages/markdown/src/example-document.out.md index bb668629cb..8b4a2d12a4 100644 --- a/packages/markdown/src/example-document.out.md +++ b/packages/markdown/src/example-document.out.md @@ -176,7 +176,7 @@ Check out these autolinks: [https://example.com](https://example.com) and [mailt

This is raw HTML that gets preserved

-Inline HTML like highlighted text can be handled too. +Inline HTML like \highlighted text\ can be handled too. ### Reference Links diff --git a/packages/markdown/src/from-portable-text/escape-plain-text.ts b/packages/markdown/src/from-portable-text/escape-plain-text.ts new file mode 100644 index 0000000000..3b20631608 --- /dev/null +++ b/packages/markdown/src/from-portable-text/escape-plain-text.ts @@ -0,0 +1,726 @@ +import {isPortableTextSpan} from '@portabletext/toolkit' +import type { + ArbitraryTypedObject, + PortableTextMarkDefinition, + PortableTextSpan, +} from '@portabletext/types' +import LinkifyIt from 'linkify-it' + +/** + * The CommonMark ASCII punctuation set. Only these characters can be + * backslash-escaped into a literal without changing the parsed text. + */ +const ASCII_PUNCTUATION = /[!-/:-@[-`{-~]/ + +const ENTITY_REFERENCE = /&(?:[a-zA-Z][a-zA-Z0-9]*|#[0-9]+|#[xX][0-9a-fA-F]+);/g +const BACKSLASH_BEFORE_PUNCTUATION = new RegExp( + `\\\\(?=${ASCII_PUNCTUATION.source})`, + 'g', +) +const TILDE_RUN = /~{2,}/g +const HTML_LIKE_ANGLE_BRACKET = /<(?=[a-zA-Z/!?])/g +const BRACKET_BEFORE_LINK_OPEN = /\](?=[([])/g + +/** + * markdown-it's own emphasis-flanking rule classifies a delimiter's + * neighbor as punctuation using Unicode's `P` (Punctuation) and `S` + * (Symbol) categories, not just ASCII: an emoji, an em dash, or a CJK + * character flanks a `*`/`_` run exactly like an ASCII punctuation + * character does, so testing ASCII alone would fabricate emphasis next to + * non-ASCII text the real reparse wouldn't create. Matched against one + * full code point at a time (a surrogate pair for an astral character + * like an emoji), never a lone UTF-16 code unit, which the category + * classes never match. + */ +const UNICODE_PUNCTUATION_OR_SYMBOL = /^(?:\p{P}|\p{S})$/u + +// Configured with no options, exactly like markdown-it constructs its own +// `md.linkify` instance: same default schemas (http/https/ftp/'//'/mailto) +// and the same built-in TLD list, so a range this reports as a link is a +// range the real reparse will claim too. +const linkify = new LinkifyIt() + +type LeafPiece = + | {kind: 'text'; raw: string; isLinkLabel: boolean; markSignature: string} + | {kind: 'hardBreak'} + | {kind: 'opaque'} + +/** + * Stands in for one opaque child (an inline object, or a leaf a custom + * renderer will replace) in a joined line's text. It can never match a + * hazard's trigger character, so it safely walls off an in-progress + * construct (an ordered-list marker, a ref-def label) without needing a + * dedicated flag, while still counting as real, non-whitespace content for + * emphasis-flanking purposes. + */ +const OPAQUE_CHAR = '\0' + +/** Which leaf (by index into the flat `pieces` list) a line character came + * from, and at what offset into that leaf's *prepared* text (raw text, with + * a link-label leaf's brackets/backslashes already doubled). `null` marks + * an opaque character: it has no leaf-local home, so no edit can target it. */ +interface LineChar { + pieceIndex: number + offset: number +} + +/** A single-position splice against a joined line's raw text: remove + * `deleteCount` characters at `at` and put `insert` in their place. A pure + * insertion (a backslash escape) has `deleteCount: 0`. */ +interface Edit { + at: number + deleteCount: number + insert: string + /** + * Set for an entity-reference or backtick escape: both change what + * markdown-it's inline parser hands to its linkify pass (an entity + * decodes first, a backtick can open a code span first), so the linkify + * mask - built from this line's raw, undecoded text - can never be + * trusted to have already accounted for them. Kept regardless of any + * linkify claim overlapping it. + */ + bypassLinkifyMask?: boolean +} + +/** + * Plans the escaped replacement for every plain-text leaf a block's children + * will produce, in the exact left-to-right order `renderText` visits them + * (mirroring `buildMarksTree`'s own `text.split('\n')` leaf splitting). + * + * Escaping runs ahead of rendering, over the flat span sequence: some + * hazards only exist across a leaf boundary (an ordered-list marker, a + * ref-def label, an emphasis run) because an annotation or decorator mark + * that introduces no markup of its own splices its children in seamlessly. + * The plan works line by line (a block's children joined into text, split + * at hard breaks) rather than leaf by leaf: each line's leaves are joined + * into one string first, opaque children (inline objects, or leaves a + * custom renderer will replace) masked with a sentinel that can't match any + * hazard, and every hazard - inline and line-start alike - is detected once + * against that real, complete line, with true left/right context on both + * sides. Detected hazards become position-tracked edits against the line's + * raw text, which are then split back into each contributing leaf's own + * escaped text; only that composition step is leaf-scoped. + * + * A joined line's text that markdown-it's own linkify pass (bundled as + * `linkify-it`) would claim as a bare URL or email is masked from most + * edits: the linkify carve-out promises that substring round-trips + * byte-identical, gaining only a link mark, so escaping inside it would + * corrupt text linkify is about to claim as a link's visible text. An + * entity-reference or backtick escape is never masked (see `computeLinkifyMask` + * for why), and a claim spliced across a decorator boundary is never masked + * in the first place. + * + * `isHeading` is set for ATX headings: only the first joined line sits + * inside the `# ` prefix an ATX heading can never be reparsed as a block + * construct within, so line-leading hazards are skipped there; a hard + * break's later lines are ordinary markdown lines and get the full + * line-start battery. That first line carries a line-*end* hazard of its + * own instead: a trailing `#`-run reads back as the heading's own optional + * closing sequence. + * + * `isListItem` is set when the block renders as list-item content: a + * `[ ] `/`[x] `/`[X] ` at the very start of the first joined line reads + * back as a GFM task-list checkbox, regardless of the list's own item type. + * + * `hardBreakOutputHasNewline` says whether the renderer's actual hard-break + * output contains a newline. A custom `hardBreak` can render to something + * with no newline of its own (eg `() => '
'`), in which case the + * leaves on either side of it land on the same rendered line, not two: a + * hard break like that can't be planned as a line boundary, so it's walled + * off as an opaque segment instead, the same protection an inline object's + * unknown rendered text already gets. + */ +export function planLeafEscaping( + children: ReadonlyArray, + markDefs: ReadonlyArray, + options: { + isHeading: boolean + isListItem: boolean + hardBreakOutputHasNewline: boolean + }, +): Array { + const linkMarkKeys = new Set( + markDefs.filter((def) => def._type === 'link').map((def) => def._key), + ) + // Every markDef key a span's marks can reference (any annotation, not + // just link): what's left after removing those from a span's `marks` is + // its decorator set, the only marks `buildMarksTree` reliably renders as + // delimiters by default (an unregistered annotation type falls back to + // passing its children through unchanged, same as an unknown decorator). + const markDefKeys = new Set(markDefs.map((def) => def._key)) + const pieces: Array = [] + + for (const child of children) { + if (isPortableTextSpan(child)) { + const isLinkLabel = (child.marks ?? []).some((mark) => + linkMarkKeys.has(mark), + ) + // Sorted so two spans carrying the same decorators in a different + // order still compare equal: `buildMarksTree` nests by decorator + // identity, not by a span's own array order, so it produces the same + // markup boundaries either way. + const markSignature = (child.marks ?? []) + .filter((mark) => !markDefKeys.has(mark)) + .sort() + .join(',') + const lines = child.text.split('\n') + lines.forEach((line, index) => { + if (index > 0) { + pieces.push( + options.hardBreakOutputHasNewline + ? {kind: 'hardBreak'} + : {kind: 'opaque'}, + ) + } + pieces.push({kind: 'text', raw: line, isLinkLabel, markSignature}) + }) + } else { + pieces.push({kind: 'opaque'}) + } + } + + const pieceOutputs: Array = pieces.map(() => '') + + let lineIndex = 0 + let lineText = '' + let lineChars: Array = [] + let lineIsLinkLabelChar: Array = [] + let lineMarkSignature: Array = [] + + const flushLine = () => { + processLine({ + text: lineText, + chars: lineChars, + isLinkLabelChar: lineIsLinkLabelChar, + markSignature: lineMarkSignature, + lineIndex, + isHeading: options.isHeading, + isListItem: options.isListItem, + pieceOutputs, + }) + lineIndex++ + lineText = '' + lineChars = [] + lineIsLinkLabelChar = [] + lineMarkSignature = [] + } + + for (let pieceIndex = 0; pieceIndex < pieces.length; pieceIndex++) { + const piece = pieces[pieceIndex] + + if (!piece || piece.kind === 'hardBreak') { + flushLine() + continue + } + + if (piece.kind === 'opaque') { + lineText += OPAQUE_CHAR + lineChars.push(null) + lineIsLinkLabelChar.push(false) + lineMarkSignature.push('') + continue + } + + // A link label's brackets and backslashes are escaped unconditionally, + // up front: a label must stay bracket-balanced regardless of context, + // so this doesn't depend on anything the line-level hazard scan below + // discovers. + const prepared = piece.isLinkLabel + ? escapeLinkLabelBrackets(piece.raw) + : piece.raw + + for (let offset = 0; offset < prepared.length; offset++) { + lineText += prepared[offset] + lineChars.push({pieceIndex, offset}) + lineIsLinkLabelChar.push(piece.isLinkLabel) + lineMarkSignature.push(piece.markSignature) + } + } + flushLine() + + const escaped: Array = [] + pieces.forEach((piece, index) => { + if (piece.kind === 'text') { + escaped.push(pieceOutputs[index] ?? '') + } + }) + return escaped +} + +function processLine(args: { + text: string + chars: Array + isLinkLabelChar: Array + markSignature: Array + lineIndex: number + isHeading: boolean + isListItem: boolean + pieceOutputs: Array +}): void { + const {text, chars, isLinkLabelChar, markSignature, pieceOutputs} = args + + const linkifyMask = computeLinkifyMask( + text, + chars, + isLinkLabelChar, + markSignature, + ) + const edits = [ + ...collectInlineEdits(text, isLinkLabelChar), + ...collectLineStartEdits(text, args), + ].filter((edit) => edit.bypassLinkifyMask || !isMasked(edit, linkifyMask)) + + applyEdits(text, chars, edits, pieceOutputs) +} + +/** + * Marks every character of this line that markdown-it's linkify pass would + * claim as part of a bare URL or email. Link-label and opaque characters + * are blanked out first: a link label's visible text sits inside `[...]` + * markup real linkify never reconsiders, and an opaque child's rendered + * text is unknown at plan time, so neither should join or seed a match. + * + * The probe only sees this line's raw, undecoded text, one hazard pass + * ahead of markdown-it's own pipeline: it runs linkify against inline + * tokenization and entity decoding, not before them. A claim survives only + * if it lies entirely inside one run of identical decorator marks: a + * decorator boundary crossing it splices that decorator's delimiters + * (`**`, `` ` ``, ...) into the middle of the range real linkify would see, + * which breaks the very claim being trusted. An annotation-only boundary + * (a link's own label text is already excluded above; any other + * annotation type falls back to rendering with no delimiters at all, + * same as an unregistered decorator) never splices, so it can't invalidate + * a claim either. + */ +function computeLinkifyMask( + text: string, + chars: Array, + isLinkLabelChar: Array, + markSignature: Array, +): Array { + const mask = new Array(text.length).fill(false) + + // Every schema `linkify-it`'s default config recognizes - `http(s):`, + // `ftp:`, `//`, `www.`, and a `user@host` email - needs a `.`, `:`, or + // `@` somewhere in the line; skipping the match call on a line with + // none of those is the cheap majority-case exit, not a heuristic that + // could miss a real claim. + if (!/[.:@]/.test(text)) { + return mask + } + + let probe = '' + for (let index = 0; index < text.length; index++) { + probe += chars[index] === null || isLinkLabelChar[index] ? ' ' : text[index] + } + + const matches = linkify.match(probe) ?? [] + for (const match of matches) { + const signature = markSignature[match.index] + let staysWithinOneMarkRun = true + for (let index = match.index; index < match.lastIndex; index++) { + if (markSignature[index] !== signature) { + staysWithinOneMarkRun = false + break + } + } + if (!staysWithinOneMarkRun) { + continue + } + for (let index = match.index; index < match.lastIndex; index++) { + mask[index] = true + } + } + return mask +} + +function isMasked(edit: Edit, mask: ReadonlyArray): boolean { + const end = edit.at + Math.max(edit.deleteCount, 1) + for (let index = edit.at; index < end; index++) { + if (mask[index]) { + return true + } + } + return false +} + +/** Rewrites a line's raw text into each contributing leaf's escaped text by + * walking it once, left to right, applying at most one edit per position. + * Every hazard is keyed off its own trigger character - a backslash, a + * tilde, a backtick, an `&`, a `<`, a `*`/`_`, a `]`, or (line-start only, + * one hazard per line) a `#`, `>`, `[`, `-`/`+`/`*`, the `.`/`)` after an + * ordered-list marker's digits, `=`, 4 spaces, or a tab - and no two of + * those characters coincide at one position, so two edits can never target + * the same position. */ +function applyEdits( + text: string, + chars: ReadonlyArray, + edits: ReadonlyArray, + pieceOutputs: Array, +): void { + const editsByPosition = new Map() + for (const edit of edits) { + if (editsByPosition.has(edit.at)) { + throw new Error( + `Two hazard edits targeted the same position (${edit.at}); ` + + 'hazard trigger characters are assumed disjoint by construction.', + ) + } + editsByPosition.set(edit.at, edit) + } + + let index = 0 + while (index < text.length) { + const edit = editsByPosition.get(index) + const owner = chars[index] + + if (edit) { + if (owner) { + pieceOutputs[owner.pieceIndex] = + (pieceOutputs[owner.pieceIndex] ?? '') + edit.insert + } + if (edit.deleteCount > 0) { + index += edit.deleteCount + continue + } + } + + if (owner) { + pieceOutputs[owner.pieceIndex] = + (pieceOutputs[owner.pieceIndex] ?? '') + (text[index] ?? '') + } + index++ + } +} + +/** + * Hazards that can appear anywhere on a line: emphasis/strikethrough runs, + * a backtick, an entity reference, an HTML/autolink-shaped `<`, a literal + * backslash before punctuation, and a `]` immediately before `(`/`[` + * (which would otherwise read back as a link/image open). + */ +function collectInlineEdits( + text: string, + isLinkLabelChar: ReadonlyArray, +): Array { + const edits: Array = [] + + for (const match of text.matchAll(BACKSLASH_BEFORE_PUNCTUATION)) { + const at = match.index ?? 0 + // A link label's backslashes were already doubled unconditionally + // while preparing its text; doubling them again here would flip their + // parity back to unescaped. + if (!isLinkLabelChar[at]) { + edits.push({at, deleteCount: 0, insert: '\\'}) + } + } + + for (const match of text.matchAll(TILDE_RUN)) { + const start = match.index ?? 0 + for (let index = start; index < start + match[0].length; index++) { + edits.push({at: index, deleteCount: 0, insert: '\\'}) + } + } + + for (let index = 0; index < text.length; index++) { + if (text[index] === '`') { + // Every backtick is escaped outright: even a lone one can pair with + // another lone backtick elsewhere to open a code span, which the + // linkify mask can't see coming - a code span forms during inline + // tokenization, before linkify ever runs - so this bypasses it. + edits.push({ + at: index, + deleteCount: 0, + insert: '\\', + bypassLinkifyMask: true, + }) + } + } + + for (const match of text.matchAll(ENTITY_REFERENCE)) { + // An entity reference decodes before linkify runs, so a masked range + // built from this line's raw text can't already account for it. + edits.push({ + at: match.index ?? 0, + deleteCount: 0, + insert: '\\', + bypassLinkifyMask: true, + }) + } + + for (const match of text.matchAll(HTML_LIKE_ANGLE_BRACKET)) { + edits.push({at: match.index ?? 0, deleteCount: 0, insert: '\\'}) + } + + edits.push(...collectEmphasisEdits(text)) + + for (const match of text.matchAll(BRACKET_BEFORE_LINK_OPEN)) { + const at = match.index ?? 0 + // A link label's `]` was already escaped unconditionally while + // preparing its text (`escapeLinkLabelBrackets`); escaping it again + // here would double the backslash and reopen the label early on + // reparse, same reasoning as the backslash rule above. + if (!isLinkLabelChar[at]) { + edits.push({at, deleteCount: 0, insert: '\\'}) + } + } + + return edits +} + +function isWhitespace(char: string | undefined): boolean { + return char === undefined || /\s/.test(char) +} + +function isPunctuation(char: string | undefined): boolean { + return char !== undefined && UNICODE_PUNCTUATION_OR_SYMBOL.test(char) +} + +/** + * The full code point sitting immediately before `index`: two UTF-16 code + * units for an astral character (eg an emoji) whose low surrogate lands at + * `index - 1`, one otherwise. + */ +function codePointBefore(text: string, index: number): string | undefined { + if (index <= 0) { + return undefined + } + if ( + index >= 2 && + isLowSurrogate(text[index - 1]) && + isHighSurrogate(text[index - 2]) + ) { + return text.slice(index - 2, index) + } + return text[index - 1] +} + +/** + * The full code point sitting immediately at `index`: two UTF-16 code units + * for an astral character whose high surrogate lands at `index`, one + * otherwise. + */ +function codePointAt(text: string, index: number): string | undefined { + if (index >= text.length) { + return undefined + } + if (isHighSurrogate(text[index]) && isLowSurrogate(text[index + 1])) { + return text.slice(index, index + 2) + } + return text[index] +} + +function isHighSurrogate(char: string | undefined): boolean { + if (char === undefined) { + return false + } + const code = char.charCodeAt(0) + return code >= 0xd800 && code <= 0xdbff +} + +function isLowSurrogate(char: string | undefined): boolean { + if (char === undefined) { + return false + } + const code = char.charCodeAt(0) + return code >= 0xdc00 && code <= 0xdfff +} + +function isLeftFlanking( + before: string | undefined, + after: string | undefined, +): boolean { + if (isWhitespace(after)) { + return false + } + return !isPunctuation(after) || isWhitespace(before) || isPunctuation(before) +} + +function isRightFlanking( + before: string | undefined, + after: string | undefined, +): boolean { + if (isWhitespace(before)) { + return false + } + return !isPunctuation(before) || isWhitespace(after) || isPunctuation(after) +} + +/** + * Finds `*`/`_` runs CommonMark would treat as flanking delimiters, using + * each run's true neighbors on the joined line (the start/end of the line + * itself counts as whitespace, matching the spec's treatment of line + * boundaries). + */ +function collectEmphasisEdits(text: string): Array { + const edits: Array = [] + let index = 0 + + while (index < text.length) { + const char = text[index] + + if (char !== '*' && char !== '_') { + index++ + continue + } + + let end = index + while (end < text.length && text[end] === char) { + end++ + } + + const before = codePointBefore(text, index) + const after = codePointAt(text, end) + const leftFlanking = isLeftFlanking(before, after) + const rightFlanking = isRightFlanking(before, after) + + const canOpen = + char === '_' + ? leftFlanking && (!rightFlanking || isPunctuation(before)) + : leftFlanking + const canClose = + char === '_' + ? rightFlanking && (!leftFlanking || isPunctuation(after)) + : rightFlanking + + if (canOpen || canClose) { + for (let position = index; position < end; position++) { + edits.push({at: position, deleteCount: 0, insert: '\\'}) + } + } + + index = end + } + + return edits +} + +/** + * Hazards that only matter at the start (or, for a handful of whole-line + * constructs, the start *and* end) of a line: headings, blockquotes, list + * markers, ref-defs, setext underlines, thematic breaks, indented code, and + * a list item's own GFM task-checkbox prefix. A fence needs no branch of + * its own here: the inline backtick/tilde escaping every line already + * neutralizes the run a fence needs, so it can never open one on reparse. + * The remaining branches are mutually exclusive by construction + * (each targets a disjoint leading character) and return as soon as one + * matches, mirroring how CommonMark itself commits to one block-start + * interpretation per line; the checkbox branch above is the one exception, + * since a list item's checkbox prefix and, say, its heading marker are two + * independent hazards that can both apply to the same first line. + */ +function collectLineStartEdits( + text: string, + context: {isHeading: boolean; isListItem: boolean; lineIndex: number}, +): Array { + const edits: Array = [] + const isFirstLine = context.lineIndex === 0 + + // CommonMark allows up to 3 leading spaces before a block marker without + // affecting how it's parsed, so every hazard below (including the GFM + // task-checkbox the parser's own pre-pass looks for, after its own + // leading-whitespace trim) checks what follows them; the escape itself + // still has to land right before the marker, not at the front of those + // spaces (a backslash-space isn't an escape). + const leadingSpaces = /^ {0,3}/.exec(text)?.[0].length ?? 0 + const rest = text.slice(leadingSpaces) + + if (context.isListItem && isFirstLine && /^\[[ xX]\] /.test(rest)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + } + + if (context.isHeading && isFirstLine) { + // A closing sequence must be preceded by a space, unless it's *all* + // the heading has: then the space ATX headings require after their + // opening `#`s stands in for it. + const closingSequence = /^(?:(.*[ \t]))?(#+[ \t]*)$/.exec(text) + if (closingSequence) { + edits.push({ + at: closingSequence[1]?.length ?? 0, + deleteCount: 0, + insert: '\\', + }) + } + return edits + } + + const orderedListMarker = /^ {0,3}(\d{1,9})([.)])(?=[ \t]|$)/.exec(text) + if (orderedListMarker) { + edits.push({ + at: orderedListMarker[0].length - 1, + deleteCount: 0, + insert: '\\', + }) + return edits + } + + if (/^#{1,6}(?:[ \t]|$)/.test(rest)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + if (rest.startsWith('>')) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + if (/^\[[^\]\n]*\]:/.test(rest)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + if (/^[-+*](?:[ \t]|$)/.test(rest)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + // CommonMark 4.1 allows interior spaces/tabs between a thematic break's + // delimiter characters. + if (/^ {0,3}([-*_])(?:[ \t]*\1){2,}[ \t]*$/.test(text)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + if (/^ {0,3}=+[ \t]*$/.test(text)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + if (/^ {0,3}-+[ \t]*$/.test(text)) { + edits.push({at: leadingSpaces, deleteCount: 0, insert: '\\'}) + return edits + } + + if (/^ {4}/.test(text)) { + // A numeric character reference decodes to the same literal character + // during inline parsing, after block structure (and its indented-code + // -block rule, 4 columns of leading whitespace) has already been + // decided. + edits.push({at: 0, deleteCount: 1, insert: ' '}) + return edits + } + + const tabIndent = /^ {0,3}\t/.exec(text) + if (tabIndent) { + // A tab advances to the next multiple of 4 columns, so even 0-3 + // leading spaces before one reaches the indented-code-block + // threshold; encoding the tab itself (not the spaces before it) is + // enough to break that count. + edits.push({at: tabIndent[0].length - 1, deleteCount: 1, insert: ' '}) + return edits + } + + return edits +} + +/** + * Text rendered inside a link label needs every `[`, `]` and `\` escaped + * unconditionally, on top of the general-purpose hazard escaping every + * line gets: a link label must stay bracket-balanced, and any literal + * backslash in it needs protecting regardless of what follows (unlike + * plain text, where only a backslash immediately before punctuation is a + * hazard). + */ +function escapeLinkLabelBrackets(text: string): string { + return text.replace(/[[\]\\]/g, (char) => `\\${char}`) +} diff --git a/packages/markdown/src/from-portable-text/list-item-first-block.ts b/packages/markdown/src/from-portable-text/list-item-first-block.ts new file mode 100644 index 0000000000..7dffb16be7 --- /dev/null +++ b/packages/markdown/src/from-portable-text/list-item-first-block.ts @@ -0,0 +1,26 @@ +import type {PortableTextBlock} from '@portabletext/types' + +/** + * Blocks currently known to be a list item's first content block: it shares + * its first line with the list marker (and, for a task item, its GFM + * checkbox), which changes how `renderBlock` plans line-start hazard + * escaping. Internal to this package so the signal never reaches the + * public `Serializable`/`RenderNode` types a custom renderer's `.d.ts` + * would otherwise expose it through. + * + * A block is marked right before rendering it; the `renderNode` call that + * dispatches to `renderBlock` consumes the membership on the way past so a + * later, unrelated render of the same object (still possible - `renderNode` + * accepts any `TypedObject`) doesn't inherit a stale claim. + */ +const listItemFirstBlocks = new WeakSet() + +export function markListItemFirstBlock(block: PortableTextBlock): void { + listItemFirstBlocks.add(block) +} + +export function consumeListItemFirstBlock(block: PortableTextBlock): boolean { + const isListItemFirstBlock = listItemFirstBlocks.has(block) + listItemFirstBlocks.delete(block) + return isListItemFirstBlock +} diff --git a/packages/markdown/src/from-portable-text/render-node.ts b/packages/markdown/src/from-portable-text/render-node.ts index 98b59c9d84..9b16b4e30a 100644 --- a/packages/markdown/src/from-portable-text/render-node.ts +++ b/packages/markdown/src/from-portable-text/render-node.ts @@ -15,22 +15,97 @@ import type { PortableTextSpan, TypedObject, } from '@portabletext/types' -import {defaultKeyGenerator} from '../key-generator' +import {planLeafEscaping} from './escape-plain-text' +import { + consumeListItemFirstBlock, + markListItemFirstBlock, +} from './list-item-first-block' import type {PortableTextRenderers, RenderNode, Serializable} from './types' -interface SerializedBlock { - _key: string - children: string - index: number - isInline: boolean - node: PortableTextBlock | PortableTextListItemBlock -} +/** + * ATX headings are single-line, inline-only leaf blocks: an ATX heading's + * first line sits inside its `# ` prefix and can never be reparsed as a + * block construct, so line-leading hazards never apply there. A hard + * break's later lines are ordinary markdown lines outside that prefix and + * get the full line-start battery, same as any other block's continuation. + */ +const HEADING_STYLES = new Set(['h1', 'h2', 'h3', 'h4', 'h5', 'h6']) export const createRenderNode = ( renderers: PortableTextRenderers, listIndexMap: Map, listDepthMap: Map, ): RenderNode => { + // Keyed by the actual `@text` node objects `buildMarksTree` produces, not + // by render order: a custom type/mark renderer can call `renderNode` with + // a synthetic node of its own mid-block (eg to render a placeholder), and + // that call must not shift which escaped string a later real leaf gets. + // A synthetic node was never planned, so it's absent from the map and + // renders its own raw text. + const escapedTextByNode = new WeakMap() + + // Computed once per document render: a custom `hardBreak` can render to + // something with no newline of its own (eg `() => '
'`), in which + // case a `\n` inside a span's text is not a real line boundary in the + // rendered output, and `planLeafEscaping` needs to know that up front. + const hardBreakOutputHasNewline = renderers.hardBreak().includes('\n') + + // Shared by `renderBlock` and `renderListItem`, whatever the block's + // style: both flatten `node.children` into the same text/mark tree and + // need it escaped and keyed before rendering it. + function renderBlockChildren( + node: PortableTextBlock | PortableTextListItemBlock, + isHeading: boolean, + isListItem = false, + ): string { + const chunks = planLeafEscaping(node.children ?? [], node.markDefs ?? [], { + isHeading, + isListItem, + hardBreakOutputHasNewline, + }) + const tree = buildMarksTree(node) + assignEscapedText(tree, chunks) + + return tree + .map((child, i) => + renderNode({node: child, isInline: true, index: i, renderNode}), + ) + .join('') + } + + // Walks the tree in the same left-to-right, opaque-object-skipping order + // `planLeafEscaping` used to produce `chunks`, so each `@text` leaf gets + // keyed to the chunk planned for it. + function assignEscapedText( + nodes: ReadonlyArray< + ToolkitNestedPortableTextSpan | ToolkitTextNode | TypedObject + >, + chunks: ReadonlyArray, + ): void { + let pointer = 0 + + const visit = ( + node: ToolkitNestedPortableTextSpan | ToolkitTextNode | TypedObject, + ) => { + if (isPortableTextToolkitTextNode(node)) { + if (node.text !== '\n') { + const escaped = chunks[pointer] + if (escaped !== undefined) { + escapedTextByNode.set(node, escaped) + } + pointer++ + } + return + } + + if (isPortableTextToolkitSpan(node)) { + node.children.forEach(visit) + } + } + + nodes.forEach(visit) + } + function renderNode(options: Serializable): string { const {node, index, isInline} = options @@ -43,7 +118,7 @@ export const createRenderNode = ( } if (isPortableTextBlock(node)) { - return renderBlock(node, index, isInline) + return renderBlock(node, index, isInline, consumeListItemFirstBlock(node)) } if (isPortableTextToolkitTextNode(node)) { @@ -65,19 +140,17 @@ export const createRenderNode = ( typeof renderer === 'function' ? renderer : renderer[node.listItem] const itemHandler = handler || renderers.unknownListItem - // Build the text content from the block - const tree = buildMarksTree(node) - const textContent = tree - .map((child, i) => { - return renderNode({node: child, isInline: true, index: i, renderNode}) - }) - .join('') - - let children = textContent + let children: string if (node.style && node.style !== 'normal') { - // Wrap any other style in whatever the block component says to use + // Wrap any other style in whatever the block component says to use. + // `renderNode` would recurse straight back into `renderListItem` if + // `blockNode` still carried `listItem`, so it's stripped from the + // copy; `markListItemFirstBlock` restores the list-item context + // (line-start hazard escaping, the GFM checkbox prefix) onto that + // same copy so `renderBlock` picks it up via `consumeListItemFirstBlock`. const {listItem: _listItem, ...blockNode} = node + markListItemFirstBlock(blockNode) children = renderNode({ node: blockNode, index, @@ -86,6 +159,8 @@ export const createRenderNode = ( }) // Strip trailing newlines from block styles - list item component handles spacing children = children.replace(/\n+$/, '') + } else { + children = renderBlockChildren(node, false, true) } return itemHandler({ @@ -120,16 +195,21 @@ export const createRenderNode = ( node: PortableTextBlock, index: number, isInline: boolean, + isListItem: boolean, ): string { - const {_key, ...props} = serializeBlock({node, index, isInline, renderNode}) - const style = props.node.style || 'normal' + const style = node.style || 'normal' + const children = renderBlockChildren( + node, + HEADING_STYLES.has(style), + isListItem, + ) const handler = typeof renderers.block === 'function' ? renderers.block : renderers.block[style] const block = handler || renderers.unknownBlockStyle - return block({...props, value: props.node, renderNode}) + return block({index, isInline, children, value: node, renderNode}) } function renderText(node: ToolkitTextNode): string { @@ -137,7 +217,7 @@ export const createRenderNode = ( return renderers.hardBreak() } - return node.text + return escapedTextByNode.get(node) ?? node.text } function renderCustomBlock( @@ -157,22 +237,3 @@ export const createRenderNode = ( return renderNode } - -function serializeBlock( - options: Serializable, -): SerializedBlock { - const {node, index, isInline, renderNode} = options - const tree = buildMarksTree(node) - - const renderedChildren = tree.map((child, i) => - renderNode({node: child, isInline: true, index: i, renderNode}), - ) - - return { - _key: node._key || defaultKeyGenerator(), - children: renderedChildren.join(''), - index, - isInline, - node, - } -} diff --git a/packages/markdown/src/from-portable-text/renderers/marks.ts b/packages/markdown/src/from-portable-text/renderers/marks.ts index 41054b184c..28338b0550 100644 --- a/packages/markdown/src/from-portable-text/renderers/marks.ts +++ b/packages/markdown/src/from-portable-text/renderers/marks.ts @@ -1,5 +1,5 @@ import type {TypedObject} from '@portabletext/types' -import {escapeImageAndLinkText, escapeImageAndLinkTitle} from '../../escape' +import {escapeImageAndLinkTitle} from '../../escape' import type {PortableTextMarkRenderer} from '../types' /** @@ -15,10 +15,38 @@ export const DefaultStrongRenderer: PortableTextMarkRenderer = ({children}) => `**${children}**` /** + * Renders a `code` decorator from the raw span text, bypassing the escaped + * `children`: code content is verbatim, never markdown syntax. The + * backtick fence is widened past the longest run of backticks already in + * the text (CommonMark: the fence must be longer than any run it encloses). + * + * A space is padded on each side when the content starts or ends with a + * backtick, so the fence and the content's own backtick never merge into + * one run, and also when the content starts and ends with a space and + * isn't all whitespace (`text.trim() !== ''`; an all-space code span has + * nothing else for CommonMark's strip rule to leave behind, so padding it + * would only add visible spaces), because CommonMark itself would + * otherwise strip one such space per side on reparse; the padding + * pre-compensates for that strip. + * * @public */ -export const DefaultCodeRenderer: PortableTextMarkRenderer = ({children}) => - `\`${children}\`` +export const DefaultCodeRenderer: PortableTextMarkRenderer = ({text}) => { + const fence = '`'.repeat(longestBacktickRun(text) + 1) + const touchesBacktick = text.startsWith('`') || text.endsWith('`') + const wouldBeStripped = + text.startsWith(' ') && text.endsWith(' ') && text.trim() !== '' + const padding = touchesBacktick || wouldBeStripped ? ' ' : '' + return `${fence}${padding}${text}${padding}${fence}` +} + +function longestBacktickRun(text: string): number { + let longest = 0 + for (const run of text.match(/`+/g) ?? []) { + longest = Math.max(longest, run.length) + } + return longest +} /** * @public @@ -61,11 +89,11 @@ export const DefaultLinkRenderer: PortableTextMarkRenderer = ({ const encodedHref = href.replace(/["<>() ]/g, (char) => { return `%${char.charCodeAt(0).toString(16).toUpperCase()}` }) - return `[${escapeImageAndLinkText(children)}](${encodedHref})` + return `[${children}](${encodedHref})` } // For normal URLs, don't encode parentheses - Markdown handles balanced parens fine - return `[${escapeImageAndLinkText(children)}](${href}${title ? ` "${escapeImageAndLinkTitle(title)}"` : ''})` + return `[${children}](${href}${title ? ` "${escapeImageAndLinkTitle(title)}"` : ''})` } // Return children without link when URL is unsafe diff --git a/packages/markdown/src/from-portable-text/renderers/type.ts b/packages/markdown/src/from-portable-text/renderers/type.ts index a04d583aaf..68d1598cbc 100644 --- a/packages/markdown/src/from-portable-text/renderers/type.ts +++ b/packages/markdown/src/from-portable-text/renderers/type.ts @@ -1,10 +1,12 @@ import {isTypedObject} from '@portabletext/schema' +import {isPortableTextBlock} from '@portabletext/toolkit' import type {PortableTextBlock, TypedObject} from '@portabletext/types' import { escapeImageAndLinkText, escapeImageAndLinkTitle, escapeTableCell, } from '../../escape' +import {markListItemFirstBlock} from '../list-item-first-block' import type {PortableTextTypeRenderer} from '../types' /** @@ -387,15 +389,23 @@ export const DefaultListRenderer: PortableTextTypeRenderer<{ const indentWidth = value.kind === 'task' ? 2 : marker.length const indent = ' '.repeat(indentWidth) - const renderedBlocks = item.content.map((block, blockIndex) => ({ - isNestedList: (block as TypedObject)._type === 'list', - text: renderNode({ - node: block as TypedObject, - index: blockIndex, - isInline: false, - renderNode, - }), - })) + const renderedBlocks = item.content.map((block, blockIndex) => { + // Only the first block shares its first line with the marker (and, + // for a task item, its GFM checkbox); later blocks render on their + // own indented lines. + if (blockIndex === 0 && isPortableTextBlock(block)) { + markListItemFirstBlock(block) + } + return { + isNestedList: (block as TypedObject)._type === 'list', + text: renderNode({ + node: block as TypedObject, + index: blockIndex, + isInline: false, + renderNode, + }), + } + }) const [first, ...rest] = renderedBlocks // Trim trailing whitespace from empty items so `- ` becomes `-`. diff --git a/packages/markdown/src/portable-text-to-markdown.fuzz.test.ts b/packages/markdown/src/portable-text-to-markdown.fuzz.test.ts new file mode 100644 index 0000000000..eeea7fd4d1 --- /dev/null +++ b/packages/markdown/src/portable-text-to-markdown.fuzz.test.ts @@ -0,0 +1,431 @@ +import {createTestKeyGenerator} from '@portabletext/test' +import {isPortableTextBlock, isPortableTextSpan} from '@portabletext/toolkit' +import type {PortableTextBlock, TypedObject} from '@portabletext/types' +import {describe, expect, test} from 'vitest' +import {portableTextToMarkdown} from './from-portable-text/portable-text-to-markdown' +import {markdownToPortableText} from './to-portable-text/markdown-to-portable-text' + +/** + * A tiny deterministic PRNG (mulberry32): the fuzz corpus is generated from + * fixed seeds, so a failure is always reproducible by re-running this file, + * and CI never sees a flake from `Math.random()`. + */ +function mulberry32(seed: number): () => number { + let state = seed + return () => { + state = (state + 0x6d2b79f5) | 0 + let t = Math.imul(state ^ (state >>> 15), 1 | state) + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t + return ((t ^ (t >>> 14)) >>> 0) / 4294967296 + } +} + +const HOSTILE_ALPHABET = [ + ' ', + '\t', + ...'0123456789-+*#>=.()[]:\\`~&(random: () => number, items: ReadonlyArray): T { + const item = items[Math.floor(random() * items.length)] + if (item === undefined) { + throw new Error('Expected a non-empty array') + } + return item +} + +function randomInt(random: () => number, min: number, max: number): number { + return min + Math.floor(random() * (max - min + 1)) +} + +function randomLeafText(random: () => number): string { + const length = randomInt(random, 0, 6) + let text = '' + for (let i = 0; i < length; i++) { + text += pick(random, HOSTILE_ALPHABET) + } + return text +} + +function randomWhitespaceOnlyText(random: () => number): string { + const length = randomInt(random, 1, 4) + let text = '' + for (let i = 0; i < length; i++) { + text += pick(random, [' ', '\t']) + } + return text +} + +function randomOpaqueSafeText(random: () => number): string { + const length = randomInt(random, 1, 4) + let text = '' + for (let i = 0; i < length; i++) { + text += pick(random, OPAQUE_SAFE_ALPHABET) + } + return text +} + +type LeafKind = 'text' | 'link' | 'opaque' + +interface GeneratedCase { + context: Context + leafTexts: Array + leafKinds: Array + expectedText: string +} + +/** + * A hard break's own markup (two trailing spaces before the newline) is + * indistinguishable from a genuine trailing space right before it, and an + * empty line after one doesn't round-trip either (there's no next line to + * break to): both are real CommonMark ambiguities a text-escaping fix + * can't resolve, not something in scope here. Only a hard break landing + * between two leaves that both carry real, non-whitespace content right at + * the boundary is generated, so the fuzz oracle only has to account for + * the two *documented* exceptions (leading/trailing trim of the whole + * text, and a linkified substring keeping its text). + */ +function canPlaceHardBreakBetween(before: string, after: string): boolean { + return ( + before.length > 0 && + after.length > 0 && + !/[ \t]$/.test(before) && + !/^[ \t]/.test(after) + ) +} + +function generateCase(random: () => number): GeneratedCase | null { + const context = pick(random, CONTEXTS) + const isHeading = /^h[1-6]$/.test(context) + const leafCount = randomInt(random, 1, 3) + + const leafTexts: Array = [] + for (let i = 0; i < leafCount; i++) { + const roll = random() + if (roll < 0.15) { + leafTexts.push(randomWhitespaceOnlyText(random)) + } else { + leafTexts.push(randomLeafText(random)) + } + } + + const leafKinds: Array = leafTexts.map(() => 'text') + const variantRoll = random() + if (variantRoll < 0.15) { + // An opaque object stands in for an inline object or a leaf a custom + // renderer replaces: at plan time its rendered text is unknown, so a + // hazard scan can't run inline edits over it, only wall it off from + // its neighbors. Its own text is a plain-letter fill so *it* never + // introduces a hazard the oracle would have to model separately. + const index = randomInt(random, 0, leafTexts.length - 1) + leafTexts[index] = randomOpaqueSafeText(random) + leafKinds[index] = 'opaque' + } else if (variantRoll < 0.3 && leafTexts.length >= 3) { + // A link label goes through the same hazard scan as plain text (plus + // unconditional bracket/backslash escaping), so an interior index is + // fair game; the first and last are skipped because the label's own + // `[`/`)` becomes the line's true first/last rendered character - the + // first is a pre-existing gap in line-start hazard planning, and the + // last would defeat the oracle's own leading/trailing whitespace trim, + // neither of which is what this variant is fuzzing. + const index = randomInt(random, 1, leafTexts.length - 2) + const candidate = leafTexts[index] ?? '' + const precedingText = leafTexts[index - 1] ?? '' + if ( + candidate.length > 0 && + !/^[ \t]*$/.test(candidate) && + // A label ending in one of the characters + // `escapeLinkLabelBrackets` escapes, right before a leaf starting + // with `(`/`[`, chains into the unrelated, pre-existing + // `]`-before-link-open hazard and double-escapes the label's own + // bracket - not what this variant is fuzzing. + !/[[\]\\]$/.test(candidate) && + // A leaf ending in `!` right before a link reads back as an image + // (`![...]`), a pre-existing gap this variant isn't fuzzing either. + !precedingText.endsWith('!') + ) { + leafKinds[index] = 'link' + } + } + + // Headings are single-line ATX constructs: a hard break inside one forces + // a structural split (a second block) on reparse, which is a documented, + // separately-tested exception, not a text-identity fuzz property. Every + // other context supports CommonMark's lazy paragraph continuation, so a + // hard break there round-trips inside one block - except a table cell, + // which has no newline syntax of its own (a cell's hard break renders as + // `
`, and default `html.inline: 'skip'` drops it on reparse instead + // of decoding it back into a newline): a separate, pre-existing limit on + // table-cell content, not a text-escaping concern. + if ( + !isHeading && + context !== 'table-cell' && + leafCount > 1 && + leafKinds.every((kind) => kind === 'text') && + random() < 0.4 + ) { + const breakBefore = randomInt(random, 1, leafCount - 1) + const before = leafTexts[breakBefore - 1] ?? '' + const after = leafTexts[breakBefore] ?? '' + if (canPlaceHardBreakBetween(before, after)) { + leafTexts[breakBefore] = `\n${after}` + } + } + + const joined = leafTexts.join('') + if (joined.length === 0) { + return null + } + + // The indented-code-block escape (` `/` `) preserves the *encoded* + // character, not the leading run around it: CommonMark's ordinary 0-3 + // leading-space block indentation trim still applies to whatever literal + // space/tab precedes it. Reproducing that interaction is already the + // directed tests' job (the `roundTripCorpus` includes `' indented'` + // and `'\tx'`); the fuzz oracle only models the simple, whole-text trim. + const firstLine = joined.split('\n')[0] ?? '' + if (/^ {0,3}\t/.test(firstLine) || /^ {4}/.test(firstLine)) { + return null + } + + // A task item's own checkbox (`[ ] `/`[x] `/`[X] `) is a fixed-width + // marker our own parser strips outright, unlike the variable-width + // leading-space indentation every other context trims: the content's + // own leading whitespace, right after that marker, survives untouched. + const expectedText = + context === 'list-task' + ? joined.replace(/[ \t]+$/, '') + : joined.replace(/^[ \t]+/, '').replace(/[ \t]+$/, '') + if (expectedText.length === 0) { + return null + } + + return {context, leafTexts, leafKinds, expectedText} +} + +function buildPortableText( + generated: GeneratedCase, + keyGenerator: () => string, +): Array { + const linkKey = generated.leafKinds.includes('link') + ? keyGenerator() + : undefined + + const children = generated.leafTexts.map((text, index) => { + const kind = generated.leafKinds[index] + + if (kind === 'opaque') { + return {_type: OPAQUE_PROBE_TYPE, _key: keyGenerator(), text} + } + + return { + _type: 'span' as const, + _key: keyGenerator(), + text, + marks: + kind === 'link' && linkKey !== undefined + ? [linkKey] + : // An unrendered mark on every leaf but the last stands in for an + // annotation/decorator the schema doesn't know how to render: it + // splices its children's text back together without adding markup + // of its own. + index < generated.leafTexts.length - 1 + ? ['unrenderedMark'] + : [], + } + }) + + const style = generated.context.startsWith('h') + ? generated.context + : generated.context === 'blockquote' + ? 'blockquote' + : 'normal' + + const block = { + _type: 'block' as const, + _key: keyGenerator(), + style, + markDefs: + linkKey === undefined + ? [] + : [{_key: linkKey, _type: 'link', href: 'https://example.com'}], + children, + } + + if (generated.context === 'list-bullet') { + const result = [{...block, listItem: 'bullet', level: 1}] + return result + } + if (generated.context === 'list-number') { + const result = [{...block, listItem: 'number', level: 1}] + return result + } + if (generated.context === 'list-task') { + const result = [{...block, listItem: 'task', level: 1, checked: false}] + return result + } + if (generated.context === 'table-cell') { + const result = [ + { + _type: 'table', + _key: keyGenerator(), + headerRows: 0, + rows: [ + { + _key: keyGenerator(), + _type: 'row', + cells: [{_key: keyGenerator(), _type: 'cell', value: [block]}], + }, + ], + }, + ] + return result + } + + return [block] +} + +function extractRenderedText( + generated: GeneratedCase, + portableText: Array, +): string { + if (generated.context === 'table-cell') { + const table = portableText[0] as unknown as { + rows: Array<{cells: Array<{value: Array}>}> + } + const cellBlock = table.rows[0]?.cells[0]?.value[0] + if (!cellBlock || !isPortableTextBlock(cellBlock)) { + throw new Error('Expected a text block inside the table cell') + } + return blockText(cellBlock) + } + + const block = portableText[0] + if (!block || !isPortableTextBlock(block)) { + throw new Error('Expected the first node to be a portable text block') + } + return blockText(block) +} + +function blockText(block: PortableTextBlock): string { + return block.children + .filter(isPortableTextSpan) + .map((span) => span.text) + .join('') +} + +const SEEDS = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] +const CASES_PER_SEED = 200 + +describe('portableTextToMarkdown fuzz (seeded, deterministic)', () => { + test('PT -> MD -> PT text identity, modulo leading/trailing whitespace trim', () => { + const failures: Array<{ + seed: number + caseIndex: number + generated: GeneratedCase + markdown: string + actual: string + }> = [] + + let attempted = 0 + + for (const seed of SEEDS) { + const random = mulberry32(seed) + for (let caseIndex = 0; caseIndex < CASES_PER_SEED; caseIndex++) { + const generated = generateCase(random) + if (!generated) { + continue + } + attempted++ + + const keyGenerator = createTestKeyGenerator() + const portableText = buildPortableText(generated, keyGenerator) + + let markdown: string + try { + markdown = portableTextToMarkdown(portableText, { + types: { + [OPAQUE_PROBE_TYPE]: ({value}) => (value as {text: string}).text, + }, + }) + } catch (error) { + failures.push({ + seed, + caseIndex, + generated, + markdown: ``, + actual: '', + }) + continue + } + + let actual: string + try { + const reparsed = markdownToPortableText(markdown) + actual = extractRenderedText(generated, reparsed) + } catch (error) { + failures.push({ + seed, + caseIndex, + generated, + markdown, + actual: ``, + }) + continue + } + + if (actual !== generated.expectedText) { + failures.push({seed, caseIndex, generated, markdown, actual}) + } + } + } + + if (failures.length > 0) { + const sample = failures + .slice(0, 10) + .map( + (failure) => + `seed ${failure.seed} case ${failure.caseIndex} (${failure.generated.context}): leaves ${JSON.stringify(failure.generated.leafTexts)} -> markdown ${JSON.stringify(failure.markdown)} -> got ${JSON.stringify(failure.actual)}, expected ${JSON.stringify(failure.generated.expectedText)}`, + ) + .join('\n') + expect.fail( + `${failures.length} of ${attempted} fuzz cases failed text identity:\n${sample}`, + ) + } + }) +}) diff --git a/packages/markdown/src/portable-text-to-markdown.test.ts b/packages/markdown/src/portable-text-to-markdown.test.ts index 6c59533524..b4af62a7e7 100644 --- a/packages/markdown/src/portable-text-to-markdown.test.ts +++ b/packages/markdown/src/portable-text-to-markdown.test.ts @@ -7,7 +7,9 @@ import {createTestKeyGenerator} from '@portabletext/test' import { isPortableTextBlock, isPortableTextListItemBlock, + isPortableTextSpan, } from '@portabletext/toolkit' +import type {PortableTextBlock, TypedObject} from '@portabletext/types' import {describe, expect, test} from 'vitest' import {defaultSchema} from './default-schema' import {portableTextToMarkdown} from './from-portable-text/portable-text-to-markdown' @@ -335,6 +337,57 @@ describe(portableTextToMarkdown.name, () => { 'foo [b\\\\\\]ar](https://example.com) baz', ) }) + + test('link label containing "](" round-trips instead of double-escaping into a lost label and destination', () => { + // The label's own `]` is already escaped once by + // `escapeLinkLabelBrackets`; the line-level `]`-before-`(` rule must + // skip it too, or the second backslash reopens the label early on + // reparse. + const inKeys = createTestKeyGenerator() + const linkKey = inKeys() + const portableText = [ + { + _type: 'block', + _key: inKeys(), + style: 'normal', + children: [ + {_type: 'span', _key: inKeys(), text: 'a](b', marks: [linkKey]}, + ], + markDefs: [ + {_key: linkKey, _type: 'link', href: 'https://example.com'}, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('[a\\](b](https://example.com)') + + const outKeys = createTestKeyGenerator() + const outBlockKey = outKeys() + const outLinkKey = outKeys() + const outSpanKey = outKeys() + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + expect(reparsed).toEqual([ + { + _type: 'block', + _key: outBlockKey, + style: 'normal', + markDefs: [ + {_key: outLinkKey, _type: 'link', href: 'https://example.com'}, + ], + children: [ + { + _type: 'span', + _key: outSpanKey, + text: 'a](b', + marks: [outLinkKey], + }, + ], + }, + ]) + }) }) describe('hard breaks', () => { @@ -375,6 +428,77 @@ describe(portableTextToMarkdown.name, () => { expect(portableTextToMarkdown(portableText)).toBe('foo \nbar') }) + test('an emphasis run spanning a custom hard break with no newline of its own round-trips instead of turning into markup', () => { + // `
` carries no newline: the leaves on either side land on the + // same rendered line, so the two `_` can pair up into an `em` on + // reparse unless the plan walls the break off instead of treating it + // as a real line boundary. + const portableText = [ + { + _type: 'block', + _key: 'k0', + style: 'normal', + children: [{_type: 'span', _key: 'k1', text: 'a _\n_ b', marks: []}], + markDefs: [], + }, + ] + + const markdown = portableTextToMarkdown(portableText, { + hardBreak: () => '
', + }) + expect(markdown).toBe('a \\_
\\_ b') + + const reparsed = markdownToPortableText(markdown, { + html: {inline: 'text'}, + }) + const block = reparsed.at(0) + if (!block || !isPortableTextBlock(block)) { + throw new Error('Expected the first node to be a portable text block') + } + expect( + block.children + .filter(isPortableTextSpan) + .map((span) => span.text) + .join(''), + ).toBe('a _
_ b') + }) + + test('the same underscore shape stays unescaped and round-trips with the default hard break, unaffected by the sentinel fix', () => { + // markdown-it itself never pairs a delimiter run across a `hardbreak` + // token, so the default renderer (a real newline) never needed + // escaping here; the sentinel only changes behavior for a renderer + // whose hard-break output carries no newline. + const portableText = [ + { + _type: 'block', + _key: 'k0', + style: 'normal', + children: [{_type: 'span', _key: 'k1', text: 'a _\n_ b', marks: []}], + markDefs: [], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('a _ \n_ b') + + const reparsed = markdownToPortableText(markdown) + const block = reparsed.at(0) + if (!block || !isPortableTextBlock(block)) { + throw new Error('Expected the first node to be a portable text block') + } + expect( + block.children + .filter(isPortableTextSpan) + .map((span) => span.text) + .join(''), + ).toBe('a _\n_ b') + expect( + block.children.every( + (child) => !isPortableTextSpan(child) || !child.marks?.includes('em'), + ), + ).toBe(true) + }) + test('multiple hard breaks from explicit PT', () => { const portableText = [ { @@ -644,6 +768,86 @@ describe(portableTextToMarkdown.name, () => { expect(portableTextToMarkdown(portableText)).toBe(markdown) }) + test('a bullet item whose own content starts with a checkbox-shaped run after leading whitespace round-trips instead of losing it to a GFM task marker', () => { + // The parser's own checkbox pre-pass reads a list item's inline + // content *after* CommonMark's leading-whitespace trim, same as + // every other line-start hazard - so a checkbox-shaped run has to + // be found (and escaped) there too, not only at column 0. + const keyGenerator = createTestKeyGenerator() + const text = ' [x] y' + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'normal', + listItem: 'bullet', + level: 1, + markDefs: [], + children: [{_type: 'span', _key: keyGenerator(), text, marks: []}], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('- \\[x] y') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const block = reparsed.at(0) + if (!block || !isPortableTextBlock(block)) { + throw new Error('Expected the first node to be a portable text block') + } + expect(isPortableTextListItemBlock(block)).toBe(true) + expect( + block.children + .filter(isPortableTextSpan) + .map((span) => span.text) + .join(''), + ).toBe('[x] y') + }) + + test('a checkbox-shaped run at the start of a styled (non-normal) list item survives instead of being eaten as a checkbox', () => { + // A styled list item's first block is rendered through the same + // block-style renderer (`DefaultH1Renderer` here) as a non-list + // block, so it has to keep its list-item escaping context on that + // path too, not just for a `normal`-style item. + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'h1', + listItem: 'bullet', + level: 1, + markDefs: [], + children: [ + { + _type: 'span', + _key: keyGenerator(), + text: '[ ] todo', + marks: [], + }, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('- # \\[ ] todo') + + const reparsed = markdownToPortableText(markdown) + const combinedText = reparsed + .map((node) => + isPortableTextBlock(node) + ? node.children + .filter(isPortableTextSpan) + .map((span) => span.text) + .join('') + : '', + ) + .join('') + expect(combinedText).toBe('[ ] todo') + }) + test('renders unchecked task from explicit portable text', () => { expect( portableTextToMarkdown([ @@ -1094,10 +1298,16 @@ describe(portableTextToMarkdown.name, () => { }) test('not supported by deserializer', () => { + // Unsupported by the schema, the image syntax survives as literal + // paragraph text instead of an image node, so it goes through the + // same leaf escaping as any other text and gains the same + // `]`-before-`(` protection a literal link pattern would. const portableText = markdownToPortableText(markdownIn, { schema: compileSchema(defineSchema({})), }) - expect(portableTextToMarkdown(portableText)).toBe(markdownIn) + expect(portableTextToMarkdown(portableText)).toBe( + 'foo\n\n![alt text\\](https://example.com/image.png)\n\nbar', + ) }) test('image with brackets in alt text', () => { @@ -1286,10 +1496,11 @@ describe(portableTextToMarkdown.name, () => { }) test('not supported by deserializer', () => { + // Same fallback-to-literal-text path as the block-image case above. const portableText = markdownToPortableText(markdown, { schema: compileSchema(defineSchema({})), }) - const markdownOut = 'foo ![alt text](https://example.com/image.png) bar' + const markdownOut = 'foo ![alt text\\](https://example.com/image.png) bar' expect(portableTextToMarkdown(portableText)).toBe(markdownOut) }) }) @@ -2874,6 +3085,1013 @@ describe(portableTextToMarkdown.name, () => { }) }) + describe('plain text escaping', () => { + // Each string round-trips as the sole text of a single span in a + // `normal` block: PT -> MD -> PT should reproduce the same block, with + // the span text byte-identical to what went in. + const roundTripCorpus = [ + '*bar*', + '_bar_', + '`code`', + '~~bar~~', + 'foo', + '&', + '\\*bar\\*', + '# heading', + '> quote', + '- item', + '1. item', + '1) item', + '---', + '```js', + '~~~', + ' indented', + '> [!NOTE]', + '-', + '+', + '*', + '\tx', + // Non-ASCII neighbors: markdown-it's emphasis-flanking rule classifies + // punctuation using Unicode's `P`/`S` categories, not ASCII alone, so + // a `_` run flanked by an emoji, an em dash, or a CJK character has to + // stay escaped exactly like one flanked by ASCII punctuation does. + '😀_a_😀', + '—_a_—', + '中_a_中', + ] + + function normalBlock(blockKey: string, spanKey: string, text: string) { + return { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [], + children: [{_type: 'span', _key: spanKey, text, marks: []}], + } + } + + // An unrendered, schema-less mark on every span but the last stands in + // for the annotation/decorator that splices the leaves together + // without introducing markup of its own. + function leavesBlock( + texts: Array, + overrides: {style?: string; listItem?: string; level?: number} = {}, + ) { + const keyGenerator = createTestKeyGenerator() + return { + _type: 'block', + _key: keyGenerator(), + style: overrides.style ?? 'normal', + ...(overrides.listItem + ? {listItem: overrides.listItem, level: overrides.level ?? 1} + : {}), + markDefs: [], + children: texts.map((text, index) => ({ + _type: 'span', + _key: keyGenerator(), + text, + marks: index < texts.length - 1 ? ['unrenderedAnnotation'] : [], + })), + } + } + + function firstBlock(blocks: Array) { + const block = blocks.at(0) + if (block === undefined || !isPortableTextBlock(block)) { + throw new Error('Expected the first block to be a portable text block') + } + return block + } + + function blockText(block: PortableTextBlock): string { + return block.children + .filter(isPortableTextSpan) + .map((span) => span.text) + .join('') + } + + test.each(roundTripCorpus)('%s round-trips byte-identical', (text) => { + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + + expect(reparsed).toEqual([normalBlock(outKeys(), outKeys(), text)]) + }) + + test('a link pattern in plain text round-trips, with the bare URL inside it gaining a link mark', () => { + const text = '[bar](https://example.com)' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('[bar\\](https://example.com)') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + const blockKey = outKeys() + const firstSpanKey = outKeys() + const linkKey = outKeys() + expect(reparsed).toEqual([ + { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [ + {_key: linkKey, _type: 'link', href: 'https://example.com'}, + ], + children: [ + {_type: 'span', _key: firstSpanKey, text: '[bar](', marks: []}, + { + _type: 'span', + _key: outKeys(), + text: 'https://example.com', + marks: [linkKey], + }, + {_type: 'span', _key: outKeys(), text: ')', marks: []}, + ], + }, + ]) + expect(blockText(firstBlock(reparsed))).toBe(text) + }) + + test('an autolink-shaped literal round-trips, with the bare URL inside it gaining a link mark', () => { + const text = '' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + expect(blockText(firstBlock(reparsed))).toBe(text) + }) + + test('a link reference definition round-trips, with its bare URL gaining a link mark', () => { + const text = '[id]: https://example.com' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\[id]: https://example.com') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + const blockKey = outKeys() + const firstSpanKey = outKeys() + const linkKey = outKeys() + expect(reparsed).toEqual([ + { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [ + {_key: linkKey, _type: 'link', href: 'https://example.com'}, + ], + children: [ + {_type: 'span', _key: firstSpanKey, text: '[id]: ', marks: []}, + { + _type: 'span', + _key: outKeys(), + text: 'https://example.com', + marks: [linkKey], + }, + ], + }, + ]) + }) + + test('a setext-underline-shaped line after a hard break round-trips instead of becoming a heading underline', () => { + const text = 'text\n===' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('text \n\\===') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + expect(reparsed).toEqual([normalBlock(outKeys(), outKeys(), text)]) + }) + + test('a bare URL round-trips byte-identical and gains a link mark (linkify carve-out)', () => { + const text = 'https://example.com' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe(text) + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + const blockKey = outKeys() + const linkKey = outKeys() + const spanKey = outKeys() + expect(reparsed).toEqual([ + { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [{_key: linkKey, _type: 'link', href: text}], + children: [{_type: 'span', _key: spanKey, text, marks: [linkKey]}], + }, + ]) + }) + + describe('leading spaces before a block marker', () => { + // CommonMark allows up to 3 leading spaces before a block marker + // without affecting how the line is parsed. + test.each([ + [' > q', ' \\> q', '> q'], + [' # head', ' \\# head', '# head'], + [' - item', ' \\- item', '- item'], + [' [x]: y', ' \\[x]: y', '[x]: y'], + [' ---', ' \\---', '---'], + [' ***', ' \\***', '***'], + [' ===', ' \\===', '==='], + [' 1. item', ' 1\\. item', '1. item'], + [' 1) item', ' 1\\) item', '1) item'], + ])( + '%s round-trips instead of losing its text to the block marker', + (text, expectedMarkdown, expectedBlockText) => { + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe(expectedMarkdown) + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + // The leading space is block indentation, which CommonMark's own + // paragraph parsing trims; it isn't part of the fixpoint claim. + expect(blockText(firstBlock(reparsed))).toBe(expectedBlockText) + }, + ) + }) + + describe('a heading ending in a space and a hash run', () => { + test("the trailing hash run round-trips instead of being read as the heading's closing sequence", () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'h1', + markDefs: [], + children: [ + {_type: 'span', _key: keyGenerator(), text: 'x #', marks: []}, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('# x \\#') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + expect(reparsed).toEqual([ + { + _type: 'block', + _key: outKeys(), + style: 'h1', + markDefs: [], + children: [ + {_type: 'span', _key: outKeys(), text: 'x #', marks: []}, + ], + }, + ]) + }) + }) + + describe('a dash-underline-shaped line', () => { + test('after a hard break round-trips instead of becoming a setext h2 underline', () => { + const text = 'a\n--' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('a \n\\--') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + expect(reparsed).toEqual([normalBlock(outKeys(), outKeys(), text)]) + }) + + test('a single trailing dash with a space round-trips as plain text, not a bullet or an underline', () => { + const text = '- ' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\- ') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const block = firstBlock(reparsed) + expect(isPortableTextListItemBlock(block)).toBe(false) + // The trailing space is trimmed by CommonMark's own paragraph + // parsing; it isn't part of the fixpoint claim. + expect(blockText(block)).toBe('-') + }) + }) + + describe('linkify mask boundaries', () => { + // markdown-it's linkify pass runs *after* inline tokenization and + // entity decoding, over whatever text and marks those steps already + // produced - not over this line's raw, undecoded source. A probe run + // against the raw text can claim a range the real reparse won't. + + test('an entity reference immediately before an email-shaped run round-trips instead of leaking through the linkify claim', () => { + const text = '&x@y.co' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\&x@y.co') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + expect(blockText(firstBlock(reparsed))).toBe(text) + }) + + test('a backtick immediately after a URL-shaped run round-trips instead of losing text to a code span', () => { + const text = '//a.co`>x`' + const inKeys = createTestKeyGenerator() + const portableText = [normalBlock(inKeys(), inKeys(), text)] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('//a.co\\`>x\\`') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + expect(blockText(firstBlock(reparsed))).toBe(text) + }) + + test('a linkify claim spliced by a rendered mark boundary round-trips instead of masking the delimiters it would corrupt', () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'normal', + markDefs: [], + children: [ + {_type: 'span', _key: keyGenerator(), text: 'www.exa', marks: []}, + { + _type: 'span', + _key: keyGenerator(), + text: 'mple.co/a*b*c', + marks: ['strong'], + }, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('www.exa**mple.co/a\\*b\\*c**') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + expect(blockText(firstBlock(reparsed))).toBe('www.example.co/a*b*c') + }) + }) + + describe('custom renderers', () => { + test("a synthetic text node from a custom type renderer does not steal a sibling leaf's escaped text", () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'normal', + markDefs: [], + children: [ + {_type: 'span', _key: keyGenerator(), text: 'a', marks: []}, + {_type: 'inlineMarker', _key: keyGenerator()}, + {_type: 'span', _key: keyGenerator(), text: '*bar*', marks: []}, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText, { + types: { + inlineMarker: ({renderNode}) => + renderNode({ + node: {_type: '@text', text: 'TICK'}, + isInline: true, + index: 0, + renderNode, + }), + }, + }) + + expect(markdown).toBe('aTICK\\*bar\\*') + }) + }) + + describe('cross-leaf hazards', () => { + test('an unrendered annotation cannot splice a digit and ". item" into an ordered-list marker', () => { + const keyGenerator = createTestKeyGenerator() + const blockKey = keyGenerator() + const firstSpanKey = keyGenerator() + const secondSpanKey = keyGenerator() + const portableText = [ + { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [], + children: [ + { + _type: 'span', + _key: firstSpanKey, + text: '1', + marks: ['unrenderedAnnotation'], + }, + {_type: 'span', _key: secondSpanKey, text: '. item', marks: []}, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('1\\. item') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.style).toBe('normal') + expect(isPortableTextListItemBlock(block)).toBe(false) + expect(blockText(block)).toBe('1. item') + }) + + test('an unrendered annotation cannot splice "[foo]" and "(bar)" into a link', () => { + const keyGenerator = createTestKeyGenerator() + const blockKey = keyGenerator() + const firstSpanKey = keyGenerator() + const secondSpanKey = keyGenerator() + const portableText = [ + { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [], + children: [ + { + _type: 'span', + _key: firstSpanKey, + text: '[foo]', + marks: ['unrenderedAnnotation'], + }, + {_type: 'span', _key: secondSpanKey, text: '(bar)', marks: []}, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('[foo\\](bar)') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.markDefs).toEqual([]) + expect(blockText(block)).toBe('[foo](bar)') + }) + + test('an unrendered annotation cannot splice "[x]" and ": y" into a link reference definition', () => { + const keyGenerator = createTestKeyGenerator() + const blockKey = keyGenerator() + const firstSpanKey = keyGenerator() + const secondSpanKey = keyGenerator() + const portableText = [ + { + _type: 'block', + _key: blockKey, + style: 'normal', + markDefs: [], + children: [ + { + _type: 'span', + _key: firstSpanKey, + text: '[x]', + marks: ['unrenderedAnnotation'], + }, + {_type: 'span', _key: secondSpanKey, text: ': y', marks: []}, + ], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\[x]: y') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.markDefs).toEqual([]) + expect(blockText(block)).toBe('[x]: y') + }) + + test('a ref-def label split anywhere before the colon round-trips (label and colon in one leaf)', () => { + const portableText = [leavesBlock(['[', 'x]: y'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\[x]: y') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.markDefs).toEqual([]) + expect(blockText(block)).toBe('[x]: y') + }) + + test('a ref-def label split anywhere before the colon round-trips (colon and destination split again)', () => { + const portableText = [leavesBlock(['[', 'x]:', 'y'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\[x]:y') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.markDefs).toEqual([]) + expect(blockText(block)).toBe('[x]:y') + }) + + test('a bullet marker split before its own space round-trips instead of becoming a list item', () => { + const portableText = [leavesBlock(['-', ' x'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\- x') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(isPortableTextListItemBlock(block)).toBe(false) + expect(blockText(block)).toBe('- x') + }) + + test('an entity reference split across leaves round-trips instead of decoding to the bare character', () => { + const portableText = [leavesBlock(['&', 'amp;'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\&') + + const reparsed = markdownToPortableText(markdown) + expect(blockText(firstBlock(reparsed))).toBe('&') + }) + + test('an HTML/autolink-shaped `<` split across leaves round-trips instead of opening a tag', () => { + const portableText = [leavesBlock(['<', 'p'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\ { + const portableText = [leavesBlock(['\\', '*'])] + + const markdown = portableTextToMarkdown(portableText) + + const reparsed = markdownToPortableText(markdown) + expect(blockText(firstBlock(reparsed))).toBe('\\*') + }) + + test('an emphasis run split across leaves round-trips instead of turning into markup', () => { + const portableText = [leavesBlock(['*', 'm', '*'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\*m\\*') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.markDefs).toEqual([]) + expect(blockText(block)).toBe('*m*') + }) + + test('a strikethrough run split across leaves round-trips instead of turning into markup', () => { + const portableText = [leavesBlock(['a~', '~b~', '~c'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('a\\~\\~b\\~\\~c') + + const reparsed = markdownToPortableText(markdown) + expect(blockText(firstBlock(reparsed))).toBe('a~~b~~c') + }) + + test('a setext-underline run accumulated across a hard break and a second leaf round-trips instead of turning the preceding line into a heading', () => { + const portableText = [leavesBlock(['x\n=', '='])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('x \n\\==') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.style).toBe('normal') + expect(blockText(block)).toBe('x\n==') + }) + + test('a thematic-break run accumulated across leaves round-trips instead of becoming a thematic break', () => { + const portableText = [leavesBlock(['-', '--'])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('\\---') + + const reparsed = markdownToPortableText(markdown) + expect(blockText(firstBlock(reparsed))).toBe('---') + }) + }) + + test.each(roundTripCorpus)( + '%s stays unescaped inside a widened code span', + (text) => { + const inKeys = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: inKeys(), + style: 'normal', + markDefs: [], + children: [{_type: 'span', _key: inKeys(), text, marks: ['code']}], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + + expect(reparsed).toEqual([ + { + _type: 'block', + _key: outKeys(), + style: 'normal', + markDefs: [], + children: [{_type: 'span', _key: outKeys(), text, marks: ['code']}], + }, + ]) + }, + ) + + describe('code span padding', () => { + test('content padded with a space on both sides round-trips instead of losing one space per side', () => { + const text = ' x ' + const inKeys = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: inKeys(), + style: 'normal', + markDefs: [], + children: [{_type: 'span', _key: inKeys(), text, marks: ['code']}], + }, + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('` x `') + + const reparsed = markdownToPortableText(markdown, { + keyGenerator: createTestKeyGenerator(), + }) + const outKeys = createTestKeyGenerator() + expect(reparsed).toEqual([ + { + _type: 'block', + _key: outKeys(), + style: 'normal', + markDefs: [], + children: [{_type: 'span', _key: outKeys(), text, marks: ['code']}], + }, + ]) + }) + }) + + describe('canonical output bytes', () => { + test.each([ + ['foo # bar', 'foo # bar'], + ['foo - bar', 'foo - bar'], + ['foo > bar', 'foo > bar'], + ['a * b', 'a * b'], + ])('%s stays bare', (text, expectedMarkdown) => { + const keyGenerator = createTestKeyGenerator() + const portableText = [normalBlock(keyGenerator(), keyGenerator(), text)] + + expect(portableTextToMarkdown(portableText)).toBe(expectedMarkdown) + }) + }) + + describe('single-span grammar fixes', () => { + describe('thematic breaks with interior spaces or tabs', () => { + test.each(['-- -', '_ _ _'])( + '%s round-trips instead of becoming a thematic break', + (text) => { + const portableText = [leavesBlock([text])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe(`\\${text}`) + + const reparsed = markdownToPortableText(markdown) + expect(blockText(firstBlock(reparsed))).toBe(text) + }, + ) + }) + + describe('a heading whose text is only a hash run', () => { + test.each(['#', '##'])( + '%s round-trips instead of reparsing as an empty heading', + (text) => { + const portableText = [leavesBlock([text], {style: 'h1'})] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe(`# \\${text}`) + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.style).toBe('h1') + expect(blockText(block)).toBe(text) + }, + ) + + test('a hash immediately followed by other text is not over-escaped', () => { + const portableText = [leavesBlock(['#', 'x'], {style: 'h1'})] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('# #x') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.style).toBe('h1') + expect(blockText(block)).toBe('#x') + }) + + test('a hash run after real heading text and a space still round-trips', () => { + const portableText = [leavesBlock(['x #'], {style: 'h1'})] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('# x \\#') + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.style).toBe('h1') + expect(blockText(block)).toBe('x #') + }) + }) + + describe('a heading with a hard break in its text', () => { + test("only the first line is inside the heading's `# ` prefix", () => { + const portableText = [leavesBlock(['a\n- b'], {style: 'h1'})] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe('# a \n\\- b') + + const reparsed = markdownToPortableText(markdown) + expect(reparsed).toHaveLength(2) + const [heading, continuation] = reparsed + if ( + heading === undefined || + continuation === undefined || + !isPortableTextBlock(heading) || + !isPortableTextBlock(continuation) + ) { + throw new Error('Expected two portable text blocks') + } + expect(heading.style).toBe('h1') + expect(blockText(heading)).toBe('a') + // The second raw line is no longer inside the heading's `# ` + // prefix, so it's an ordinary line: it must not misparse as a + // bullet list item, and its text (the hard break's continuation) + // must survive. + expect(isPortableTextListItemBlock(continuation)).toBe(false) + expect(blockText(continuation)).toBe('- b') + }) + }) + + describe('a GFM task-list checkbox at the start of list-item content', () => { + test.each(['[x] done', '[X] done', '[ ] done'])( + '%s round-trips instead of being consumed as a checkbox', + (text) => { + const portableText = [ + leavesBlock([text], {listItem: 'bullet', level: 1}), + ] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe(`- \\${text}`) + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(block.listItem).toBe('bullet') + expect('checked' in block).toBe(false) + expect(blockText(block)).toBe(text) + }, + ) + + test('a list item that already is a task keeps its own checkbox and its content bare', () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'normal', + listItem: 'task', + level: 1, + checked: true, + markDefs: [], + children: [ + {_type: 'span', _key: keyGenerator(), text: 'done', marks: []}, + ], + }, + ] + + expect(portableTextToMarkdown(portableText)).toBe('- [x] done') + }) + }) + }) + + describe('linkify awareness', () => { + test.each([ + 'https://e.co#~~', + 'https://e.co#_', + 'www.e.co/a_b', + 'foo@e.co', + ])( + '%s round-trips byte-identical, not escaped inside the linkified range', + (text) => { + const portableText = [leavesBlock([text])] + + const markdown = portableTextToMarkdown(portableText) + expect(markdown).toBe(text) + + const reparsed = markdownToPortableText(markdown) + const block = firstBlock(reparsed) + expect(blockText(block)).toBe(text) + const linkSpan = block.children.find(isPortableTextSpan) + expect(linkSpan?.marks).toHaveLength(1) + const linkKey = linkSpan?.marks?.at(0) + const markDef = (block.markDefs ?? []).find( + (def) => def._key === linkKey, + ) + expect(markDef?._type).toBe('link') + }, + ) + }) + + describe('context', () => { + test('heading', () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'h1', + markDefs: [], + children: [ + {_type: 'span', _key: keyGenerator(), text: '*bar*', marks: []}, + ], + }, + ] + + expect(portableTextToMarkdown(portableText)).toBe('# \\*bar\\*') + }) + + test('list item', () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'normal', + listItem: 'bullet', + level: 1, + markDefs: [], + children: [ + {_type: 'span', _key: keyGenerator(), text: '*bar*', marks: []}, + ], + }, + ] + + expect(portableTextToMarkdown(portableText)).toBe('- \\*bar\\*') + }) + + test('blockquote', () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'block', + _key: keyGenerator(), + style: 'blockquote', + markDefs: [], + children: [ + { + _type: 'span', + _key: keyGenerator(), + text: '# heading text', + marks: [], + }, + ], + }, + ] + + expect(portableTextToMarkdown(portableText)).toBe('> \\# heading text') + }) + + test('table cell, including a literal backslash before a pipe', () => { + const keyGenerator = createTestKeyGenerator() + const cellBlock = (text: string) => ({ + _type: 'block', + _key: keyGenerator(), + style: 'normal', + markDefs: [], + children: [{_type: 'span', _key: keyGenerator(), text, marks: []}], + }) + const table = { + _type: 'table', + _key: keyGenerator(), + headerRows: 0, + rows: [ + { + _key: keyGenerator(), + _type: 'row', + cells: [ + { + _key: keyGenerator(), + _type: 'cell', + value: [cellBlock('*bar*')], + }, + ], + }, + { + _key: keyGenerator(), + _type: 'row', + cells: [ + { + _key: keyGenerator(), + _type: 'cell', + value: [cellBlock('a\\|b')], + }, + ], + }, + ], + } + + expect(portableTextToMarkdown([table])).toBe( + ['| |', '| --- |', '| \\*bar\\* |', '| a\\\\\\|b |'].join('\n'), + ) + }) + + test('callout', () => { + const keyGenerator = createTestKeyGenerator() + const portableText = [ + { + _type: 'callout', + _key: keyGenerator(), + tone: 'note', + content: [ + { + _type: 'block', + _key: keyGenerator(), + style: 'normal', + markDefs: [], + children: [ + { + _type: 'span', + _key: keyGenerator(), + text: '*bar*', + marks: [], + }, + ], + }, + ], + }, + ] + + expect(portableTextToMarkdown(portableText)).toBe( + '> [!NOTE]\n> \\*bar\\*', + ) + }) + }) + }) + describe('zero-config round-trip', () => { test('MD -> PT -> MD round-trip is stable for a document exercising code, image, horizontal rule, HTML, and a callout', () => { const markdown = [ diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 893308b462..1725295c56 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -697,6 +697,9 @@ importers: '@portabletext/toolkit': specifier: ^6.0.0 version: 6.0.0 + linkify-it: + specifier: ^5.0.2 + version: 5.0.2 markdown-it: specifier: ^14.3.0 version: 14.3.0 @@ -713,6 +716,9 @@ importers: '@sanity/tsconfig': specifier: catalog:tooling version: 2.1.0 + '@types/linkify-it': + specifier: ^5.0.0 + version: 5.0.0 '@types/markdown-it': specifier: ^14.1.2 version: 14.1.2 @@ -7158,9 +7164,6 @@ packages: resolution: {integrity: sha512-/vlFKAoH5Cgt3Ie+JLhRbwOsCQePABiU3tJ1egGvyQ+33R/vcwM2Zl2QR/LzjsBeItPt3oSVXapn+m4nQDvpzw==} engines: {node: '>=14'} - linkify-it@5.0.0: - resolution: {integrity: sha512-5aHCbzQRADcdP+ATqnDuhhJ/MRIqDkZX5pyjFHRRysS8vZ5AbqGEoFIb6pYHPZ+L/OC2Lc+xT8uHVVR5CAK/wQ==} - linkify-it@5.0.2: resolution: {integrity: sha512-ONTm2jCMAVZjgQa/Fy1kScXsuOoF5NPTsoFBdE1KVIZ2vAh/r9+Bqo+0jINCBYnavTPQZz38QzFTme79ENoN3Q==} @@ -13653,7 +13656,7 @@ snapshots: magicast: 0.5.3 obug: 2.1.4 tinyrainbow: 3.1.0 - vitest: 4.1.11(@types/node@20.19.25)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-istanbul@4.1.11)(@vitest/coverage-v8@4.1.11)(jsdom@27.2.0)(vite@8.2.0(@types/node@20.19.25)(esbuild@0.28.1)(jiti@2.7.0)(terser@5.44.1)(tsx@4.23.12)(yaml@2.8.3)) + vitest: 4.1.11(@types/node@24.12.2)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-istanbul@4.1.11)(@vitest/coverage-v8@4.1.11)(jsdom@27.2.0)(vite@8.2.0(@types/node@24.12.2)(esbuild@0.28.1)(jiti@2.7.0)(terser@5.44.1)(tsx@4.23.12)(yaml@2.8.3)) transitivePeerDependencies: - supports-color @@ -13669,9 +13672,9 @@ snapshots: obug: 2.1.4 std-env: 4.0.0 tinyrainbow: 3.1.0 - vitest: 4.1.11(@types/node@20.19.25)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-istanbul@4.1.11)(@vitest/coverage-v8@4.1.11)(jsdom@27.2.0)(vite@8.2.0(@types/node@20.19.25)(esbuild@0.28.1)(jiti@2.7.0)(terser@5.44.1)(tsx@4.23.12)(yaml@2.8.3)) + vitest: 4.1.11(@types/node@24.12.2)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-istanbul@4.1.11)(@vitest/coverage-v8@4.1.11)(jsdom@27.2.0)(vite@8.2.0(@types/node@24.12.2)(esbuild@0.28.1)(jiti@2.7.0)(terser@5.44.1)(tsx@4.23.12)(yaml@2.8.3)) optionalDependencies: - '@vitest/browser': 4.1.11(vite@8.2.0(@types/node@20.19.25)(esbuild@0.28.1)(jiti@2.7.0)(terser@5.44.1)(tsx@4.23.12)(yaml@2.8.3))(vitest@4.1.11) + '@vitest/browser': 4.1.11(vite@8.2.0(@types/node@24.12.2)(esbuild@0.28.1)(jiti@2.7.0)(terser@5.44.1)(tsx@4.23.12)(yaml@2.8.3))(vitest@4.1.11) '@vitest/expect@4.1.11': dependencies: @@ -15483,10 +15486,6 @@ snapshots: lilconfig@3.1.3: {} - linkify-it@5.0.0: - dependencies: - uc.micro: 2.1.0 - linkify-it@5.0.2: dependencies: uc.micro: 2.1.0 @@ -15573,7 +15572,7 @@ snapshots: dependencies: argparse: 2.0.1 entities: 4.5.0 - linkify-it: 5.0.0 + linkify-it: 5.0.2 mdurl: 2.0.0 punycode.js: 2.3.1 uc.micro: 2.1.0