fix: escape markdown syntax in plain span text during serialization - #3183
fix: escape markdown syntax in plain span text during serialization#3183christianhg wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: 2a860ef The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Bundle StatsWarning 1 significant change. @portabletext/markdown🔴 All scenario measurements (7)🗺️
Significant means at least 1.0 KB and 1% gzip, or at least 5 ms and 10% import time. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
66ce7ec to
c4de779
Compare
c4de779 to
3941c77
Compare
`renderText` returned span text unchanged, so Portable Text whose text contains literal markdown punctuation re-parsed as markup: a span reading `*bar*` came back as an `em` span, a leading `#` became a heading, and a text of ` [x]: y` vanished entirely as a link reference definition. Escaping happens at the text leaf, before mark renderers wrap their delimiters, planned per block by `planLeafEscaping`: leaves are grouped into lines at hard breaks (only when the hard-break renderer's output carries a newline; otherwise the break is an inline opaque segment), each line is joined into one string with a per-character map back to its owning leaf (custom- rendered segments become opaque sentinels), all hazard detection runs once against that joined line, and the resulting edits map back into per-leaf escaped strings keyed by leaf identity in a `WeakMap`, so a custom renderer rendering a synthetic text node cannot shift sibling escapes. Detection on the joined line makes leaf boundaries irrelevant by construction: constructs spliced across leaves through unrendered marks (an ordered-list marker, a reference-definition label, an entity, an emphasis run) are seen whole. Inline hazards escape everywhere in non-code text; line-leading hazards (headings, blockquotes, list markers, GFM task checkboxes, setext underlines, thematic breaks with interior spaces, indented code, reference definitions, alerts) escape only at real line starts with CommonMark's 0-3 space tolerance. Ranges that markdown-it's linkifier will claim (bare URLs, emails, `www.`) are not escaped: text identity holds and the re-parse adds a link mark. Entity-reference and backtick escapes bypass that mask, since both alter what the parser sees before linkify runs, and a claim interrupted by a rendered mark boundary is not honored. The `code` decorator receives raw text and widens its backtick delimiters past internal runs, padding when stripping would occur; the link renderer's own bracket escaping is gone (leaf escaping covers it, existing link expectations unchanged); table pipe escaping counts consecutive backslashes. The round-trip contract this pins: span text survives PT->MD->PT byte-for-byte, except linkified substrings (text kept, mark added) and markdown-inherent whitespace trimming. Pinned by a directed corpus and a committed seeded fuzz test (mulberry32, 2,000 cases per run over the hazard alphabet, multi-leaf splits, whitespace-only leaves, rendered and unrendered marks). Known inherent edges, both fuzz-discovered pathological shapes: an entity or backtick inside an explicit-scheme URL path corrupts under any policy, and a `<` before an inline object whose rendered output starts with a letter can form a tag; both are on record with repros. `linkify-it` becomes a declared dependency (previously reachable only transitively).
3941c77 to
2a860ef
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 2a860ef. Configure here.
| const edits = [ | ||
| ...collectInlineEdits(text, isLinkLabelChar), | ||
| ...collectLineStartEdits(text, args), | ||
| ].filter((edit) => edit.bypassLinkifyMask || !isMasked(edit, linkifyMask)) |
There was a problem hiding this comment.
Linkify mask drops pre-linkify markup
High Severity
The linkify mask suppresses emphasis and strikethrough escapes inside a claimed URL, but markdown-it runs those inline rules before linkify. Paired _, *, or ~~ in a path can be consumed as markup, so characters vanish instead of the substring keeping its text and only gaining a link mark. bypassLinkifyMask already exists for the same reason on entities and backticks, and is not set on these edits.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 2a860ef. Configure here.
1 participant
Portable Text whose span text contains literal markdown punctuation does not survive a trip through
portableTextToMarkdownand back: a span reading*bar*re-parses as emphasis, a leading#becomes a heading, and a text of[x]: ydisappears entirely, consumed as a link reference definition. For agent pipelines that read and edit Portable Text as markdown, this is silent text corruption in the middle of the edit loop, and it was the documented exception holding the round-trip contract back.Escaping happens at the text leaf, before mark renderers wrap their delimiters, so the escaper never touches generated syntax. The engine plans per block: leaves are grouped into lines at hard breaks, each line is joined into one string with a per-character map back to its owning leaf, all hazard detection runs once against that joined line, and the edits map back into per-leaf escaped strings keyed by leaf identity in a
WeakMap. Detecting on the joined line makes leaf boundaries irrelevant by construction, a hazard spliced across spans through an unrendered mark is seen whole, and a custom renderer rendering its own synthetic text node cannot shift sibling escapes. Minimal noise was a design goal, since agents read this output: mid-line#,-,>, lone brackets, and spaced asterisks all stay bare, pinned by a canonical-bytes table that catches over-escaping the round-trip tests cannot see.Ranges the linkifier will claim are deliberately not escaped (text identity holds, the re-parse adds a link mark), with two precision rules: entity and backtick escapes bypass the mask because they change what the parser sees before linkify runs, and a claim interrupted by a rendered mark boundary is not honored. The
codedecorator now receives raw text and widens its backtick delimiters past internal runs; the link renderer's own bracket escaping is gone, with the existing link test expectations byte-identical as the composition guard.The contract the docs now state without an asterisk: span text survives PT → MD → PT byte-for-byte, except linkified substrings and CommonMark-inherent whitespace trimming. Pinned by a directed corpus and a committed seeded fuzz test, the anti-regression tripwire this surface lacked. Two pathological fuzz-discovered edges have no correct markdown representation and are on record as known limits rather than fixed (entities/backticks inside an explicit-scheme URL path;
<immediately before an inline object whose rendered output starts with a letter). Markdown output for text containing significant punctuation gains backslashes; anyone diffing stored markdown output will see that shift.Note
Medium Risk
Touches core PT→MD serialization with complex CommonMark edge cases; output bytes change for punctuation-heavy plain text, though behavior is heavily tested (directed corpus + seeded fuzz).
Overview
Portable Text → Markdown now backslash-escapes literal markdown punctuation in plain span text so PT → MD → PT preserves span text byte-for-byte (e.g.
*bar*serializes as\*bar\*instead of becoming emphasis on reparse). Escaping is planned per block before mark renderers run: leaves are joined line-by-line, hazards are detected on the full line (including patterns split across spans), and results are keyed to text nodes via aWeakMapso custom renderers get pre-escapedchildren; the defaultcodemark uses rawtextand widened backtick fences instead.The planner is context-aware (headings, list-item first lines, blockquotes, table cells) and uses linkify-it to skip escaping inside bare URLs/emails/
www.(text stays identical; reparse may add alinkmark). Table cell pipe escaping now respects backslash parity. Docs and the changeset state the updated round-trip fixpoint and documented exceptions.Breaking-ish output change: stored markdown diffs will show more backslashes where span text contained significant punctuation; link label escaping is centralized (duplicate escaping removed from the default link renderer).
Reviewed by Cursor Bugbot for commit 2a860ef. Bugbot is set up for automated code reviews on this repo. Configure here.