feat(rtf-codec): hand-written RTF reader and writer against the content pivot

[Mearman]

Adds packages/rtf-codec, a hand-written RTF reader and writer against the shared document-schema.js pivot. Works towards #83; it does not close it — see "What is not here" below, and the scope comment on the issue itself.

RTF is the odd one out structurally: it is tokenised plain text with a brace-nested group and destination model, not XML, so nothing in ooxml.js/odf.js was reusable. The package carries its own byte lexer, its own destination state machine, its own \uN/\ucN handling with code-page fallback, and its own parsers for the five header mini-formats. Everything is implemented against Microsoft's own RTF Specification 1.9.1 (March 2008), and each module cites the section it implements by name; the primary source is the copy in Microsoft's Office protocol documentation archive, https://officeprotocoldoc.z19.web.core.windows.net/files/Archive_References/%5BMSFT-RTF%5D.pdf.

Status: under active development, using the same wording odf.js's README already uses for a package whose scope is real but bounded. The read and write paths below are implemented and tested; the package has not yet been run against a real-world corpus.

What reads

Groups, destinations and {\* ignorable destinations per the spec's own reader conventions. Control words (32-letter name cap, 10-digit signed parameter, one-space delimiter), control symbols, \'hh, \binN. \uN/\ucN with the ANSI fallback skip — including "any control word or symbol counts as one character" and "a brace ends the skippable run early" — and {\upr}/{\*\ud} pairs. Code pages: \ansi/\mac/\pc/\pca, \ansicpgN, and a font's own \cpgN/\fcharsetN, across the Windows, OEM and Macintosh single-byte pages plus UTF-8.

All five header tables: \fonttbl (face name, family, per-font code page), \colortbl (RGB, auto colour at index 0), \stylesheet (paragraph style names and heading levels), \listtable/\listoverridetable (resolving \lsN → \listidN → the level's \levelnfcN/\levelstartatN), plus \info and the document page geometry.

Body: paragraphs (\par, \pard, alignment, indents, spacing, \slN/\slmultN, \pagebb), runs (\b, \i, every \ul variant, \strike, \fN, \fsN, \cfN, \v), tables via the \trowd/\cellxN/\cell/\row marks with multi-paragraph cells, lists via \lsN/\ilvlN, PNG and JPEG pictures with goal-size and scaling, HYPERLINK fields including the \l anchor switch, \page, and the special-character vocabulary.

What writes

Everything above has a write path. Header tables are minted from what the document actually uses — a font entry per distinct family, a colour per distinct colour, a heading N style per heading level, a \listtable/\listoverridetable pair per list — and the body then references them by index. Output is deterministic (byte-identical for one input) and pure 7-bit ASCII: every non-ASCII character leaves as \uN with a one-character ? fallback under a single \uc1, which is what the spec itself recommends and what makes the output diffable.

Both encodings are exposed the way this family already does it: readRtf/writeRtf over DocumentTree, readRtfContent/writeRtfContent over ContentDocument, plus rtfCodec/rtfContentCodec z.codec() pairs and a {\rtf magic-byte RtfBytesSchema.

Diagnostics

Every construct the reader decides not to place reports a named code rather than disappearing — that matters more in RTF than in the XML formats, because the spec requires a reader to ignore what it does not recognise, so a silent drop is indistinguishable from never having seen the construct. src/diagnostics-coverage.test.ts produces each code from a real input and fails if any code in the table has no reachable emit site.

The counterpart is that a skip which loses nothing stays silent: legacy {\*\pn} numbering superseded by \lsN, the {\nonshppict} copy Word itself refuses to read, and the font-table subgroups the header parser already consumed would otherwise emit a warning per paragraph in a real document and bury the drops that matter.

What is not here

• Headers, footers, footnotes, endnotes, annotations — the flat ContentDocument has no position for them, and a footnote's real home is the tree-only definitions table a codec producing the flat form cannot reach.
• Fidelity constructs (constructStart/constructEnd, run-level extents) — neither read nor written.
• The residue channel — document-schema.js's SourceFormatSchema is a closed enum with no rtf member, so this codec has no legal residue value to emit. Adding one is an additive change to that package.
• \lfolevel override-level semantics — an override is followed to its \listidN and the list's own levels are used, so a per-level start-at or format override is ignored.
• East Asian DBCS code pages and code page 42; metafile and bitmap pictures; a picture with no stated size; nested tables; embedded and drawing objects; multiple sections; superscript/subscript; cell borders and vertical merge; right-to-left text.
• Registry wiring, tracked in Wire rtf-codec into documents.js's conversion registry and pathfinder #847: rtf is not yet a DocumentFormat member, so it is not reachable from convertDocument, the CLI, the MCP server, or the web UI. That change touches five tables in documents.js and three exhaustive conversion-matrix tests, and is deliberately separate — exactly as epub-codec's own engine wiring was (Wire epub into the conversion engine, CLI, MCP server, and web UI #802).

Verification

CI is green on this PR: Commitlint, Lint, Typecheck, Test, Test (workerd), Smoke test, Knip and Audit all pass — https://github.com/ExaDev/documents.js/actions/runs/33716756164. Locally the same tasks pass, with 146 unit tests, the workerd suite, and the smoke suite against the built ESM and CJS artifacts.

(The pull_request trigger did not fire on the first few pushes to this branch, which is why earlier revisions of this description pointed at workflow_dispatch runs instead; it fired normally once the branch was rebased onto current main, and the run above is the real PR check.)

The package is Worker-isomorphic, with isomorphic: true in its lint config; the workers suite deliberately exercises the two places a Node-only shortcut would have been tempting — src/base64.ts's hand-written encoders (instead of Buffer.from(...).toString("base64")) and src/codepage.ts's own tables (instead of the Node-only iconv-lite, which is banned by name alongside every third-party RTF library).

The code-page tables were generated from Python's own codec library and verified byte-for-byte against it rather than transcribed by hand.

Checklist against CONTRIBUTING's "Adding a package to the workspace"

1. packages/rtf-codec/ with the sibling scripts convention — done.
2. Sibling dependency as an ordinary semver range ("document-schema.js": "^5.4.0"), not workspace:* — done.
3. Added to pnpm-workspace.yaml's minimumReleaseAgeExclude — done.
4. eslint.config.ts calling packageLintConfig with isomorphic: true plus a test:workers suite — done.
5. _typecheck:attw in its scripts and its typecheck run list — done.
6. npm trusted publisher for rtf-codec pointing at this repository's ci.yml — not done, and needs a human: it is an npmjs.com setting, and per CONTRIBUTING it must be configured before this package's first release attempt, since a misconfigured package early in the topological order blocks every other package's release too.
7. Listed in the root README's package table and its Worker-isomorphic list — done.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
🔒 Security Review	✅ Completed 2026-09-03T05:18:46.751275Z	28e5ae1	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.