ir!: the validation constraints WA Web enforces, not just field shapes (schemaVersion 2.0.0)

[jlucaso1]

Contract break — schemaVersion 1.0.0 → 2.0.0. Migration at the bottom; four mechanical changes for a 1.x consumer.

Summary

The IR describes what fields a stanza has. That is enough to read a well-formed stanza and not enough to produce one, or to know why a real client rejected the one you produced. This adds the rules WA Web's own parsers enforce — symmetric by construction, so they serve a parser and an emitter equally.

Constraint	Where in the IR	Recovered
Echo rules — an answer's from must equal the request's to	assertions[].kind == "reference" + referencePath; referencePath on a field for the optional form	584
Pinned values — type="admin", matched="true", code=429	literalValue on ParsedField	1784
Per-RPC error vocabulary — a closed, per-RPC set	errorArms (+ errorEnvelope) and kind on a variant	645 arms, 104 side-typed
Response enums — the legal values, not a bare "type": "enum"	enumRef / enumKeys on ParsedField	151
Notification action unions — the payload inside the envelope	notifications[].actions, and NOTIF_ACTIONS in the reference notif.rs	47 arms, 184 shape elements
Typed content leaves — an integer/enum/byte-range element body, not a coarse contentType	type + byteLength/byteMin-byteMax/literalValue on a content field	PreKeyBundle <registration> (4 bytes) <type> (1) <id> (3); typeElementValue pinned to 05

Why

Each of these has cost a consumer real debugging time:

• build_iq_error hardcoded from="s.whatsapp.net", so every error answering a request addressed to g.us was unparseable — the client raised SmaxParsingFailure instead of surfacing the error. ~50 error sites, mis-filed for months as "the mock does not implement X".
• A successful promote was answered <participant type="200"> — a status code where the role goes — and the client rejected the whole response of an operation that had already been applied.
• An empty batch was answered 404 item-not-found, which matches no branch of that RPC. The vocabulary is per-RPC and there was no way to know it except reading each parser.
• The disappearing-message timer was read as ephemeralDuration (the create payload's spelling) when the action field is duration — every timer reported as 0. And not_ephemeral normalises into ephemeral with duration: 0, so a NOT_EPHEMERAL branch is dead code.

Answerable from the IR alone

// 1. "For RPC X, what must an error response's `from` be?"  → the request's `to`.
{ "kind": "reference", "name": "from", "referencePath": ["to"] }

// 2. "What does BatchGetGroupInfo accept?"  → a closed set of PAIRS; note no 404.
{ "tag": "BatchGetGroupInfoResponseClientError", "kind": "client_error",
  "errorArms": [ { "name": "IQErrorBadRequest",    "code": 400, "text": "bad-request" },
                 { "name": "IQErrorRateOverlimit", "code": 429, "text": "rate-overlimit" } ] }

// 3. "What pins a successful promote?"  → optional type="admin".
{ "method": "maybeAttrString", "name": "type", "wireName": "type",
  "literalValue": "admin", "required": false }

// 4. "Which wire tags map to group action `ephemeral`, and what does each set?"
{ "wireTag": "ephemeral",     "actionType": "ephemeral",
  "fields": [{ "name": "duration", "wireName": "expiration", "type": "integer" }] }
{ "wireTag": "not_ephemeral", "actionType": "ephemeral",
  "constantFields": [{ "name": "duration", "value": 0 }] }

Design notes

The error vocabulary is pairs, not two lists. errorCodes: [400, 501] beside errorTexts: ["bad-request", "feature-not-implemented"] admits four combinations where the parser accepts two — an emitter picking one from each list sends 400 feature-not-implemented, unparseable. 117 variants were ambiguous that way. errorArms pairs them, and the flat lists were removed rather than kept as a convenience view: a convenient view of an unsound shape is still unsound.

A two-level error carries an envelope. SetResponsePreKeySuccessVnameFailure wraps <error code="207"> around an inner <error> whose text the disjunction pins. Pins are partitioned by the node they are read from, so the envelope holds only the outer node's and the inner node's shared range stays on the arms — mixing them would say code 207 must also fall in 400–599.

The error side comes from the codes, not the name. WA spells the arms inconsistently, and SetResponsePreKeySuccessVnameFailure reads as a success while asserting type="error". Classification takes the parser's root discriminator first, then refines by code range. A variant whose codes settle nothing stays a plain error (45 of them) rather than being guessed into a side, and an error-ish name on a parser that asserts type="result" is a structured non-happy outcome (alternative, 2 of them) — never an error, because the wire discriminator outranks the name in both directions.

Required and optional pins are different constraints. A literal is a hard discriminator — present and exact or the variant does not match. An optionalLiteral guards nothing, so it records no assertion; its value still rides on the field. On a Bool presence flag the pin is always conditional: required: true describes the flag the parser always produces, never the wire attribute it reports on.

The action union resolves through the handler's own helpers. The w:gp2 arms delegate to module-local functions and the minifier hoists nearly every accessor into a local, so the reader inlines helpers (bounded), follows var bindings in statement order, and builds a scope per return path — sibling branches routinely rebind the same minified name to different accessors. A branch-only field is optional; a key bound to different wire reads in two branches is dropped rather than resolved to the first.

Missing beats wrong, consistently. Ambiguity is refused rather than guessed — a constant table bound twice, a helper name reused by a nested function, a child mapped under one name with two wire tags. Anything seen but not structurally resolvable is counted under dropsByReason, keyed per parser site, so "no constraint here" and "a constraint we failed to extract" never look alike.

Guards

• Round-trip (crates/wa-scan/tests/iq_roundtrip.rs) — for every IQ success shape, build a stanza from the IR alone and check the recorded constraints accept it. 140 shapes, 118 constrained, 180 nested pins exercised, against a group-addressed request so a hardcoded s.whatsapp.net fails. It proves the constraints are mutually satisfiable and correctly placed — not that none is missing, since emitter and checker read the same list; the module doc says so.
• Named canaries — the promote type="admin", both blocklist matched pins, and a floor on the from → to echo. An aggregate stays healthy while one variant quietly loses one pin; a named assertion cannot.
• Floor guard — diagnostics.iq.constraints and diagnostics.notif.{actions,actionShapes} are regression-checked; the counters may not shrink without --allow-shrink. It fired four times during review: three legitimate reductions (de-duplicated counters, phantom account_sync arms) and once on a real regression of mine, before it reached generated/.
• Per-domain drop counts — dropsByReason now exists for incoming and notif as well as iq. A constraint seen and not recovered is counted in the domain whose artifact it belongs to, so the number describes the file beside it: notif reports the one action enum table (create.reason) it cannot name, incoming reports 0.

Migration from 1.x

1. AssertionKind gained reference — handle or ignore it. (A closed-enum consumer rejects the document: validating the current iq/index.json against the 1.0 schema fails 579 times.)
2. ResponseVariantKind gained client_error / server_error — a variant that was error may now be either. Match all three, or use is_error().
3. A variant's errorCodes / errorTexts / errorCodeMin / errorCodeMax / errorClass are gone → errorArms (+ errorEnvelope).
4. ContentType gained integer — a <registration> whose body is a number used to be reported as string.

Also non-breaking but worth knowing: enum accessors now decode to type: "enum" rather than "string" (486 fields already did; 28 disagreed), and maybeAttrX derives its type from attrX, which fixed a PN user JID reported as a bare string.

Review

Thirty-one rounds, 173 findings from Codex (153) and CodeRabbit (20), all applied or answered with evidence; 170 threads resolved plus 3 raised in review bodies, one declined on the record (see below).

Several corrected wrong values already committed, and a majority of the later rounds were regressions introduced by earlier fixes in the same review — the recurring shape being a predicate that enumerated accessor spellings instead of deriving them from wap. That bug was found at five separate sites; the fifth is why the "byte-identical codegen" line below now reads differently.

One finding was declined on the record — constraint enforcement in the generated parsers; see the follow-ups below.

Follow-ups, deliberately not in this PR

Each of these came up during the work or the review, was measured rather than guessed, and is separable. None blocks this change.

1. Enforce the constraints in the reference codegen. This PR makes the constraints expressible; nothing consumes them yet. The generated parser decodes and validates nothing at all: 2178 literalValue, 169 intMin, 88 byteLength, 108 byteMin, 281 enumRef all go unchecked, and assertions are used only to dispatch a union, never to reject. Codex asked for byte-content validation specifically; enforcing only the newest constraint kind would leave a consumer rejecting an out-of-range byte payload while still accepting code=999 on a field pinned to 429. The design question is the surface — a validate() pass, a Result from the parser, typed errors — and it should be answered once for every kind.

2. Compile the generated Rust in CI. generated/**/*.rs is gitignored and CI only checks that it parses (syn::parse_file). Building it against a real wacore is what catches type errors, and it has: a smoke crate found 133 of them in an earlier round. Two of this PR's own bugs are the argument: the contentUint fields reached generated code, parsed fine as Rust, and decoded binary as decimal text; and an optional JID would have emitted a Jid initializer for an Option<Jid> field — syntactically valid, type-incorrect, and latent only because no such field exists yet. A tiny throwaway crate in CI would close both.

3. previousDocIds — the mex doc-id history. WA rotates a mex operation's docId and the current artifact can only ever show the newest. Measured across the 32 committed versions of generated/mex/index.json: 139 operations, 15 have rotated, 20 retired doc ids are recoverable from a one-shot backfill of git history. It cannot live in mex/index.json — that file must stay a pure function of the pinned bundles for the determinism gate — so it needs a separate accumulator outside the reproducibility chain. wasm.lock.json is the precedent. Shape: operationName → { docId → {firstSeen, lastSeen} }.

4. Pin the bootloader maps. The extracted structured data (rsrcMap / compMap / bxData / consistency.rev) is what the discovery path actually depends on, and none of it is pinned today. Pin the extraction, not the raw HTML — nonces and timestamps make the page byte-unstable, so hashing it would fail the determinism gate for reasons that have nothing to do with the protocol.

5. A real dataflow pass for the action reader, or a documented scope boundary. actions.rs has grown a hand-rolled JS scope/control-flow analysis, and the last six rounds of review have been a stream of constructs it does not model (do…while breaks, finally writes, for initializers, conditional-expression writes, object spreads, by-name callbacks, accessor aliases). Three are still open, recorded on their threads. generated/notif/index.json has not moved across those six rounds, and patching construct-by-construct has itself introduced regressions in five of them — so the class wants a real pass, or an explicit statement of what the reader does and does not model, rather than another round.

6. The constraints that are counted but still lost. dropsByReason now names them rather than hiding them, which is the point, but they are real gaps: 37 IQ enum tables WA composes at runtime (babelHelpers.extends({}, base, {error: "error"})), and 1 w:gp2 action table (create.reason) that is a module-local object rather than a nameable export. Both need a small evaluator rather than a structural matcher.

Validation

• cargo fmt --all --check; clippy --workspace --all-targets and --all-features --all-targets, both -D warnings

• cargo test --workspace — 571 tests, 119 added here; cargo test -p wa-ir --features schema

• cargo check -p wa-ir --target wasm32-unknown-unknown; cargo build -p whatspec --no-default-features

• scripts/validate-schemas.py generated → 12/12

• determinism: update --bundles <pinned> --check → 27 committed artifact(s) up to date

• the reference Rust codegen gains 26 lines over main (23 071 → 23 097), verified by regenerating from both and diffing

  For most of this review that line read "byte-identical", and that was a bug reporting itself as good news: wa-codegen still enumerated three content spellings, so the contentUint fields the scanners recovered were carried in the IR and dropped at generation. The unchanged output was the symptom. The added lines are those fields — registration: u64, r#type: u64, id: u64 and their reads — now reaching generated code.

  Two rounds later a third defect surfaced in the same file: 108 JID fields the IR marks required: false were declared Jid and parsed with ok_or_else("missing …"), because optionality was derived from the accessor spelling rather than from the data. The generated parser rejected responses the IR says may omit them.

  And one round after the fields arrived they were still decoded wrong: contentUint(N) is N big-endian bytes, not decimal text (contentInt is the text one), so the first version of those reads ran content_str().parse() over binary and yielded 0 every time. They decode from content_bytes() now, and byteLength records the width.

Summary by CodeRabbit

• New Features
  • Added richer notification payload extraction with per-tag action unions, constants, nested children mapping, and improved optional/required recovery.
  • Added expanded IQ constraint validation, including echoed request-value rules and structured typed error/outcome modeling.
  • Introduced deeper diagnostics and coverage reporting for both IQ constraints and notification action-union extraction.
• Bug Fixes
  • Improved field type detection across additional accessor spellings (enums, integers, bytes, content, and JIDs).
  • Ensured unsupported/ambiguous constraints are tracked as dropped rather than silently approximated.
• Documentation
  • Updated validation-constraints and cache filename/integrity guidance.
• Tests
  • Added/expanded regression and roundtrip validation to enforce the new constraint and notification metrics.