Pin the icebird absent-column contract at the SQL surface (#778)

[philcunliffe]

Documents the icebird-backed absent-column contract deferred from PR #740, and lands executeSql + collect SQL-surface tests that pin it.

Nothing here was described from reading the code. Every claim below is a run I made and its output. Where a prior description (including the issue's own) turned out to be wrong, that is called out.

What the contract actually is

Over an icebird-backed partition that physically lacks a declared column (the normal state after the v7 additive bump, LLP 0032), nothing throws, and the value read is path-dependent:

query	value	JSON
SELECT git_remote FROM t	null	{"git_remote":null}
SELECT git_remote AS gr FROM t	null	{"gr":null}
SELECT git_remote, 1 AS n FROM t	null	{"git_remote":null,"n":1}
SELECT id, git_remote FROM t	undefined	{"id":1} (key dropped)
SELECT git_remote FROM t WHERE date >= '2026-01-01'	undefined	{}
SELECT * FROM t	key absent from the row	{"id":1,"date":"2026-05-26"}

The discriminator is the size of the scan's hint column set, not the shape of the SELECT list. Squirreling routes a one-hint-column scan through scanColumn (execute.js, gated on plan.hints.columns?.length === 1, no aggregate required) and withSchemaColumns null-normalizes the hole there. Two hint columns take the row path and read undefined.

null and undefined are therefore both live readings of the same absent cell, and neither is "the" value. A consumer cannot distinguish "no value" from "column predates this partition" from the read.

Two things the issue predicted that measurement contradicts

1. A non-identifier sibling does not throw and does not change the value. The issue (carrying PR The union's absent-column contract is undefined-or-throws, not null (#731) #740's parquet reasoning) predicted SELECT git_remote, 1 AS n FROM t collapses the fast path and throws. It reads null, because a literal reads no column, so the hint set is still one column and the query never leaves the scanColumn path.
2. A WHERE on an unrelated column silently flips the projection's value. SELECT git_remote FROM t reads null; adding WHERE date >= '...' makes the identical projection read undefined, because the predicate's column joins the hint set. This shape was not anticipated anywhere.

Why it never throws (the mechanism, verified)

icebird builds each row with squirreling's asyncRow(row, rowColumns) where rowColumns = scanColumns ?? columns, i.e. the requested list (node_modules/icebird/src/sql/icebergDataSource.js:91-92,186). asyncRow sets cells[key] = () => Promise.resolve(obj[key]) for every requested key, so the cell exists and resolves to undefined; resolved is the raw object, which has no such key. I confirmed the row shape directly:

row.cells keys    : [ 'id', 'git_remote' ]
row.resolved keys : [ 'id' ]
'git_remote' in resolved: false
'git_remote' in cells   : true

A parquet-backed partition builds asyncRow over Object.keys(data[0]), the row's physical keys (src/core/query/parquet-source.js:111-113), so the cell does not exist and evaluating it throws ColumnNotFoundError. That is the whole difference. On icebird, ORDER BY, GROUP BY, DISTINCT, an expression, and an aggregate over the absent column all answer where the parquet union throws:

SELECT git_remote || 'x' AS e FROM t              -> { e=null }
SELECT id FROM t ORDER BY git_remote              -> { id=1 }
SELECT DISTINCT git_remote FROM t                 -> { git_remote=null }
SELECT git_remote, COUNT(*) AS n GROUP BY ...     -> { git_remote=null, n=1 }
SELECT COUNT(git_remote) AS n FROM t              -> { n=0 }
SELECT MAX(git_remote) AS m FROM t                -> { m=null }

A correctness hole the measurement exposed

The maintainer asked for a test on "a WHERE/aggregate on the absent column". Running it produced a wrong answer.

withSchemaColumns.scan forwarded options verbatim. LLP 0098#wrapper-duties already requires the wrapper to strip a predicate naming a declared-but-physically-absent column, but only scanColumn implemented it. icebird converts such a predicate to a hyparquet filter over a column its schema never had, filters nothing away, and still reports appliedWhere: true, so the engine trusts the stream and does not re-filter.

Before the fix (3 fixture shapes, same queries):

=== one-narrow (ONE partition lacking git_remote: no union in the way) ===
  SELECT id FROM t WHERE git_remote IS NOT NULL   -> 1 row(s): id=1     <-- WRONG
  SELECT id FROM t WHERE git_remote = 'zzz'       -> 1 row(s): id=1     <-- WRONG
  SELECT id, date FROM t WHERE git_remote = 'zzz' -> 1 row(s): id=1 ... <-- WRONG

=== two-narrow (TWO partitions, both lacking it) ===
  SELECT id FROM t WHERE git_remote IS NOT NULL   -> 0 row(s)
  SELECT id FROM t WHERE git_remote = 'zzz'       -> 0 row(s)

=== drifted (one with, one without) ===
  SELECT id FROM t WHERE git_remote IS NOT NULL   -> 1 row(s): id=2
  SELECT id FROM t WHERE git_remote = 'zzz'       -> 0 row(s)

Direct scan probe on the lone partition, before the fix:

source.scan({ where: git_remote = 'x' })
  appliedWhere: true   <-- the lie
  rows: [{"id":1,"date":"2026-05-26"}]

Two or more partitions hid it, because createDataSource then wraps unionSources, whose own per-partition gate (LLP 0015#multi-partition-union) fires first. The exposed shape is the ordinary one: a fresh install with a single client.

The fix mirrors the gate already in the wrapper's scanColumn: when the predicate names a column the wrapped source does not advertise, drop it along with limit/offset (only meaningful post-filter) and report appliedWhere: false / appliedLimitOffset: false. A predicate the source can satisfy is still pushed and still claimed, so ordinary filtered reads keep their pushdown.

After the fix, all three shapes agree:

=== one-narrow ===
  SELECT id FROM t WHERE git_remote IS NOT NULL   -> 0 row(s)
  SELECT id FROM t WHERE git_remote = 'zzz'       -> 0 row(s)
  SELECT id FROM t WHERE git_remote IS NULL       -> 1 row(s): id=1

I consider this a bug fix against an already-settled contract (LLP 0098#wrapper-duties states the duty; the union already honours it on its row path), not a new design call. If you disagree, this is the part to descope and I will resubmit the tests alone.

How to re-run this yourself

npm test                                                     # 4095 pass, 0 fail, 1 skipped
node --test test/core/ai-gateway-absent-column-sql.test.js    # 12 pass
npm run typecheck                                            # clean
npm run smoke -- gateway_claude_capture                       # ok
npm run smoke -- gateway_codex_capture                        # ok

The regression pins genuinely bite. Reverting only dataset.js and re-running the new file:

ok 1..8, 11, 12
not ok 9  - absent column: a lone icebird partition answers predicates on the column it lacks
not ok 10 - absent column: the wrapper reports appliedWhere false for a predicate it had to strip
# tests 12 / pass 10 / fail 2

The other ten pass either way, which is the point: they describe the contract, not the fix.

What is in this PR

• test/core/ai-gateway-absent-column-sql.test.js (new, 12 tests). executeSql + collect over a staged icebird cache in two shapes: one partition lacking the column (the no-union path), and a drifted pair. Every value asserted exactly (strictEqual against null / undefined, key presence, the JSON rendering), never through a tolerant ?? null. That tolerance is what let the mechanism be described wrongly five times while the suite stayed green; the existing ai-gateway-dataset.test.js pin uses ?? null and cannot tell the two apart.
• llp/0240-icebird-absent-column-contract.decision.md (new). Records the table, the mechanism, and the gate.
• llp/0015 and llp/0098 gain forward-refs only. Both are Active/Accepted, so per CLAUDE.md nothing they settled is rewritten. LLP 0015's union section is being corrected separately for the parquet backing in PR The union's absent-column contract is undefined-or-throws, not null (#731) #740; this branch does not touch it, so the two should not conflict.
• dataset.js: the row-path predicate gate, plus corrected block comment. The old comment claimed "a row object that lacks the key simply reads as null", which is the sixth wrong description and is now removed.
• message_projector.js: the v7 column comment now says what a read of one yields, and says not to branch on it.

LLP number

0240. Computed, not guessed: git ls-tree -r over all 49 refs/remotes/origin/* refs plus llp/tombstones/. Highest claimed anywhere is 0239, and 0240 appears in no ref. Unclaimed gaps below the maximum are 0047, 0048, 0082, 0126, 0127, 0221, 0227; I took the next number above the maximum rather than backfill a gap, since a gap can be reserved by work not yet pushed and this repo has already had to renumber one collision (#775).

What I could not establish

• SELECT *, git_remote FROM t over a drifted union mis-assigns a value into a neighbouring declared column (gateway_id came back holding the git_remote value). That is a star-expansion defect above this layer, I did not chase it, and the tests deliberately do not cover it. Noted in LLP 0240's Consequences. It probably deserves its own issue.
• I did not verify any of this against a real daemon-populated cache, only against appendRowsToSourceTable-staged icebird tables. That is the same fixture route the existing ai-gateway-dataset.test.js pins use, and it exercises the real createQueryStorageService / icebird / withSchemaColumns / squirreling stack, but it is not an acceptance run.
• Whether the null/undefined split is worth removing (making the row path null-normalize too, so one value is the answer) is a design call I did not make. It would change query output, and LLP 0240 documents the split as it stands rather than deciding against it.

Fixes #778

[philcunliffe]

Verdict: clean

Reviewed manually in a dedicated worktree at db76a3b with a real npm install
(icebird 0.8.22, hyparquet 1.28.2, squirreling from lockfile). codex is not
installed here and code-review is not invocable, so this was a hand review
plus independent measurement. No blockers. No preferences left open.

The scepticism this PR asks for is warranted, so I did not read the contract
back off the code. I built my own fixture and drove it, then re-ran everything
against origin/master to see the pre-fix shape for myself.

1. Re-measured the contract independently

Own probe (three rows in the narrow partition, two in the wide one, so the row
counts differ from the test fixture and a copied expectation could not pass by
accident). Every row of LLP 0240 #contract reproduced exactly, on both the
lone and drifted shapes:

query	measured value	measured rendering	LLP 0240
SELECT git_remote FROM t	null	{"git_remote":null}	match
SELECT git_remote AS gr FROM t	null	{"gr":null}	match
SELECT git_remote, 1 AS n FROM t	null	{"git_remote":null,"n":1}	match
SELECT id, git_remote FROM t	undefined	{"id":1}	match
SELECT git_remote FROM t WHERE date >= '...'	undefined	{}	match
SELECT * FROM t	key absent	{"id":1,"date":"..."}	match

Nothing threw: ||, ORDER BY, DISTINCT, GROUP BY, COUNT, MAX over the
absent column all answered. The path-dependence is real and reproduced: adding
WHERE date >= '2026-01-01' to an unchanged SELECT git_remote flips it from
null to undefined, while a literal sibling (, 1 AS n) does not.

I also verified the two mechanism claims the doc rests on rather than taking
them on trust:

• execute.js:284 gates the fast path on plan.hints.columns?.length === 1,
  with no aggregate required. The doc's discriminator is stated correctly.
• icebird sql/icebergDataSource.js:92,186 yields asyncRow(row, rowColumns)
  with rowColumns = scanColumns ?? columns, i.e. the requested list, and
  asyncRow sets resolved: obj. collect() (execute/utils.js:60-79) takes
  its all-materialized fast path and reads row.resolved[col], which has no
  entry. So "the pre-materialized resolved map that collect() reads simply
  has no entry for it" is right down to the field name.
• The parquet contrast is structurally correct too:
  src/core/query/parquet-source.js:111 builds its columns from
  Object.keys(data[0]), the physical keys, exactly as the doc says.

My observation agrees with LLP 0240 everywhere. No discrepancy found.

2. The where fix

Reproduced the defect on origin/master with my own fixture, lone partition:

SELECT id FROM t WHERE git_remote = 'zzz'      -> [{"id":1},{"id":3}]   (every row)
SELECT id FROM t WHERE git_remote IS NOT NULL  -> [{"id":1},{"id":3}]   (every row)
SELECT id FROM t WHERE git_remote IS NULL      -> [{"id":1},{"id":3}]   (right by luck)

and confirmed the union masks it (drifted was already correct on master), and
that COUNT(*) ... WHERE git_remote = 'zzz' was already 0 on master because it
routes through scanColumn, which had the gate. That is a clean confirmation of
the two-path story, not just of the bug.

At db76a3b all of these are correct.

Does it strip exactly the right predicates? Yes, and it cannot over-strip
into wrong results. Stripping is unconditionally safe: the wrapper reports
appliedWhere: false, so execute.js:355 re-applies the predicate over the
returned rows. The only cost of an over-strip is lost pushdown, never a wrong
row. Verified by adversarial probe, all correct on both shapes:

• predicate mixing a present and an absent column
  (date = '...' AND git_remote IS NULL)
• stripped predicate + LIMIT 1 where the only match is the last row, the
  exact "silently drop matching rows" shape the stripped limit/offset guards
  against: returned the late match, not empty
• LIMIT 2 OFFSET 1 under a stripped predicate
• a declared column absent from every partition (head_sha)
• IN / OR over the absent column
• qualified identifiers (t.git_remote), where whereColumns returns null and
  the whole predicate is declined: still correct, engine re-filters

The "present in some partitions, not others" risk is not present. With two
or more partitions createDataSource wraps unionSources, whose columns is
the physical superset, so canPushWhere returns true and the predicate is
forwarded to the union, which applies its own per-partition gate. Measured:
drifted WHERE git_remote IS NOT NULL returns exactly [{"id":2}], and
IS NULL exactly the two narrow rows. No legitimate predicate is dropped.

appliedWhere honesty, both branches. Stripped branch: hardcoded
false/false, and it must be, because icebird would otherwise return
appliedWhere true and appliedLimitOffset: canPushOffset (true, since the
inner call passes no where) for a filter the caller asked for and did not get.
Pushable branch: delegates verbatim, which is honest because the gate condition
is precisely that every predicate column is physically present. A legacy source
returning no flags degrades to falsy, so the engine re-filters. Also confirms
execute.js:342 ("applied limit/offset without applying where") can no longer
fire here. The returned object matches squirreling's ScanResults interface
exactly (rows(), appliedWhere, appliedLimitOffset).

Worth noting the fix is better than the scanColumn gate it mirrors: the
stripped branch keeps options.columns, so the projection pushdown survives,
where scanColumn drops everything but column/signal.

Tests 9 and 10 confirmed. Reverting only dataset.js to origin/master
and leaving everything else at PR head:

not ok 9  - a lone icebird partition answers predicates on the column it lacks
not ok 10 - the wrapper reports appliedWhere false for a predicate it had to strip
# tests 12  # pass 10  # fail 2

Exactly those two, and no others.

3. LLP 0240 numbering and forward-refs

• Number is free. 0240 appears on origin/fix/issue-778 and nowhere else:
  swept all 49 remote branches, origin/master, and llp/tombstones/. The
  neighbourhood is as expected: 0228 on fix/issue-742, 0229 on
  fix/issue-544, 0230 on fix/issue-614, 0231-0239 on
  feat/proxy-mode-capture (PR Attach Claude Code by proxy so Remote Control keeps working (LLP 0231-0235) #782). 0240 is the first free number.
• Every anchor resolves. 0240#contract, 0240#where-gate, 0240#pinned
  are explicit <a id=> anchors; 0098#wrapper-duties exists; 0015 has no
  explicit anchors but #multi-partition-union is the slug of its
  ## Multi-partition union heading, and 0032#capture of ## Capture. All
  relative links resolve, in both directions.
• Forward-refs only. 0015 and 0098 are +8/-0 and +7/-0: pure
  additions, nothing they settled was rewritten. The blockquote
  **Extended by [LLP NNNN].** form matches the convention already in use in
  those very two files (0015 already carries the same shape for 0034 and 0054).
  Status: Accepted matches 0220-0226. llp/0000 does not enumerate individual
  docs, so no index update was owed.

4. Test quality: the tests genuinely discriminate

Mutation test. Removing the undefined -> null normalization inside
scanColumn (a one-line mutation, the whole reason the two paths differ) breaks
five of the twelve:

not ok 2, 3, 4, 8, 11   # tests 12  # pass 7  # fail 5

So the null assertions are load-bearing and not tolerantly written. I also
confirmed node:assert/strict's deepEqual is deepStrictEqual and does
distinguish undefined from null both inside arrays and as object values, so
tests 3, 8, 9 and 11 discriminate too, not only the explicit strictEqual ones.
Combined with the revert result above, every one of the 12 tests fails under
some wrong behaviour it is meant to exclude. None is a free pass.

5. Conventions

No em dashes (U+2014) anywhere in the diff, including the LLP prose. No
statement-terminating semicolons (the three ; hits are prose inside comments).
No @typedef, no inline import('...') types; the test file declares its type
imports at the top with @import and uses repo-root-anchored .js specifiers.
@refs are attached with no blank line breaking attachment, and each says
something the filename does not.

Checks

• npm test: 4095 pass, 0 fail, 1 skipped (4096 total)
• npm run typecheck: clean
• npm run smoke -- gateway_claude_capture: ok
• npm run smoke -- gateway_codex_capture: ok
• new file alone: 12 pass, 0 fail

Findings

One, editorial, fixed.

• (low, editorial) llp/0240-icebird-absent-column-contract.decision.md:134
  • The final Consequences bullet records an observed star-expansion defect and
    says it "is left unaddressed here", but pointed at no tracking issue. SELECT *, <col> over a drifted union mis-assigns a value into a neighbouring column #788
    had already been filed for it and cites LLP 0240 in return, so the link was
    missing in one direction only. In a doc whose entire subject is a mechanism
    that went five rounds undocumented, an observed-and-deferred defect should
    say where it went.
  • Fixed in bca70d2e75ffb8fdc7970a3a7455050c71e43a4c: one sentence citing
    SELECT *, <col> over a drifted union mis-assigns a value into a neighbouring column #788. Editorial link only; nothing the doc settled changed,
    which is the class LLP 0156 keeps permissible on an Accepted doc. Verified
    present in the pushed tree.

Not treated as findings, per scope: the star-expansion defect itself (#788),
and whether to null-normalize the row path (LLP 0240 documents it as an open
design call rather than deciding it, which I think is the right call: the two
values are observationally equivalent for every consumer that treats both as
absent, and normalizing would be a behaviour change wanting its own doc).

Nothing needs a human. The correctness fix is real, minimal, correctly
scoped, and pinned by tests that fail without it.

[philcunliffe]

Round 2 of 2 - delta review. Clean. Round 1's clearance stands.

Round 1 reviewed db76a3b and returned clean. This round reviews only what moved the head since then.

The delta. git diff db76a3b..bca70d2 is exactly one commit (bca70d2), one file, +2 -1: the closing bullet of llp/0240-icebird-absent-column-contract.decision.md gains one sentence, It is tracked as [hyparam/hypaware#788](https://github.com/hyparam/hypaware/issues/788). No code, test, or other doc changed. Nothing else is in the diff.

The reference is accurate. Issue #788 exists, is OPEN, and carries exactly one label, neutral:fix. Its title, "SELECT *, <col> over a drifted union mis-assigns a value into a neighbouring column", is the same defect the 0240 bullet defers, and its body reproduces the concrete symptom the bullet describes (SELECT *, git_remote FROM ai_gateway_messages returning a row where gateway_id held git_remote's value). It back-references this PR, points at withSchemaColumns in dataset.js and at test/core/ai-gateway-absent-column-sql.test.js as starting points, and carries forward #778's caution about reproducing before explaining. The tracking pointer is correct, not decorative.

Hygiene.

• node --test test/core/llp-ref-hygiene.test.js at head: 11/11 pass, so every @ref in the tree still resolves. 0240's cited anchors are all present (#contract, #where-gate, #pinned) and are reached by four annotations across dataset.js, message_projector.js, and the new test.
• No em dash on the added line. The whole of llp/0240 is pure ASCII.
• 0240 is Status: Accepted, but it is unmerged and landing in this same PR, and a tracking-issue link is the forward-ref/editorial class of edit the repo convention explicitly permits. It does not touch what the doc settled.

CI. Green at bca70d2 on the commit itself, not merely at the PR ref: test (22), test (24), typecheck (22), typecheck (24), duplicate-numbers all success.

Not repeated, by design. A one-sentence doc addition cannot move the measured contract, the where-forwarding fix, or the tests, so round 1's measurement work (the independent fixture, the six-row #contract table, the master-side reproduction, the revert and mutation tests, the corpus-wide 0240 collision check) was not re-run.

codex is not installed and code-review is not invocable here; this was reviewed manually.

Nothing needs a human.

[philcunliffe]

Re-triage after the #789 merge: clean, held for human as before

Head c0206d2 (merge of origin/master into fix/issue-778). All checks below were run in a fresh worktree at that head with a real npm install, not read off the resolver's report.

The merged withSchemaColumns.scan does both duties

Verified in hypaware-core/plugins-workspace/ai-gateway/src/dataset.js:

• Gate (LLP 0240#where-gate): pushable computed at L203, the strip of where/limit/offset at L207-209, honest flags pushable && inner.* at L211-212.
• Alignment (LLP 0241#alignment): scanColumns read from options.columns at L202, before the strip, so the gate never narrows the advertised list; alignRows(inner.rows(), scanColumns) wraps the stream at L217.

The composition test genuinely bites in both directions

test/core/ai-gateway-absent-column-sql.test.js (now 13 tests, all pass at head; full suite 4195 pass 0 fail 1 skipped, typecheck clean). Revert experiments, each restored afterward:

• Alignment reverted (both call sites, dataset.js:217 and src/core/query/union-source.js:152, back to the raw stream): tests 7 and 11 fail, test 11 with exactly the predicted ColumnNotFoundError: Column "git_remote" not found. Available columns: id, date out of the engine's re-filter. So with the gate kept but padding gone, SELECT * FROM t WHERE git_remote IS NULL stops answering.
• Gate reverted (pushable = true on the row path only): tests 9, 10 and 11 fail; the = 'zzz' star case returns the row again.

So the two behaviours are load-bearing on each other exactly as the resolver claimed, and test 11 pins the intersection neither PR covered.

Judgement on the LLP 0240 amendment

What the merge did to 0240 (Accepted): added an "Amended by LLP 0241 (alignment), which landed first" blockquote that records the superseded reading verbatim, changed the one SELECT * table cell ("key absent from the row" became "undefined, under a key that exists (LLP 0241)"), and extended the Consequences (#788 now fixed by 0241; the composition dependency). One test assertion moved with it. The JSON rendering column did not move, so LLP 0241's byte-identical claim holds; I re-measured the star row at head and confirmed key presence and the unchanged rendering.

Recommendation: keep the amendment; a maintainer should ratify it when approving the merge. Reasons:

1. 0240 reaches master for the first time through this very PR. Landing it verbatim would publish a table wrong on arrival, which is the exact failure mode (a confidently wrong description of this mechanism) the doc exists to end.
2. Nothing was silently rewritten: the blockquote preserves the pre-amendment measurement in full and names the doc that invalidated it, which is the spirit of the forward-ref rule.
3. The alternative (0240 verbatim, correction only in 0241) does not work well here because 0241 does not carry the six-row table; readers would have to compose two docs to get one correct row.

Two caveats for the maintainer: the edited cell is inside what the doc settled, so this is at the edge of the convention's letter, and "Amended by" is not one of the named forward-ref verbs (Extended-by / Superseded-by). If strict immutability is preferred, the minimal correction is to restore the cell text and move the corrected reading into the blockquote; I have deliberately not made that change either way.

Everything else

• LLP numbering: swept every refs/remotes/origin/* ref plus llp/tombstones/. 0240 exists only on fix/issue-778; 0241 is one document (identical filename) inherited from master on the branches that merged it. No collision, and the duplicate-numbers CI check agrees.
• Anchors: 0241#alignment exists on master's copy (<a id="alignment">, line 78); node --test test/core/llp-ref-hygiene.test.js passes 11/11 at head.
• CI: all 9 check runs at commit c0206d2048be2fc98f363cd40e4fb4b8ccca0d6f are success (test 22/24 x2, typecheck 22/24 x2, duplicate-numbers).
• Body: Fixes #778 trailer intact; both prior review markers (db76a3b clean, bca70d2 clean) remain as comments.
• Hygiene: no em dashes and no semicolons in the resolver-touched files.
• Open issues: no open neutral:fix issue is owed; the only open items matching the searches are Follow-up: document the icebird-backed absent-column contract deferred from PR #740 #778 itself (closed by this PR on merge) and the unrelated Follow-up: deferred review findings from PR #585 #588. SELECT *, <col> over a drifted union mis-assigns a value into a neighbouring column #788 is addressed by 0241 on master.

No blocker, nothing unresolved. Marker appended to the body at head c0206d2048be2fc98f363cd40e4fb4b8ccca0d6f. Still held for human approval; not merging.

[philcunliffe]

Triage at head 36c77e1: one blocker class, held as stuck

What neutral was doing. Re-triage of PR #787 at head 36c77e186daeeaf8cb1aaf5e72edd7493beafcd0, after the two review rounds (both clean at db76a3b/bca70d2) and the c0206d2 re-triage were exhausted and the head moved twice more: the 32578580 conflict resolution when the proxy-mode batch and PR #740 merged, then the 36c77e1 merge of a now-green master (bringing PR #821 / issue #820). All checks re-run in a fresh worktree at this head with a real npm install: full suite 4270 pass / 0 fail / 1 skipped, typecheck clean, the PR's own 13 tests green, union-source.test.js 21/21 green. The question this pass focused on, per the contract area's history, was whether the PR's factual claims still match master after LLP 0241 and PR #821.

The icebird half still holds. Every row of LLP 0240's #contract table, the hint-set discriminator, the #where-gate, and the gate-plus-padding composition are accurate at this head and pinned by the 13 green tests in test/core/ai-gateway-absent-column-sql.test.js. The c0206d2 amendment correctly moved the one SELECT * cell that LLP 0241 changed. No problem there.

The blocker: the parquet-union contrast is now false, in two places

The mechanism prose contrasts icebird with a parquet union that throws. That contrast was true when written, survived the 32578580 merge (whose base still carried PR #740's pre-0241 text), and became false at this head when the 36c77e1 merge brought in PR #821, which re-corrected LLP 0015 to the post-LLP-0241 padded contract. The c0206d2 amendment blockquote scoped itself to "exactly one cell of the table" and does not cover these:

1. llp/0240-icebird-absent-column-contract.decision.md:78-85: "the cell does not exist and anything evaluating it throws ColumnNotFoundError. Hence, on icebird, ORDER BY, GROUP BY, DISTINCT, an expression, and an aggregate over the absent column all answer ... where the parquet union throws." On this very tree, none of those shapes throws over the parquet union: unionSources pads every row to the advertised list (LLP 0241 §alignment, src/core/query/union-source.js), and test/core/union-source.test.js:602 ("evaluating a column one partition lacks answers with undefined") pins WHERE extra = 'x', max(extra), ORDER BY extra, coalesce, and non-identifier siblings all answering, over two real parquet partitions, green at this head.
2. hypaware-core/plugins-workspace/ai-gateway/src/dataset.js:170-171 (JSDoc added in the 32578580 conflict resolution): "LLP 0015#multi-partition-union states the parquet-backed contract, which is a different one (undefined-or-throws) and does not govern this dataset." LLP 0015 at this head states the opposite: an absent column "reads as undefined", every row-path read agrees, and the evaluating shapes answer (the "Corrected again (master is red: four union-source absent-column tests fail, blocking every open PR #820)" note in llp/0015-query-and-datasets.spec.md). The comment mischaracterizes the doc it cites, and the two contracts are no longer meaningfully "different" on the row path.

Why this is a production risk rather than a style nit. LLP 0240 lands on master as Status: Accepted through this PR, asserting throw behaviour that green tests on the same commit contradict, so the corpus would carry two Accepted docs in direct contradiction about one mechanism. This repo's convention directs implementers to read and trust the tagged LLP before changing a subsystem, and a confidently wrong description of exactly this mechanism is what produced the #820 red-master incident (PR #740's tests pinned a description the tree no longer matched). Post-merge, the correction is expensive by design: an Accepted doc needs a new LLP plus forward-ref (the ceremony PR #824 / LLP 0261 is currently performing for a smaller instance of the same error in llp/0032). Pre-merge, it is a small edit of the kind this branch already made once (the c0206d2 amendment).

Non-blockers (recorded, not gating)

• PR body staleness (preference): the body's contract table still shows SELECT * FROM t as "key absent from the row", the same "where the parquet union throws" paragraph, and "12 tests" (now 13). The body is a dated measurement record with triage markers, but it becomes the merge-commit context; a refresh alongside the blocker fix would be cheap.
• Amendment ratification (carried forward from the c0206d2 triage note, preference/decision): the conflict resolution edited a settled table cell inside Accepted LLP 0240 and used "Amended by", which is not one of the named forward-ref verbs. The recommendation to keep it stands; a maintainer should ratify it when approving, or ask for the strict-immutability form (restore the cell, move the corrected reading into the blockquote).
• llp/0015 forward-ref wording (minor): the blockquote this PR adds says the union section "describes the parquet-backed sources it was written against"; post-Union absent-column tests pin the pre-LLP-0241 contract, so master is red (#820) #821 that section describes the current tree. Nothing false is asserted; tightening it can ride along with the blocker fix.

What unsticks this

One decision: how to correct the stale contrast before LLP 0240 lands Accepted. Either

• (a) extend the existing "Amended by LLP 0241" blockquote (or the paragraph itself, since the doc has not yet reached master) to cover the mechanism contrast: the parquet source still builds rows from physical keys at src/core/query/parquet-source.js, but since LLP 0241 the union pads, so the evaluating shapes answer with undefined on both backings, and the remaining icebird-specific fact is the scanColumn null path and the silent appliedWhere: true filter behaviour; and reword dataset.js:170-171 to match what LLP 0015 now states, or
• (b) rule the historical framing acceptable as-is and merge, accepting the contradiction into the Accepted corpus, or
• (c) take the PR's own descope offer and land the tests plus a corrected doc separately.

Neutral's recommendation is (a): it is the same treatment the conflict resolution already gave the table cell, and it keeps the doc's promise (measured, not derived) intact.

Reply with a comment on this PR (or push to the branch); neutral monitors this thread and will re-engage with your guidance on its next tick.