Pushed-down WHERE agrees with the engine on null cells

[philcunliffe]

Fixes #36.

whereToParquetFilter's output replaces engine-side WHERE (icebergDataSource sets appliedWhere: true whenever the conversion succeeds), so it has to select exactly the rows the engine selects. On columns containing nulls it did not, because the two evaluators disagreed:

• squirreling (applyBinaryOp) returns false for any comparison with a null operand, so a null cell never satisfies a bare comparison and always satisfies a negated one.
• hyparquet before 1.28.2 evaluated $lt/$lte/$gt/$gte with raw JS operators, where a null cell coerces to 0 (null <= new Date() is true); and it evaluates $ne as !equals(null, target), which is true (correct MongoDB semantics, but not what SQL wants).

So WHERE at <= TIMESTAMP '...' on a nullable column returned the null rows too, and WHERE NOT (n < 7) dropped the null rows it should keep. Wrong answers, not slow ones.

What changed

1. Require hyparquet >= 1.28.2. Its range operators now use MongoDB type bracketing: a null cell never satisfies $lt/$lte/$gt/$gte. That makes every bare comparison ($eq included) agree with the engine with no guard at all, so the common col < value predicate keeps its bare shape and prunes exactly as before.

2. Guard the two remaining disagreements, which are MongoDB semantics rather than hyparquet bugs:

• Bare col != value: $ne is true on a null cell in Mongo, but the engine excludes null rows. Converted to {$and: [{col: {$ne: null}}, {col: {$ne: value}}]}.
• Negated comparisons: NOT (n < 7) flips to $gte, which is false on a null cell, but the engine's NOT over a false comparison keeps null rows. Converted to {$or: [{col: {$eq: null}}, <flipped>]}. The exception is NOT (a = b) → $ne, already true on a null cell, which stays bare.

3. Fall back for a comparison against a literal NULL. WHERE n = NULL folded to {n: {$eq: null}}, which means "is null" to hyparquet and returned exactly the null rows; the engine answers it false for every row. IS NULL / IS NOT NULL are unaffected - they are a different AST node and were already correct.

4. Fall back for a TEXT cast of a non-primitive. foldCast used String(val), but the engine JSON-stringifies objects, so CAST(TIMESTAMP '2026-08-11T00:00:00Z' AS TEXT) folded to a locale date string where the engine produces "\"2026-08-11T00:00:00.000Z\"". squirreling's stringify helper is not exported, so this falls back rather than copying it.

Everything else already agreed and is untouched: =, IS NULL, IS NOT NULL, IN, NOT IN (including a NULL inside the list), and NOT (a = b).

Not a pushdown regression

Every case that pushed down before still pushes down - appliedWhere stays true. Bare comparisons keep their original unguarded shape, so row-group and file-level pruning are unchanged for them; the #20 file-level bounds pruning and #21 row-group pruning suites pass unchanged, including their dataFilesRead() and byte-count assertions.

The $or shape used for negated comparisons does give up pruning on row groups that contain nulls, since $or can only skip when every branch can. Negated bounds are rare, and the alternative is wrong rows.

Tests

New test/sql/whereFilter.equivalence.test.js builds a real 3-row Iceberg table with nullable int, timestamptz and string columns and runs 37 predicates both ways - through icebergQuery (pushdown) and through executeSql over the same records (engine truth) - asserting the row sets match. This is the invariant the converter has to hold, tested end to end through the actual parquet reader rather than at the filter-shape level. 13 of the 37 fail on master with hyparquet 1.28.1.

test/sql/whereFilter.test.js gains unit coverage for the guard shapes, the NULL-literal fallback, and the TEXT-cast fallback.

Full suite: 682 pass, lint and typecheck clean.

Downstream

hyparam/hypaware#721 is blocked on this - it adopts this converter in place of its own, and eleven of its shipped column specs are nullable TIMESTAMP (logs.timestamp, logs.observedTimestamp, traces.startTimestamp, traces.endTimestamp among them). It needs a release to bump to.