Add exact uniqueness (D02) validation for Parquet

[hadley]

Implements the individual-column and composite primary-key duplicate checks over Parquet data, with an implementation tuned for large files.

Approach

• Streaming, column-projected reads via the low-level typed column reader — no Arrow dependency, only the columns a check needs are read.
• In-memory exact hashing rather than an external sort: a duplicate is a key already seen. Three key shapes, each with its own fast path:
  • a single scalar column hashes bare i64s (no allocation);
  • a single byte column hashes its value bytes directly, as a zero-copy slice into the decoded page (ByteArray moved, not cloned or re-copied);
  • a composite key is length-framed so its columns can't collide.
• hashbrown table + chunked arena, reserved up front from the footer row count — no incremental rehashing and no reallocation spikes.
• Footer statistics settle the check without a scan where they can.
• Checks run in parallel (rayon).

Benchmarks

10M-row file; numeric id, unique string code, and a (string, string) composite primary key. Peak RSS is worst-case (all-unique).

check (10M rows)	time	peak RSS
numeric id	0.08 s	155 MB
string code	0.36 s	347 MB
composite (a, b)	0.58 s	450 MB
all three (in parallel)	0.72 s	854 MB

The earlier row-materializing external-sort approach took ~14 s for the same three checks — roughly 16–19× slower. Memory is proportional to the number of distinct keys (in-RAM), the trade for dropping the spill.

Fixes #66

[gaborcsardi]

Without going into the details of the implementation too much, I think there are two things here worth discussing.

Comparing the physical types instead of the logical types

This is a neat trick and I am actually surprised that it works at all. I suspect that it is not going to work for the new VARIANT and possibly GEOMETRY types, and one could argue that it already does not work for the JSON type. Probably also does not work in some edge cases, e.g. for DECIMAL the same value can be encoded differently, and NaN values may also have different encodings.

So I think it would make sense to explicitly restrict the checks to the logical types we deem safe. This is most current types I think, maybe not JSON depending on what we think about two JSON values being the same.

It is also worth checking (again) that we handle type matching to unknown logical types correctly. I thought about this when I wrote the type validation, but it is worth going back to it again, now that we know more about what we are trying to do. I can do this.

Missing values

I am not sure if we handle missing values correctly here.

For a PRIMARY KEY we already get an error from another check, I think, so maybe it is not worth doing the uniqueness check at all? Nevertheless if we still do the uniqueness checks, which should not report the second missing value as a duplicate. The same for composite keys as well, I think.

For a UNIQUE column missing values are allowed, I assume, so we should definitely not report the second missing value as a duplicate.