[WIP] Rust Acceleration Kernel POC#208

Draft

siddharthteotia wants to merge 43 commits into

feat/rust-native-poc-li

Commits on May 30, 2026

Add design docs for Rust acceleration initiative

authored and
siddharthteotia
committed
Add pinot-native module — Rust SUM kernel + JNI bindings

authored and
siddharthteotia
committed
Route eligible aggregations through native engine (WIP)

authored and
siddharthteotia
committed
Make NativeSumAggregationFunctionTest checkstyle-clean

authored and
siddharthteotia
committed
Replace autovec SUM kernel with explicit SIMD intrinsics

authored and
siddharthteotia
committed
Add ASF license headers to design docs, Cargo manifests, README

authored and
siddharthteotia
committed
Add JMH microbenchmark for native vs Java SUM(LONG)

siddharthteotia
and
claude
committed
Add scalar Rust SUM kernel JMH variant for three-way attribution

siddharthteotia
and
claude
committed
Lock Phase 1 agg scope, document SwissTable sharing

siddharthteotia
and
claude
committed
Add Rust SUM kernels and JNI entry points for INT/FLOAT/DOUBLE

siddharthteotia
and
claude
committed
Route SUM(INT/FLOAT/DOUBLE) through native engine

siddharthteotia
and
claude
committed
Port pinot-native module to li-pinot base (version + BlockValSet drift)

siddharthteotia
and
claude
committed
Add per-type scalar JNI entries for three-way SUM attribution

siddharthteotia
and
claude
committed
Extend SUM JMH to per-type matrix, write back §11.B

siddharthteotia
and
claude
committed

Commits on May 31, 2026

Commits on Jun 4, 2026

Commits on Jun 8, 2026

Commits on Jun 19, 2026

Fix checkstyle: NativeAggKind field ordinalByte -> _ordinalByte

Siddharth Teotia
and
claude
committed
Plan step (1a): add f64-accumulator SUM kinds for Pinot group-by parity

Siddharth Teotia
and
claude
committed
Design: lock native server combine + B1 seam for GROUP BY

Siddharth Teotia
and
claude
committed
Plan step (1b): wire native segment GROUP BY into the SSE operator

Siddharth Teotia
and
claude
committed
Design §19.0: mark step 1b done, set native combine as next

Siddharth Teotia
and
claude
committed
Design §22: full session log for 2026-06-19

Siddharth Teotia
and
claude
committed
Add native GROUP BY JMH harness + backend toggle

Siddharth Teotia
and
claude
committed
Design §22.8.1: capture combine threading + raw-value-key Q&A verbatim

Siddharth Teotia
and
claude
committed

Commits on Jun 20, 2026

Add gated attribution profiling to native GROUP BY (§22.9 lever 0)

Siddharth Teotia
and
claude
committed
Task #54: native server-combine core — radix-partitioned parallel merge

Siddharth Teotia
and
claude
committed
Foundation #1: dict-direct backend for dict-encoded segment keys

Siddharth Teotia
and
claude
committed
Foundation #2: multi-key support — generic driver + packed-key encoder

Siddharth Teotia
and
claude
committed
Foundation #4a: CanonicalF32/F64 keys (Task #45) — FP GROUP BY keys

Siddharth Teotia
and
claude
committed
Design §23: SOTA GROUP BY foundation — key-encoding architecture + build log

Siddharth Teotia
and
claude
committed
Foundation #4b: STRING arena — variable-length GROUP BY keys (Task #53)

Siddharth Teotia
and
claude
committed
Design §23: mark STRING arena (#4b) done, key-type foundation complete

Siddharth Teotia
and
claude
committed
Step 5 wiring (first slice): native server combine callable from Java (LONG keys)

Siddharth Teotia
and
claude
committed
Design §23: log the combine JNI wiring first slice (LONG keys)

Siddharth Teotia
and
claude
committed

Commits on Jun 22, 2026

Native combine: unify FFI/bridge over all key types (LONG/DOUBLE/STRING)

Siddharth Teotia
and
claude
committed
Design §24-§26: combine result-path, verbatim session log, limiting machinery

Siddharth Teotia
and
claude
committed
Design §27: boundary model + Pattern A/B + boundary-2-first decision (verbatim)

Siddharth Teotia
and
claude
committed
Native combine: ORDER BY top-K / result selection core (boundary-2, step 1)

Siddharth Teotia
and
claude
committed