diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c0d0864..4659ac06 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Documentation +- Persistence guide and Valkey comparison no longer describe the removed + WAL v2 (`src/persistence/wal.rs`); both now document WAL v3 + (`src/persistence/wal_v3/`: segmented files, per-record LSN, lz4 FPI, + off-loop fsync agent). + ### Fixed - **Data-dir deleted under a running server now latches a degraded state instead of error-looping the persistence tick (#366).** When `--dir` diff --git a/docs/comparison-valkey.md b/docs/comparison-valkey.md index b6ff7f4a..af7aa0f2 100644 --- a/docs/comparison-valkey.md +++ b/docs/comparison-valkey.md @@ -26,7 +26,7 @@ The right mental model is **Valkey = horizontal moat (ecosystem + governance), M | **In-memory data structure** | **`DashTable`** — custom segmented hash table = Dragonfly-style directory→segments→buckets + hashbrown-style Swiss-Table 16-way SIMD probing (`src/storage/dashtable/`). | **New open-addressing hashtable in 8.1** — 64-byte buckets, `serverObject` embeds key+value, replaces classic `dict`+`dictEntry`. Saves 20-30 B/KV vs Redis 7.2. | Both made the same architectural realization (cache-line-aligned, embedded entries). Moon's is per-shard; Valkey's is global. | | **Per-key memory footprint** | `CompactKey` 23 B inline SSO; `CompactValue` 16 B SSO with inline TTL; `HeapString` for ≥24 B strings; `listpack` / `intset` / B+tree sorted sets — all custom. **27-35 % less RSS than Redis 8.6 at ≥1 KB values** (Moon's own bench). | Embstr threshold raised to **128 B** in 9.1 (PR #1726/#2516) — 20 % less for short strings. Listpack, intset, quicklist, ziplist retained from Redis. | Different optimizations: Moon wins on large values; Valkey 9.1 wins on tiny strings. Below 64 B Valkey may be tighter post-9.1 — needs head-to-head re-bench. | | **Persistence — snapshot** | **Forkless compartmentalized snapshot** (`src/persistence/snapshot.rs`). Serializes DashTable segments one at a time as cooperative tasks in the shard event loop; segment-level COW via per-snapshot overflow buffer for keys modified in not-yet-serialized segments. RDB v2 format (`RRDSHARD` magic, last_lsn, created_at_unix_ms for PITR). | **`fork()` + COW** RDB child process. Memory spike during snapshot is the well-known production gotcha. RDB binary format. | **Clean architectural win for Moon.** No fork = no 2× RSS spike, predictable tail latencies during snapshot, works in container / non-MMU environments. | -| **Persistence — WAL/AOF** | **Per-shard WAL v2** (`src/persistence/wal.rs`) — `RRDWAL` magic, 32 B header, CRC32-framed blocks, batched 1 ms fsync via `FileIo` trait (uring on Linux). Multi-part AOF + RDB preamble + BGREWRITEAOF on monoio. 100 % crash-recovery validated. Three fsync policies. | **Single global AOF** written by one background thread via POSIX `write` / `fdatasync`. Multi-part AOF (base + incremental, since Redis 7.0). RDB preamble. Three fsync policies. | Moon's per-shard WAL eliminates the global write-serialization point — advantage grows with pipeline depth (2.75× over Redis at p=64 per Moon's bench). | +| **Persistence — WAL/AOF** | **Per-shard WAL v3** (`src/persistence/wal_v3/`) — `RRDWAL` magic, 64 B segment header, per-record LSN, checksummed records, lz4-compressed FPI, 16 MB segments, batched 1 ms flush + off-loop fsync agent. Multi-part AOF + RDB preamble + BGREWRITEAOF on monoio. 100 % crash-recovery validated. Three fsync policies. | **Single global AOF** written by one background thread via POSIX `write` / `fdatasync`. Multi-part AOF (base + incremental, since Redis 7.0). RDB preamble. Three fsync policies. | Moon's per-shard WAL eliminates the global write-serialization point — advantage grows with pipeline depth (2.75× over Redis at p=64 per Moon's bench). | | **Tiered storage** | **Tiered NVMe disk-offload** with cold-index, async write-back, read-through. Eviction spills under `maxmemory` instead of deleting. | None in core; not on roadmap. (Redis Enterprise has Flash; Valkey OSS does not.) | Moon-only feature; high differentiator for large-working-set / cost-sensitive workloads. | | **Replication** | PSYNC2 + per-shard shared replication backlog (`SharedBacklog` array; `evaluate_psync_shared` ANDs cover-check across shards). Async stream of WAL bytes. `WAIT` supported. No multi-master. | PSYNC2, single replication backlog, diskless RDB streaming, `WAIT`. No multi-master in OSS (CRDT only in Redis Enterprise). | Moon's per-shard backlog is the natural complement to per-shard WAL — but cross-shard PSYNC quorum logic is more complex. Both lack open-source active-active. | | **Clustering** | Redis Cluster compatible: 16384 slots, gossip (PING / PONG / MEET / FailoverAuthReq+Ack — magic `0x52656469` = "Redi"), 2048-byte slot bitmap per node, MOVED/ASK, election. | Same 16384-slot model. **9.0 added atomic slot migration**, multi-DB-in-cluster. **9.1 added CLUSTERSCAN.** 2000-node target. | Valkey's clustering is more battle-tested and has more recent feature work (atomic slot migration is genuinely valuable). Moon has feature parity at the protocol level but less operational mileage. | @@ -133,7 +133,7 @@ Two strategic plays look high-value: - `src/storage/dashtable/mod.rs` — Dragonfly + Swiss-Table architecture. - `src/storage/db.rs` — `Database`, `CompactKey`, `CompactValue`, `HeapString`, listpack/intset thresholds. - `src/persistence/snapshot.rs` — forkless segment-COW snapshot engine. -- `src/persistence/wal.rs` — WAL v2 format, `RRDWAL` magic, CRC32 block frames. +- `src/persistence/wal_v3/` — WAL v3 format, `RRDWAL` magic, per-record LSN, checksummed records. - `src/persistence/aof.rs` — multi-part AOF, fsync policies. - `src/replication/master.rs` — PSYNC2 with shared per-shard backlogs. - `src/cluster/mod.rs`, `src/cluster/gossip.rs` — Cluster bus, gossip, MOVED / ASK. diff --git a/docs/guides/persistence.md b/docs/guides/persistence.md index f0ef2fc3..855a200d 100644 --- a/docs/guides/persistence.md +++ b/docs/guides/persistence.md @@ -35,12 +35,17 @@ Unlike Redis's single global AOF file, Moon writes a separate WAL per shard. Thi The advantage grows with pipeline depth because each shard appends independently with no lock contention. -### WAL v2 format +### WAL v3 format -Moon uses WAL v2 with: +Moon uses WAL v3 (`src/persistence/wal_v3/`; v2 was removed) with: +- **Segmented files** (16MB default, `--wal-segment-size`) with a 64-byte + header carrying epoch, redo LSN, and base LSN +- **Per-record LSNs** — the foundation for PITR and CDC cursors - **Checksums** for corruption detection -- **Block framing** for crash recovery +- **Full-page images (FPI)** with lz4 compression for torn-page recovery - **Corruption isolation** per shard (one shard's corruption does not affect others) +- **Off-loop fsync** — a per-shard sync agent thread owns the fsync so the + shard event loop never blocks on durability waits The hot-path cost of WAL append is ~5ns (`buf.extend_from_slice()`), with batch `write_all` every 1ms tick and fsync on the configured schedule.