-
Notifications
You must be signed in to change notification settings - Fork 0
release(v0.7.0): Replication GA for multi-shard masters #333
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
4938ecd
4f2ea61
3856b8c
f71b3f4
af82a4e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,10 @@ | ||
| # Releases | ||
|
|
||
| ## v0.7.0 — 2026-07-15 | ||
| milestones: v0.7.0 "Replication GA for multi-shard masters" (soak-gated tag; v0.6.1 hardening folded in) | ||
| waivers: none — the v0.6.0 `shardslice-migration` waiver (was: expires 2026-08-01) is **retired**; lock-free cross-shard read work is now tracked as an open GA gap (`XSHARD-READ-01`, ROADMAP R4), not a time-boxed waiver. | ||
|
Comment on lines
+3
to
+5
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win Synchronize the waiver status with the production-contract ledger. This release entry says the 🤖 Prompt for AI Agents |
||
| evidence: Real async replication with WAIT/ACK across all six data planes (KV, vector/text, graph, WS, MQ, temporal); multi-shard master → single-shard streaming replica; every plane crash-durable under kill -9. Plus full v0.6.1 hardening (WAL v3 storage-kernel M1–M4) and a supply-chain CI gate (cargo audit + cargo deny). Release gate — 24h replication kill-9 soak (REPL-SOAK-01), zero acked-write loss: `SOAK-PASS duration=86400s cycles=114 acked=82044 inflight=7 master_kills=57 replica_kills=57` (2026-07-15; run dir moon-dev:~/moon-soak/runs/20260714-141946; RC main e2d87893; 114 alternating master/replica kill -9 cycles, 82,044 WAIT-acked writes preserved). Known limitation: streaming replica is single-shard only. Replica relative-TTL semantics: verbatim replication (no PEXPIREAT rewrite) + role-agnostic active expiry — absolute-rewrite lands in v0.7.1 (task #71b). | ||
|
|
||
| ## v0.6.0 — 2026-07-08 | ||
| milestones: v0.6.0-release (PR #249, six workstreams) | ||
| waivers: shardslice-migration (expires 2026-08-01; retirement planned with v0.7 L4 cross-shard-read work) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,42 +30,88 @@ Moon implements PSYNC2-compatible replication with per-shard WAL streaming and p | |
| - `WAIT N timeout` reflects real replica ACKs (1s `REPLCONF ACK` cadence). | ||
| - `master_link_status` in `INFO replication` reflects the handshake state — use it to detect a failed REPLICAOF. | ||
| - `CLIENT LIST TYPE replica` has no predicate yet; returns all clients. | ||
| - WS.\*/MQ.\* planes are **not replicated yet** (the master logs one warning | ||
| when a replica is attached); vector/text/graph planes replicate fully. | ||
| - **All six data planes replicate** (v0.7 GA): KV, vector/text index, graph, | ||
| workspace (WS.\*), message-queue (MQ.\*), and temporal. A full resync ships | ||
| each plane's snapshot; the live stream carries every plane's effect records. | ||
|
|
||
| ### Set up a replica | ||
|
|
||
| ### Start the leader (must be --shards 1 in v0.1.x) | ||
| **Topology (v0.7):** the **master** runs any `--shards N` (multi-core writer); each | ||
| **replica** runs `--shards 1`. Scale reads by adding replicas, not replica shards. | ||
| Replication is initiated at runtime with the `REPLICAOF` command — there is no | ||
| startup flag; operators script it after the replica is up (e.g. via an init hook). | ||
|
|
||
| #### 1. Start the master (any shard count) | ||
|
|
||
| ```bash | ||
| ./target/release/moon --port 6379 --shards 1 | ||
| ./target/release/moon --port 6379 --shards 4 --appendonly yes --appendfsync always | ||
| ``` | ||
|
|
||
| ### Start the replica (any shard count) | ||
| #### 2. Start the replica (must be --shards 1) | ||
|
|
||
| ```bash | ||
| ./target/release/moon --port 6380 --shards 4 | ||
| ./target/release/moon --port 6380 --shards 1 --appendonly yes | ||
| ``` | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| ### Connect the replica to the leader | ||
| #### 3. Attach the replica to the master | ||
|
|
||
| ```bash | ||
| redis-cli -p 6380 REPLICAOF 127.0.0.1 6379 | ||
| ``` | ||
|
|
||
| ### Verify link status | ||
| The replica performs a full resync (one merged Redis-format RDB across all planes), | ||
| then applies the live stream. Replicas are **read-only**: writes return | ||
| `-READONLY` (`INFO replication` reports `slave_read_only:1`). | ||
|
Comment on lines
+68
to
+70
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win Do not imply standard Redis replica interoperability. The supplied architecture contract describes the full-resync payload as Also applies to: 100-103 🤖 Prompt for AI Agents |
||
|
|
||
| #### 4. Verify the link | ||
|
|
||
| ```bash | ||
| redis-cli -p 6380 INFO replication | grep master_link_status | ||
| # Expect: master_link_status:up | ||
| # Expect: master_link_status:up (anything else = handshake not complete) | ||
| ``` | ||
|
|
||
| ### Acknowledged writes with WAIT | ||
|
|
||
| `WAIT numreplicas timeout` blocks until at least `numreplicas` replicas have ACKed | ||
| every write issued on the connection (replicas send `REPLCONF ACK` on a ~1 s | ||
| cadence). Combine it with `appendfsync always` on both sides for a **zero-RPO, | ||
| cross-node durable write**: | ||
|
|
||
| ```bash | ||
| redis-cli -p 6379 SET k v | ||
| redis-cli -p 6379 WAIT 1 1000 # returns the number of replicas that ACKed within 1000 ms | ||
| ``` | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| On the master, `INFO replication` lists each replica's `offset` and `lag`; on the | ||
| replica it reports `slave_repl_offset`. See the [tuning guide](tuning.md#replication-durability) | ||
| for the durability/latency trade-offs. | ||
|
|
||
| ### Promote a replica (failover) | ||
|
|
||
| ```bash | ||
| redis-cli -p 6380 REPLICAOF NO ONE # replica stops applying and becomes a writable master | ||
| ``` | ||
|
|
||
| Repoint surviving replicas at the new master with `REPLICAOF <new-host> <new-port>`. | ||
| There is no automatic replication failover outside cluster mode (see below). | ||
|
|
||
| ### Replication features | ||
|
|
||
| - **All six data planes** — KV, vector/text index, graph, WS, MQ, temporal (v0.7 GA) | ||
| - **PSYNC2 protocol** — compatible with Redis replication clients | ||
| - **Per-shard WAL streaming** — each shard streams its own WAL independently (once connected) | ||
| - **Partial resync** — reconnecting replicas resume from where they left off via the replication backlog | ||
| - **Lazy backlog** — the replication backlog is only allocated when the first replica handshake begins (REPLCONF), saving ~12 MB baseline memory | ||
| - **Per-shard WAL streaming** — each master shard streams its own WAL; a multi-shard master merges them into one exactly-once feed | ||
| - **Partial resync** — single-shard masters resume reconnecting replicas from the backlog window; multi-shard masters answer every reconnect with a full resync | ||
| - **Lazy backlog** — allocated only when the first replica handshake begins (REPLCONF), saving ~12 MB baseline memory | ||
| - **Validated** — 24 h continuous-load kill-9 soak (alternating master/replica restarts), zero loss of any WAIT-acknowledged write | ||
|
|
||
| !!! warning "Replica TTL semantics (v0.7)" | ||
| Relative-expire commands (`EXPIRE`, `SETEX`, `PEXPIRE`, `GETEX` with a relative | ||
| TTL) replicate verbatim rather than being rewritten to absolute `PEXPIREAT`, and | ||
| replicas run their own active-expiry cycle. Under normal clock sync keys expire | ||
| correctly on both sides, but master/replica clock skew can shift a relative-TTL | ||
| key's expiry moment by up to that skew. For exact cross-node expiry parity, set | ||
| absolute deadlines with `PEXPIREAT`. Absolute-rewrite + role-gated expiry land in | ||
| v0.7.1. | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| ## Cluster mode | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
The soak is documented as a tag gate, but the supplied release workflow does not enforce it for v0.7.0.
CHANGELOG.md#L22-L25: either wire REPL-SOAK-01 into an enforcing release step or describe it as completed/manual evidence.docs/PRODUCTION-CONTRACT.md#L129-L129: remove the claim that the row gates the v0.7.0 tag unless the workflow is changed to block that tag.📍 Affects 2 files
CHANGELOG.md#L22-L25(this comment)docs/PRODUCTION-CONTRACT.md#L129-L129🤖 Prompt for AI Agents