Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,13 @@ jobs:

- name: Python-frame JS cross-check (zero-dep independent reader, full round-trip)
run: node tools/frame-crosscheck.mjs

# Narrow on purpose: catches ONE of the five matrix incidents behind
# LAB-1400 — a version in the SDK Overview table stated as a snapshot,
# true when written and false at the next release. The other four (a ✅ on
# dead code, a ship-status boolean, two footnote regressions) need a
# reader, not a regex (decisions/matrix-version-verification.md).
- name: SDK Overview versions are floors, not snapshots
run: |
python3 tools/test_check_version_floors.py
python3 tools/check-version-floors.py
58 changes: 58 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,64 @@ All notable changes to the CacheKit Protocol Specification.

## [Unreleased]

### SDK Feature Matrix

- Consolidated ten conflicting open matrix PRs into one code-verified end-state
(LAB-1400), regenerated from current SDK code rather than from the stale PR
diffs. Cells that **reversed** — check these if you built on them: key
rotation (py/rs ✅ → ❌ fleet-wide; `rotate_key()` is a `NotImplemented`
stub, and cachekit-py's importable PyO3 `KeyRotationState` succeeds while
rotating nothing), Rust `::secure` preset and Rust sync support (both ✅ →
never existed), Builder API (py/ts ✅ → ❌), hardware-acceleration detection
(rs ✅ → not re-exported; ts N/A → ❌), TypeScript Arrow (🔜 → ❌), and
Python's encrypted read path (documented fail-closed → **fail-open by
default**), and `cache.secure.wrap()` in TypeScript (implied encryption → no
guarantee at all; LAB-513, CWE-311). New rows: Retry, Graceful degradation,
Cross-instance L1 invalidation (LAB-520), client-L1 stale-while-revalidate
(LAB-728), Orjson serializer, tamper/wrong-key failure mode, `secure`-API
enforcement, plus an Observability section (LAB-275). Supersedes protocol#25,
#28, #29, #31, #32, #33, #35, #37, #40, #43 — per-PR fold verdicts below.

- **Every version-keyed claim re-verified against published artifacts**, after an
expert-panel review found the first pass had introduced two new false cells of
the very class it was fixing. `cachekit-rs` 0.6.0 published 74 minutes before
that pass's final commit, so six Rust reliability cells shipped marked 🚧
unreleased when the tier was in fact released and **on by default**; and
"cachekit-ts ships the protocol-1.1 `bin` flip as of 0.1.5" was false —
published `@cachekit-io/cachekit@0.1.5` carries dependency pins byte-identical
to 0.1.4's, on `cachekit-core-ts@0.1.2` (native addons embed core **0.2.0**)
and `cachekit-core-wasm@0.1.1` (core **0.3.0**), so TypeScript emits legacy on
both paths. The matrix now carries a per-artifact rollout table with the
embedded-core evidence, and the method is recorded in
[decisions/matrix-version-verification.md](decisions/matrix-version-verification.md):
registry metadata establishes which artifact is current, and where an embedded
dependency decides the claim the `.crate`/`.tgz` is opened. Versions in the SDK
Overview are now floors (`X+`), enforced by
[`tools/check-version-floors.py`](tools/check-version-floors.py) in `verify.yml`
— four failures of this one mechanical class in six weeks.

- Footnote namespace repaired: markers `¹`–`⁴` were each defined **twice** with
unrelated content (Cache Backends and Protocol Compliance), so half the
evidence pointers in the file resolved to the wrong note — including the
"version cells are floors" note. The Protocol Compliance block is now `¹⁴`–`¹⁷`
and every marker is defined exactly once.

#### Per-PR fold verdicts (LAB-1400)

| PR | Ticket | Verdict |
| :--- | :--- | :--- |
| [#25](https://github.com/cachekit-io/protocol/pull/25) | LAB-423 | Incorporated — `spec/wire-format.md` lacked the CI-enforcement note; both enforcement points now named |
| [#28](https://github.com/cachekit-io/protocol/pull/28) | LAB-274 | Incorporated incl. the intent-preset semantics table; rejected its stale "rs has no circuit breaker" line |
| [#29](https://github.com/cachekit-io/protocol/pull/29) | LAB-275 | Partly incorporated (key rotation, hardware accel, Observability, serializer rows, MSRV 1.85); versions / ts-Workers / rs-stampede / interop claims rejected as stale |
| [#31](https://github.com/cachekit-io/protocol/pull/31) | LAB-520 | Incorporated as-is |
| [#32](https://github.com/cachekit-io/protocol/pull/32) | LAB-426 | Incorporated — rs Workers locking + TTL was still missing from `main` |
| [#33](https://github.com/cachekit-io/protocol/pull/33) | LAB-427 | Already on `main`; would have reintroduced a stale rs-Redis-lock ❌ |
| [#35](https://github.com/cachekit-io/protocol/pull/35) | LAB-518 | Incorporated; rejected its "backpressure stays ❌" line (LAB-729) |
| [#37](https://github.com/cachekit-io/protocol/pull/37) | LAB-430 | Already on `main`; same stale-cell problem as #33 |
| [#40](https://github.com/cachekit-io/protocol/pull/40) | LAB-751 | Incorporated — `main` still claimed "SWR forced off" on Workers |
| [#43](https://github.com/cachekit-io/protocol/pull/43) | LAB-728 | Incorporated; extended the py cell, which understated its gate (needs an explicit `ttl=`) |
| [#17](https://github.com/cachekit-io/protocol/pull/17) | — | Out of scope, left open |

### Specs

- StorageEnvelope `compressed_data` canonical encoding flipped from MessagePack
Expand Down
2 changes: 1 addition & 1 deletion decisions/key-rotation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
| **Date** | 2026-07-23 |
| **Ticket** | LAB-516 (filed by the LAB-275 cross-SDK feature-gap audit) |
| **Normative spec** | [`spec/encryption.md` → Key Rotation (Keyring)](../spec/encryption.md#key-rotation-keyring) — the spec section owns the rules; this record owns the rationale and runbooks. |
| **Implementation** | Not yet shipped in any SDK — tracked as LAB-516 sub-issues. The [feature matrix](../sdk-feature-matrix.md) rotation row is corrected to ❌ fleet-wide by the LAB-275 audit PR ([protocol#29](https://github.com/cachekit-io/protocol/pull/29)) and flips per SDK only as each implementation ships. |
| **Implementation** | Not yet shipped in any SDK — tracked as LAB-516 sub-issues. `ZeroKnowledgeEncryptor::rotate_key()` returns `NotImplemented` (`cachekit-core/src/encryption/core.rs:492`). The [feature matrix](../sdk-feature-matrix.md#encryption) rotation row reads ❌ fleet-wide as of the LAB-1400 consolidation and flips per SDK only as each implementation ships. |

---

Expand Down
68 changes: 68 additions & 0 deletions decisions/matrix-version-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
**[Protocol](../README.md)** > **Decisions** > **Matrix Version Verification**

# Decision Record: Verify the Feature Matrix Against Published Artifacts, Not Branches

| | |
| :--- | :--- |
| **Status** | Accepted |
| **Date** | 2026-08-04 |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
| **Ticket** | LAB-1400 (consolidation of ten conflicting `sdk-feature-matrix.md` PRs) |
| **Applies to** | [`sdk-feature-matrix.md`](../sdk-feature-matrix.md), and any version- or ship-status claim in [`spec/`](../spec) |

---

## Context

[`sdk-feature-matrix.md`](../sdk-feature-matrix.md) is a trust surface: readers use it to decide what they can rely on without reading four SDKs. In six weeks it produced **six** failures of one family — a cell asserting something the reader would act on that the shipped artifacts did not support:

| # | Failure | Direction |
| :--- | :--- | :--- |
| LAB-388 | "SWR ✓" for orphaned, never-called code | claimed more than shipped |
| LAB-998 | interop/v1 ship-status false | claimed less than shipped |
| LAB-1400 pass 1 | six Rust reliability cells marked 🚧 unreleased — `cachekit-rs` 0.6.0 had published 74 minutes earlier | claimed less than shipped |
| LAB-1400 pass 1 | "cachekit-ts ships the `bin` flip as of 0.1.5" — the published 0.1.5 pins a NAPI addon embedding `cachekit-core` 0.2.0 | claimed more than shipped |
| LAB-1400 pass 2 | a floor-scoping instruction that, read literally, told maintainers to strip the seven floors living outside the Overview table | would have reopened LAB-998 |
| LAB-1400 pass 3 | "a reader built against core ≤ 0.3.0 rejects `bin`", with fleet-upgrade sequencing advice derived from it — contradicted by `cachekit-core/tests/dual_decode.rs`, which asserts pre-flip readers accept `bin` | invented a migration risk |

**Four of the six were committed by the audit that existed to remove the first two.** That is the signal, and it is not carelessness: each was written by someone holding the correct general rule and applying it to a claim they had not opened the artifact for. Reading `main` tells you what the *next* release will contain, never what the current one does. Reading a plausible mechanism tells you what *could* happen, never what the test asserts. The gap in both cases is where the trust bug lives.

Two properties make it worse than ordinary staleness:

- **It is bidirectional.** A ❌ hiding a shipped feature makes users hand-roll what they already have; a ✅ they cannot install makes them plan around vapour. Neither is safer.
- **It rots without an edit.** A registry release falsifies a cell while the file is untouched. `cachekit-rs` 0.6.0 falsified six cells and a footnote in one event.

## Decision

**A version- or ship-status claim in the matrix is verified against the published artifact, or it is not made.**

1. **Registry metadata is the floor, not the proof.** Latest version and publish timestamp come from the registry API (crates.io, npm, PyPI). That establishes *which* artifact is current — nothing more.

2. **When an embedded or transitive dependency version decides the claim, open the artifact.** Downloading and inspecting is mandatory, not optional:
- Rust — fetch the `.crate`, read the *published* `Cargo.toml` for `[features] default` and dependency requirements, and list `src/` for the modules the claim names.
- npm — read the exact `dependencies` pins from the registry document (caret-free pins do **not** float), fetch the `.tgz`, and `strings` the `.node` / `.wasm` for the embedded `cachekit-core-X.Y.Z`. A source-level pin bump in a monorepo does **not** mean the consumed binary was republished.
- Python — the wheel's bundled extension, when a core version decides the claim.

3. **A merged PR is not a shipped feature.** Cite the release that carries it, not the PR that landed it. `git log` and tags describe intent; the registry describes reality.

4. **A version answering "which release do I need" is a floor (`X+`), never a snapshot.** A floor stays true as new releases publish; a snapshot is wrong the moment the next one lands and silently misleads until someone notices. That covers the SDK Overview table, the Compliance Status table, and the Architecture Notes release bullets. A version that is instead **evidence about one specific artifact** under rule 2 — an embedded `cachekit-core-0.2.0` in one `.node` binary, a caret-free `0.1.2` npm pin, a historical statement — stays bare, because appending `+` to it would make it false.

The CI guard polices only the SDK Overview column. The other floors are reviewer discipline: distinguishing "which release do I need" from "what is inside this artifact" needs the claim's intent, which a regex cannot read.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

5. **Record the verification date** next to the claim. A floor plus a date is auditable; a bare number is a guess with a decimal point.

6. **For a behavioural claim, cite the executed test — not a mechanism you traced.** Reading a code path and reasoning "therefore X rejects Y" produces claims that are plausible and wrong; three of the six incidents above were exactly that. Where a test already asserts the behaviour, cite the test (`cachekit-core/tests/dual_decode.rs` settles the dual-read question in one file). Where none does, trace the **whole** path including its error handling — a claim about what an SDK does on failure is worthless if the operation runs inside a `catch` you did not read — and say which layer you checked. If the two disagree, the test wins.

## Consequences

- Refreshing the matrix costs a handful of artifact downloads. That is the price of the document meaning anything, and it is minutes.
- CI partially enforces rule 4: [`.github/workflows/verify.yml`](../.github/workflows/verify.yml) runs [`tools/check-version-floors.py`](../tools/check-version-floors.py), which fails on a non-floor version in the **SDK Overview table only**, and [`tools/test_check_version_floors.py`](../tools/test_check_version_floors.py), a 15-case mutation suite that runs first so the guard cannot silently degrade to reporting OK.

- **Be honest about the guard's reach: it catches one of the six incidents above** — the `cachekit-rs` 0.5.0 snapshot that sat in the Overview table against a published 0.6.0. LAB-388 (a ✅ on dead code), LAB-998 (a ship-status boolean) and both LAB-1400 regressions (which lived in footnotes and Reliability cells) are all invisible to it, and it cannot tell whether a floor is *accurate*. Rules 1–3 and 5, and every floor outside that one table, remain reviewer discipline.

This matters more than it looks. An earlier revision of this record and of `verify.yml` both claimed the guard encoded "the one failure mode that recurred four times" — which would tell the next auditor that CI has this covered when it does not. An overclaiming gate is worse than no gate, because it converts a known gap into an assumed-safe one. The guard's own first version also passed a snapshot hidden behind an ASCII-hyphen placeholder and rejected a valid backticked floor; the mutation suite exists because of that.

## Rejected alternatives

- **Generate the version table from the registries in CI.** Removes the class outright, but the matrix's value is the *prose* — "shipped but unreachable", "on by default since 0.6.0" — which no generator produces. A generated table beside hand-written prose would drift from it, trading one inconsistency for another.
- **Drop versions from the matrix entirely.** "Which release do I need?" is the question the document is most often opened to answer.
- **Trust the SDK repos' own CHANGELOGs.** They record merges, and release-please tags can precede or follow publication. `cachekit-ts` 0.1.5's changelog lists the core-0.4.0 bump ([cachekit-ts#91](https://github.com/cachekit-io/cachekit-ts/pull/91)) while its published tree still pins the old addon — the changelog is exactly how the false claim was produced.
Loading
Loading