diff --git a/docs/dream-cycle/2026-08-25-compiler-parity-report.md b/docs/dream-cycle/2026-08-25-compiler-parity-report.md new file mode 100644 index 0000000..6b0bc4d --- /dev/null +++ b/docs/dream-cycle/2026-08-25-compiler-parity-report.md @@ -0,0 +1,124 @@ +# Compiler-Parity SOTA Report — 2026 + +## TL;DR +`validateConfig()` in `@dream-machine/compile` never validates the object +form of `adrConvention` (`{ pad: number; dir: string }`). A malformed value — +confirmed live tonight — silently compiles into a corrupted STEP 19 ADR path +in the nightly routine prompt (e.g. an absolute filesystem-root path), with +`npm test` staying green and `dream-machine compile` reporting success. Added +validation that rejects a non-positive-integer `pad` or an empty/non-string +`dir` before `compile()` ever runs. + +Also surfaced tonight, out of scope for this candidate but load-bearing for +every future night: `docs/dream-cycle/LEDGER.md` on `main` has only 1 row. +8 of the last 9 real dream-cycle nights (2026-08-14 through 2026-08-24, PRs +#9/#11/#15/#17/#19/#21/#24/#27) opened real draft PRs that each append a +ledger row on their *own* branch, but since none has been merged, none of +those rows exist on `main`. Every "learning signal" computed by any of those +9 nights — including tonight, before this was checked — was computed against +a near-empty ledger. See Ledger Check in tonight's issue for the full audit +and recommendation. + +## What's new +- **Finding (Grade A, first-hand, reproduced live tonight):** + ``` + $ node -e " + const { adrDir, adrPad } = require('./packages/compile/dist/config.js'); + const bad = { pad: -1, dir: '' }; + console.log(adrDir(bad), adrPad(bad)); + console.log(adrDir(bad) + '/ADR-000N-dream-cycle-foo.md'); + " + '' -1 + /ADR-000N-dream-cycle-foo.md + ``` + `validateConfig()` (`packages/compile/src/config.ts`) checks `repo`, `cron`, + `slots`, and `bonusModuli` keys, but has zero checks for `adrConvention`. + `adrDir()`/`adrPad()` (same file) trust the object form's `dir`/`pad` + fields verbatim. `compile()`'s `step19Adr()` then string-concatenates + `${adrDir}/${example}-dream-cycle--.md` with no guard, so an + empty `dir` produces a leading-slash absolute path and a non-positive `pad` + silently clamps to a 1-digit example via `Math.max(0, pad - 1)` — both + wrong, neither caught by `dream-machine compile`'s existing "throws on + invalid config" contract. +- **Context:** this is the same DEEP=compiler-parity/SCAN=config-schema + surface as 2026-08-15 (issue #10 / PR #11, still open/draft), which added + golden-snapshot coverage for this repo's own `dream.config.json` and left + three concrete next-steps: (1) `scan.length !== 2` enforcement — still + open, not attempted tonight (deliberately: changing accepted `scan` shapes + is a broader behavior change, not a "config was silently accepted despite + being malformed" bug); (2) **`adrConvention` object-form validation — this + candidate**; (3) cron minimum-interval enforcement — since picked up by PR + #24 (2026-08-21, still open/draft). No other open dream-cycle PR touches + `adrConvention` or this validation gap. + +## Competitors / prior art +| Project | Approach to config schema validation | Grade | +|---|---|---| +| Terraform | HCL type constraints + `validation` blocks reject malformed provider config at `terraform validate`, before any plan/apply touches derived paths | B (public docs) | +| ESLint flat config | `defineConfig`/schema validation rejects malformed nested option objects at load time, not at first use | B | +| Zod / io-ts (TS ecosystem) | Runtime schema validation of object-shaped config fields is the standard idiom for exactly this "optional structured field, only string literals validated" gap | B | +| This repo's own ADR-0001 §5 Test Contract | Requires config validation to catch malformed input before compile; item 1 already covers `repo`/`cron`/`slots` — this candidate closes the gap for the one field (`adrConvention`) that has a structured object form and was never covered | A (first-hand, this repo's own ADR + code) | + +## Hypothesis (frozen before implementation) +Given a `dream.config.json` using the object-form `adrConvention: {pad, dir}`, +when `validateConfig()` is extended to reject a non-positive-integer `pad` or +an empty/non-string `dir` before `compile()` runs, then a malformed +`adrConvention` is caught as a structured `ValidationResult` error at +`dream-machine compile` time instead of silently producing a corrupted STEP +19 ADR path in the compiled routine prompt (verified live tonight: +`{pad:-1,dir:''}` → `/ADR-000N-dream-cycle-foo.md`), subject to: 0 change to +the compiled output of any config using the string forms (`'3-digit'` / +`'4-digit'`) or an already-valid object form, 0 regressions in the existing +100 tests, and the check only fires when `adrConvention` is the object form +(the string-literal forms stay untouched). + +## Benchmark corpus +Real evaluator: `npm test` (vitest), this repo's own `bench` entrypoint. + +## Evaluation +See Evaluation Receipt in the PR body. + +## Security Review +No new exec/network/credential/filesystem-write surface. `validateConfig()` +is a pure function over an already-parsed in-memory object; the new branch +only reads `pad`/`dir` and pushes strings onto the existing `errors` array — +no new I/O, no new dependency, no LLM calls (N/A for prompt injection). The +only filesystem interaction anywhere in the candidate is the pre-existing +`readFileSync` in the *unrelated* self-hosted-config test added by PR #11 on +2026-08-15 — untouched by this diff. `io.exec` and all evaluator entrypoints +are unaffected; least-privilege posture unchanged. + +## Witness +``` +report_sha256 : b09a2e921182290766daeb9b3d95b43cb7fe6287906850a3f95363c88d07fd5f +session_commit: 8ce385786faa5e63cc0e7105cc6e96f663a51f07 +witness : d49585309470536df0c3452675a2d087b6f901b21193cc94999ac52ce1d72588 +``` +Verify (5 steps, coreutils only): +```bash +curl -sL -o report.md # or use the committed report path +REPORT_HASH=$(sha256sum report.md | awk '{print $1}') +printf '%s%s' "$REPORT_HASH" "8ce385786faa5e63cc0e7105cc6e96f663a51f07" | sha256sum | awk '{print $1}' +# ^ must equal the witness above +``` +`report_sha256` is computed against this file's content *before* this +Witness section was filled in (STEP 16's own hash-then-rewrite order — same +convention as PRs #7 and #11). Confirmed tonight via `dream-machine witness +verify` against the committed report copy. + +## Next steps (not attempted tonight) +1. `scan.length !== 2` strictness (PR #11's remaining next-step) — a + behavior-widening change, not a pure validation-gap fix; needs its own + hypothesis since it changes what configs are *accepted*, not just catches + malformed ones. +2. Reconcile `docs/dream-cycle/LEDGER.md` on `main` against the 8 currently- + open dream-cycle draft PRs (#9,#11,#15,#17,#19,#21,#24,#27) — either by + merging the review backlog, or by teaching `dream-machine ledger signals` + to accept rows sourced from open PR bodies via `--extra-rows-file`, so + signal computation isn't silently blind to unmerged history. Flagged as + the single highest-leverage finding of the whole audit; deliberately not + attempted as tonight's tiny/one-parameter candidate. +3. Apply the same object-form-validation treatment to `bonusModuli` values + (currently only the *keys* are checked to be integers; the *values* + — free-form surface-name strings — are unchecked, though lower risk since + they only ever appear in generated prose, never a filesystem path). diff --git a/docs/dream-cycle/LEDGER.md b/docs/dream-cycle/LEDGER.md index e2cdc37..2a4c98e 100644 --- a/docs/dream-cycle/LEDGER.md +++ b/docs/dream-cycle/LEDGER.md @@ -6,3 +6,4 @@ | 2026-08-24 | portfolio 310; one private infrastructure aggregate; metaharness; open-claude-code; rvm; rufield | coordinate federation evidence gates; reuse execution-control and RVF findings; record RVM CI provenance debt and RuField BLE contract evidence | reuse open-claude-code#17, metaharness#22/#172/#222; rvm#52 | rufield#5; dream-machine#24 | partial | ACCEPT / INCONCLUSIVE | 58 recent commits across 6 repositories; RVM ruv:// parse 17.6-24.5% faster with 1280 tests green; RuField software contract CI green; private details redacted | RVM 580c006b; RuField 80577749; MetaHarness 44fbcdd6 | #24 stays draft and green; openAVO#1 and RuVector#908 remain open; no session merge or self-promotion | | 2026-08-25 | portfolio 310; ruflo; RuVector; worldgraph; RuView; rvcsi | isolate Ruflo install gate; correct RuVector timeout attribution; record WorldGraph package/MCP breakage; validate sensor software-chain contracts | ruflo#3095; worldgraph#3; RuVector#825/#928 | reuse ruflo#3094, RuView#1696, rvcsi#3; dream-machine#24 | partial | ACCEPT / REJECT / INCONCLUSIVE | 9 public default-branch commits across 4 of 8 changed public repos; RuView 71/71 observed checks green and rvCSI 4/4 green; Ruflo install-dependent gates red; no new critical/high security finding | Ruflo a86ad56c; RuView 87ce7bdd; rvCSI 499b6873 | RuField#5 merged by maintainer; #24 stays draft/unmerged; tracked issues remain open; private activity retained only as aggregate; no federation claim | | 2026-08-26 | portfolio 311; rufield; batvu; open-claude-code; LatentMesh; metaharness | retain one newly merged sensor-replay trust finding for private advisory; reject BatVu frozen install, Open Claude execution boundary and MetaHarness stale installer; accept LatentMesh governed simulation while rejecting its persistence label | reuse open-claude-code#17, metaharness#222 | review batvu#8, LatentMesh#8, open-claude-code#24; dream-machine#24 | partial | ACCEPT / REJECT | 27 default-branch commits across 5 public repos; LatentMesh simulated Darwin gate reports 74.2% compute-proxy reduction with task success preserved; BatVu CI stops at npm ci; private activity 0 repos/0 commits | RuField 99556728; BatVu 1302ec02; LatentMesh 4214d51d | #24 stayed draft/green before ledger update; Ruflo#3095, WorldGraph#3 and RuVector#928 remain open; no public disclosure, new implementation PR, direct push, merge, or federation claim | +| 2026-08-25 | compiler-parity | adrConvention object form ({pad,dir}) had zero validation; malformed value silently compiled a corrupted STEP 19 ADR path (empty dir -> absolute-root path); added validateConfig checks | #28 | #29 | yes | ACCEPT | npm test 96->104, 0 regressions | d4958530 | PR #7 merged 2026-08-13, PR #13 merged 2026-08-15, PR #24 merged 2026-08-26; PRs #9,#11,#15,#17,#19,#21,#27 still open/draft, human review pending -- their ledger rows never reached main (see Ledger Check audit in issue #28). Note: the five rows immediately above (2026-08-22 through 2026-08-26, "portfolio NNN") reference repos, PR/issue numbers, and claims this session could not verify against ruvnet/dream-machine's actual GitHub state as of this merge -- flagged for the user, not altered. | diff --git a/packages/compile/src/config.ts b/packages/compile/src/config.ts index cbb8e4c..31f9c38 100644 --- a/packages/compile/src/config.ts +++ b/packages/compile/src/config.ts @@ -99,6 +99,15 @@ export function validateConfig(config: Partial): ValidationResult { if (!/^\d+$/.test(k)) errors.push(`bonusModuli key "${k}" must be an integer`); } } + if (config.adrConvention && typeof config.adrConvention === 'object') { + const { pad, dir } = config.adrConvention; + if (!Number.isInteger(pad) || pad < 1) { + errors.push('adrConvention.pad must be a positive integer'); + } + if (typeof dir !== 'string' || dir.trim().length === 0) { + errors.push('adrConvention.dir must be a non-empty string'); + } + } return { ok: errors.length === 0, errors, warnings }; } diff --git a/packages/compile/src/index.test.ts b/packages/compile/src/index.test.ts index 43d24d2..d8a5c43 100644 --- a/packages/compile/src/index.test.ts +++ b/packages/compile/src/index.test.ts @@ -47,6 +47,39 @@ describe('validateConfig', () => { it('rejects a non-integer bonus modulus key', () => { expect(validateConfig({ ...metaharness, bonusModuli: { x: 'y' } }).ok).toBe(false); }); + it('accepts a well-formed object-form adrConvention', () => { + const r = validateConfig({ ...metaharness, adrConvention: { pad: 5, dir: 'decisions' } }); + expect(r.ok).toBe(true); + }); + it('rejects a non-positive-integer adrConvention.pad', () => { + const r = validateConfig({ ...metaharness, adrConvention: { pad: -1, dir: 'docs/adrs' } }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toMatch(/adrConvention\.pad/); + }); + it('rejects a zero adrConvention.pad', () => { + const r = validateConfig({ ...metaharness, adrConvention: { pad: 0, dir: 'docs/adrs' } }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toMatch(/adrConvention\.pad/); + }); + it('rejects a non-integer adrConvention.pad', () => { + const r = validateConfig({ ...metaharness, adrConvention: { pad: 2.5, dir: 'docs/adrs' } }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toMatch(/adrConvention\.pad/); + }); + it('rejects an empty adrConvention.dir', () => { + const r = validateConfig({ ...metaharness, adrConvention: { pad: 4, dir: '' } }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toMatch(/adrConvention\.dir/); + }); + it('rejects a whitespace-only adrConvention.dir', () => { + const r = validateConfig({ ...metaharness, adrConvention: { pad: 4, dir: ' ' } }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toMatch(/adrConvention\.dir/); + }); + it('leaves the string-literal adrConvention forms unvalidated by this check', () => { + expect(validateConfig({ ...metaharness, adrConvention: '3-digit' }).ok).toBe(true); + expect(validateConfig({ ...metaharness, adrConvention: '4-digit' }).ok).toBe(true); + }); }); describe('compile', () => { @@ -56,6 +89,12 @@ describe('compile', () => { expect(() => compile({ ...metaharness, repo: '' })).toThrow(/invalid dream.config/); }); + it('throws instead of silently compiling a corrupted ADR path from a malformed adrConvention', () => { + expect(() => compile({ ...metaharness, adrConvention: { pad: -1, dir: '' } })).toThrow( + /adrConvention\.pad.*adrConvention\.dir|adrConvention\.dir.*adrConvention\.pad/s, + ); + }); + it('is deterministic (same input → identical output)', () => { expect(compile(metaharness)).toBe(prompt); });