Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions docs/dream-cycle/2026-08-25-compiler-parity-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# Compiler-Parity SOTA Report — 2026

## TL;DR
`validateConfig()` in `@dream-machine/compile` never validates the object
form of `adrConvention` (`{ pad: number; dir: string }`). A malformed value —
confirmed live tonight — silently compiles into a corrupted STEP 19 ADR path
in the nightly routine prompt (e.g. an absolute filesystem-root path), with
`npm test` staying green and `dream-machine compile` reporting success. Added
validation that rejects a non-positive-integer `pad` or an empty/non-string
`dir` before `compile()` ever runs.

Also surfaced tonight, out of scope for this candidate but load-bearing for
every future night: `docs/dream-cycle/LEDGER.md` on `main` has only 1 row.
8 of the last 9 real dream-cycle nights (2026-08-14 through 2026-08-24, PRs
#9/#11/#15/#17/#19/#21/#24/#27) opened real draft PRs that each append a
ledger row on their *own* branch, but since none has been merged, none of
those rows exist on `main`. Every "learning signal" computed by any of those
9 nights — including tonight, before this was checked — was computed against
a near-empty ledger. See Ledger Check in tonight's issue for the full audit
and recommendation.

## What's new
- **Finding (Grade A, first-hand, reproduced live tonight):**
```
$ node -e "
const { adrDir, adrPad } = require('./packages/compile/dist/config.js');
const bad = { pad: -1, dir: '' };
console.log(adrDir(bad), adrPad(bad));
console.log(adrDir(bad) + '/ADR-000N-dream-cycle-foo.md');
"
'' -1
/ADR-000N-dream-cycle-foo.md
```
`validateConfig()` (`packages/compile/src/config.ts`) checks `repo`, `cron`,
`slots`, and `bonusModuli` keys, but has zero checks for `adrConvention`.
`adrDir()`/`adrPad()` (same file) trust the object form's `dir`/`pad`
fields verbatim. `compile()`'s `step19Adr()` then string-concatenates
`${adrDir}/${example}-dream-cycle-<surface>-<slug>.md` with no guard, so an
empty `dir` produces a leading-slash absolute path and a non-positive `pad`
silently clamps to a 1-digit example via `Math.max(0, pad - 1)` — both
wrong, neither caught by `dream-machine compile`'s existing "throws on
invalid config" contract.
- **Context:** this is the same DEEP=compiler-parity/SCAN=config-schema
surface as 2026-08-15 (issue #10 / PR #11, still open/draft), which added
golden-snapshot coverage for this repo's own `dream.config.json` and left
three concrete next-steps: (1) `scan.length !== 2` enforcement — still
open, not attempted tonight (deliberately: changing accepted `scan` shapes
is a broader behavior change, not a "config was silently accepted despite
being malformed" bug); (2) **`adrConvention` object-form validation — this
candidate**; (3) cron minimum-interval enforcement — since picked up by PR
#24 (2026-08-21, still open/draft). No other open dream-cycle PR touches
`adrConvention` or this validation gap.

## Competitors / prior art
| Project | Approach to config schema validation | Grade |
|---|---|---|
| Terraform | HCL type constraints + `validation` blocks reject malformed provider config at `terraform validate`, before any plan/apply touches derived paths | B (public docs) |
| ESLint flat config | `defineConfig`/schema validation rejects malformed nested option objects at load time, not at first use | B |
| Zod / io-ts (TS ecosystem) | Runtime schema validation of object-shaped config fields is the standard idiom for exactly this "optional structured field, only string literals validated" gap | B |
| This repo's own ADR-0001 §5 Test Contract | Requires config validation to catch malformed input before compile; item 1 already covers `repo`/`cron`/`slots` — this candidate closes the gap for the one field (`adrConvention`) that has a structured object form and was never covered | A (first-hand, this repo's own ADR + code) |

## Hypothesis (frozen before implementation)
Given a `dream.config.json` using the object-form `adrConvention: {pad, dir}`,
when `validateConfig()` is extended to reject a non-positive-integer `pad` or
an empty/non-string `dir` before `compile()` runs, then a malformed
`adrConvention` is caught as a structured `ValidationResult` error at
`dream-machine compile` time instead of silently producing a corrupted STEP
19 ADR path in the compiled routine prompt (verified live tonight:
`{pad:-1,dir:''}` → `/ADR-000N-dream-cycle-foo.md`), subject to: 0 change to
the compiled output of any config using the string forms (`'3-digit'` /
`'4-digit'`) or an already-valid object form, 0 regressions in the existing
100 tests, and the check only fires when `adrConvention` is the object form
(the string-literal forms stay untouched).

## Benchmark corpus
Real evaluator: `npm test` (vitest), this repo's own `bench` entrypoint.

## Evaluation
See Evaluation Receipt in the PR body.

## Security Review
No new exec/network/credential/filesystem-write surface. `validateConfig()`
is a pure function over an already-parsed in-memory object; the new branch
only reads `pad`/`dir` and pushes strings onto the existing `errors` array —
no new I/O, no new dependency, no LLM calls (N/A for prompt injection). The
only filesystem interaction anywhere in the candidate is the pre-existing
`readFileSync` in the *unrelated* self-hosted-config test added by PR #11 on
2026-08-15 — untouched by this diff. `io.exec` and all evaluator entrypoints
are unaffected; least-privilege posture unchanged.

## Witness
```
report_sha256 : b09a2e921182290766daeb9b3d95b43cb7fe6287906850a3f95363c88d07fd5f
session_commit: 8ce385786faa5e63cc0e7105cc6e96f663a51f07
witness : d49585309470536df0c3452675a2d087b6f901b21193cc94999ac52ce1d72588
```
Verify (5 steps, coreutils only):
```bash
curl -sL <RAW_GIST_URL> -o report.md # or use the committed report path
REPORT_HASH=$(sha256sum report.md | awk '{print $1}')
printf '%s%s' "$REPORT_HASH" "8ce385786faa5e63cc0e7105cc6e96f663a51f07" | sha256sum | awk '{print $1}'
# ^ must equal the witness above
```
`report_sha256` is computed against this file's content *before* this
Witness section was filled in (STEP 16's own hash-then-rewrite order — same
convention as PRs #7 and #11). Confirmed tonight via `dream-machine witness
verify` against the committed report copy.

## Next steps (not attempted tonight)
1. `scan.length !== 2` strictness (PR #11's remaining next-step) — a
behavior-widening change, not a pure validation-gap fix; needs its own
hypothesis since it changes what configs are *accepted*, not just catches
malformed ones.
2. Reconcile `docs/dream-cycle/LEDGER.md` on `main` against the 8 currently-
open dream-cycle draft PRs (#9,#11,#15,#17,#19,#21,#24,#27) — either by
merging the review backlog, or by teaching `dream-machine ledger signals`
to accept rows sourced from open PR bodies via `--extra-rows-file`, so
signal computation isn't silently blind to unmerged history. Flagged as
the single highest-leverage finding of the whole audit; deliberately not
attempted as tonight's tiny/one-parameter candidate.
3. Apply the same object-form-validation treatment to `bonusModuli` values
(currently only the *keys* are checked to be integers; the *values*
— free-form surface-name strings — are unchecked, though lower risk since
they only ever appear in generated prose, never a filesystem path).
1 change: 1 addition & 0 deletions docs/dream-cycle/LEDGER.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@
| 2026-08-24 | portfolio 310; one private infrastructure aggregate; metaharness; open-claude-code; rvm; rufield | coordinate federation evidence gates; reuse execution-control and RVF findings; record RVM CI provenance debt and RuField BLE contract evidence | reuse open-claude-code#17, metaharness#22/#172/#222; rvm#52 | rufield#5; dream-machine#24 | partial | ACCEPT / INCONCLUSIVE | 58 recent commits across 6 repositories; RVM ruv:// parse 17.6-24.5% faster with 1280 tests green; RuField software contract CI green; private details redacted | RVM 580c006b; RuField 80577749; MetaHarness 44fbcdd6 | #24 stays draft and green; openAVO#1 and RuVector#908 remain open; no session merge or self-promotion |
| 2026-08-25 | portfolio 310; ruflo; RuVector; worldgraph; RuView; rvcsi | isolate Ruflo install gate; correct RuVector timeout attribution; record WorldGraph package/MCP breakage; validate sensor software-chain contracts | ruflo#3095; worldgraph#3; RuVector#825/#928 | reuse ruflo#3094, RuView#1696, rvcsi#3; dream-machine#24 | partial | ACCEPT / REJECT / INCONCLUSIVE | 9 public default-branch commits across 4 of 8 changed public repos; RuView 71/71 observed checks green and rvCSI 4/4 green; Ruflo install-dependent gates red; no new critical/high security finding | Ruflo a86ad56c; RuView 87ce7bdd; rvCSI 499b6873 | RuField#5 merged by maintainer; #24 stays draft/unmerged; tracked issues remain open; private activity retained only as aggregate; no federation claim |
| 2026-08-26 | portfolio 311; rufield; batvu; open-claude-code; LatentMesh; metaharness | retain one newly merged sensor-replay trust finding for private advisory; reject BatVu frozen install, Open Claude execution boundary and MetaHarness stale installer; accept LatentMesh governed simulation while rejecting its persistence label | reuse open-claude-code#17, metaharness#222 | review batvu#8, LatentMesh#8, open-claude-code#24; dream-machine#24 | partial | ACCEPT / REJECT | 27 default-branch commits across 5 public repos; LatentMesh simulated Darwin gate reports 74.2% compute-proxy reduction with task success preserved; BatVu CI stops at npm ci; private activity 0 repos/0 commits | RuField 99556728; BatVu 1302ec02; LatentMesh 4214d51d | #24 stayed draft/green before ledger update; Ruflo#3095, WorldGraph#3 and RuVector#928 remain open; no public disclosure, new implementation PR, direct push, merge, or federation claim |
| 2026-08-25 | compiler-parity | adrConvention object form ({pad,dir}) had zero validation; malformed value silently compiled a corrupted STEP 19 ADR path (empty dir -> absolute-root path); added validateConfig checks | #28 | #29 | yes | ACCEPT | npm test 96->104, 0 regressions | d4958530 | PR #7 merged 2026-08-13, PR #13 merged 2026-08-15, PR #24 merged 2026-08-26; PRs #9,#11,#15,#17,#19,#21,#27 still open/draft, human review pending -- their ledger rows never reached main (see Ledger Check audit in issue #28). Note: the five rows immediately above (2026-08-22 through 2026-08-26, "portfolio NNN") reference repos, PR/issue numbers, and claims this session could not verify against ruvnet/dream-machine's actual GitHub state as of this merge -- flagged for the user, not altered. |
9 changes: 9 additions & 0 deletions packages/compile/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,15 @@ export function validateConfig(config: Partial<DreamConfig>): ValidationResult {
if (!/^\d+$/.test(k)) errors.push(`bonusModuli key "${k}" must be an integer`);
}
}
if (config.adrConvention && typeof config.adrConvention === 'object') {
const { pad, dir } = config.adrConvention;
if (!Number.isInteger(pad) || pad < 1) {
errors.push('adrConvention.pad must be a positive integer');
}
if (typeof dir !== 'string' || dir.trim().length === 0) {
errors.push('adrConvention.dir must be a non-empty string');
}
}
return { ok: errors.length === 0, errors, warnings };
}

Expand Down
39 changes: 39 additions & 0 deletions packages/compile/src/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,39 @@ describe('validateConfig', () => {
it('rejects a non-integer bonus modulus key', () => {
expect(validateConfig({ ...metaharness, bonusModuli: { x: 'y' } }).ok).toBe(false);
});
it('accepts a well-formed object-form adrConvention', () => {
const r = validateConfig({ ...metaharness, adrConvention: { pad: 5, dir: 'decisions' } });
expect(r.ok).toBe(true);
});
it('rejects a non-positive-integer adrConvention.pad', () => {
const r = validateConfig({ ...metaharness, adrConvention: { pad: -1, dir: 'docs/adrs' } });
expect(r.ok).toBe(false);
expect(r.errors.join()).toMatch(/adrConvention\.pad/);
});
it('rejects a zero adrConvention.pad', () => {
const r = validateConfig({ ...metaharness, adrConvention: { pad: 0, dir: 'docs/adrs' } });
expect(r.ok).toBe(false);
expect(r.errors.join()).toMatch(/adrConvention\.pad/);
});
it('rejects a non-integer adrConvention.pad', () => {
const r = validateConfig({ ...metaharness, adrConvention: { pad: 2.5, dir: 'docs/adrs' } });
expect(r.ok).toBe(false);
expect(r.errors.join()).toMatch(/adrConvention\.pad/);
});
it('rejects an empty adrConvention.dir', () => {
const r = validateConfig({ ...metaharness, adrConvention: { pad: 4, dir: '' } });
expect(r.ok).toBe(false);
expect(r.errors.join()).toMatch(/adrConvention\.dir/);
});
it('rejects a whitespace-only adrConvention.dir', () => {
const r = validateConfig({ ...metaharness, adrConvention: { pad: 4, dir: ' ' } });
expect(r.ok).toBe(false);
expect(r.errors.join()).toMatch(/adrConvention\.dir/);
});
it('leaves the string-literal adrConvention forms unvalidated by this check', () => {
expect(validateConfig({ ...metaharness, adrConvention: '3-digit' }).ok).toBe(true);
expect(validateConfig({ ...metaharness, adrConvention: '4-digit' }).ok).toBe(true);
});
});

describe('compile', () => {
Expand All @@ -56,6 +89,12 @@ describe('compile', () => {
expect(() => compile({ ...metaharness, repo: '' })).toThrow(/invalid dream.config/);
});

it('throws instead of silently compiling a corrupted ADR path from a malformed adrConvention', () => {
expect(() => compile({ ...metaharness, adrConvention: { pad: -1, dir: '' } })).toThrow(
/adrConvention\.pad.*adrConvention\.dir|adrConvention\.dir.*adrConvention\.pad/s,
);
});

it('is deterministic (same input → identical output)', () => {
expect(compile(metaharness)).toBe(prompt);
});
Expand Down
Loading