diff --git a/.claude/skills/changelog/SKILL.md b/.claude/skills/changelog/SKILL.md new file mode 100644 index 000000000..908e326f1 --- /dev/null +++ b/.claude/skills/changelog/SKILL.md @@ -0,0 +1,78 @@ +--- +name: Write Changelog +description: Write a concise developer changelog focused on breaking changes and new features with before/after examples +--- + +## Write Changelog + +Write a changelog entry for the current branch or a specified set of commits. + +### Steps + +1. Determine scope: use `git log main..HEAD --oneline` (or the specified range) to identify commits. +2. For each changed file, use `detect_changes` to identify what shifted and its risk level. +3. Identify breaking changes: anything that removes or renames a public symbol, changes a function signature, alters config structure, or requires a migration step. +4. Identify new features: new public APIs, new CLI flags, new config fields, new behaviour behind existing APIs. +5. Skip internal refactors, test-only changes, and lint fixes unless they affect callers. +6. Write the file to `changelog/YYYY-MM-DD_.md` using today's date (check current date from the environment or `date +%F`). + +### Output Format + +``` +# + +## Summary + +One or two sentences. What changed and why. + +--- + +## Breaking change: + + + +| What | Before | After | +|------|--------|-------| +| ... | ... | ... | + +Before: +```go +// old usage +``` + +After: +```go +// new usage +``` + +--- + +## New: + + + +```go +// minimal usage example +``` + +--- + +## Bug fixes + +- ****: one-line description. + +--- + +## Recommended additions *(optional)* + +- Doc or test gaps worth filing. +``` + +### Rules + +- **Brief**: each section should fit on a screen. Cut prose, not examples. +- **Examples first**: before/after code blocks beat paragraph explanations. +- **Breaking changes go first**, even if there are many new features. +- **Omit sections** that have no content — don't write empty headers. +- File name slug is lowercase, words separated by underscores, ≤5 words (e.g. `2026-04-27_executor_jd_migration.md`). +- Use the before/after table for structural changes (config shape, struct fields, interface methods); use code blocks for API or invocation changes. diff --git a/.claude/skills/debug-issue/SKILL.md b/.claude/skills/debug-issue/SKILL.md new file mode 100644 index 000000000..ef1b38a2e --- /dev/null +++ b/.claude/skills/debug-issue/SKILL.md @@ -0,0 +1,27 @@ +--- +name: Debug Issue +description: Systematically debug issues using graph-powered code navigation +--- + +## Debug Issue + +Use the knowledge graph to systematically trace and debug issues. + +### Steps + +1. Use `semantic_search_nodes` to find code related to the issue. +2. Use `query_graph` with `callers_of` and `callees_of` to trace call chains. +3. Use `get_flow` to see full execution paths through suspected areas. +4. Run `detect_changes` to check if recent changes caused the issue. +5. Use `get_impact_radius` on suspected files to see what else is affected. + +### Tips + +- Check both callers and callees to understand the full context. +- Look at affected flows to find the entry point that triggers the bug. +- Recent changes are the most common source of new issues. + +## Token Efficiency Rules +- ALWAYS start with `get_minimal_context(task="")` before any other graph tool. +- Use `detail_level="minimal"` on all calls. Only escalate to "standard" when minimal is insufficient. +- Target: complete any review/debug/refactor task in ≤5 tool calls and ≤800 total output tokens. diff --git a/.claude/skills/integration-tests/SKILL.md b/.claude/skills/integration-tests/SKILL.md new file mode 100644 index 000000000..7c03b4ff6 --- /dev/null +++ b/.claude/skills/integration-tests/SKILL.md @@ -0,0 +1,106 @@ +--- +name: Integration Tests +description: Run local devenv integration/smoke tests, including Chainlink node (-cl) variants +--- + +## Integration Tests + +All commands run from `build/devenv` unless noted otherwise. + +### Test matrix (from CI workflows) + +**Standalone tests** (`test-smoke.yaml`): +- `TestE2ESmoke_Basic` — profile: `standard.profile` +- `TestE2ESmoke_Basic_OneExecPerChain` — profile: `standard.one-exec-per-chain.profile` +- `TestE2ESmoke_TokenVerification` — profile: `standard.profile` +- `TestE2ESmoke_ExtraArgsV2` — profile: `standard.profile` +- `TestE2ESmoke_ChainStatus` — profile: `standard.profile` +- `TestE2ESmoke_JobQueue` — profile: `standard.profile` +- `TestE2ESmoke_Replay` — profile: `standard.profile` +- `TestE2EReorg` — profile: `standard.src-auto-mine.profile` +- `TestChaos_AggregatorOutageRecovery` — profile: `standard.profile` +- Phased `TestE2ESmoke_Basic` — profile: `phased.profile` + +**Chainlink node tests** (`test-cl-smoke.yaml`): +- `TestE2ESmoke_Basic` — profile: `standard.clnode.profile` +- `TestE2ESmoke_Basic_OneExecPerChain` — profile: `standard.one-exec-per-chain.profile` (clnode variant) +- `TestE2ESmoke_TokenVerification` — profile: `standard.clnode.profile` +- `TestE2ESmoke_ExtraArgsV2` — profile: `standard.clnode.profile` +- `TestHA_CrossComponentDown` — profile: `standard.ha.clnode.profile` +- Phased `TestE2ESmoke_Basic` — profile: `phased.clnode.profile` + +When the user describes a CI failure, match it to the test name and profile above to know what to run locally. + +--- + +### The preferred way: `ccv test` + +`ccv test` handles build, env startup, and test execution in one command. Run from `build/devenv`. + +```bash +# Full cycle (build images, start env, run test, all output to log): +ccv test --profile standard.clnode.profile --pattern TestE2ESmoke_Basic --timeout 20m --log /tmp/test.log + +# Skip rebuild if images are already up-to-date: +ccv test --profile standard.profile --pattern TestE2ESmoke_Basic --build=false --log /tmp/test.log + +# Named suite aliases (no --pattern needed): +ccv test smoke --profile standard.profile --log /tmp/test.log +ccv test load --profile standard.profile --log /tmp/test.log + +# Against an already-running environment (no profile, no build): +ccv test --pattern TestE2ESmoke_Basic --build=false --log /tmp/test.log +``` + +**Key flags:** + +| Flag | Default | Notes | +|------|---------|-------| +| `--profile` | — | Profile file (`.profile`) that selects env config + type | +| `--pattern` | — | Raw Go test pattern; mutually exclusive with suite name | +| `--build` | `true` | Build Docker images first; pass `--build=false` to skip | +| `--timeout` | `0` (unlimited) | Go test timeout | +| `--log ` | — | Write ALL output to file; terminal shows only progress lines | + +The `--log` flag redirects at the OS fd level — it captures docker build output, env startup, CTF framework subprocess output, and go test output. Always use it when running from Claude Code to avoid token-heavy output. + +> ⚠️ Do NOT use CI-only profiles (e.g. `standard.clnode.ci.profile`) locally — they reference CI-specific image tags and paths that fail locally. Use the non-CI variants (`standard.clnode.profile`, `phased.clnode.profile`, etc.). + +--- + +### Manual steps (if not using `ccv test`) + +#### 1. Rebuild Docker images + +```bash +ccv down +just build-docker-dev +``` + +#### 2. For Chainlink node (-cl) tests only: update the chainlink repo + +The `-cl` environment loads the `chainlink` repo from the directory next to `chainlink-ccv`. It must reference the current commit of this repo. + +```bash +COMMIT=$(git -C /path/to/chainlink-ccv rev-parse HEAD) +cd /path/to/chainlink # must be a sibling of chainlink-ccv +go get github.com/smartcontractkit/chainlink-ccv@$COMMIT +gomods tidy +``` + +#### 3. Start the environment + +```bash +ccv up --profile standard.profile +ccv up --profile standard.clnode.profile # CL node variant +ccv up --profile phased.profile # phased runtime +``` + +Only one environment at a time — overlapping startups collide on Docker resources. + +#### 4. Run the tests + +```bash +cd build/devenv/tests/e2e +go test -v -timeout 15m -count=1 -run TestE2ESmoke_Basic +``` diff --git a/.kiro/steering/code-review-graph.md b/.kiro/steering/code-review-graph.md new file mode 100644 index 000000000..0a3e5c8ec --- /dev/null +++ b/.kiro/steering/code-review-graph.md @@ -0,0 +1,38 @@ + +## MCP Tools: code-review-graph + +**IMPORTANT: This project has a knowledge graph. ALWAYS use the +code-review-graph MCP tools BEFORE using Grep/Glob/Read to explore +the codebase.** The graph is faster, cheaper (fewer tokens), and gives +you structural context (callers, dependents, test coverage) that file +scanning cannot. + +### When to use graph tools FIRST + +- **Exploring code**: `semantic_search_nodes` or `query_graph` instead of Grep +- **Understanding impact**: `get_impact_radius` instead of manually tracing imports +- **Code review**: `detect_changes` + `get_review_context` instead of reading entire files +- **Finding relationships**: `query_graph` with callers_of/callees_of/imports_of/tests_for +- **Architecture questions**: `get_architecture_overview` + `list_communities` + +Fall back to Grep/Glob/Read **only** when the graph doesn't cover what you need. + +### Key Tools + +| Tool | Use when | +|------|----------| +| `detect_changes` | Reviewing code changes — gives risk-scored analysis | +| `get_review_context` | Need source snippets for review — token-efficient | +| `get_impact_radius` | Understanding blast radius of a change | +| `get_affected_flows` | Finding which execution paths are impacted | +| `query_graph` | Tracing callers, callees, imports, tests, dependencies | +| `semantic_search_nodes` | Finding functions/classes by name or keyword | +| `get_architecture_overview` | Understanding high-level codebase structure | +| `refactor_tool` | Planning renames, finding dead code | + +### Workflow + +1. The graph auto-updates on file changes (via hooks). +2. Use `detect_changes` for code review. +3. Use `get_affected_flows` to understand impact. +4. Use `query_graph` pattern="tests_for" to check coverage. diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 000000000..905d84484 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,11 @@ +{ + "mcpServers": { + "code-review-graph": { + "command": "code-review-graph", + "args": [ + "serve" + ], + "type": "stdio" + } + } +} diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..01247865c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,232 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Primary Objective + +The main focus of work in this repo is **multi-service refactoring** guided by **enforcing layer boundaries**. When suggesting or making changes: + +- Prefer solutions that tighten the dependency graph rather than loosen it. The `depguard` rules in `.golangci.yaml` encode the intended layering — treat violations as design signals, not just lint errors. +- `protocol/` is the foundation. Types and abstractions that belong to the protocol layer should live there, not leak into service-specific packages. +- `verifier/` and `executor/` should depend on `protocol/` and `common/` abstractions, not on each other or on `chainlink/v2`. +- When refactoring spans multiple services, look for the right shared abstraction in `common/` or `pkg/` before duplicating logic. +- Prefer changing interfaces and boundaries over working around them with wrappers or adapters. + +## Build Commands + +This project uses [`just`](https://github.com/casey/just) as a command runner. Run `just --list` to see all targets. + +```bash +just install-go-tools # Install golangci-lint, mockery, protoc-gen-go, buf, etc. +just build # Build all services (also available per-service: cd aggregator && just build) +just generate # Run go generate + mockery to regenerate mocks +just mock # Regenerate mocks only (uses .mockery.yaml) +just fmt # Format all Go files via golangci-lint +just tidy # Tidy go.mod across all modules +``` + +Individual service builds (each service directory has its own Justfile): +```bash +cd aggregator && just build # Production image: aggregator:latest +cd aggregator && just build-dev # Dev image with hot-reload: aggregator:dev +``` + +## Test Commands + +```bash +just test # All tests: -race -shuffle on -fullpath -v +just test short="-short" # Unit tests only (skip integration/slow tests) +just test-coverage # All tests with coverage → coverage.out +just test-coverage short="-short" coverage_file="cov.out" +``` + +To run a single test: +```bash +go test -v -run TestMyTestName ./path/to/package/... +``` + +Dev environment tests (requires devenv to be running): +```bash +cd build/devenv/tests/services && go test -v -run TestService +cd build/devenv/tests/e2e && go test -v -run TestE2ESmoke +``` + +## Lint + +```bash +just lint # Run golangci-lint (v2.6.0, configured in .golangci.yaml) +just lint fix # Run with --fix to auto-correct issues +just shellcheck # Lint all .sh files +``` + +Key `depguard` rules enforced by the linter: +- `protocol/` may only import stdlib + `chainlink-ccv/protocol` (no `chainlink/v2`) +- `executor/` and `verifier/` packages may not import `chainlink/v2` +- Test files must use `stretchr/testify`, not `test-go/testify` + +## Code Generation + +Protocol buffers: `just generate` (uses `buf`) +Mocks: `just mock` (uses mockery v2, config in `.mockery.yaml`) +OpenAPI client for indexer: generated via `oapi-codegen` + +## Architecture + +Chainlink CCV is a **modular cross-chain verification and execution system**. Messages created on a source chain are verified offchain, indexed, and then executed on a destination chain. + +### Data Flow + +``` +Source Chain → OnRamp → [Verifiers] → [Indexer] → [Executor] → OffRamp → Destination Chain +``` + +### Core Services + +**`protocol/`** — Pure Go protocol abstractions with no `chainlink/v2` dependency. Contains message types, hashing, signature verification, finality, and chain status primitives. This is the canonical definition of protocol types. + +**`verifier/`** — Reads messages from source chains and produces verification results. Multiple implementations: +- Committee (DON-based, default): aggregates from multiple nodes +- Token/USDC: wraps CCTP token transfers +- ZK, Custom: pluggable via build arg `VERIFIER_TYPE=token` + +**`aggregator/`** — Aggregates committee verifier results from multiple nodes into a single result. Uses PostgreSQL for storage. + +**`indexer/`** — Central query API (REST/OpenAPI) over PostgreSQL. Executors use it to fetch verification results. Exposes `IndexerAPI`; client in `pkg/indexerclient/`. + +**`executor/`** — Fetches verified messages from the indexer and executes them on destination chains. Includes leader election (`pkg/leaderelector/`) and a priority message heap (`pkg/message_heap/`). + +**`bootstrap/`** — Shared startup logic used by all services: keystore lifecycle, Job Distributor (JD) integration, DB setup, and health-check HTTP server. + +**`pricer/`** — Standalone service that updates gas/token prices on-chain. Example of a product-specific binary (PSB). + +### Supporting Packages + +- `common/` — Shared utilities: JD client, batching, metrics interfaces, committee logic +- `pkg/chainaccess/` — Source/destination chain readers and contract transmitters +- `pkg/indexerclient/` — Generated client for the indexer REST API +- `cli/` — CLI tools for chain status and job queue queries +- `internal/mocks/` — Mockery-generated mocks (do not edit manually) + +### Dev Environment (`build/devenv/`) + +Full local environment managed by the `ccv` CLI: +```bash +cd build/devenv && just cli # Build the ccv CLI binary +./ccv --help # Manage the full local stack +``` + +The devenv spins up: 2 Anvil chains, 4 Chainlink nodes, Job Distributor, all services, and an observability stack (Prometheus, Loki, Grafana). + +- `environment.go` — Full environment setup (~2000 LOC); start here to understand how services are wired together +- `config.go` / `env.toml` — Environment configuration +- `build/devenv/services/` — Per-service config templates + +#### Phased Component Runtime (`build/devenv/runtime/`) + +The devenv has a component-based startup path (`--env-mode phased`) that replaces the monolithic `NewEnvironment`. It uses a 4-phase execution model with write-once output keys. + +**Config**: phased mode uses the dedicated `build/devenv/env-phased.toml` (run `--env-mode phased up env-phased.toml`), a standalone copy of `env.toml` that omits the monolith-only keys (`cl_nodes_funding_eth`, `cl_nodes_funding_link`, `high_availability`, `[cldf]`). Monolith mode keeps using `env.toml` (+ `env-cl.toml`). Keep the shared sections of the two files in sync until they are formally split. + +**Entry point**: `NewEnvironmentPhased` in `build/devenv/environment_phased.go` — calls `devenvruntime.NewEnvironment` with the component registry. + +**Runtime package** (`build/devenv/runtime/`): +- `environment.go` — Orchestrates phases 1–4. Within a phase, every component receives `maps.Clone(phaseSnapshot)` of outputs at phase-start (siblings cannot see each other). `mergeNoOverwrite` enforces write-once keys across all phases. +- `component.go` — `Phase1Component` through `Phase4Component` interfaces. Each `RunPhaseN` returns `(map[string]any, []Effect, error)`. +- `effects.go` — `Effect` interface + `FundingEffect`, `JobProposalEffect`, `CLNodeConfigEffect`. +- `registry.go` — `Register(key, factory)` for specific components. + +**Critical constraint**: ALL Phase N components receive the same phase-start snapshot clone. A Phase 2 component **cannot** see another Phase 2 component's outputs — only outputs from phases 1 through N-1. + +**Blank component imports**: All `_ "...components/..."` blank imports that trigger component `init()` registration belong in `build/devenv/environment.go`, not in any other file. This is because `environment.go` is the devenv entry point; placing blank imports elsewhere either causes linter errors (if the package has no other use) or creates circular imports. + +**Components** (`build/devenv/components/`): +- `blockchains/` — Phase 1: deploys Anvil/Geth chains; publishes `"blockchains"`, `"blockchainOutputs"` +- `fake/` — Phase 1: fake services +- `jd/` — Phase 1: starts JD container; publishes `"jd"` +- `protocol_contracts/` — Phase 2: deploys contracts, configures lanes; publishes `"_env"`, `"_topology"`, `"_ds"`, `"_selectors"`, `"_impls"`, `"_cldf"`, `"_time_track"` +- `committeeccv/` — Phase 3: generates HMAC creds, launches verifiers and aggregators, configures lanes, generates committee config; publishes `"aggregators"`, `"verifiers"`, `"shared_tls_certs"` +- `executor/` — Phase 3: launches executor containers, registers with JD, generates job specs; emits `FundingEffect` + `JobProposalEffect`; publishes `"executor"` +- `pricer/` — Phase 3: launches pricer service; emits `FundingEffect` +- `indexer/` — Phase 4: launches indexer containers; reads `"aggregators"` from Phase 3 +- `tokenverifier/` — Phase 4: launches token verifier containers + +**Effect executor** (`build/devenv/effect_executor.go`): Runs after each phase. Executes `CLNodeConfigEffect` → `FundingEffect` → `JobProposalEffect` in fixed order. + +**Current phase output map**: + +| Phase | Component | Key | Type | +|-------|-----------|-----|------| +| 1 | blockchains | `"blockchains"` | `[]*blockchain.Input` | +| 1 | blockchains | `"blockchainOutputs"` | `[]*blockchain.Output` | +| 1 | jd | `"jd"` | `*jobs.JDInfrastructure` | +| 2 | protocol_contracts | `"_env"` | `*deployment.Environment` | +| 2 | protocol_contracts | `"_topology"` | `*ccvdeployment.EnvironmentTopology` | +| 2 | protocol_contracts | `"_ds"` | `datastore.MutableDataStore` | +| 3 | committeeccv | `"aggregators"` | `[]*services.AggregatorInput` | +| 3 | committeeccv | `"shared_tls_certs"` | `*services.TLSCertPaths` | +| 3 | executor | `"executor"` | `[]*executorsvc.Input` | + +**Remaining extraction order** (tracked in task list): +1. Extract `protocol_contracts` as Phase 3 component (plan finalized) +2. Extract `CommitteeCCV` as full Phase 3 component — bundles aggregators + CL nodes + verifiers; CL nodes are internal, not a separate component +3. Misc cleanups: `runPhase` helper to DRY phases 2–4, `unclaimedKeys` deduplication, generic `decode` helper, `slices.Sorted(maps.Keys(...))` replacement + +### Key Conventions + +- All services are bootstrapped via the `bootstrap/` package; new services should follow the same pattern. +- Configuration is TOML-based; see `config.example.toml` in each service directory. +- Database migrations use `goose`; migration files live in each service's `migrations/` directory. +- Each service has its own `go.mod` (multi-module repo); `gomods` is used to run commands across all modules. + +## Agent skills + +### Issue tracker + +Issues and PRDs are tracked as local markdown files under `.scratch//`, with triage state in a `Status:` line. See `docs/agents/issue-tracker.md`. + +### Triage labels + +Uses the five canonical triage roles (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`) — default strings, no overrides. See `docs/agents/triage-labels.md`. + +### Domain docs + +Single-context layout: `CONTEXT.md` + `docs/adr/` at repo root. See `docs/agents/domain.md`. + + +## MCP Tools: code-review-graph + +**IMPORTANT: This project has a knowledge graph. ALWAYS use the +code-review-graph MCP tools BEFORE using Grep/Glob/Read to explore +the codebase.** The graph is faster, cheaper (fewer tokens), and gives +you structural context (callers, dependents, test coverage) that file +scanning cannot. + +### When to use graph tools FIRST + +- **Exploring code**: `semantic_search_nodes` or `query_graph` instead of Grep +- **Understanding impact**: `get_impact_radius` instead of manually tracing imports +- **Code review**: `detect_changes` + `get_review_context` instead of reading entire files +- **Finding relationships**: `query_graph` with callers_of/callees_of/imports_of/tests_for +- **Architecture questions**: `get_architecture_overview` + `list_communities` + +Fall back to Grep/Glob/Read **only** when the graph doesn't cover what you need. + +### Key Tools + +| Tool | Use when | +|------|----------| +| `detect_changes` | Reviewing code changes — gives risk-scored analysis | +| `get_review_context` | Need source snippets for review — token-efficient | +| `get_impact_radius` | Understanding blast radius of a change | +| `get_affected_flows` | Finding which execution paths are impacted | +| `query_graph` | Tracing callers, callees, imports, tests, dependencies | +| `semantic_search_nodes` | Finding functions/classes by name or keyword | +| `get_architecture_overview` | Understanding high-level codebase structure | +| `refactor_tool` | Planning renames, finding dead code | + +### Workflow + +1. The graph auto-updates on file changes (via hooks). +2. Use `detect_changes` for code review. +3. Use `get_affected_flows` to understand impact. +4. Use `query_graph` pattern="tests_for" to check coverage. diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 000000000..29fd1f4c5 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,100 @@ +# Domain Context: Chainlink CCV + +## Overview + +**Chainlink CCV** (Cross-Chain Verification) is a modular cross-chain verification and execution system. Messages created on a source chain are verified offchain, indexed, and then executed on a destination chain. + +## Primary Design Principle + +**Enforce layer boundaries** through dependency isolation. The `depguard` rules in `.golangci.yaml` encode the intended layering — treat violations as design signals, not just lint errors. + +### Layer Model + +``` +protocol/ ← foundation (pure Go, no chainlink/v2) + ↑ + ├─ verifier/, executor/, aggregator/ (can depend on protocol/ + common/) + ├─ common/ (shared utilities, abstractions) + ├─ pkg/ (cross-service utilities) + └─ bootstrap/ (startup logic) +``` + +**Key rules:** +- `protocol/` may only import stdlib + `chainlink-ccv/protocol` (no `chainlink/v2`) +- `executor/` and `verifier/` packages may not import `chainlink/v2` +- Prefer changing interfaces and boundaries over working around them with wrappers or adapters + +## Data Flow + +``` +Source Chain → OnRamp → [Verifiers] → [Indexer] → [Executor] → OffRamp → Destination Chain +``` + +## Core Services + +### `protocol/` +Pure Go protocol abstractions. Contains message types, hashing, signature verification, finality, and chain status primitives. This is the canonical definition of protocol types — types that belong here should not leak into service-specific packages. + +### `verifier/` +Reads messages from source chains and produces verification results. Multiple pluggable implementations: +- **Committee** (DON-based, default): aggregates from multiple nodes +- **Token/USDC**: wraps CCTP token transfers +- **ZK, Custom**: pluggable via build arg `VERIFIER_TYPE=token` + +### `aggregator/` +Aggregates committee verifier results from multiple nodes into a single result. Uses PostgreSQL for storage. + +### `indexer/` +Central query API (REST/OpenAPI) over PostgreSQL. Executors use it to fetch verification results. Exposes `IndexerAPI`; client in `pkg/indexerclient/`. + +### `executor/` +Fetches verified messages from the indexer and executes them on destination chains. Includes leader election (`pkg/leaderelector/`) and a priority message heap (`pkg/message_heap/`). + +### `bootstrap/` +Shared startup logic used by all services: keystore lifecycle, Job Distributor (JD) integration, DB setup, and health-check HTTP server. All services should follow the bootstrap pattern for initialization. + +### `pricer/` +Standalone service that updates gas/token prices on-chain. Example of a product-specific binary (PSB). + +## Supporting Packages + +- `common/` — Shared utilities: JD client, batching, metrics interfaces, committee logic +- `pkg/chainaccess/` — Source/destination chain readers and contract transmitters +- `pkg/indexerclient/` — Generated client for the indexer REST API +- `cli/` — CLI tools for chain status and job queue queries +- `internal/mocks/` — Mockery-generated mocks (do not edit manually) + +## Dev Environment + +### Profile + +A TOML file that encodes a complete, valid devenv configuration: the runtime mode (`legacy` or `phased`), an ordered list of config files, an optional output file path, and an optional description. Profiles are the canonical way to invoke `ccv up`; ad-hoc comma-separated config lists are the lower-level primitive profiles build on. + +### Phased Component Runtime + +The devenv (`build/devenv/`) uses a 4-phase execution model with write-once output keys: + +**Phase 1**: Blockchains, Job Distributor +**Phase 2**: Protocol contracts, lanes +**Phase 3**: Committee CCV (verifiers, aggregators), executor, pricer +**Phase 4**: Indexer, token verifier + +**Critical constraint**: Within a phase, every component receives the same phase-start snapshot. A Phase N component **cannot** see another Phase N component's outputs — only outputs from phases 1 through N-1. + +**Blank component imports**: All `_ "...components/..."` blank imports that trigger component `init()` registration belong in `build/devenv/environment.go`, not elsewhere. + +## Key Conventions + +- All services are bootstrapped via the `bootstrap/` package +- Configuration is TOML-based; see `config.example.toml` in each service directory +- Database migrations use `goose`; migration files live in each service's `migrations/` directory +- Multi-module repo: each service has its own `go.mod`; use `gomods` to run commands across all modules +- Code generation: Protocol buffers via `buf`, mocks via `mockery` v2 + +## Refactoring Guidance + +When suggesting changes: +1. **Prefer solutions that tighten the dependency graph** rather than loosen it +2. **Look for the right shared abstraction** in `common/` or `pkg/` before duplicating logic across services +3. **Enforce boundaries first, then optimize** — never work around layer violations with adapters +4. **Depguard violations are design signals** — they indicate where boundaries need refinement, not quick workarounds diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 000000000..4fd5fb6ea --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1 @@ +# Architecture Decision Records diff --git a/docs/agents/domain.md b/docs/agents/domain.md new file mode 100644 index 000000000..c97d6a6db --- /dev/null +++ b/docs/agents/domain.md @@ -0,0 +1,51 @@ +# Domain Docs + +How the engineering skills should consume this repo's domain documentation when exploring the codebase. + +## Before exploring, read these + +- **`CONTEXT.md`** at the repo root, or +- **`CONTEXT-MAP.md`** at the repo root if it exists — it points at one `CONTEXT.md` per context. Read each one relevant to the topic. +- **`docs/adr/`** — read ADRs that touch the area you're about to work in. In multi-context repos, also check `src//docs/adr/` for context-scoped decisions. + +If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The producer skill (`/grill-with-docs`) creates them lazily when terms or decisions actually get resolved. + +## File structure + +Single-context repo (most repos): + +``` +/ +├── CONTEXT.md +├── docs/adr/ +│ ├── 0001-event-sourced-orders.md +│ └── 0002-postgres-for-write-model.md +└── src/ +``` + +Multi-context repo (presence of `CONTEXT-MAP.md` at the root): + +``` +/ +├── CONTEXT-MAP.md +├── docs/adr/ ← system-wide decisions +└── src/ + ├── ordering/ + │ ├── CONTEXT.md + │ └── docs/adr/ ← context-specific decisions + └── billing/ + ├── CONTEXT.md + └── docs/adr/ +``` + +## Use the glossary's vocabulary + +When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids. + +If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/grill-with-docs`). + +## Flag ADR conflicts + +If your output contradicts an existing ADR, surface it explicitly rather than silently overriding: + +> _Contradicts ADR-0007 (event-sourced orders) — but worth reopening because…_ diff --git a/docs/agents/issue-tracker.md b/docs/agents/issue-tracker.md new file mode 100644 index 000000000..a2f08fb07 --- /dev/null +++ b/docs/agents/issue-tracker.md @@ -0,0 +1,19 @@ +# Issue tracker: Local Markdown + +Issues and PRDs for this repo live as markdown files in `.scratch/`. + +## Conventions + +- One feature per directory: `.scratch//` +- The PRD is `.scratch//PRD.md` +- Implementation issues are `.scratch//issues/-.md`, numbered from `01` +- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings) +- Comments and conversation history append to the bottom of the file under a `## Comments` heading + +## When a skill says "publish to the issue tracker" + +Create a new file under `.scratch//` (creating the directory if needed). + +## When a skill says "fetch the relevant ticket" + +Read the file at the referenced path. The user will normally pass the path or the issue number directly. diff --git a/docs/agents/triage-labels.md b/docs/agents/triage-labels.md new file mode 100644 index 000000000..b716855d4 --- /dev/null +++ b/docs/agents/triage-labels.md @@ -0,0 +1,15 @@ +# Triage Labels + +The skills speak in terms of five canonical triage roles. This file maps those roles to the actual label strings used in this repo's issue tracker. + +| Label in mattpocock/skills | Label in our tracker | Meaning | +| -------------------------- | -------------------- | ---------------------------------------- | +| `needs-triage` | `needs-triage` | Maintainer needs to evaluate this issue | +| `needs-info` | `needs-info` | Waiting on reporter for more information | +| `ready-for-agent` | `ready-for-agent` | Fully specified, ready for an AFK agent | +| `ready-for-human` | `ready-for-human` | Requires human implementation | +| `wontfix` | `wontfix` | Will not be actioned | + +When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table. + +Edit the right-hand column to match whatever vocabulary you actually use.