gather: deterministic context-pack tool (aft_gather)

[iceteaSA]

gather: deterministic context-pack tool (aft_gather)

What

One new tool — aft_gather — assembles a bounded "context pack" (ranked, deduped, budgeted verbatim code evidence) in a single call. It replaces the multi-turn search → outline → zoom → callgraph read chain an agent otherwise runs to build context around a question or a symbol.

Two modes (mutually exclusive):

• question: "how does X work?" — seeds from handle_semantic_search (same pipeline as aft_search, all lanes/fallbacks)
• symbol + filePath — seeds from the callgraph (impact depth-1 callers + call_tree depth-1 callees)

Seeds expand one hop through the callgraph, dedupe by canonicalized (file, symbol) with seeds winning, and render via render_symbol_within_budget until a hard line budget (default 400, cap 800) is spent. Everything past the cut appears as one-line stubs under ## Beyond budget (zoom to expand) — nothing is silently dropped.

Why

Agents burn serial turns assembling context: search, then outline the hits, then zoom the symbols, then chase callers. Each turn round-trips through the model. A pack returns evidence (verbatim bodies with file:line headers), not conclusions — the agent reasons over it directly, ready to attach to a subagent dispatch.

Measured on a real config repo (3 questions, one call each vs. the manual chain):

• "how does the lane-verdict cache flow end-to-end" → complete 4-file chain (cache shapes, verdict logic, write/read, plugin deny hook) in one pack, used=283/400. Manual baseline: 5-6 tool calls.
• "how does score-tap decide when to write an executor row" → the full decision chain (event filter → dedup → context sources → verdict parse → DB insert) in one pack at budget=200.
• An unrehearsed cross-file question → correct core symbols plus their docstrings, first try.

Independently reproduced on the aft codebase itself: question: "how does bash output compression dispatch pick a compressor" → seeds=15, used=226/400, one pack assembling the full dispatch chain (compress → gate → compress_with_registry_exit_code 20-compressor array → Compressor trait → install path → subc mirror) that otherwise takes a 4–5-call search→zoom chain.

Honest degradation

The pack never lies about its own quality:

• While the semantic index is building, long NL queries degrade to lexical-only file-level hits. The pack renders them as visible file:line (no containing symbol) stubs and flags the header with degraded=semantic-index-building (partial results — retry when index ready) — detected via the response's semantic_status field, cleared as soon as one real seed resolves. No blocking or retry inside the tool.
• Grep-fallback hits ({file, line_text, line}, no symbol name) resolve to their containing symbol by line containment — definitions, call sites, and comment hits all upgrade to the enclosing symbol. Hits with no containing symbol stay visible as stubs.
• Unresolved external/stdlib callees collapse to one summary line ((N unresolved external calls omitted)) instead of drowning the stub list; unresolved seeds and callers are never suppressed.

Implementation

• crates/aft/src/commands/gather.rs — Rust-side composition: calls handle_semantic_search / impact_result / call_tree_result / render_symbol_within_budget directly (shared &AppContext, no bridge round-trips, no parallel reimplementation of search).
• Wiring: main.rs dispatch arm, subc_translate.rs mapping, TS factory packages/opencode-plugin/src/tools/gather.ts + registration (same tier as aft_callgraph — depends on the callgraph store).
• No new dependencies, no config surface beyond the tool args, no LLM calls, no caching.
• Follows the tri-state honest-reporting convention (protocol.rs Response doc-comment): success:false+code for un-performable calls (e.g. invalid_request on a bad mode combo), success:true with a visible degraded/stub pack for partial results — never a bare empty success.

Tests

22 unit tests in gather.rs, including red-checked regressions (each confirmed to fail against pre-fix code): mid-codepoint truncation panic, duplicate-symbol line-anchored resolution, abs/rel path dedupe, containing-symbol resolution via a real TreeSitterProvider, callee-only stub suppression driven through the production build_pack path, and degradation-flag presence/absence/mixed cases.

Limitations (deliberate scope)

• 1-hop expansion only — multi-hop was deliberately excluded: the budget math and ranking get harder and the packs get noisier.
• Callgraph-dependent by design: neighbor expansion quality follows the callgraph store's freshness, same as aft_callgraph.
• Budget counted in lines, not tokens — matches aft's existing budget idiom across zoom/outline.

---

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

---

Summary by cubic

Adds aft_gather, a one-call, deterministic context-pack builder that returns ranked, deduped, verbatim code within a fixed line budget, replacing the multi-step search→outline→zoom→callgraph chain. Exposed as gather in Rust and aft_gather in @opencode (tier: "all").

• New Features

  • Two modes: question (semantic seeds) or symbol + filePath (impact callers + call-tree callees). 1‑hop expansion. Dedupe by normalized (file, symbol) with seeds winning.
  • Hard line budget (default 400, max 800). Overflow listed under “Beyond budget”. Unresolved external callees collapse to one summary line; seeds/callers stay as visible stubs.
  • Grep‑fallback hits resolve to containing symbols; hits with no containing symbol render as visible stubs.
  • Header flags semantic-index-building degradation and neighbors=skipped(callgraph-unavailable) when the callgraph store is down.
  • Robustness: repo‑relative paths resolve against project root (incl. filePath in translate). Abs/rel paths normalize for dedupe. Query truncation is Unicode‑safe. No new dependencies. 22 tests. README adds aft_gather. Tool gated in plugin ALL_ONLY_TOOLS.

• Migration

  • Call with { question: "how does X work?" } or { symbol, filePath }; optional budget 1–800 (default 400).
  • Neighbor expansion uses the callgraph store; required in symbol mode. In question mode, neighbors are skipped with a header notice if unavailable.

^{Written for commit 48424b4. Summary will update on new commits.}

Greptile Summary

This PR introduces aft_gather, a single-call context-pack tool that replaces the manual search → outline → zoom → callgraph chain an agent otherwise runs. It seeds from either a natural-language question (semantic search, top-k 15) or a symbol+filePath (impact callers + call-tree callees), expands one hop, deduplicates by (file, symbol) with seeds winning, and renders within a hard line budget (default 400, cap 800).

• New crates/aft/src/commands/gather.rs (1 460 lines) — full Rust implementation with build_pack, render_symbol_section, resolve_containing_symbol, Unicode-safe truncate_str, and 22 unit tests.
• New packages/opencode-plugin/src/tools/gather.ts — TS wrapper with mode validation and budget coercion; registered at the all surface tier.
• Wiring in subc_translate.rs and a single dispatch arm in main.rs; no new dependencies.

Confidence Score: 5/5

Safe to merge; the two findings are minor polish items that do not affect correctness on any reachable call path.

The core logic — budget accounting, deduplication, degradation flags, stub ordering — is correct and well-tested. Previously flagged issues are all resolved. The two remaining findings are a misleading doc comment on a deliberately-unused field, and a missing .max(1) clamp that the TS schema already enforces; neither causes incorrect output on the intended call paths.

No files require special attention; gather.rs is the natural focus given its size, and both suggestions there are cosmetic.

Important Files Changed

Filename	Overview
crates/aft/src/commands/gather.rs	New 1460-line gather command implementing two-mode context-pack assembly with dedup, budget accounting, degradation flags, and 22 unit tests. Core logic is sound; one dead field with a misleading doc comment and a missing lower-bound clamp on budget.
packages/opencode-plugin/src/tools/gather.ts	Clean TS tool wrapper with mode validation matching the Rust translate layer, schema constraints (min:1, max:800 on budget), and correct error propagation.
crates/aft/src/subc_translate.rs	Adds translate_gather with three-way mode validation and proper filePath resolution via resolve_path_from_project_root; matches the Rust-side error messages.
crates/aft/src/main.rs	Adds a single dispatch arm for the gather command; straightforward wiring.
packages/opencode-plugin/src/index.ts	Registers aft_gather in the ALL_ONLY_TOOLS set and spreads gatherTools into the tool map; consistent with how aft_callgraph is registered.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Agent
    participant Plugin as TS Plugin
    participant Rust as handle_gather
    participant Search as semantic_search
    participant CG as CallgraphStore
    participant Render as render_symbol_within_budget

    Agent->>Plugin: aft_gather args
    Plugin->>Rust: callToolCall gather

    alt question mode
        Rust->>Search: "query top_k=15"
        Search-->>Rust: results + semantic_status
        Rust->>Rust: resolve grep-fallback hits
        Rust->>CG: impact + call_tree per seed
        CG-->>Rust: callers and callees
    else symbol mode
        Rust->>CG: "impact_result depth=1"
        CG-->>Rust: callers
        Rust->>CG: "call_tree_result depth=1"
        CG-->>Rust: callees
    end

    Rust->>Rust: deduplicate by file and symbol
    Rust->>Rust: order seeds first then neighbors

    loop each candidate
        Rust->>Render: render with per_symbol_budget
        Render-->>Rust: section or stub
    end

    Rust->>Rust: build header with final used count
    Rust-->>Plugin: text pack_content
    Plugin-->>Agent: pack_content string

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Agent
    participant Plugin as TS Plugin
    participant Rust as handle_gather
    participant Search as semantic_search
    participant CG as CallgraphStore
    participant Render as render_symbol_within_budget

    Agent->>Plugin: aft_gather args
    Plugin->>Rust: callToolCall gather

    alt question mode
        Rust->>Search: "query top_k=15"
        Search-->>Rust: results + semantic_status
        Rust->>Rust: resolve grep-fallback hits
        Rust->>CG: impact + call_tree per seed
        CG-->>Rust: callers and callees
    else symbol mode
        Rust->>CG: "impact_result depth=1"
        CG-->>Rust: callers
        Rust->>CG: "call_tree_result depth=1"
        CG-->>Rust: callees
    end

    Rust->>Rust: deduplicate by file and symbol
    Rust->>Rust: order seeds first then neighbors

    loop each candidate
        Rust->>Render: render with per_symbol_budget
        Render-->>Rust: section or stub
    end

    Rust->>Rust: build header with final used count
    Rust-->>Plugin: text pack_content
    Plugin-->>Agent: pack_content string

Loading

_{Reviews (7): Last reviewed commit: "gather: deterministic context-pack tool ..." | Re-trigger Greptile}

[cubic-dev-ai]

4 issues found across 7 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[iceteaSA]

Follow-up on the two maintainability notes from the Greptile summary (they weren't separate review threads, so noting here) — both addressed in cf09b9b9:

• degraded expression flagged as always-false in the non-empty-seeds path — intentional: degraded only applies when zero symbol-level seeds resolve (a building index yields no seeds; a ready index with seeds is never degraded). Logic unchanged; added a comment on the why so it doesn't read as dead code.
• callee-suppression string-match coupling — extracted UNRESOLVED_MARKER ("(symbol not resolved)") and CALLEE_PROVENANCE_PREFIX ("callee-of-") consts, referenced by both the producer (render_symbol_section err arm / collect_callees_for_seed) and the consumer (build_pack suppression guard) so the match can't silently drift. Only test-site literals remain, intentionally — a test pointing at the const couldn't catch const drift.

Matched/rendered text is byte-identical; 23/23 gather tests green.