What happened
Issue #6571 explicitly stated "CONTRIBUTING.md in the plugin repo documents the pi-specific gotchas" and linked to fullsend-ai/pi-xai-vertex. The code agent (run 32781317760) read 17 internal files during its planning phase but never visited the external repo. Its transcript shows zero gh repo view, WebFetch, or Read calls targeting fullsend-ai/pi-xai-vertex. The agent pinned v0.1.0 as specified in the issue without verifying whether that version's auth model was compatible with the sandbox. The human reviewer later discovered v0.1.0 used interactive OAuth (auth.oauth), which requires persisted credentials impossible in an ephemeral sandbox, and bumped to v0.2.0. This was the single largest rework item on the PR — the human had to independently test the extension, discover the auth failure, identify v0.2.0 as the fix, recompute the SHA256 digest, and verify the new version end-to-end.
What could go better
The code agent's planning phase should identify external repositories and URLs referenced in the issue body and attempt to read their key documentation (README, CONTRIBUTING.md, changelog, release notes). In this case, reading the external repo's CONTRIBUTING.md or release notes would have revealed the v0.1.0 → v0.2.0 auth model change and likely prevented the rework. I'm confident this is the right improvement because: (1) the issue explicitly directed the reader to the external doc, (2) the transcript confirms the agent never attempted it, and (3) the information was available in that repo. The sandbox does have network restrictions that prevented tarball downloads, but gh API access to read repo contents should be available since the agent already uses gh extensively.
Proposed change
In the code-implementation skill (or the code agent definition at agents/code.md) in fullsend-ai/agents, add guidance to the planning phase that instructs the agent to: (1) scan the issue body for references to external repositories (GitHub repo URLs, org/repo mentions, or explicit directions like "see CONTRIBUTING.md in repo X"), (2) use gh api or gh repo view to read key files (README.md, CONTRIBUTING.md, CHANGELOG.md, latest release notes) from those repos, and (3) incorporate any compatibility constraints, version-specific gotchas, or breaking changes discovered into its implementation plan before writing code. When the sandbox prevents access, the agent should flag the unverified external reference in its PR description (complementing the fallback behavior proposed in #4624).
Validation criteria
On the next 3 code agent runs where the issue body references an external repository, the agent's transcript should show at least one attempt to read that repo's documentation. If the read succeeds, the agent should incorporate the findings into its plan. If it fails due to access restrictions, the PR description should flag the unverified reference. Measure by reviewing transcripts of code runs on issues that link external repos.
Generated by retro agent from fullsend-ai/fullsend#6572
What happened
Issue #6571 explicitly stated "CONTRIBUTING.md in the plugin repo documents the pi-specific gotchas" and linked to
fullsend-ai/pi-xai-vertex. The code agent (run 32781317760) read 17 internal files during its planning phase but never visited the external repo. Its transcript shows zerogh repo view,WebFetch, orReadcalls targetingfullsend-ai/pi-xai-vertex. The agent pinned v0.1.0 as specified in the issue without verifying whether that version's auth model was compatible with the sandbox. The human reviewer later discovered v0.1.0 used interactive OAuth (auth.oauth), which requires persisted credentials impossible in an ephemeral sandbox, and bumped to v0.2.0. This was the single largest rework item on the PR — the human had to independently test the extension, discover the auth failure, identify v0.2.0 as the fix, recompute the SHA256 digest, and verify the new version end-to-end.What could go better
The code agent's planning phase should identify external repositories and URLs referenced in the issue body and attempt to read their key documentation (README, CONTRIBUTING.md, changelog, release notes). In this case, reading the external repo's CONTRIBUTING.md or release notes would have revealed the v0.1.0 → v0.2.0 auth model change and likely prevented the rework. I'm confident this is the right improvement because: (1) the issue explicitly directed the reader to the external doc, (2) the transcript confirms the agent never attempted it, and (3) the information was available in that repo. The sandbox does have network restrictions that prevented tarball downloads, but
ghAPI access to read repo contents should be available since the agent already usesghextensively.Proposed change
In the
code-implementationskill (or the code agent definition atagents/code.md) infullsend-ai/agents, add guidance to the planning phase that instructs the agent to: (1) scan the issue body for references to external repositories (GitHub repo URLs, org/repo mentions, or explicit directions like "see CONTRIBUTING.md in repo X"), (2) usegh apiorgh repo viewto read key files (README.md, CONTRIBUTING.md, CHANGELOG.md, latest release notes) from those repos, and (3) incorporate any compatibility constraints, version-specific gotchas, or breaking changes discovered into its implementation plan before writing code. When the sandbox prevents access, the agent should flag the unverified external reference in its PR description (complementing the fallback behavior proposed in #4624).Validation criteria
On the next 3 code agent runs where the issue body references an external repository, the agent's transcript should show at least one attempt to read that repo's documentation. If the read succeeds, the agent should incorporate the findings into its plan. If it fails due to access restrictions, the PR description should flag the unverified reference. Measure by reviewing transcripts of code runs on issues that link external repos.
Generated by retro agent from fullsend-ai/fullsend#6572