fix(#6557): detect scaffold content drift in repos status by fullsend-ai-coder[bot] · Pull Request #6559 · fullsend-ai/fullsend

fullsend-ai-coder · 2026-08-24T18:39:15Z

Summary

Add scaffold content-drift detection to repos status so that template changes (new jobs, permissions, thin-caller restructuring) are reported even when the ref string matches the manifest. For branch-ref targets like fullsend_ref: main, the ref never changes, so template drift was previously invisible.

Related Issue

Closes #6557

Changes

Add ExpectedScaffoldContent() in install.go — renders expected scaffold files from the manifest's resolved config using BuildScaffoldFiles, shared by both the status and converge paths
Add checkScaffoldContentDrift() in status.go — compares installed scaffold content against expected templates, normalising refs via replaceShimRef before comparison to avoid false positives from ref-format differences
Call checkScaffoldContentDrift from checkRepoStatus after existing component and ref drift checks
Update populateInstalledRepo test helper to use BuildScaffoldFiles for realistic scaffold content so existing tests remain accurate with content-drift detection enabled
Add 5 new tests covering workflow content drift, thin-caller content drift, no-drift baseline, branch-ref detection, and ref-only difference (no false positive)

Testing

All 35 existing TestStatus_* tests pass unchanged
5 new content-drift tests pass
Full internal/repos test suite passes with -race
100% coverage on both new functions (ExpectedScaffoldContent, checkScaffoldContentDrift)
go vet clean

Closes #6557

Post-script verification

Branch is not main/master (agent/6557-scaffold-content-drift)
Secret scan passed (gitleaks — 1ad2c76a216f6776e2e97a1bef7739fd863ce457..HEAD)
PR body secret scan passed (gitleaks — no-git)

fullsend-ai-review · 2026-08-24T18:42:31Z

🤖 Finished Review · ✅ Success · Started 6:42 PM UTC · Completed 7:02 PM UTC

Commit: 6435025 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $7.16

codecov · 2026-08-24T18:42:45Z

Codecov Report

❌ Patch coverage is 81.03448% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/repos/status.go	75.00%	7 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

fullsend-ai-review · 2026-08-24T19:02:02Z

Looks good to me

Previous run

Looks good to me

Previous run (2)

Review

Findings

Medium

[error-handling] internal/repos/status.go:282 — checkScaffoldContentDrift can set status.Error (on ExpectedScaffoldContent failure or GetFileContent server error), but checkRepoStatus does not check status.Error after the call and continues to the FULLSEND_GCP_REGION read. If that read also fails, the region error overwrites the scaffold error, losing diagnostic information. Every other status.Error assignment in checkRepoStatus is followed by an early return, but the new helper function breaks this pattern.
Remediation: After the checkScaffoldContentDrift call, check if status.Error is non-empty and return early: if status.Error != "" { return status }

Low

[false-positive-drift] internal/repos/install.go:253 — ExpectedScaffoldContent always constructs InstallConfig with VendorBinary as its zero value (false). For repos installed with --vendor, the installed scaffold files use local workflow refs while ExpectedScaffoldContent produces remote refs. The replaceShimRef normalization only matches the remote-ref pattern, so vendored local-ref syntax will not be normalized and content drift will be falsely reported. The doc comment documents RunnerTags and PrebuiltScaffoldFiles as omitted fields but does not mention VendorBinary.
Remediation: Add VendorBinary to the limitation note in the doc comment.
[test-tautology] internal/repos/status_test.go:42 — The refactored populateInstalledRepo now generates installed scaffold content via BuildScaffoldFiles — the same function that ExpectedScaffoldContent calls internally. This makes the pre-existing "no drift" tests tautological for content drift: both sides produce identical output by construction. The new content-drift-specific tests mitigate this for the positive-drift case, but the "no false positives when content matches" assertion is now weaker.
Remediation: Consider keeping at least one test that constructs installed content independently to verify normalization and comparison work correctly end-to-end.

Next steps:

/fs-fix — agent addresses review findings automatically
/fs-fix <your instruction> — agent fixes with your specific guidance
Push commits directly — review re-runs automatically on push
/fs-fix-stop — disable automatic fix runs for this PR

Previous run (3)

Review

Findings

Medium

[error-handling-idiom] internal/repos/status.go:393 — The workflow-path fallback loop in checkScaffoldContentDrift treats all GetFileContent errors as "not found" and continues to the next path. The established pattern in readWorkflowContent (ref_ops.go:99-111) and ProbeComponents distinguishes forge.IsNotFound errors from transient API failures (rate limiting, server errors). Silently swallowing non-404 errors here could mask real problems that the rest of the codebase would surface.
Remediation: Check forge.IsNotFound(readErr) before continuing; propagate unexpected errors or skip the content-drift check for that file.
[stale-doc] docs/cli/repos.md:192 — The DRIFT column description says "Fields that differ from the manifest" but repos status now also reports scaffold content drift (template body mismatches with Expected: "current template"). Content drift is not a manifest field difference — it is a template rendering comparison. The column description is now incomplete for operators interpreting drift output.
Remediation: Update the DRIFT column description to mention content drift.

Low

[error-handling] internal/repos/status.go:375 — checkScaffoldContentDrift silently swallows errors from ExpectedScaffoldContent. If BuildScaffoldFiles fails (e.g., template rendering error), the content-drift check is silently skipped with no indication to the caller that the check was incomplete.
Remediation: Consider setting status.Error when ExpectedScaffoldContent returns an error.
[spelling-convention] internal/repos/status.go:403 — The placeholder string NORMALISED_REF and comments use British English "normalised", but the codebase consistently uses American English "normalized" (internal/normevent/ package and other occurrences).
Remediation: Rename to NORMALIZED_REF and update comments.
[error-handling-idiom] internal/repos/status_test.go:55 — populateInstalledRepo now calls panic() on BuildScaffoldFiles failure. No other test helper in this package uses panic — existing helpers are designed so they structurally cannot fail.
Remediation: Consider accepting *testing.T and using t.Fatal instead.
[variable-naming] internal/repos/status.go:378 — forgeName := cfg.Forge is a single-use alias. The sibling function checkRepoStatus passes cfg.Forge directly to callsites.
Remediation: Use cfg.Forge directly.
[incomplete-parity] internal/repos/install.go:255 — ExpectedScaffoldContent omits InstallConfig fields that the converge path populates (RunnerTags, PrebuiltScaffoldFiles). Currently only GitHub is wired so not an immediate defect, but becomes one when GitLab status support is added.
Remediation: Document the limitation in the function's doc comment.
[stale-doc] docs/guides/getting-started/operations.md:111 — The repos status description says "detect missing or drifted components and ref drift" but does not mention the new content drift detection.
Remediation: Update to include content drift.
[stale-doc] docs/guides/getting-started/repo-management.md:214 — Drift types listed as "component drift and scaffold ref drift" without mentioning scaffold content drift.
Remediation: Add scaffold content drift to the list.

ggallen · 2026-08-24T21:19:26Z

/fs-fix Rebase and fix all the review issues.

fullsend-ai-coder · 2026-08-24T21:21:16Z

🤖 Finished Fix · ❌ Failure · Started 9:21 PM UTC · Completed 9:30 PM UTC

Commit: 6435025 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.82

fullsend-ai-coder · 2026-08-24T21:30:13Z

⚠️ Post-fix script failed — Signed-off-by rejected (exit code 1)

The fix agent completed, but the post-fix script failed before finishing.

Workflow run: https://github.com/fullsend-ai/fullsend/actions/runs/32778875041

Details:
Agent commit contains a Signed-off-by trailer. Agents must not use 'git commit -s' or append Signed-off-by trailers.
Please check the workflow logs for full details and retry with /fs-fix if appropriate.

ggallen · 2026-08-24T23:32:46Z

/fs-fix - Rebase and fix all review issues.

CRITICAL: Do not use a signed-off by trailer on the commit.

fullsend-ai-coder · 2026-08-24T23:34:37Z

🤖 Finished Fix · ✅ Success · Started 11:34 PM UTC · Completed 11:43 PM UTC

Commit: 6435025 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.24

fullsend-ai-coder · 2026-08-24T23:43:03Z

🔧 Fix agent — iteration 1 (human-triggered)

Addressed all 9 review findings (2 medium, 7 low). Fixed error handling to match codebase idioms (forge.IsNotFound checks, status.Error propagation), corrected British-to-American spelling, replaced panic with t.Fatal in test helper, removed single-use alias, documented incomplete parity limitation, and updated three documentation files to mention scaffold content drift. All tests pass with -race. Rebase could not be performed due to sandbox network restrictions.

Fixed (9):

error-handling-idiom: workflow-path fallback loop treats all errors as not-found (internal/repos/status.go): Added forge.IsNotFound(readErr) check before continuing in the workflow-path loop and the non-workflow path, propagating unexpected errors (rate limiting, server errors) via status.Error. Matches the established pattern in readWorkflowContent.
stale-doc: DRIFT column description incomplete (docs/cli/repos.md): Updated DRIFT column description to mention scaffold files whose template content has changed, in addition to manifest field differences.
error-handling: ExpectedScaffoldContent errors silently swallowed (internal/repos/status.go): Split the combined err/nil check into separate branches. When ExpectedScaffoldContent returns an error, status.Error is now set with a descriptive message before returning.
spelling-convention: NORMALISED_REF uses British English (internal/repos/status.go): Renamed NORMALISED_REF to NORMALIZED_REF and updated all comments from 'normalised'/'normalise' to 'normalized'/'normalize' to match codebase convention (American English).
error-handling-idiom: populateInstalledRepo uses panic instead of t.Fatal (internal/repos/status_test.go): Changed populateInstalledRepo to accept testing.TB as first parameter with t.Helper(), replaced panic with t.Fatalf. Updated all 30+ call sites to pass t.
variable-naming: forgeName is a single-use alias (internal/repos/status.go): Removed the forgeName := cfg.Forge alias and passed cfg.Forge directly to replaceShimRef calls.
incomplete-parity: ExpectedScaffoldContent omits some InstallConfig fields (internal/repos/install.go): Added a Limitation paragraph to the doc comment documenting that RunnerTags and PrebuiltScaffoldFiles are omitted and will need resolution when GitLab status support is added.
stale-doc: repos status description in operations.md missing content drift (docs/guides/getting-started/operations.md): Updated the repos status command description to include scaffold content drift alongside component and ref drift.
stale-doc: drift types in repo-management.md missing content drift (docs/guides/getting-started/repo-management.md): Added scaffold content drift to the drift types listed in the detecting and reconciling configuration drift section.

Tests: passed

Next steps:

/fs-review — request a re-review of the changes
/fs-fix <your instruction> — run another fix pass with specific guidance
Push commits directly — review re-runs automatically on push
_{Updated by fullsend fix agent}

- Check forge.IsNotFound before continuing in workflow-path fallback loop, propagating unexpected errors via status.Error (matches readWorkflowContent pattern) - Set status.Error when ExpectedScaffoldContent returns an error instead of silently skipping the content-drift check - Rename NORMALISED_REF to NORMALIZED_REF and update comments to use American English spelling consistently - Change populateInstalledRepo to accept testing.TB and use t.Fatal instead of panic, matching test helper conventions - Remove single-use forgeName alias, pass cfg.Forge directly - Document RunnerTags/PrebuiltScaffoldFiles omission in ExpectedScaffoldContent doc comment - Update DRIFT column description in docs/cli/repos.md to mention scaffold content drift - Update repos status description in operations.md and repo-management.md to mention scaffold content drift Addresses review feedback on #6559

fullsend-ai-review · 2026-08-24T23:45:04Z

🤖 Review · ❌ Terminated · Started 11:45 PM UTC · Ended 12:02 AM UTC

Commit: a7f9736 · View workflow run →

github-actions · 2026-08-24T23:45:17Z

Site preview

Preview: https://28a78c27-site.fullsend-ai.workers.dev

Commit: ed6204e7600cb12b7cafad343f599fb05c17f134

fullsend-ai-review · 2026-08-25T00:02:49Z

🤖 Finished Review · ✅ Success · Started 11:45 PM UTC · Completed 12:02 AM UTC

Commit: a7f9736 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $6.51

fullsend-ai-coder · 2026-08-25T00:04:49Z

🤖 Finished Fix · ✅ Success · Started 12:04 AM UTC · Completed 12:16 AM UTC

Commit: a7f9736 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.46

fullsend-ai-coder · 2026-08-25T00:16:06Z

🔧 Fix agent — iteration 2 (bot-triggered)

Addressed all 3 review findings: added early return after checkScaffoldContentDrift for error-handling consistency, documented VendorBinary in the limitation note, and added an independent-construction test to address the test-tautology concern.

Fixed (3):

error-handling: checkScaffoldContentDrift can set status.Error but checkRepoStatus does not check afterward (internal/repos/status.go): Added early return after checkScaffoldContentDrift call when status.Error is non-empty, matching the pattern used by every other status.Error assignment in checkRepoStatus. Prevents the subsequent region read from overwriting scaffold content drift errors.
false-positive-drift: ExpectedScaffoldContent doc comment omits VendorBinary from limitation note (internal/repos/install.go): Added VendorBinary to the limitation note alongside RunnerTags and PrebuiltScaffoldFiles, documenting that vendor-mode repos may see false content-drift reports until vendor-mode status support is added.
test-tautology: populateInstalledRepo using BuildScaffoldFiles makes no-drift tests tautological (internal/repos/status_test.go): Added TestStatus_NoContentDrift_IndependentInstalledContent which constructs installed content independently by rendering scaffold files once, then using regex replacement to change refs to SHA-annotated format (not a second BuildScaffoldFiles call). This exercises the replaceShimRef normalization and bytes.Equal comparison end-to-end.

Tests: passed

Next steps:

/fs-review — request a re-review of the changes
/fs-fix <your instruction> — run another fix pass with specific guidance
Push commits directly — review re-runs automatically on push
_{Updated by fullsend fix agent}

- Add early return after checkScaffoldContentDrift when status.Error is set, matching the pattern used by every other error assignment in checkRepoStatus. Prevents the region read from overwriting scaffold errors. - Add VendorBinary to the limitation note in ExpectedScaffoldContent's doc comment, documenting the false-positive risk for vendor-mode repos. - Add TestStatus_NoContentDrift_IndependentInstalledContent which constructs installed content independently (via regex ref replacement, not BuildScaffoldFiles) to verify normalization and comparison logic end-to-end, addressing the test-tautology concern. Addresses review feedback on #6559

fullsend-ai-review · 2026-08-25T00:18:00Z

🤖 Finished Review · ✅ Success · Started 12:17 AM UTC · Completed 12:32 AM UTC

Commit: 775c4a4 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.52

Superseded by updated review

Add content-drift detection to `repos status` so that template changes (new jobs, permissions, thin-caller restructuring) are reported even when the ref string matches the manifest. Previously, status only checked scaffold files for presence and ref equality, making template changes invisible for branch-ref targets like `fullsend_ref: main`. Add `ExpectedScaffoldContent` in install.go — a shared function that renders expected scaffold files using `BuildScaffoldFiles` with the manifest's resolved config. The status path calls this and compares each rendered file against the installed content. Refs are normalised via `replaceShimRef` before comparison so ref-format differences do not produce false content-drift reports. Update the `populateInstalledRepo` test helper to use `BuildScaffoldFiles` for realistic scaffold content, ensuring existing no-drift tests remain accurate. Closes #6557

- Check forge.IsNotFound before continuing in workflow-path fallback loop, propagating unexpected errors via status.Error (matches readWorkflowContent pattern) - Set status.Error when ExpectedScaffoldContent returns an error instead of silently skipping the content-drift check - Rename NORMALISED_REF to NORMALIZED_REF and update comments to use American English spelling consistently - Change populateInstalledRepo to accept testing.TB and use t.Fatal instead of panic, matching test helper conventions - Remove single-use forgeName alias, pass cfg.Forge directly - Document RunnerTags/PrebuiltScaffoldFiles omission in ExpectedScaffoldContent doc comment - Update DRIFT column description in docs/cli/repos.md to mention scaffold content drift - Update repos status description in operations.md and repo-management.md to mention scaffold content drift Addresses review feedback on #6559

- Add early return after checkScaffoldContentDrift when status.Error is set, matching the pattern used by every other error assignment in checkRepoStatus. Prevents the region read from overwriting scaffold errors. - Add VendorBinary to the limitation note in ExpectedScaffoldContent's doc comment, documenting the false-positive risk for vendor-mode repos. - Add TestStatus_NoContentDrift_IndependentInstalledContent which constructs installed content independently (via regex ref replacement, not BuildScaffoldFiles) to verify normalization and comparison logic end-to-end, addressing the test-tautology concern. Addresses review feedback on #6559

fullsend-ai-review · 2026-08-25T00:36:42Z

🤖 Finished Review · ✅ Success · Started 12:36 AM UTC · Completed 12:48 AM UTC

Commit: ed6204e · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.20

fullsend-ai-coder Bot requested a review from a team as a code owner August 24, 2026 18:39

fullsend-ai-coder Bot added the ready-for-review Agent PR ready for human review label Aug 24, 2026

fullsend-ai-coder Bot assigned ggallen Aug 24, 2026

fullsend-ai-coder Bot temporarily deployed to dev August 24, 2026 18:39 Inactive

fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 24, 2026

ggallen added the fullsend-fix Enables automatic bot-triggered fix runs on human-authored PRs label Aug 24, 2026

fullsend-ai-coder Bot temporarily deployed to dev August 24, 2026 23:43 Inactive

github-actions Bot deployed to site-preview August 24, 2026 23:45 View deployment

This comment was marked as outdated.

Sign in to view

fullsend-ai-review Bot removed the requires-manual-review Review requires human judgment label Aug 25, 2026

fullsend-ai-coder Bot temporarily deployed to dev August 25, 2026 00:16 Inactive

github-actions Bot deployed to site-preview August 25, 2026 00:17 View deployment

This comment was marked as outdated.

Sign in to view

fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Aug 25, 2026

fullsend-ai-coder Bot added 3 commits August 24, 2026 20:34

ggallen force-pushed the agent/6557-scaffold-content-drift branch from 775c4a4 to ed6204e Compare August 25, 2026 00:34

ggallen temporarily deployed to dev August 25, 2026 00:35 — with GitHub Actions Inactive

github-actions Bot deployed to site-preview August 25, 2026 00:36 View deployment

fullsend-ai-review Bot approved these changes Aug 25, 2026

View reviewed changes

ggallen approved these changes Aug 25, 2026

View reviewed changes

ggallen added this pull request to the merge queue Aug 25, 2026

Merged via the queue into main with commit 8e2eff7 Aug 25, 2026
42 checks passed

ggallen deleted the agent/6557-scaffold-content-drift branch August 25, 2026 01:00

ggallen mentioned this pull request Aug 25, 2026

repos install (converge) does not detect or repair scaffold content drift #6576

Closed

Uh oh!

Conversation

fullsend-ai-coder Bot commented Aug 24, 2026

Summary

Related Issue

Changes

Testing

Post-script verification

Uh oh!

fullsend-ai-review Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

fullsend-ai-review Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review

Findings

Medium

Low

Review

Findings

Medium

Low

Uh oh!

ggallen commented Aug 24, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 24, 2026

Uh oh!

ggallen commented Aug 24, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 24, 2026

🔧 Fix agent — iteration 1 (human-triggered)

Tests: passed

Uh oh!

fullsend-ai-review Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Site preview

Uh oh!

This comment was marked as outdated.

Uh oh!

fullsend-ai-review Bot commented Aug 25, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 25, 2026

🔧 Fix agent — iteration 2 (bot-triggered)

Tests: passed

Uh oh!

fullsend-ai-review Bot commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

fullsend-ai-review Bot commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

fullsend-ai-review Bot commented Aug 24, 2026 •

edited

Loading

codecov Bot commented Aug 24, 2026 •

edited

Loading

fullsend-ai-review Bot commented Aug 24, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 24, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 24, 2026 •

edited

Loading

fullsend-ai-review Bot commented Aug 24, 2026 •

edited

Loading

github-actions Bot commented Aug 24, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 25, 2026 •

edited

Loading

fullsend-ai-review Bot commented Aug 25, 2026 •

edited

Loading

fullsend-ai-review Bot commented Aug 25, 2026 •

edited

Loading