chore(coderabbit): stop incremental reviews to cut add-on spend

.agents/skills/backlog-sweep/SKILL.md

@@ -5,7 +5,7 @@ title: Backlog Sweep Skill
 status: active
 owner: eng
 canonical: true
-last_verified: 2026-08-28
+last_verified: 2026-09-03
 doc_type: skill
 scope: repo-wide
 review_interval_days: 90
@@ -86,8 +86,14 @@ record that looks stale from outside is routinely a live holder inside a long
 browser suite.
 
 **State the usage reality before starting.** One shipped PR costs roughly 3% of
-the weekly usage window, and every push to it triggers another round of bot
-reviews whose findings then cost replies and often another push. Two issues is
+the weekly usage window, and every push to it triggers another Codex review,
+whose findings then cost replies and often another push. Claude is not a
+per-push cost: `.github/workflows/claude.yml` fires on `opened` and
+`ready_for_review` only, so a Claude re-review is opt-in via `@claude review`.
+CodeRabbit should not be a per-push cost either — it is configured to review
+the opening push and the closeout head only — but PR #2236 observed a run on
+every push, all refused by the spending cap, so budget for the attempt until
+ADR 0066's open question is settled. Two issues is
 the default because the cost is dominated by review rounds, not by the first
 implementation. **Refuse a batch size above 4.** Say that plainly and stop
 rather than clamping silently — an operator who asked for 6 needs to know they
@@ -543,7 +549,7 @@ changed.` Review the lifecycle and install scripts in the diff first, then
   `pnpm issue:review --pr <pr> --issue <n>`.
 - **The babysit:** sweep every feedback surface — top-level comments, review
   bodies, inline threads, annotations, failing logs. **Batch fixes into single
-  pushes**, because every push costs another bot review round. Reply before
+  pushes**, because every push costs another Codex review round. Reply before
   resolving, in the two canonical forms: `Fixed in <commit> — <what changed>`
   and `Won't fix: <technical reason why>`. Drive to READY on both projections,
   `pr:feedback-state` clean first, then `pr:ready-state`.

.claude/skills/backlog-sweep/SKILL.md

@@ -5,7 +5,7 @@ title: Backlog Sweep Skill
 status: active
 owner: eng
 canonical: true
-last_verified: 2026-08-28
+last_verified: 2026-09-03
 doc_type: skill
 scope: repo-wide
 review_interval_days: 90
@@ -86,8 +86,14 @@ record that looks stale from outside is routinely a live holder inside a long
 browser suite.
 
 **State the usage reality before starting.** One shipped PR costs roughly 3% of
-the weekly usage window, and every push to it triggers another round of bot
-reviews whose findings then cost replies and often another push. Two issues is
+the weekly usage window, and every push to it triggers another Codex review,
+whose findings then cost replies and often another push. Claude is not a
+per-push cost: `.github/workflows/claude.yml` fires on `opened` and
+`ready_for_review` only, so a Claude re-review is opt-in via `@claude review`.
+CodeRabbit should not be a per-push cost either — it is configured to review
+the opening push and the closeout head only — but PR #2236 observed a run on
+every push, all refused by the spending cap, so budget for the attempt until
+ADR 0066's open question is settled. Two issues is
 the default because the cost is dominated by review rounds, not by the first
 implementation. **Refuse a batch size above 4.** Say that plainly and stop
 rather than clamping silently — an operator who asked for 6 needs to know they
@@ -543,7 +549,7 @@ changed.` Review the lifecycle and install scripts in the diff first, then
   `pnpm issue:review --pr <pr> --issue <n>`.
 - **The babysit:** sweep every feedback surface — top-level comments, review
   bodies, inline threads, annotations, failing logs. **Batch fixes into single
-  pushes**, because every push costs another bot review round. Reply before
+  pushes**, because every push costs another Codex review round. Reply before
   resolving, in the two canonical forms: `Fixed in <commit> — <what changed>`
   and `Won't fix: <technical reason why>`. Drive to READY on both projections,
   `pr:feedback-state` clean first, then `pr:ready-state`.

.coderabbit.yaml

@@ -69,8 +69,8 @@ reviews:
         Report an ordering or lifecycle finding when the trace shows a documented lifecycle invariant violation, such as a bypassed prerequisite, duplicate execution, deadlock, incorrect ownership, or missed settlement, or when focused coverage no longer reaches the changed path.
 
   # Generated, vendored, and machine-written trees. Reviewing them spends
-  # reviewed-file capacity (the Pro+ fair-usage ladder is finite; no usage-based
-  # add-on is enabled) on diffs no human authored.
+  # billable reviewed files (the usage-based add-on is enabled, so over-limit
+  # reviews bill 25¢ per reviewed file) on diffs no human authored.
   path_filters:
     - "!pnpm-lock.yaml"
     - "!**/pnpm-lock.yaml"
@@ -102,10 +102,24 @@ reviews:
     # Empty means the default branch flow only.
     base_branches: []
     drafts: false
-    auto_incremental_review: true
-    # Keep the vendor default of 5. A threshold of 2 paused 16 of the first 29
-    # PRs after rollout, including six PRs with only 2-4 total commits. Five
-    # avoids treating one normal fix round as an active-development burst while
-    # still bounding long agent commit sequences. `@coderabbitai review`
-    # requests one exact-head closeout review after a pause.
+    # Off since 2026-09-02 (ADR 0066). The usage-based add-on is enabled
+    # (Automatic mode, $500/month cap) and the cap was reached on 2026-09-02
+    # with 16 days left in the cycle. The 30-day team view read that day: 672
+    # review events, 365 billed, 3.9 review events per PR, ~5.5 billed files
+    # per billed push. CodeRabbit bills per push delta, so every agent fix
+    # round re-bills the same files. With this off, CodeRabbit should review a
+    # PR once when it opens, and the head-bound `@coderabbitai review` request
+    # in `docs/notes/pr-ready-state.md` buys the final head one more review.
+    # Observed on PR #2236, though: every push still started a run. Until
+    # that is explained, treat "no run on an intermediate push" as the intent
+    # rather than the measured behaviour; ADR 0066 holds the dated tally, the
+    # evidence, and the organization-level override that follows from it.
+    # Intermediate pushes lose CodeRabbit's review. Codex still reviews every
+    # push automatically; Claude is opt-in via `@claude review`, because
+    # `.github/workflows/claude.yml` triggers only on `opened` and
+    # `ready_for_review`. No required gate changes, because CodeRabbit's check
+    # was already advisory; ADR 0066 records the residual this does cost.
+    auto_incremental_review: false
+    # The vendor default of 5, kept. It now only matters if incremental
+    # reviews are re-enabled; the org-level global override pins it too.
     auto_pause_after_reviewed_commits: 5

docs/adr/0066-coderabbit-replaces-bugbot-third-reviewer.md

@@ -3,7 +3,7 @@ title: CodeRabbit replaces Cursor BugBot as the third PR review bot
 status: active
 owner: eng
 canonical: true
-last_verified: 2026-09-02
+last_verified: 2026-09-03
 scope: ci/process
 date: 2026-08
 doc_type: adr
@@ -14,7 +14,8 @@ garden_lane: adrs-architecture
 # ADR 0066 — CodeRabbit replaces Cursor BugBot as the third PR review bot
 
 **Status:** Accepted (Aug 2026) — BugBot disabled on 2026-08-31; the live
-compatibility paths retired on 2026-09-02.
+compatibility paths retired on 2026-09-02; amended 2026-09-02 (usage add-on
+enabled, incremental reviews off).
 **Scope:** ci/process
 
 ## Context
@@ -114,6 +115,10 @@ BugBot after reviewing these results.
 | Graphite Agent      | Flat $20–40/month, unlimited reviews          | $20–40                                                   |
 | Drop to two bots    | —                                             | $0, minus third-bot coverage                             |
 
+The CodeRabbit row records the 2026-08-18 estimate. The add-on was later
+enabled and its real cost overran that row — see the 2026-09-02 amendment
+below.
+
 CodeRabbit's public-repo terms were verified against the vendor's docs and
 the installed org's billing page on 2026-08-18. The Open Source tier gives
 public repos **Pro+ features free**, with no application step and no license
@@ -148,9 +153,10 @@ Replace BugBot with CodeRabbit as the third advisory reviewer.
    deferred: the operator decided on 2026-08-18 to feel the rate-limit
    friction first, at flat seat cost, and enable the add-on only if that
    friction proves annoying. Until then, over-limit reviews wait instead of
-   billing. The $0 OSS tier (Pro+ features, ~1/hour, manual-trigger-only
-   under 10 stars, no add-on) remains the documented fallback if spend must
-   return to zero.
+   billing. **Superseded 2026-09-02** — the add-on was enabled in Automatic
+   mode with a $500/month cap; see the amendment below. The $0 OSS tier (Pro+
+   features, ~1/hour, manual-trigger-only under 10 stars, no add-on) remains
+   the documented fallback if spend must return to zero.
 2. **Commit `.coderabbit.yaml`** (done 2026-08-19 in PR #1927): use the `chill`
    profile, `path_filters` that exclude lockfiles and generated trees, and
    `auto_pause_after_reviewed_commits: 5`. The initial value of 2 paused normal
@@ -159,6 +165,10 @@ Replace BugBot with CodeRabbit as the third advisory reviewer.
    normal fix round as active development. The ship and babysit closeout
    requests one manual review for an exact head when the automatic review is
    stale or missing after the optional check becomes terminal.
+   **Amended 2026-09-02** — `auto_incremental_review: false` means no
+   automatic review follows a push, so the closeout request no longer waits
+   for one; the pause threshold now only matters if incremental reviews
+   return.
 3. **Run both reviewers for two weeks** (complete 2026-08-31). Compare their
    findings on the same PRs. Use the result to confirm or reverse the decision.
 4. **Disable BugBot** (complete 2026-08-31) and **retire the compatibility
@@ -182,6 +192,123 @@ Replace BugBot with CodeRabbit as the third advisory reviewer.
    cross-bot forensic evidence, create a narrow issue-scoped query for that
    decision.
 
+## Amendment 2026-09-02 — usage add-on enabled, incremental reviews off
+
+This corrects the "add-on deferred" record above. Decision step 1, the cost
+table, and the first Consequences bullet describe the state as of 2026-08-18
+and stay in place as history.
+
+- **The add-on was enabled, not deferred.** It runs in Automatic mode with a
+  $500/month cap. The enabling date was not recorded; the billing page showed
+  it active on 2026-09-02.
+- **The vendor renamed the plan tiers.** Checked 2026-09-02,
+  docs.coderabbit.ai/management/plans calls the tier this org holds **Team**
+  (formerly Pro+): $48/developer/month annual, $60 month-to-month, 8 PR
+  reviews/hour, 300 files per review. Pro is now Essentials. The rate-limit
+  notice on PR #2236 reported `Plan: Team` for that reason — the same seat
+  under a new name, not a downgrade. Sections above this amendment keep the
+  Pro+ name they were written with.
+- **The cap was reached on 2026-09-02**, sixteen days before the cycle resets
+  on 2026-09-18. Past the cap only the free refill runs — 1 review/hour for
+  the sole PR author, who sits permanently in the 90+ reviews/7-days tier —
+  and every other push is blocked.
+- **30-day Review usage, team view, read 2026-09-02:** 672 review events, 365
+  continued with credits (billed), 48 blocked, 3.9 review events per PR, ~5.5
+  billed files per continued review. CodeRabbit meters pushes, not commits or
+  files: each push past the hourly refill bills every file in that push's
+  delta at 25¢, so agent fix rounds re-bill the same files several times per
+  PR.
+- **Decision:** set `reviews.auto_review.auto_incremental_review: false` in
+  `.coderabbit.yaml`. CodeRabbit then reviews a PR once when it opens and once
+  more at closeout, when the ready-state flow posts the head-bound
+  `@coderabbitai review` request that
+  [`../notes/pr-ready-state.md`](../notes/pr-ready-state.md) already defines.
+  Intermediate pushes get no automatic CodeRabbit review. Codex still reviews
+  every push automatically, but Claude does not: `.github/workflows/claude.yml`
+  triggers on `opened` and `ready_for_review` only, so a Claude re-review is
+  opt-in via `@claude review`. The coverage an intermediate push keeps is
+  therefore Codex alone. No required gate changes, because CodeRabbit's own
+  check was already advisory — but read the accepted residual below for what this
+  does cost. Expected: review events per PR fall from ~3.9 to ~2, each changed
+  file bills at most twice per PR, and spend lands at an estimated
+  $200–$250/month at current volume.
+- **The repository file governs this key.** The organization-level Global
+  overrides applied on 2026-09-02 pin `reviews.profile`,
+  `request_changes_workflow`, `auto_review.enabled`, `auto_review.drafts`, and
+  `auto_review.auto_pause_after_reviewed_commits`. They do not set
+  `auto_incremental_review`, so `.coderabbit.yaml` decides it and
+  `scripts/coderabbit-config.test.mjs` is its only guard. **Follow-up, now
+  indicated rather than conditional:** the operator adds
+  `auto_incremental_review: false` to the Global overrides. PR #2236 measured
+  the repository file on its own and it did not hold — see the next bullet.
+- **Measured on PR #2236: the repository file alone did not suppress the
+  attempt.** This bullet is the single tally; the runbooks point here rather
+  than carrying their own counts, which went stale every time the PR pushed.
+  As of 2026-09-03, five pushes each drew a CodeRabbit run within roughly 75
+  seconds, with `auto_incremental_review: false` in force on the branch the
+  runs read (`Path: .coderabbit.yaml`, `Review profile: CHILL`):
+
+  | Push          | Head        | Run ID      | Observed             |
+  | ------------- | ----------- | ----------- | -------------------- |
+  | 1, opening    | `733fa0b83` | `3fdfc30d…` | 2026-09-02 14:40:44Z |
+  | 2             | `242f47afb` | `5e05ac36…` | 2026-09-02 15:51:39Z |
+  | 3             | `cb273d4e2` | `9739f061…` | 2026-09-02 15:57:49Z |
+  | 4             | `32b60c787` | `beabff8f…` | 2026-09-02 16:51:26Z |
+  | 5, base merge | `95c3b6dc7` | `c21f85af…` | 2026-09-03 06:43:23Z |
+
+  All five runs were refused by the spending cap, so nothing was billed. What
+  set run 5 apart was its scope, not its outcome: runs 1-4 each reported the
+  full base-to-head diff, while run 5 reported an incremental
+  `4aef1be09`..`95c3b6dc7` delta, the first to do so. One confound stays open:
+  the opening review never completed either, and CodeRabbit may retry an
+  unfinished review when the head moves regardless of this key. A clean
+  measurement needs a PR whose opening review completed, so it waits for the
+  2026-09-18 cap reset.
+
+- **The closeout request is the vendor-documented path here, checked
+  2026-09-02.** docs.coderabbit.ai/configuration/auto-review states that
+  `auto_incremental_review: false` reviews "only when a PR is first opened.
+  Subsequent pushes will be ignored until you trigger the review manually", and
+  names `@coderabbitai review` as that manual trigger.
+  docs.coderabbit.ai/reference/review-commands lists "when automatic reviews
+  are disabled" as a use case for the same command, describes it as triggering
+  "an incremental review of new changes only", and notes it spends one review
+  from the allowance. The "applicable only when automatic reviews are paused"
+  line in PR #2236's refusal came attached to a rate-limited command, so it is
+  not evidence against the documented path.
+- **Confirmed 2026-09-02: the closeout request works.** The head-bound request
+  posted against `4aef1be09` at `21:24:54Z` produced a completed CodeRabbit
+  review at `21:31:58Z` carrying four inline findings — the first review this
+  PR obtained. That settles the open question above: the command is applicable
+  under `auto_incremental_review: false`, and the "applicable only when
+  automatic reviews are paused" line really was boilerplate on a rate-limited
+  refusal. One question remains for the 2026-09-18 re-measure: why pushes
+  still trigger runs at all when the documentation says they are ignored. The
+  incremental range on run 5 is the sharpest clue — it is the shape an
+  incremental review takes, not the shape of a retry of an unfinished opening
+  review.
+- **Rejected here:** raising the cap keeps the per-push meter and buys more
+  duplicate reviews; tightening `path_filters` saves little because the billed
+  unit is an already-small push delta. The $0 OSS tier stays the fallback if
+  spend must return to zero.
+- **Accepted residual:** the closeout request is now the final scheduled
+  request in the ready-state flow, and readiness never waits for it.
+  `summarizeCodeRabbitReviewGate` returns `required: false`, `ready` is
+  `required.ready` in `scripts/pr/pr-ready-state-core.mjs`, and
+  `pr-ready-state.test.mjs` pins "never awaits a pending CodeRabbit check for
+  readiness" — so a head can reach all-clear with its CodeRabbit signal still
+  `requested`. That was already true before this change; incremental review
+  only made it unlikely to bite, because the final head usually already
+  carried an automatic review. The partial cover stays as
+  [`../notes/pr-ready-state.md`](../notes/pr-ready-state.md) states it: if the
+  requested review lands while the PR is still under watch, rerun
+  `pr:feedback-state` and handle its findings before all-clear. Making the
+  CodeRabbit signal blocking is a separate decision this ADR does not take.
+- **Revisit trigger:** closeout-only coverage missing defects that incremental
+  reviews would have caught, heads merging with the closeout review still only
+  `requested`, or spend still tracking toward the cap after the 2026-09-18
+  reset.
+
 ## Alternatives considered
 
 - **Greptile** — strongest precision on the current Martian snapshot, 0% false
@@ -224,6 +351,8 @@ Replace BugBot with CodeRabbit as the third advisory reviewer.
   ~$180/month in heavy months, with path filters shrinking the file counts.
   BugBot's per-run meter, which scaled with agent iteration, is gone from the
   default path either way, and the $0 OSS fallback caps downside.
+  **Superseded 2026-09-02**: the add-on was on, and add-on spend reached its
+  $500/month cap rather than staying under ~$180. See the amendment below.
 - The stack's third seat changes character from precision-quiet to
   high-recall. CodeRabbit's known weakness is verbosity; the quiet/chill
   profiles, path filters, and pause-after-reviewed-commits are the levers, and
@@ -237,7 +366,10 @@ Replace BugBot with CodeRabbit as the third advisory reviewer.
   the add-on. Keep `auto_pause_after_reviewed_commits` at 5 and request one
   head-bound `@coderabbitai review` at closeout when the automatic review is
   stale or missing. This spends the ladder on review rounds that matter while
-  preventing duplicate requests for the same head.
+  preventing duplicate requests for the same head. **Amended 2026-09-02**:
+  agent fix bursts now do neither — incremental auto-review is off, so they
+  get no CodeRabbit review at all and the closeout request is the second and
+  last one per PR.
   `@coderabbitai rate limit` reports remaining capacity without consuming
   a review.
 - The 2026-08-31 open-PR sweep found one current-head Cursor finding, on PR
@@ -327,6 +459,10 @@ Replace BugBot with CodeRabbit as the third advisory reviewer.
   add-on availability (paid plans only). The installed org's billing page
   confirmed the default Pro+ trial and the 2026-08-18 paid upgrade. All
   checked 2026-08-18.
+- Usage add-on state and spend: the installed org's CodeRabbit Subscription
+  and Usage pages, read 2026-09-02 — add-on Automatic with a $500/month cap,
+  cap reached that day, cycle reset 2026-09-18, and the 30-day Review usage
+  figures (672 / 365 / 48 / 3.9 / ~5.5) in the amendment above.
 - Post-rollout pause sample, queried from GitHub on 2026-08-21: 16 of the 29
   PRs created after `.coderabbit.yaml` merged carried CodeRabbit's generated
   pause marker. Six of those PRs had only 2-4 total commits. Two of the 29 PRs