Skip to content

[HOLD Github-Actions 97] [No QA] Skip AI reviews when they already ran for the PR head commit - #98016

Draft
neil-marcellini wants to merge 2 commits into
mainfrom
neil-skip-ai-review-same-commit
Draft

[HOLD Github-Actions 97] [No QA] Skip AI reviews when they already ran for the PR head commit#98016
neil-marcellini wants to merge 2 commits into
mainfrom
neil-skip-ai-review-same-commit

Conversation

@neil-marcellini

@neil-marcellini neil-marcellini commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Explanation of Change

(Neil's AI agent)

Today, marking a PR ready for review runs both AI reviewers again on the exact same commit they already reviewed, reposting identical findings. This gates each reviewer on whether it has already completed for the PR's current head commit.

The marker is a commit status on the head SHA — ai-review-completed/claude and ai-review-completed/codex — which is the only durable record keyed by commit. Reactions and comments are not tied to a commit, and workflow-run history is not either (comment-triggered runs record github.sha as the base branch head, not the PR head). Both workflows gain two steps around the existing ones:

  • Check for an existing review of this commit resolves the head SHA and sets skip=true if a success status with that context is already on it. Every step that costs money or posts to the PR is gated on skip != 'true'.
  • Record review completion sets that status once the review finishes, against the SHA captured before the review started. If the author pushed mid-review, the status lands on the commit that was actually reviewed, so the next event correctly triggers a fresh review.

The status is named ai-review-completed/* rather than ai-review/* because it sits directly beside the review job in the PR's checks list. A neighbouring green row called ai-review/claude reads like a second review verdict; ai-review-completed/claude reads as the record of a past run, which is what it is.

The status is recorded whether or not the review found anything — re-running on an unchanged commit would only repost identical comments, so "already completed" is the right skip condition rather than "already passed".

Two knock-on changes in claude-review.yml, both required by the skip:

  • Added the @claude review comment trigger. Auth and Web-Expensify already have it; App did not. A skip with no manual override would leave no way to force a re-review, so this is a prerequisite rather than a nice-to-have. It also closes the on-demand inconsistency called out in the issue. Codex already had /codex-review.
  • Replaced dorny/paths-filter with a gh pr view --json files filter. paths-filter reads the pull_request event payload and has no PR context on an issue_comment event, so it cannot survive the new trigger without being handed an explicit base and ref (which in turn needs a full fetch of the PR head that pull_request_target deliberately does not do). One API call gives the same file list for both event types.

The shared scripts live in the toolkit action so Auth and Web-Expensify use the same implementation: Expensify/GitHub-Actions#97. This PR must not merge until that one does — the toolkit refs here point at that PR's branch commit and need re-pinning to the merged SHA first.

Sibling PRs: https://github.com/Expensify/Auth/pull/23558, https://github.com/Expensify/Web-Expensify/pull/55193

Fixed Issues

$ https://github.com/Expensify/Expensify/issues/669298
PROPOSAL: N/A

Tests

(Neil's AI agent)

CI-only change with no product surface, so there is nothing to test in the app and no unit tests apply. actionlint passes on both workflows (only the pre-existing unknown-blacksmith-*-label warning), and oxfmt reports both formatted.

Not yet run end to end — the toolkit PR is not merged. Planned, on this PR itself:

  1. Return this PR to draft and comment @claude review; wait for the review to finish.
  2. Confirm an ai-review-completed/claude status appears on the head commit in the checks list, linking to the run.
  3. Mark the PR ready for review — both review and codex_review should run and skip their review steps (Codex's own status is recorded when /codex-review runs, so verify it independently in step 6).
  4. Push a new commit and mark ready again — both reviewers should run normally and record new statuses.
  5. Comment @claude review on the already-reviewed commit — the review should run, confirming the manual override still works.
  6. Comment /codex-review twice on the same commit — the second should skip.
  7. On a PR touching only docs/**/*.md, confirm the docs reviewer still runs (verifying the replacement path filter).
  • Verify that no errors appear in the JS console

Offline tests

N/A — CI configuration only.

QA Steps

No QA — this changes GitHub Actions workflows only and has no staging or production surface.

  • Verify that no errors appear in the JS console

@neil-marcellini neil-marcellini changed the title [No QA] Skip AI reviews when they already ran for the PR head commit [HOLD Github-Actions 97] [No QA] Skip AI reviews when they already ran for the PR head commit Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant