[DO NOT MERGE] Test harness for the AI review skip gate - #98
[DO NOT MERGE] Test harness for the AI review skip gate#98neil-marcellini wants to merge 6 commits into
Conversation
|
(Neil's AI agent) Gate verified on all four events; results recorded in the Manual Tests section of #97. Closing without merging. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d30e9112bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| ```yaml | ||
| permissions: | ||
| statuses: write |
There was a problem hiding this comment.
For a private caller repository, copying this workflow-level permission block sets every unspecified permission to none, so shouldSkipReview.sh cannot perform its GET /repos/{owner}/{repo}/pulls/{pull_number} request and the review job fails before reaching the gate. GitHub documents that this endpoint requires the fine-grained Pull requests: read permission, while unauthenticated access is limited to public resources; add pull-requests: read here and to the test workflow. (workflow permission semantics, pull-request endpoint permissions)
Useful? React with 👍 / 👎.
| fi | ||
|
|
||
| # The combined status endpoint returns only the most recent status per context. | ||
| STATE=$(gh api "/repos/$REPO/commits/$HEAD_SHA/status" --jq ".statuses[] | select(.context == \"$CONTEXT\") | .state") |
There was a problem hiding this comment.
In repositories where more than one page of commit-status contexts exists and this review context is not on the first page, the filter returns an empty STATE even though a successful marker exists, causing the expensive review and its findings to be posted again. The combined-status endpoint exposes page/per_page, and gh api only follows subsequent pages when --paginate is supplied, so this lookup needs pagination or a targeted equivalent. (combined-status endpoint, gh api --paginate)
Useful? React with 👍 / 👎.
1 participant
Details
(Neil's AI agent)
Throwaway PR used to exercise the two scripts added in #97 on a real pull request. It carries that PR's branch plus
.github/workflows/testSkipGate.yml, which stands in for an AI reviewer: it callsshouldSkipReview.sh, echoes the gate decision, and callsrecordReviewComplete.shunder the same conditions the consumer workflows use.It uses
pull_request(notpull_request_target) so the workflow definition is read from this branch's head rather thanmain, and it writes the status under the contextai-review-completed/testso it cannot collide with a real reviewer.Close this without merging once the gate is verified.
Related Issues
https://github.com/Expensify/Expensify/issues/669298
Manual Tests
(Neil's AI agent)
See the run history on this PR.