test(codex-ws): de-flake semaphore-tail test (widen noise floor 25ms->75ms)#1736
Open
chopratejas wants to merge 1 commit into
Open
test(codex-ws): de-flake semaphore-tail test (widen noise floor 25ms->75ms)#1736chopratejas wants to merge 1 commit into
chopratejas wants to merge 1 commit into
Conversation
…ke CI test_concurrent_compression_has_no_semaphore_tail takes p99 as the max of only ~12 concurrent samples, so one CI scheduler/GC outlier (observed 29ms / 22x on a shared runner) tripped the 25ms floor. The real regression signal is p99 ~2433ms (success criterion <250ms), so a 75ms floor absorbs the jitter while staying 3.3x below the regression threshold. Passes locally; no behavior change.
Contributor
PR governanceThis PR does not yet satisfy the required template fields:
Please update the PR body, or move the PR back to draft while it is still in progress. |
github-actions
Bot
added
the
status: needs author action
JerrettDavis
approved these changes
Jul 3, 2026
JerrettDavis
left a comment
JerrettDavis
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed the flake adjustment. The updated 75ms floor is still materially below the 250ms regression threshold and far below the historical semaphore-tail signal, while accounting for the small-sample max/p99 behavior on shared CI runners. Test-only change; this looks ready.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test_concurrent_compression_has_no_semaphore_tailfailed on main CI (shard 1) with p50=1ms, p99=29ms, ratio=22x. It's flaky, not a regression:p99is the max of only ~12 concurrent samples, so one CI scheduler/GC outlier dominates it.Widen the noise floor 25ms -> 75ms: absorbs CI jitter, still 3.3x below the 250ms regression threshold (a genuine hundreds-of-ms tail still trips it). Comment updated. No production code change.
Does not block the v0.29.0 publish (release.yml runs no pytest), but greens main CI.
🤖 Generated with Claude Code