fix(ci): stop the startup import budget guard flaking on Repo checks - #95853
fix(ci): stop the startup import budget guard flaking on Repo checks#95853posthog[bot] wants to merge 2 commits into
Conversation
test_no_new_heavy_imports_at_setup gates on a wall-clock 100ms threshold over `python -X importtime` self-time, and fails any name missing from setup_import_baseline.txt. The baseline only listed the expensive part of the startup path, so a slow or contended runner could inflate a long-standing import past the gate and fail the job. Complete the baseline with every name measured on the setup path today, cheap ones included, and run the timing capture in its own CI step instead of inside the eight-way parallel group that saturates the runner. Generated-By: PostHog Desktop Task-Id: 85fb332a-cede-4b03-ae48-7ea17f26b14b
🦔 PostHog Review reviewed this pull requestFound 0 must fix, 1 should fix, 0 consider. Published 1 finding (view the review). Resolved comments: 1 fixed |
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
🤖 CI report
|
| Project | Preview | Updated (UTC) |
|---|---|---|
| posthog.com | Open preview | Sep 7, 2026, 1:04 AM |
The preview should be ready in about 10 minutes. Open the preview at /handbook/engineering/.
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
There was a problem hiding this comment.
Not approved yet — waiting on the conditions below.
Re-add the stamphog label to request another review once you have addressed this.
Gates denied this PR for touching CI workflow files (deny-listed infra_cicd territory) and it entered T2-never; the diff is well-reasoned but there is zero independent review (no approvals, no substantive bot/human comments) to serve as assurance for a CI-behavior change.
- Modifies .github/workflows/ci-backend.yml and .depot/workflows/ci-backend.yml (deny-listed CI/infra territory) with no independent review or approval on the current head.
- Author is a machine/bot user, so ownership and familiarity signals don't apply as assurance.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✗ | matches: infra_cicd |
| size | ✓ | 45L, 2F substantive, 192L/4F incl. docs/generated/snapshots — within ceiling |
| tier | ✗ | classified as T2-never: T2-never (192L, 4F, cross-cutting, fix) |
| stamphog 2.0.0b4 | .stamphog/policy.yml @ 62f76b5 · reviewed head 62f76b5 |
|
PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏 |
![posthog[bot]](https://avatars.githubusercontent.com/in/1203049?s=60&v=4){kind=small}
The docs and the baseline header claimed the file lists every name on the setup path, with only sub-1ms names left out. Two importtime captures of a bare django.setup() show that holds for third-party and stdlib roots but not for first-party modules: about 190 posthog/products modules in the 1-5ms range are absent. State the real scope in both places, note the residual risk (a listed-nowhere first-party module needs roughly 20x inflation in both captures to reach the threshold), and say what to do when the guard fires on a module whose import predates the change under test. Add django_redis, the one third-party root at or above 1ms that the snapshot missed. Generated-By: PostHog Desktop Task-Id: 61864223-bd6c-4bb1-b2b5-f6b38f74b0ad
There was a problem hiding this comment.
Not approved yet — waiting on the conditions below.
Re-add the stamphog label to request another review once you have addressed this.
Gates denied this as CI/workflow tooling territory, and a reviewer bot's unresolved finding shows the new baseline's documented coverage claim is materially inaccurate.
- Deny-list gate failed: edits CI workflow files (.github/workflows/ci-backend.yml, .depot/workflows/ci-backend.yml)
- Tier gate failed: classified T2-never (cross-cutting CI fix)
- Unresolved reviewer finding: baseline still omits ~190 names above the stated 1ms cutoff, contradicting the docs
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✗ | matches: infra_cicd |
| size | ✓ | 45L, 2F substantive, 202L/4F incl. docs/generated/snapshots — within ceiling |
| tier | ✗ | classified as T2-never: T2-never (202L, 4F, cross-cutting, fix) |
| stamphog 2.0.0b4 | .stamphog/policy.yml @ bafbe24 · reviewed head bafbe24 |
Problem
Repo checksfailed then passed on a rerun of the same commit three times in the last seven days, andcancel-backend-on-repo-check-failurekills the rest of Backend CI when it fails, so each flake costs a full rerun and stalls the PR.test_no_new_heavy_imports_at_setup. It gates on a wall-clock 100ms threshold overpython -X importtimeself-time and fails any name missing fromsetup_import_baseline.txt.parallel:group, so the measurement was taken while seven other CPU-heavy checks saturated the runner.Changes
setup_import_baseline.txtcarries the full measured set, cheap entries included, so runner speed can no longer decide the outcome. 106 third-party and stdlib roots (chardet,posthoganalytics,pydantic, …) plus 20 first-party modules were added.The guard keeps its teeth: with
pydanticremoved from the baseline and the threshold at 20ms, it still fails with{'pydantic': '27ms'}.Backwards compatibility with unrebased branches
A workflow edit reaches every in-flight PR at once, so both steps degrade on a branch that predates the test.
-k "not no_new_heavy_imports_at_setup".-knever errors on a non-match, so a branch without the test just runs the whole file there.repo_invariantsfile layout.How did you test this code?
pytest posthog/test/repo_invariantspasses locally, 14 tests.pydanticfrom the baseline, set the threshold to 20ms, and it failed with the expected offender.hogli ci:preflightpasses, including the.depotshadow drift check. The shadow workflow carries the same change.Automatic notifications
Docs update
docs/internal/django-startup-time.mdnow records why the baseline lists cheap names and why this guard runs alone.🤖 Agent context
Autonomy: Fully autonomous
/writing-pr-descriptions,writing-simplified-technical-english.-kwas chosen over--deselectbecause--deselecterrors on a node id that does not exist, which would break unrebased branches.gh pr list --state open --search "setup_import_baseline"and"repo invariants"found no open PR for this.Created with PostHog Desktop from this inbox report.