Added e2e runtime acceptance gate for console errors and pageerrors#29142
Added e2e runtime acceptance gate for console errors and pageerrors#291429larsons wants to merge 1 commit into
Conversation
ref https://linear.app/tryghost/issue/PLA-96 - the build-mode e2e suite boots the real production admin bundle, but no test fails on console.error / uncaught pageerror, so a view can throw at runtime while a navigation-only test stays green - exactly the class of bug that build-success and unit tests miss (and that a real bug slipped through) - adds a console-error/pageerror collector fixture (extends the Ghost fixture) with a small, explicit, commented allowlist for known-benign third-party noise - adds a smoke project/spec that navigates each major admin view (posts, editor, tags, members, comments, settings sections, activitypub, and the Tinybird-gated analytics overview + web/stats dashboard), waits for a key content locator, and asserts zero non-allowlisted runtime errors - authors the missing activitypub page object; the stats/analytics views reuse the existing analytics page objects - wires the smoke project into the analytics CI lane so Tinybird is present for the stats/analytics views; analytics views skip gracefully without it - bundler-agnostic and valuable independent of the Rolldown migration, so it targets main
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
| Command | Status | Duration | Result |
|---|---|---|---|
nx run @tryghost/admin-x-settings:test:acceptance |
✅ Succeeded | 9m 58s | View ↗ |
nx run ghost:test:ci:integration |
✅ Succeeded | 4s | View ↗ |
nx run ghost:test:integration |
✅ Succeeded | 2m 45s | View ↗ |
nx run ghost:test:legacy |
✅ Succeeded | 2m 54s | View ↗ |
nx run ghost:test:e2e |
✅ Succeeded | 2m 24s | View ↗ |
nx run-many --target=build --projects=tag:publi... |
✅ Succeeded | 1s | View ↗ |
nx run @tryghost/activitypub:test:acceptance |
✅ Succeeded | 46s | View ↗ |
nx run @tryghost/comments-ui:test:acceptance |
✅ Succeeded | 43s | View ↗ |
Additional runs (7) |
✅ Succeeded | ... | View ↗ |
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗
☁️ Nx Cloud last updated this comment at 2026-07-06 20:23:28 UTC
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #29142 +/- ##
==========================================
- Coverage 74.29% 74.26% -0.03%
==========================================
Files 1565 1565
Lines 135738 135738
Branches 16459 16452 -7
==========================================
- Hits 100845 100805 -40
- Misses 33864 33904 +40
Partials 1029 1029
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
E2E Tests FailedTo view the Playwright test report locally, run: REPORT_DIR=$(mktemp -d) && gh run download 28820101356 -n playwright-report -D "$REPORT_DIR" && npx playwright show-report "$REPORT_DIR" |
1 participant
ref https://linear.app/tryghost/issue/PLA-96
Problem
The build-mode e2e suite (CI
job_e2e_tests) boots the real production admin bundle in Docker, but no test fails onconsole.error/ uncaughtpageerror. A view can throw at runtime while a navigation-only test stays green — exactly the class of bug that build-success + unit tests miss, and that a real bug slipped through. Thestatsdashboard andadmin-x-activitypubhad thin/no real-build coverage.This gate is bundler-agnostic (part of the Rolldown migration workstream but valuable independent of it), so it targets
main.What this adds
Console-error / pageerror collector fixture —
e2e/helpers/playwright/console-errors.tsextends the Ghost base fixture with aruntimeErrorscollector wired to the authenticatedpage. Capturespageerror+console.error(React error boundaries log viaconsole.error, so boundary-swallowed throws are still caught). Small, explicit, commented allowlist for known-benign third-party noise. Failing tests print the captured errors with stacks.Smoke project/spec —
e2e/tests/admin/smoke/admin-views-smoke.test.ts. For each major admin view it navigates, waits for a key content locator (reusing existing Page Objects), and asserts (i) content rendered and (ii) zero non-allowlisted console errors / pageerrors. Covered: posts list, editor, tags, members (seeded), comments, settings (general + staff), activitypub, and the Tinybird-gated analytics overview + web/stats dashboard.Wiring — new
smokePlaywright project (excluded frommainso existing shards are unchanged); runs in the analytics CI lane (projects: analytics,smoke) so Tinybird is present for the stats/analytics views. Those viewstest.skip()gracefully whenGHOST_E2E_ANALYTICS !== 'true'.Missing page object — authored
ActivityPubPage(e2e/helpers/pages/admin/activitypub/); the stats/analytics views reuse the existing analytics page objects.Negative control
A skipped, annotated
negative controltest documents exactly how reintroducing a broken component makes the gate go red: break a view the suite renders (e.g.throwin the stats Overview render), rebuild the e2e image, run thesmokeproject — the shell may still mount, but the broken view emits apageerror/ Reactconsole.error,runtimeErrorsbecomes non-empty, andtoHaveLength(0)fails with the captured stack. Revert → green.Verification
Full smoke run against the live admin bundle requires the CI-only
ghost-e2e:localimage, so it's proven out in CI (draft). Locally, the novel collector logic was driven against a real Chromium page: a real uncaughtpageerroris captured with its stack, a realconsole.erroris captured, the allowlist drops third-party noise while keeping real crashes, andconsole.warn/console.logare ignored. Lint + types are clean.