[No QA] Migrate GitHub Actions and scripts tests from Jest to bun:test - #98402
Draft
roryabraham wants to merge 16 commits into
Draft
[No QA] Migrate GitHub Actions and scripts tests from Jest to bun:test#98402roryabraham wants to merge 16 commits into
roryabraham wants to merge 16 commits into
Conversation
These 19 files test .github/actions, .github/libs, .github/scripts and scripts/ — build and CI tooling rather than app code. Grouping them in one directory lets every config that needs to treat them differently (Jest, tsconfig, ESLint, the Bun test runner, CI path filters) use a single glob instead of enumerating each file. Renamed to the *.test.ts suffix Bun's test runner discovers, so the whole directory can be passed to `bun test` without listing files. Pure move: no file contents changed.
Adds the plumbing that lets tests/tooling/ run under `bun test` instead of Jest: - tests/tooling/tsconfig.json type-checks the directory with @types/bun (bun:test's ambient types conflict with the @types/jest the root config uses), and the root tsconfig excludes it so nothing is checked twice with the wrong types. - tests/tooling/setup.ts is preloaded to default GITHUB_REPOSITORY, mirroring jest/setup.ts for the files Jest still owns. - `test:tooling` runs the whole directory with --isolate, which gives each file a fresh module registry, so module-level state and mock.module() calls can't leak between files the way they would in Bun's default single-registry mode. - Jest ignores the directory, ESLint parses it with the new tsconfig, knip treats the files as entry points (nothing imports them), and the Bun CI job runs the new script.
These 14 files exercise .github/actions and .github/libs, whose @actions/@octokit dependencies go ESM-only in their next majors. Running them under bun:test means those imports resolve natively, so no CJS shims are needed when the upgrade lands. Mechanical changes throughout: import the test globals from bun:test, drop the `@jest-environment node` docblocks (Bun has no jsdom to opt out of), and swap Jest's two-parameter mock generics for Bun's single function-type form. Two differences needed real changes: - Bun resolves @actions/* as real ESM, whose namespace exports are read-only live bindings, so `core.getInput = mock` no longer works; these now use jest.spyOn. That also removes the tests' dependency on @src/types/utils/asMutable. - Bun has no `jest.mock(path)` automock and no advanceTimersByTimeAsync. Auto-mocked modules are replaced with explicit per-function spies, and DeployChecklistUtils' retry tests get a small local helper that yields to the microtask queue before firing the timer. postOrReplaceComment also drops jest-when, which depends on Jest's internal expect-matcher state. Its per-argument stubbing is replaced with a plain inputs record per test that throws on an undeclared input, preserving jest-when's strictness about unexpected calls.
This is the only action still ending in `module.exports = run` while
using ESM `import` statements at the top. Babel rewrote the whole file
to CommonJS for Jest, so the mix went unnoticed; Bun loads it as real
ESM and refuses it outright ("Expected CommonJS module to have a
function wrapper"), which blocks migrating its test to bun:test.
Every other action already uses `export default run`. The rebuilt bundle
changes only the three corresponding lines, and its
`require.main === require.cache[...]` entry guard - the only thing that
actually invokes the action - is unaffected.
Completes tests/tooling/ with the five files the earlier commit left on Jest, so the whole directory now runs under one runner: - bumpVersion and Git relied on `jest.mock(path)`'s automock, which Bun has no equivalent of. They now replace `fs`/`fs/promises` and `child_process` with mock.module() before importing the module under test. --isolate keeps those replacements from reaching other files. - markPullRequestsAsDeployed drops its ActionUtils mock and exercises the real getJSONInput, since the mock only reimplemented it. That means core.getInput's stub has to return strings, as the real one does, and MOBILE_EXPENSIFY_PR_LIST has to be a declared input rather than one that throws and gets swallowed. - DeployChecklistUtils' retry tests replace advanceTimersByTimeAsync, which Bun lacks, with a helper that alternates between yielding to the microtask queue and advancing the clock until the call settles. It throws rather than hanging if the call is waiting on something else. - versionUpdater, failureNotifier and detectReactComponent needed only their bun:test imports.
- @types/bun types the assertion helpers more tightly than @types/jest did, so a few mocks needed adjusting: DeployChecklistUtils' redundant listForRepo mock is folded into the spy that wrapped it, expect.stringContaining (typed `any` by bun-types) becomes toContain, and the two octokit stubs that can't carry the endpoint statics get an explicit disable with the reason. - getPullRequestIncrementalChanges stubs unset inputs as '' rather than null, matching what core.getInput actually returns. - oxfmt sorts bun:test alongside the other test-runner imports, which also reorders the imports in the existing server/ bun tests. - jest-when and @types/jest-when are dropped now that postOrReplaceComment, their only consumer, no longer uses them.
Jest's CI job passes --silent, so this output was never visible before; `bun test` has no equivalent flag, and the code under test logs enough to bury the results (2200 lines of output for a 19-file run). Stub the console in the preload instead, behind TEST_VERBOSE.
- runWithFakeTimers now advances by the exact backoff schedule the code under test declares, restoring what the Jest version pinned; advancing by an arbitrary amount had made the tests pass for any delay. It also installs the fake timers inside the try, so a synchronous throw can't leave the clock frozen for later tests. - GithubUtils re-installs its core.getInput spy per test: getCommitHistoryBetweenTags' afterEach calls jest.restoreAllMocks(), which under Jest could not reach the plain assignment this replaced. - createOrUpdateDeployChecklist pins the clock. Its assertions re-derive today's date and compare it against the title the action stamped, and Jest's global fake timers used to make that deterministic. test:tooling also pins TZ=utc, as npm test does. - isDeployChecklistLocked drops its module restore, dead under --isolate, along with the comment claiming files share a module registry. - bunTests.yml watches tests/utils/**, which five of these files import. - Git's two `not.toContain(expect.stringContaining(...))` assertions were vacuous: toContain compares with === and ignores asymmetric matchers. - Comment fixes: the @actions/github context justification described an environment that doesn't hold in CI, and CIGitLogic's preamble still described Jest, which no longer runs it. It now also records that a failure cascades, since Bun's --bail can't be scoped to one file. - Tightened the four seatbelt counts that dropped.
Moving these files out of tests/unit/ took them out of test.yml, which preDeploy.yml calls on every merge to main - so the tests covering preDeploy's own machinery (isDeployChecklistLocked, the deploy checklist, markPullRequestsAsDeployed) stopped gating the deploy they guard. Give bunTests.yml a workflow_call trigger and add it to preDeploy alongside typecheck/lint/test. That also closes the same pre-existing gap for the server tests. Split bunTests.yml into two jobs so a server-test failure no longer hides the tooling results, and so the git-heavy tooling suite runs alongside the victory-chart-renderer build rather than after it. The rest is documentation, because nothing told a contributor that this directory exists or how to run it: - tests/tooling/README.md covers how to run the suite and a single file (the leading ./ and the flags are both load-bearing), the rule for what belongs here rather than in tests/unit/, and the bun:test/Jest API gaps that shaped these files. - README.md, tests/README.md and CLAUDE.md all described Jest as the only unit-test runner. - bunfig.toml's comment claimed Jest owns all of tests/. Also: typecheck.yml's paths filter listed tsconfig.json literally, so editing tests/tooling/tsconfig.json (or server/tsconfig.json) skipped typecheck entirely.
Adding the workflow_call trigger meant this group is now evaluated for pushes to main, where github.ref is the same for every merge - so two merges in quick succession would cancel each other's run. That is the bug #41936 fixed for lint and test, and it would be worse here: confirmPassingBuild only treats 'failure' as failing, so a cancelled bunTests would pass the gate silently. Use the same SHA-qualified group the other preDeploy workflows use. Also drop GithubUtils' core.getInput stub, which had no implementation and no assertions behind it - the tests that need core stub it themselves - and move useFakeTimers inside the try that restores them.
These four transitively import @actions/core at runtime, so they would have broken under Jest the moment the ESM-only upgrade landed. Moving them now means no Jest test can reach @actions/* or @octokit/* any more, which is the whole point of this PR: nothing in tests/unit/ needs a CJS shim built for it. Per-file notes: - artifactsResolver replaces child_process, githubCLI and GithubUtils with mock.module() in place of three jest.mock() automocks. Typing the paginate mock properly also retires two no-unsafe-type-assertion disables. - generateTranslations swaps `en` per test via a fresh mock.module() rather than a getter over a mutable variable: the script imports `en` as a default binding, which Bun resolves once at link time, so a getter only fires for the first test. Its Git stubs are now spies on the three static methods the script actually calls, rather than an automock of the whole class, and the diff fixtures gained the `diffType` the real Git.diff always sets - inert at runtime, but it means the fixtures now type-check against FileDiff. - ChatGPTTranslator drops its OpenAIUtils automock. Only `promptResponses` was ever stubbed, and the constructor it was hiding just stores a key. - createRetestRequestForCP needed only its bun:test imports. test:tooling now also preloads scripts/stubReactNative.js: generateTranslations reads src/languages/en, which pulls in react-native, whose Flow syntax Bun can't parse. Jest got away with it because Babel transformed react-native. This is the same stub `bun scripts/generateTranslations.ts` already runs with - bunfig.toml's top-level preload doesn't apply to `bun test`. Counts are unchanged: the four ran 63 tests under Jest and run 63 here, so tests/tooling is 23 files and 316 tests.
…orts Migrating generateTranslationsTest ran fine under bun:test (40/40), but it can't be type-checked there. tests/tooling type-checks with @types/bun, whose ambient JSX declarations conflict with the app's React and react-native types, and the script under test imports src/languages/en - which drags ~3,000 app files into the project and reports thousands of errors that the root project, with `jest` types instead of `bun`, does not. Confirmed both directions: main's tsconfig finds no errors in src/, and swapping only the `types` array reproduces them. The test side can't narrow that graph, because the import comes from the script. Moving it needs the pure logic extracted or a separate type-check strategy, so it stays on Jest here and still has to move before @actions/* goes ESM-only. Recorded in tests/tooling/README.md alongside the constraint itself, so the next person doesn't rediscover it. ChatGPTTranslator hit the same wall for a reason that *was* fixable: it imported Locale from @src/types/onyx/Locale, which re-exports the whole @src/CONST barrel. It now uses TranslationTargetLocale from @src/CONST/LOCALES - self-contained, and the type the API it exercises actually takes. The stubReactNative preload goes away with generateTranslations, since nothing else here reaches react-native.
Codecov Report✅ All modified and coverable lines are covered by tests. |
Three unrelated failures on this PR: - `verify` rebuilt isDeployChecklistLocked's bundle and found a diff. Main recently made the html-entities patch idempotent by folding the "worklet" directive onto the "use strict" line, but the committed bundle still embeds the old two-line form, so every PR branched off current main fails this check. Rebuilt against the current patch. Not introduced here - it reproduces on plain main - but it blocks this PR, so it's fixed here. - `Compare knip issues against main` reported three new `unlisted` findings, which were the same three pre-existing ones at their new tests/tooling paths. @octokit/request-error is imported directly by six files (three of these tests plus .github/libs/GithubUtils, .github/libs/isTeamMember and getPullRequestIncrementalChanges) but only ever resolved transitively through @actions/github, so declaring it resolves the finding rather than relocating it. - `spellcheck` flagged "Backoffs" and "bunfig". The first is gone: runWithFakeTimers' parameter is now expectedDelaysMs, matching the LIST_RETRY_DELAYS_MS constant it is pinned to. The second is a real filename, so it goes in the dictionary. The Android HybridApp job also failed, at "Load files from 1Password". That is infrastructure - the iOS job on the same runner passed, and nothing here touches the build - so there is nothing to fix for it.
Three conflicts, all where main independently reworked files this branch had migrated: - createOrUpdateDeployChecklist: main replaced the hand-built fake octokit with a real one from initOctokitWithToken plus spies, added createMock in place of `as unknown as` casts, and added the mockDeployChecklistIssuesByLabel helper. Resolved by taking main's file wholesale and re-applying this branch's bun migration onto it, rather than stitching nine hunks: the memfs mock.module, the dynamic import, explicit @actions/core spies, the pinned clock, and Bun's Mock<T> in place of jest.SpiedFunction/MockedFunction/mocked. Octokit's defaults/endpoint statics are now copied onto the mocks with Object.assign, since a Bun mock doesn't carry them and paginate reads them off the method. - isDeployChecklistLocked: main's changes were jest.mocked and typed requireActual, both of which the bun version already supersedes, so the conflicting hunks take this branch's side. Main's createMock improvements merged cleanly and are kept. - eslint.seatbelt.tsv: kept this branch's renamed paths, minus the three entries main deleted outright - its createMock rework removed the last no-unsafe-type-assertion violations in failureNotifier, createOrUpdateDeployChecklist and isDeployChecklistLocked. Confirmed by running ESLint, which neither errors nor re-tightens the baseline. Verified after the merge: 276 tooling tests pass, all four tsconfig projects are clean, ESLint and knip are clean, all 24 action bundles still reproduce, generateTranslationsTest still passes under Jest, and no Jest test reaches @actions/* or @octokit/*.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Explanation of Change
This is PR 2 of 4 from the bottom-up ncc → esbuild migration plan. PR 1 (Migrate scripts from ts-node to Bun) is already merged; PR 1 and PR 2 are independent, and PR 3 (upgrading
@actions/*and@octokit/*to their ESM-only majors) needs both.It migrates the tests covering our build and deploy tooling —
.github/actions/,.github/libs/,.github/scripts/andscripts/— from Jest tobun:test. That is every test whose import graph reaches@actions/*or@octokit/*bar one (see below), which is the bar that matters: those are exactly the tests that would otherwise need hand-built CJS shims in PR 3.Why
@actions/coreand@actions/githubare ESM-only from the versions PR 3 upgrades to. Jest resolves them through Babel's CommonJS interop, so keeping these tests on Jest would mean building and maintaining hand-written CJS shims for every@actions/*and@octokit/*package they touch (which is what the original all-in-one PR ended up doing, then had to undo). Bun imports them natively, so the shims are never needed.What changed:
tests/tooling/directory and renamed them to the*.test.tssuffix Bun's runner discovers. This is the main structural decision in the PR: every config that has to treat these files differently —jest.config.js,tsconfig.json, ESLint, knip, the npm script, the CI path filters — now uses a single glob instead of a 19-entry list that has to be updated by hand each time a test is added. The first commit is a pure move so the content diff is readable on its own.npm run test:tooling(TZ=utc bun test --isolate --preload ./tests/tooling/setup.ts ./tests/tooling).--isolategives each file a fresh module registry and a freshprocess.env, so module-level state andmock.module()calls can't leak between files. That removed the need for the cross-file cleanup hooks the earlier attempt at this needed.bun:test, drop the@jest-environment nodedocblocks, swap Jest's two-parameter mock generics for Bun's single function-type form. The differences that needed real work:@actions/*as real ESM, whose namespace exports are read-only live bindings, socore.getInput = mockno longer works — these usejest.spyOninstead. That also drops these tests' dependency on@src/types/utils/asMutable.jest.mock(path)automock, sofs,fs/promisesandchild_processare replaced with explicitmock.module()calls, and auto-mocked@actions/corebecomes per-function spies.advanceTimersByTimeAsync, soDeployChecklistUtils' retry tests get a small local helper that alternates between yielding to the microtask queue and advancing the clock by the exact backoff schedule the code under test declares.jest-whenand@types/jest-when. Its only consumer waspostOrReplaceComment, and it depends on Jest's internal expect-matcher state. Its per-argument stubbing is replaced with a plain inputs record per test that throws on an undeclared input — stricter than what it replaced, which silently returnedundefined.markPullRequestsAsDeployed.tswas the only action still ending inmodule.exports = runwhile using ESMimportat the top. Babel rewrote the whole file to CommonJS for Jest so the mix went unnoticed; Bun loads it as real ESM and refuses it outright. Changed toexport default runlike every other action. The rebuilt bundle changes only the three corresponding lines and itsrequire.main === require.cache[...]entry guard — the only thing that actually invokes the action — is unaffected.tests/unit/took them out oftest.yml, whichpreDeploy.ymlcalls on every merge tomain— so the tests covering preDeploy's own machinery (isDeployChecklistLocked, the deploy checklist,markPullRequestsAsDeployed) stopped gating the deploy they guard.bunTests.ymlnow has aworkflow_calltrigger and runs in preDeploy alongside typecheck/lint/test. That also closes the same pre-existing gap for theserver/Bun tests, which have never gatedmain. It's split into two jobs so a server-test failure doesn't hide the tooling results.tests/tooling/README.md(how to run the suite and a single file, what belongs in the directory, the bun:test/Jest API gaps), and updatedREADME.md,tests/README.md,CLAUDE.mdandbunfig.toml, all of which described Jest as the only unit-test runner.The one test that couldn't move, and why
tests/unit/generateTranslationsTest.tsstays on Jest. It runs fine underbun:test— I migrated it and got 40/40 — but it cannot be type-checked there, and the reason is a constraint worth knowing about before anyone else tries:tests/tooling/type-checks with@types/bun, whose ambient JSX declarations conflict with the app's React and react-native types.scripts/generateTranslations.tsimportssrc/languages/en, so the test's type graph pulls in ~3,000 app files, and checking those withbuntypes instead ofjesttypes produces thousands of errors the root project doesn't report. I confirmed it both ways: with main'stsconfig.jsonthere are zero errors insrc/, and swapping only thetypesarray reproduces them.The test side can't narrow that graph, because the import comes from the script. Moving it needs the pure logic extracted from the script, or a separate type-check strategy — either way its own PR. It still has to move before
@actions/*goes ESM-only, so PR 3 is not fully unblocked until it does. The constraint and the reasoning are recorded intests/tooling/README.mdso this isn't rediscovered from scratch.ChatGPTTranslatorhit the same wall for a reason that was fixable: it importedLocalefrom@src/types/onyx/Locale, which re-exports the whole@src/CONSTbarrel. It now usesTranslationTargetLocalefrom@src/CONST/LOCALES, which is self-contained and is the type the API it exercises actually takes.Also deliberately left alone:
test:bun(theserver/suite) is named on a different axis fromtest:toolingand is reallytest:server, but renaming it is unrelated churn.Test counts and timing
Identical before and after: 22 files, 276 tests in
tests/tooling, all passing. Verified per group against the Jest baseline — the 19 originally-planned files ran 253 tests under Jest and run 253 here; the three added later ran 23 and run 23.End-to-end the two runners are a wash, but that's because the suite is dominated by tests that wait on things neither runner controls. Splitting those out (macOS, M-series, 5 runs each, Jest with a warm
.jest-cache, same test counts on both sides):awaitStagingDeploys— polls on a real timerCIGitLogic— drives realgit/npmsubprocessesGithubUtils), the inner dev loopSo: ~6× on the part that is actually running test code, and no meaningful change on the ~52s of real subprocess work that dominates the total.
The CPU numbers are the more striking part. For that 17-file group Jest burns 37.7s of CPU across ~10 workers to produce 3.65s of wall clock; Bun uses 0.61s of CPU in a single process — roughly 60× less machine time for the same 242 tests. That's what actually costs money on a shared CI runner, and it's why the single-file case collapses from 1.35s to 60ms: there's no worker pool to spin up and no Babel transform to cache.
Two caveats worth stating: Jest parallelises across files and
bun test --isolateis sequential in one process, so Jest is being measured at its best here; and Jest's cold-cache numbers are worse (4.44s vs 1.35s for the single file), while Bun has no cache to warm.Fixed Issues
$
PROPOSAL:
Tests
npm installin a clean checkout.npm run test:toolingand confirm it reports253 pass,0 failacross 19 files.npm test -- tests/unit tests/actionsand confirm the Jest suite still passes and no longer collects the 22 migrated files (npx jest --listTests | grep -c tests/toolingprints0).npm run typecheckand confirm the newtests/tooling/tsconfig.jsonproject passes along with the other three.npm run lintandnpm run knipand confirm both are clean — in particular thatjest-whenis no longer reported as an unused dependency andtests/tooling/setup.tsis not reported as an unused file../.github/scripts/verifyActions.shand confirm it reports "Github Actions are up to date!" — i.e. the rebuiltmarkPullRequestsAsDeployedbundle matches its source.node .github/actions/javascript/markPullRequestsAsDeployed/index.jswith no inputs set and confirm it fails withError: Input required and not supplied: PR_LIST(the entry guard still fires) rather than doing nothing.tests/tooling/README.mddocuments —TZ=utc bun test --isolate --preload ./tests/tooling/setup.ts ./tests/tooling/GithubUtils.test.ts— and confirm it passes on its own.expectedBackoffsMsvalues intests/tooling/DeployChecklistUtils.test.tsfrom2000to1999, re-run, and confirm the test fails with "did its retry schedule change?" rather than passing or hanging.This PR also exercises several of the bundled actions on itself before merge, at least
isAuthorizedContributorandverifySignedCommits, andValidate GitHub Actionsruns the bundle build in CI.Offline tests
N/A - this PR only changes test tooling and CI configuration, not app code.
QA Steps
N/A - this PR only changes internal test/CI tooling; there is no end-user-facing or staging/production behavior to QA.
PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectionAvatar, I verified the components usingAvatarare working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
N/A - test tooling change only, no app UI affected.
Android: mWeb Chrome
N/A - test tooling change only, no app UI affected.
iOS: Native
N/A - test tooling change only, no app UI affected.
iOS: mWeb Safari
N/A - test tooling change only, no app UI affected.
MacOS: Chrome / Safari
N/A - test tooling change only, no app UI affected.