ci: test against the packages the eXist-db image bundles - #865
Conversation
The workflow started the container with `--volume $(pwd)/build:/exist/autodeploy`. A bind mount replaces the directory, so every package the image ships was discarded and CI deployed only three: the freshly built eXide, the pinned roaster-1.12.0, and the pinned existdb-openapi-0.9.5. That is not the stack anyone runs. The image bundles a newer existdb-openapi and Roaster plus dashboard, monex, functx, the documentation apps, packageservice, semver-xq and templating; eXide's own README documents `docker cp`ing the xar into the container, which layers onto that set. The divergence hid a real regression for weeks (eXide#821): the pinned 0.9.5 predates existdb-openapi's /api/query error-envelope regression, so CI stayed green while every user hit a query error that eXide could not display. Assemble the autodeploy set from the image's own packages with only eXide swapped for the build under test. Dropping the two `gh release download` steps also removes the pins, which is what forced a roaster >= 1.12.1 PR (eXide#814) to fail on a bed that supplied 1.12.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@joewiz yes this is a know problem of all our Ci configs. Luckily that means there is plenty of prior art for how to approach this. The basics principles are:
Using a volume with just the deps declared in expath-pkg ensures 1. But requires determining deps on CI either manually (what we have here) or by parsing expath-pkg.xml inside the workflow. For 2. one can just drop the whole volume from CI, but then one has to ensure that the app under test is actually the xar being compiled, and not the one inherited from the base image. using None of these solutions are great, they are all hacks around the shortcomings of auto deploys and its lack of dependency resolution. Extracting the xars from the image is not the right approach imv, it fails to satisfy 1. and it's just a long-winded way to tackle 2. |
[This PR was prompted by Joe, drafted by Claude Code, and reviewed by Joe.]
Problem
The workflow started the test container with:
docker run --volume $(pwd)/build:/exist/autodeploy:ro ... existdb/existdb:latestA bind mount replaces the directory, so everything the eXist-db image ships in
/exist/autodeploywas discarded and CI deployed exactly three packages: the freshly built eXide, plus the two the workflow downloaded —roaster-1.12.0andexistdb-openapi-0.9.5.That is not the stack anyone runs.
existdb/existdb:latestbundles:eXide's own README tells contributors to
docker cp ./build/*.xar exist-ci:exist/autodeploy, which layers onto that set — so the documented local procedure and CI have been testing different stacks.The pins are now actively broken
As of today's
existdb/existdb:latest, the pinnedroaster-1.12.0cannot serialize eXide's login response, soPOST /api/auth/sessionreturns:Every spec calls
cy.loginXHRfirst, so the whole suite fails — 100% of specs, on any branch. Isolated locally against the same image, with only the Roaster version changed:POST /api/auth/sessiondevelopdevelopdevelopdevelop's last green run predates this; the next push todevelopgoes red. This PR is the fix.Why the divergence mattered before that
This is also what made eXide#821 unresolvable for two months. The pinned
existdb-openapi-0.9.5predates the/api/queryerror-envelope regression, so CI stayed green while every user on a stock eXist-db 7 hit a query error eXide could not display. Reproduced on a clean container, same eXist-db build, same query:/api/queryresponse to1 + "oops"HTTP 500+{ code, line, column, description }HTTP 200+{ error: "Invalid context-item: …" }HTTP 400+{ code, message, line, column, raw }The pins also block PRs: eXide#814 requires Roaster >= 1.12.1 and cannot pass on a bed that supplies 1.12.0.
Change
Assemble the autodeploy set from the image's own packages, with only eXide swapped for the build under test:
The two
gh release downloadsteps go away with the pins. Copying the set out and mounting it (rather thandocker cping the xar in) keeps the deployment deterministic: exactly one eXide, even across a version bump.Merge ordering
CI is red right now regardless of this PR, so this is about the shortest path back to green rather than about avoiding a regression.
Merge eXide#840 first, then this. Against the bundled existdb-openapi 0.10.0,
query_error_structured_specfails on currentdevelop— the real incompatibility this PR stops hiding, and #840 is its fix. With #840 in, this PR takes the suite to fully green on the stack users actually run.Verification
Full 39-spec suite, clean container each run,
existdb/existdb:latest(7.0.0-SNAPSHOT, build 2026-08-18):develop+ bundled packages (openapi 0.10.0): 1 failing —query_error_structured, per the ordering note abovedevelop+ eXide#840 + bundled packages: query-error specs passdevelop+ the old CI pins (roaster 1.12.0, openapi 0.9.5, no other apps): all 38 specs fail at login, per the table above