fix(db): adapt /api/storage onto existdb-openapi db-core (held) - #824
Draft
joewiz wants to merge 4 commits into
Draft
fix(db): adapt /api/storage onto existdb-openapi db-core (held)#824joewiz wants to merge 4 commits into
joewiz wants to merge 4 commits into
Conversation
…over held)
Rewrites modules/api/db.xqm as a thin adapter over existdb-openapi's
roaster-independent db-core module (imported by namespace — existdb-openapi is
already a declared dependency and registers db-core as a public XQuery module).
All resource CRUD + naming correctness is delegated in-process to db-core (no
HTTP hop, no re-auth; runs as the request user), making existdb-openapi the
single audited implementation. The /api/storage surface and eXide's response
shapes are unchanged, so the frontend is untouched.
- GET browse -> db-core:list, remapped to eXide's envelope/items (mode->
permissions, children->items, type->isCollection, modified->lastModified,
mime-type->mime); db-core does pagination + per-item writable.
- GET load (XML, non-download) -> db-core:get-resource(meta=full + serialization);
eXide's serialization param names translated (omit-xml-decl->omit-xml-declaration).
Binary load + ?download raw streaming stay eXide-side until db-core gains real
binary transport (existdb-openapi#38). externalPath kept eXide's context-path
-aware way, overriding db-core's runPath.
- PUT/POST(create|copy|move|rename)/DELETE/PATCH -> the matching db-core calls,
remapped to {status:"ok"[,path]}. DELETE passes force=true (eXide's recursive
semantics); db-core's protected-path guard (403) and copy/move overwrite guard
(409) now apply. PATCH errors classified by db-core (chmod/chown/chgrp->403,
set-mime->400).
Drops the dead config:/dbutil: imports and the unused `filter` query param
(declared + handled but never sent by the frontend). expand-xincludes is not
forwarded (unachievable for node-to-string serialization in eXist 7.0.0-beta3,
eXist-db/exist#3446; out of scope, Bug D).
Adapter compiles cleanly against a db-core-registered eXist (all 5 handlers,
namespace import resolves). HOLD cutover/merge: adopting this adapter is the
resource-naming convention flip (decoded paths -> db-core's iri-to-uri; it's.xml
not it%27s.xml), gated on the naming-contract decision, and full e2e (eXide
suite + awkward-name pass) runs on the integration instance (step 3).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Binary ?download was returning roaster:response(200, $mime, util:binary-doc(...)), which runs the value through roaster's serializer and emits its base64 TEXT — a 67-byte PNG came back as 92 bytes of base64 (corrupt). This is a pre-existing eXide bug (the prior db.xqm had the same line), not a roaster bug: roaster's own test app streams binary with response:stream-binary, the established pattern. The binary download branch now does util:binary-doc($stored) => response:stream-binary($mime, ()) — raw bytes, the same path eXist's REST server uses. XML download still serializes via db-core (a text string), so roaster:response stays correct there. Verified on stock roaster (1.12.1, unpatched): PNG PUT then ?download round-trips byte-identical; text/XML load unaffected. No roaster change needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ?download branch streamed binary docs raw (prior commit) but still returned the XML/text case via roaster:response(200, $mime, $serialized-string). Under an xml mime that runs the already-serialized string through roaster's serializer a SECOND time, escaping the angle brackets (a downloaded XML file came back as <doc>…). Now the serialized string (which still honors db-core's indent/omit-xml-decl) is converted to bytes and streamed via response:stream-binary, so it's emitted verbatim with the correct mime. Binary download and the non-download editor load are unchanged. Verified on a live eXist: XML ?download returns unescaped <doc>… as application/xml; PNG ?download stays byte-identical; editor load still returns JSON content. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-core)
The consolidated db-core (existdb-openapi#59) drops meta=full and runPath:
get-resource now returns just { path, binary, content, mime-type }. db:load's
XML branch stops requesting meta=full and reads owner/group/mode/last-modified
from dbc:properties instead. store's { stored, created } and the client-derived
externalPath were already handled; the rest is doc-comment cleanup.
Add db_resource_adapter_spec.cy.js guarding the paths the suite left unasserted:
the re-sourced metadata (db:load envelope + the properties dialog that consumes
it), serialization params driven through a load, ?download for XML and binary,
db:patch Apply, and resource-level delete. Verified by mutation — reverting to
get-resource metadata fails the envelope/properties/patch assertions while
serialization/download/store/delete stay green.
Cutover remains held per the plan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[This PR was co-authored with Claude Code. -Joe]
Summary
Rewrites
modules/api/db.xqmas a thin adapter over existdb-openapi's roaster-independentdb-coremodule. All resource CRUD and naming correctness is delegated in-process to db-core (imported by namespace — no HTTP hop, no re-auth; db-core runs as the request user), making existdb-openapi the single audited implementation of db resource handling. eXide stops carrying its ownxmldb:store/doc()/ encode logic.The
/api/storagesurface and eXide's response shapes are unchanged, so the frontend is untouched.Status — draft, held per the migration plan
This is intentionally a draft:
http://exist-db.org/api/db-core, a public module that does not yet exist on openapi'sdevelop. It is introduced by the db-core consolidation stack, still under review: existdb-openapi#55 (extracts db-core and registers it as a public XQuery module) and existdb-openapi#59 (the consolidated, binary-safeget-resourceshape this adapter consumes —{ path, binary, content, mime-type }, nometa=full, norunPath). Against the published openapi the import does not resolve, so CI is red until Missing Error for invalid logins #55 and Allow saving of empty binary docs #59 land on openapidevelopand eXide's openapi dependency floor is bumped to that release.it%27s.xml→ db-core'sfn:iri-to-uriformit's.xml), gated on the resource-naming contract decision ([bugfix] Speak decoded UTF-8 resource names on the wire existdb-openapi#54).Before un-drafting: (1) eXist-db/existdb-openapi#55 and eXist-db/existdb-openapi#59 merged; (2) bump the
http://exist-db.org/pkg/openapidependencysemver-mininexpath-pkg.xmlto the db-core-registering release; (3) resource-naming convention decision (eXist-db/existdb-openapi#54).What changed
dbc:list, remapped to eXide's envelope/items (mode→permissions,children→items,type→isCollection,modified→lastModified,mime-type→mime); db-core does pagination and per-itemwritable.dbc:get-resourcefor serialized content (serialization params translated,omit-xml-decl→omit-xml-declaration) plusdbc:propertiesforowner/group/mode/last-modified. The consolidated db-core'sget-resourcereturns just{ path, binary, content, mime-type }(nometa=full, norunPath), so metadata comes fromdbc:propertiesandexternalPathis computed eXide's context-path-aware way.?downloadraw streaming stay eXide-side until db-core gains real binary transport (existdb-openapi#38), streamed viaresponse:stream-binaryto avoid double-escaping.{ status: "ok"[, path] }. DELETE passesforce=true(eXide's recursive semantics); db-core's protected-path guard (403) and copy/move overwrite guard (409) now apply. PATCH errors are classified by db-core (chmod/chown/chgrp → 403, set-mime → 400).config:/dbutil:imports and the unusedfilterquery param (declared and handled but never sent by the frontend).Testing
Adds
cypress/e2e/db_resource_adapter_spec.cy.js(9 tests) guarding the adapter paths the broader suite leaves unasserted: the metadata re-sourced fromdbc:properties(both thedb:loadenvelope and the properties dialog that consumes it), serialization params driven through a load,?downloadfor XML and binary,db:patchApply, and resource-level delete.These were mutation-verified: reverting
db:loadto read metadata fromget-resource(the regression the consolidated contract would introduce if mishandled) fails the envelope/properties/patch assertions while serialization/download/store/delete stay green — so the guard demonstrably guards.Verified on a disposable eXist bed running the consolidated
existdb-openapi(0.9.7, from existdb-openapi#59) + roaster 1.12.0 + this build: the 9 new tests plus the genuinely adapter-relevant existing specs (file_save,dbmanager,binary_preview) are green (44/44), and the JS unit suite is 179/179.