Skip to content

fix(db): adapt /api/storage onto existdb-openapi db-core (held) - #824

Draft
joewiz wants to merge 4 commits into
eXist-db:developfrom
joewiz:feat/db-adapter-over-dbcore
Draft

fix(db): adapt /api/storage onto existdb-openapi db-core (held)#824
joewiz wants to merge 4 commits into
eXist-db:developfrom
joewiz:feat/db-adapter-over-dbcore

Conversation

@joewiz

@joewiz joewiz commented Jun 15, 2026

Copy link
Copy Markdown
Member

[This PR was co-authored with Claude Code. -Joe]

Summary

Rewrites modules/api/db.xqm as a thin adapter over existdb-openapi's roaster-independent db-core module. All resource CRUD and naming correctness is delegated in-process to db-core (imported by namespace — no HTTP hop, no re-auth; db-core runs as the request user), making existdb-openapi the single audited implementation of db resource handling. eXide stops carrying its own xmldb:store / doc() / encode logic.

The /api/storage surface and eXide's response shapes are unchanged, so the frontend is untouched.

Status — draft, held per the migration plan

This is intentionally a draft:

  • It imports http://exist-db.org/api/db-core, a public module that does not yet exist on openapi's develop. It is introduced by the db-core consolidation stack, still under review: existdb-openapi#55 (extracts db-core and registers it as a public XQuery module) and existdb-openapi#59 (the consolidated, binary-safe get-resource shape this adapter consumes — { path, binary, content, mime-type }, no meta=full, no runPath). Against the published openapi the import does not resolve, so CI is red until Missing Error for invalid logins  #55 and Allow saving of empty binary docs #59 land on openapi develop and eXide's openapi dependency floor is bumped to that release.
  • The cutover is held: adopting this adapter is also the resource-naming convention flip (eXide's full-encode it%27s.xml → db-core's fn:iri-to-uri form it's.xml), gated on the resource-naming contract decision ([bugfix] Speak decoded UTF-8 resource names on the wire existdb-openapi#54).

Before un-drafting: (1) eXist-db/existdb-openapi#55 and eXist-db/existdb-openapi#59 merged; (2) bump the http://exist-db.org/pkg/openapi dependency semver-min in expath-pkg.xml to the db-core-registering release; (3) resource-naming convention decision (eXist-db/existdb-openapi#54).

What changed

  • GET browsedbc:list, remapped to eXide's envelope/items (modepermissions, childrenitems, typeisCollection, modifiedlastModified, mime-typemime); db-core does pagination and per-item writable.
  • GET load (XML, non-download)dbc:get-resource for serialized content (serialization params translated, omit-xml-declomit-xml-declaration) plus dbc:properties for owner/group/mode/last-modified. The consolidated db-core's get-resource returns just { path, binary, content, mime-type } (no meta=full, no runPath), so metadata comes from dbc:properties and externalPath is computed eXide's context-path-aware way.
  • Binary load + ?download raw streaming stay eXide-side until db-core gains real binary transport (existdb-openapi#38), streamed via response:stream-binary to avoid double-escaping.
  • PUT / POST (create|copy|move|rename) / DELETE / PATCH → the matching db-core calls, remapped to { status: "ok"[, path] }. DELETE passes force=true (eXide's recursive semantics); db-core's protected-path guard (403) and copy/move overwrite guard (409) now apply. PATCH errors are classified by db-core (chmod/chown/chgrp → 403, set-mime → 400).
  • Drops the dead config: / dbutil: imports and the unused filter query param (declared and handled but never sent by the frontend).

Testing

Adds cypress/e2e/db_resource_adapter_spec.cy.js (9 tests) guarding the adapter paths the broader suite leaves unasserted: the metadata re-sourced from dbc:properties (both the db:load envelope and the properties dialog that consumes it), serialization params driven through a load, ?download for XML and binary, db:patch Apply, and resource-level delete.

These were mutation-verified: reverting db:load to read metadata from get-resource (the regression the consolidated contract would introduce if mishandled) fails the envelope/properties/patch assertions while serialization/download/store/delete stay green — so the guard demonstrably guards.

Verified on a disposable eXist bed running the consolidated existdb-openapi (0.9.7, from existdb-openapi#59) + roaster 1.12.0 + this build: the 9 new tests plus the genuinely adapter-relevant existing specs (file_save, dbmanager, binary_preview) are green (44/44), and the JS unit suite is 179/179.

joewiz and others added 4 commits June 10, 2026 18:16
…over held)

Rewrites modules/api/db.xqm as a thin adapter over existdb-openapi's
roaster-independent db-core module (imported by namespace — existdb-openapi is
already a declared dependency and registers db-core as a public XQuery module).
All resource CRUD + naming correctness is delegated in-process to db-core (no
HTTP hop, no re-auth; runs as the request user), making existdb-openapi the
single audited implementation. The /api/storage surface and eXide's response
shapes are unchanged, so the frontend is untouched.

- GET browse -> db-core:list, remapped to eXide's envelope/items (mode->
  permissions, children->items, type->isCollection, modified->lastModified,
  mime-type->mime); db-core does pagination + per-item writable.
- GET load (XML, non-download) -> db-core:get-resource(meta=full + serialization);
  eXide's serialization param names translated (omit-xml-decl->omit-xml-declaration).
  Binary load + ?download raw streaming stay eXide-side until db-core gains real
  binary transport (existdb-openapi#38). externalPath kept eXide's context-path
  -aware way, overriding db-core's runPath.
- PUT/POST(create|copy|move|rename)/DELETE/PATCH -> the matching db-core calls,
  remapped to {status:"ok"[,path]}. DELETE passes force=true (eXide's recursive
  semantics); db-core's protected-path guard (403) and copy/move overwrite guard
  (409) now apply. PATCH errors classified by db-core (chmod/chown/chgrp->403,
  set-mime->400).

Drops the dead config:/dbutil: imports and the unused `filter` query param
(declared + handled but never sent by the frontend). expand-xincludes is not
forwarded (unachievable for node-to-string serialization in eXist 7.0.0-beta3,
eXist-db/exist#3446; out of scope, Bug D).

Adapter compiles cleanly against a db-core-registered eXist (all 5 handlers,
namespace import resolves). HOLD cutover/merge: adopting this adapter is the
resource-naming convention flip (decoded paths -> db-core's iri-to-uri; it's.xml
not it%27s.xml), gated on the naming-contract decision, and full e2e (eXide
suite + awkward-name pass) runs on the integration instance (step 3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Binary ?download was returning roaster:response(200, $mime, util:binary-doc(...)),
which runs the value through roaster's serializer and emits its base64 TEXT — a
67-byte PNG came back as 92 bytes of base64 (corrupt). This is a pre-existing eXide
bug (the prior db.xqm had the same line), not a roaster bug: roaster's own test app
streams binary with response:stream-binary, the established pattern.

The binary download branch now does
  util:binary-doc($stored) => response:stream-binary($mime, ())
— raw bytes, the same path eXist's REST server uses. XML download still serializes
via db-core (a text string), so roaster:response stays correct there.

Verified on stock roaster (1.12.1, unpatched): PNG PUT then ?download round-trips
byte-identical; text/XML load unaffected. No roaster change needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ?download branch streamed binary docs raw (prior commit) but still returned
the XML/text case via roaster:response(200, $mime, $serialized-string). Under an
xml mime that runs the already-serialized string through roaster's serializer a
SECOND time, escaping the angle brackets (a downloaded XML file came back as
&lt;doc&gt;…). Now the serialized string (which still honors db-core's
indent/omit-xml-decl) is converted to bytes and streamed via response:stream-binary,
so it's emitted verbatim with the correct mime. Binary download and the non-download
editor load are unchanged.

Verified on a live eXist: XML ?download returns unescaped <doc>… as application/xml;
PNG ?download stays byte-identical; editor load still returns JSON content.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-core)

The consolidated db-core (existdb-openapi#59) drops meta=full and runPath:
get-resource now returns just { path, binary, content, mime-type }. db:load's
XML branch stops requesting meta=full and reads owner/group/mode/last-modified
from dbc:properties instead. store's { stored, created } and the client-derived
externalPath were already handled; the rest is doc-comment cleanup.

Add db_resource_adapter_spec.cy.js guarding the paths the suite left unasserted:
the re-sourced metadata (db:load envelope + the properties dialog that consumes
it), serialization params driven through a load, ?download for XML and binary,
db:patch Apply, and resource-level delete. Verified by mutation — reverting to
get-resource metadata fails the envelope/properties/patch assertions while
serialization/download/store/delete stay green.

Cutover remains held per the plan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant