Skip to content

feat(dev): local data persistence + seeding (state dir, fixtures, seed.ts, data pull/dump)#566

Open
netanelgilad wants to merge 5 commits into
mainfrom
claude/base44-cli-seed-feature-30br0e
Open

feat(dev): local data persistence + seeding (state dir, fixtures, seed.ts, data pull/dump)#566
netanelgilad wants to merge 5 commits into
mainfrom
claude/base44-cli-seed-feature-30br0e

Conversation

@netanelgilad

@netanelgilad netanelgilad commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Makes base44 dev local data persistent by default and adds a full seeding story — declarative fixtures, a programmatic base44/seed.ts hook with remote data-env access, and data pull/data dump snapshotting. Resolves the two loudest local-dev complaints (data wiped on every restart/entity edit; no way to establish test users with roles for RLS-gated apps) and follows the design in docs/proposals/local-data-and-seeding.md (researched against Supabase, Firebase, Wrangler, Convex, Prisma, PocketBase, Amplify).

What's in the box

Persistence (default on)

  • File-backed NeDB in a gitignored, project-relative .base44/data/ (worktree isolation for free); --fresh for a clean slate; app-id mismatch guard.
  • Entity-file edits now reload schemas and preserve data (previously dropAll() wiped everything).
  • .base44/dev.json instance descriptor (url, port, pid, per-instance admin token, seed state) + base44 dev status; groundwork for future dev ps/logs/inspect.

Seeding

  • base44/seed/users.jsonc — test app users with roles/passwords, created through the real local auth path (upsert by email; CLI login user always survives).
  • base44/seed/<entity>.jsonc — records with optional stable id (upsert) and created_by attribution for RLS-shaped data; filename↔entity resolution is case/dash/underscore-insensitive.
  • Auto-seed only when the data dir is new; base44 dev seed [--replace] on demand; base44 dev reset as the canonical wipe-and-rebuild; "seed files changed" hint on startup; never silently re-seeds.
  • Localhost admin endpoints (/_base44/dev/status|seed|reset|export) behind a per-instance token so commands work against a live server (with realtime events); offline they open the datastores directly (seed.ts runs via an ephemeral internal instance).

Programmable seeding + remote bridge

  • base44/seed.ts (Deno, exec-wrapper pattern): ctx.base44 = local service-role SDK client; ctx.remote({ dataEnv: "prod"|"dev" }) = SDK client against the linked app (dataEnv via SDK headers → X-Data-Env); ctx.log. Runs after fixtures; script stdio goes to stderr so --json stdout stays clean.
  • base44 data pull [--entity ...] [--data-env prod|dev] [--query <json>] [--limit <n>] [--out <dir>] [--force] — remote records → committed fixtures (read-only against remote, paginated).
  • base44 data dump — local data → fixtures ("freeze what I hand-crafted"); users excluded in v1.
  • All new commands honor global --json; destructive ops confirm in TTY and require --force non-interactively.

Scaffolding & docs

  • base44 create templates ship commented example seeds (users.jsonc, task.jsonc) and gitignore the state dir.
  • New docs/local-data.md topic guide; commands.md/testing.md/binary-distribution.md updates; proposal doc updated with the shipped surface; CHANGELOG entry under Unreleased.

Testing

  • 66 new tests across persistence, seeding, seed-script orchestration, and data pull/dump (dev-persistence, dev-seed, dev-seed-script, data-pull, data-dump, local-state, seed, seed-script specs) + a with-seed fixture project and a sanctioned seedScript test override.
  • Full suite: 651 passed, 12 failed — the 12 are Deno-dependent (dev.spec.ts functions tests, all of exec.spec.ts) and fail identically on clean main in this container (no Deno installed); verified via a baseline worktree run. Typecheck + Biome green. bun run build verified deno-runtime/seed.ts ships in dist/assets/.

Notes for reviewers

  • JWT helpers moved from src/cli/dev/dev-server/auth/tokens.ts to src/core/local-state/tokens.ts (old path is a re-export shim) so the core-layer seed-script runner can mint the local service token without importing cli/.
  • data pull reports total == pulled — the runtime records API returns bare arrays with no server total; human output notes "limit reached, more may exist" when a full page hits --limit.
  • Missing Deno with a seed.ts present: fixtures still apply, summary carries script: {ran:false} + warning, dev seed/dev reset exit non-zero (startup only warns — the server must not crash).
  • Related: this branch makes issue [Bug]: base44.asServiceRole in base44 dev depends on caller Authorization instead of a real service token #491 (asServiceRole in local dev) easier to close — the local service JWT plumbing now lives in core — but does not change the function-proxy header behavior.

Companion docs PRs (drafts until release): base44/skills#140, base44-dev/mintlify-docs#1582.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MdYwKUei4xxBZTZcBUmbaw

Generated by Claude Code

claude added 5 commits July 13, 2026 10:10
@claude
Add design proposal: local data persistence & seeding for base44 dev
41f2cfc 
Research-backed design for making the base44 dev entity DB persistent by
default (file-backed NeDB in a gitignored .base44/ state dir), adding
declarative seed fixtures (base44/seed/ incl. test users with roles),
a canonical dev reset / dev seed lifecycle, and a data pull/dump remote
bridge. Covers prior art (Supabase, Firebase, Wrangler, Convex, Prisma,
PocketBase, Amplify), agent-friendliness contract, and rollout phases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYwKUei4xxBZTZcBUmbaw
@claude
feat(dev): file-backed local data persistence, dev.json descriptor, a…
76cb9ba 
…nd dev status

Phase A of local data persistence & seeding for `base44 dev`:

- New core module `src/core/local-state/` (paths, Zod schemas, meta.json
  helpers, dev.json instance descriptor with pid-based stale detection).
- `Database` accepts `{ dataDir }` for file-backed NeDB collections
  (one `<name>.db` per collection under `.base44/data/`); in-memory mode
  unchanged when omitted. Bootstrap CLI user insert is now idempotent by
  email; datafiles are compacted after load.
- New `Database.reloadSchemas(entities)`: the entity watcher reloads
  schemas without dropping data (new collections created, removed ones
  detached but left on disk).
- Dev server writes `.base44/data/meta.json` ({formatVersion, appId,
  seed}) after load and refuses to start over another app's data unless
  `--fresh`; corrupt meta is warned about and treated as new.
- `.base44/dev.json` (appId, url, port, pid, dataDir, adminToken,
  startedAt, seed) written after listen, deleted on graceful shutdown.
- `base44 dev` is now a command group with a default action: new
  `--fresh` flag wipes the data dir before load; new `base44 dev status`
  subcommand reports {running, appId, url, port, startedAt, dataDir,
  seed} (JSON stdout under --json, human summary otherwise).
- Gitignore templates now exclude `.base44/`.

Tests: unit coverage for local-state; integration coverage for restart
persistence, watcher schema reload, --fresh, dev.json lifecycle, appId
mismatch guard, and dev status output. dev.spec.ts failures for
Deno-dependent tests are pre-existing in this container (Deno absent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYwKUei4xxBZTZcBUmbaw
@claude
feat(dev): seed fixtures, dev seed/reset commands, and local admin en…
30814b2 
…dpoints

Declarative seed fixtures (base44/seed/*.jsonc) for the local dev server:

- New core seed module (src/core/resources/seed/): Zod schemas for users
  and entity fixtures, readSeedFiles (seedDir config key, default "seed"),
  sha256 seed hash, and the seed summary / reset result shapes.
- Applier (src/cli/dev/dev-server/db/seed.ts): users upserted by email
  through the same document/credential helpers as local registration;
  entity records validated and stamped exactly like the entity POST route
  (service role, stable-id upserts, replace-mode truncation, per-file
  validation errors citing file + index, unknown-entity warnings, best-
  effort realtime create/update events).
- Auto-seed on dev startup when the data dir is new (first boot, --fresh,
  reset), "seed files changed" hint otherwise; seed state {hash, appliedAt}
  recorded in meta.json and dev.json. Startup seed failures log and never
  crash the server.
- Admin endpoints on the dev server (/_base44/dev/status|seed|reset),
  guarded by the per-instance x-base44-dev-admin token from dev.json.
- New commands: base44 dev seed [--replace] [--force] and base44 dev reset
  [--force] — use the running instance via the admin API or open the
  datastore offline; TTY confirmation for destructive modes, --force
  required non-interactively; --json emits the summary/reset document.
- backend-and-client template ships example base44/seed/users.jsonc and
  task.jsonc; new with-seed test fixture plus core + CLI test suites.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYwKUei4xxBZTZcBUmbaw
@claude
feat(dev): base44/seed.ts programmatic seeding and base44 data pull/dump
84bdae1 
Programmatic seed hook (base44/seed.ts):
- New Deno wrapper deno-runtime/seed.ts (exec.ts pattern): builds ctx with
  a local service-role SDK client, a remote({dataEnv}) client factory
  (X-Data-Env header for dev), and a stderr logger, then calls the script's
  default export.
- Core runner runSeedScript (src/core/seed-script/): spawns deno with piped
  stdio (stdout stays --json-clean), mints the local service JWT, fetches
  remote credentials tolerantly (ctx.remote() throws the reason when they
  are unavailable). Test seams: spawnImpl/wrapperPath + seedScript test
  override.
- JWT helpers moved to core (src/core/local-state/tokens.ts) with a
  re-export shim at the old dev-server path.
- Script step wired into all seed orchestration sites: dev-server startup
  (post-listen, single summary log, failures never crash the server), the
  admin seed/reset endpoints, and offline dev seed/reset (which now boot an
  ephemeral internal dev server — no dev.json, no startup auto-seed — and
  drive seeding through the admin endpoints). dev seed/reset exit non-zero
  when the script fails while keeping fixture results + warnings.

data commands:
- base44 data pull [--entity ...] [--data-env prod|dev] [--query <json>]
  [--limit <n>] [--out <dir>] [--force]: pages the remote runtime entities
  API (limit/skip, page size 500) and writes <kebab-entity>.jsonc fixtures
  with ids preserved; X-Data-Env: dev header for the dev data environment.
- base44 data dump [--entity ...] [--out <dir>] [--force]: live path via a
  new GET /_base44/dev/export admin endpoint, offline path straight from
  the NeDB files; _id stripped, user collection skipped (warn on request).
- Both honor --json ({entities: {Name: {pulled, total}}, wrote: [...]})
  and require --force (or TTY confirm) to overwrite existing fixtures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYwKUei4xxBZTZcBUmbaw
@claude
docs: local data & seeding topic guide, proposal status, changelog
8cae2c8 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYwKUei4xxBZTZcBUmbaw
@netanelgilad netanelgilad self-assigned this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@netanelgilad netanelgilad

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@netanelgilad @claude