Restructured Save/Upload output as a BIDS-Study tree

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 1.1.0
+
+#### 🚀 Enhancement
+
+- Save and Upload now write a BIDS-Study-shaped tree instead of a single ad hoc folder: the source video under `sourcedata/rawbids/`, the extracted clip or frame and anything rendered from it under `derivatives/clip-extractor/`, each named per BEP047 and described by its own sidecar and `dataset_description.json` ([#41](https://github.com/brain-bbqs/clip-extractor/pull/41))
+
 ## 1.0.0
 
 #### 🐛 Bug Fix

CLAUDE.md

@@ -6,6 +6,7 @@
 - Always bump the version in `package.json` appropriately when any file under `src/` (except `tests/`), `configs/`, `index.html`, or `package.json`/`package-lock.json` itself, is changed. Bump once per PR: if the version was already bumped by earlier work on the same PR/branch and it hasn't been merged yet, do not bump it again for follow-up commits on that same PR, keep adding entries under the existing top-most `CHANGELOG.md` heading instead
 - This project has no formal releases, so there is no `## Upcoming` staging section in `CHANGELOG.md`. Leave a short description of the change or addition directly under the top-most version heading (the same version just bumped in `package.json`; create the heading if it does not yet exist) under the appropriate subsection (`#### 🚀 Enhancement`, `#### 🐛 Bug Fix`, or `#### 🏠 Internal`); create the subsection if it does not yet exist; include the GitHub PR link at the end of each entry in the format `([#N](https://github.com/brain-bbqs/clip-extractor/pull/N))`
 - Keep `CHANGELOG.md` entries concise: one sentence each, naming what changed and what it means for someone using the app. Leave the reasoning, the measurements and the mechanism to the code comments and the PR
+- Prefer a single `CHANGELOG.md` entry per PR, describing the change as a whole at the level someone using the app would notice. Follow-up commits on the same PR should usually revise that entry rather than add another: a PR that ends up with a list of entries is almost always narrating its own development (including steps a later commit reversed) instead of stating where it landed. Add a second entry only for a genuinely separate change that happens to ride along. Field names, filename shapes, flag spellings and other such detail belong in the code comments and the PR description, not here
 - PR titles should be human-readable and in the past tense; they should NOT use conventional commit style
 - Keep PR descriptions short and to the point
 - End every PR description with the prompts that asked for the work, verbatim, inside a collapsed `<details>` block titled `Original prompt`; when follow-up prompts refine the same PR, append each one to that block

docs/README.md

@@ -2,20 +2,24 @@
 
 ## Live test injections
 
-| URL                                    | Expected Appearance                                      | Link                                                                               |
-| -------------------------------------- | -------------------------------------------------------- | ---------------------------------------------------------------------------------- |
-| `?test&signed_out`                     | Page as seen by a signed-out visitor                     | [Open](https://clip-extractor.brain-bbqs.org/?test&signed_out)                     |
-| `?test&num_datasets=0`                 | "Not added to any direct-upload datasets"                | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=0)                 |
-| `?test&num_datasets=1`                 | Single fake dataset, Upload enabled                      | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=1)                 |
-| `?test&num_datasets=2`                 | Dropdown of 2 fake datasets                              | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=2)                 |
-| `?test&num_datasets=1&embargoed=false` | Non-embargoed fake dataset: error card, Upload disabled  | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=1&embargoed=false) |
-| `?test&num_datasets=1&human_subjects`  | Flagged dataset: warning banner, blur tool, gated Upload | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=1&human_subjects)  |
-| `?test&mock_video`                     | A synthesized 30-frame clip loaded, as if dropped        | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video)                     |
-| `?test&mock_video=200`                 | Same, 200 frames, enough to see the trim track           | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video=200)                 |
-| `?test&mock_video_long`                | A 4-hour clip: the sliding-window timeline and width     | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video_long)                |
-| `?test&mock_video&mock_slp`            | Loaded clip with a synthesized, matching pose overlay    | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_slp)            |
-| `?test&mock_video&mock_slp&mismatch`   | Same, but refused: the SLEAP card's mismatch state       | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_slp&mismatch)   |
-| `?test&remote_listing=12`              | Browse pane, 12 fake videos across 3 fake datasets       | [Open](https://clip-extractor.brain-bbqs.org/?test&remote_listing=12)              |
+| URL                                              | Expected Appearance                                        | Link                                                                                         |
+| ------------------------------------------------ | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
+| `?test&signed_out`                               | Page as seen by a signed-out visitor                       | [Open](https://clip-extractor.brain-bbqs.org/?test&signed_out)                               |
+| `?test&num_datasets=0`                           | "Not added to any direct-upload datasets"                  | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=0)                           |
+| `?test&num_datasets=1`                           | Single fake dataset, Upload enabled                        | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=1)                           |
+| `?test&num_datasets=2`                           | Dropdown of 2 fake datasets                                | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=2)                           |
+| `?test&num_datasets=1&embargoed=false`           | Non-embargoed fake dataset: error card, Upload disabled    | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=1&embargoed=false)           |
+| `?test&num_datasets=1&human_subjects`            | Flagged dataset: warning banner, blur tool, gated Upload   | [Open](https://clip-extractor.brain-bbqs.org/?test&num_datasets=1&human_subjects)            |
+| `?test&mock_video`                               | A synthesized 30-frame clip loaded, as if dropped          | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video)                               |
+| `?test&mock_video=200`                           | Same, 200 frames, enough to see the trim track             | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video=200)                           |
+| `?test&mock_video_long`                          | A 4-hour clip: the sliding-window timeline and width       | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video_long)                          |
+| `?test&mock_video&mock_slp`                      | Loaded clip with a synthesized, matching pose overlay      | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_slp)                      |
+| `?test&mock_video&mock_slp&mismatch`             | Same, but refused: the SLEAP card's mismatch state         | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_slp&mismatch)             |
+| `?test&remote_listing=12`                        | Browse pane, 12 fake videos named `sub-01/ses-01/…` and up | [Open](https://clip-extractor.brain-bbqs.org/?test&remote_listing=12)                        |
+| `?test&mock_video&mock_ready&from_local&frame`   | Selected, described, Save/Upload enabled — no clicking     | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_ready&from_local&frame)   |
+| `?test&mock_video&mock_ready&from_local&snippet` | A marked range of a locally dropped video                  | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_ready&from_local&snippet) |
+| `?test&mock_video&mock_ready&from_ember&frame`   | A still frame of an archive-sourced `sub-01/ses-02` video  | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_ready&from_ember&frame)   |
+| `?test&mock_video&mock_ready&from_ember&snippet` | A marked range of that same archive-sourced video          | [Open](https://clip-extractor.brain-bbqs.org/?test&mock_video&mock_ready&from_ember&snippet) |
 
 **Safety**: `?test` alone, with none of the params below, is a no-op. Nothing here writes to real
 `localStorage`, nothing touches real sign-in tokens, and every fake id is chosen from a range no real
@@ -39,7 +43,28 @@ fixture, and loads it exactly as if it had been dropped onto the picker. It is t
 injection: the player, the timeline, the delivery panes and the SLEAP card are all only interesting
 once a video is on screen. Add `mock_slp` to also synthesize a matching pose model over it, drawn
 through the same overlay code a real `.slp` would be, or `mock_slp&mismatch` to make that pose
-describe a different recording, previewing the SLEAP card's mismatch refusal.
+describe a different recording, previewing the SLEAP card's mismatch refusal. Save works fully
+offline, so the resulting `.tar.gz` can be downloaded and unpacked to see the whole tree, without
+needing a real EMBER sign-in. Add `mock_ready` to skip the manual steps Save/Upload gate on entirely —
+marking a selection and typing a description — so the link lands directly on a saveable state, ready
+for one click, rather than the gated "describe it first" state `mock_video` alone previews (itself
+worth trying, since it is what a real visitor sees too). Pair it with the two choices below, the way
+the four links in the table do, rather than leaning on their defaults.
+
+**What a ready link previews**: `mock_ready` crosses two choices, and the four links in the table
+above are that grid spelled out. _Where the video came from_: `from_local` (the default) is the
+"dropped locally" case — no archive path, so the output falls back to `sub-unknown`, its derivatives
+sit in a `recording-<label>/` directory, and `dataset_description.json` gets no `SourceDatasets`
+entry, since a local file has no URL to name. `from_ember` instead makes the same mock video look as
+though it had been opened out of the archive at `sub-01/ses-02/…`, which is what a real Browse EMBER
+selection produces: a known subject and session, a `date-<label>_time-<label>/` directory, and a real
+`URL` in `SourceDatasets`. _What is selected in it_: `frame` (the default) marks a still frame, which
+needs no ffmpeg.wasm — and so no CDN — to extract; `snippet` marks a range instead, previewing the
+video sidecar's own `VideoCodec`/`VideoFrameCount` and, with `mock_slp`, a rendered overlay. Both
+land on real, mid-clip indices rather than the whole recording or the frame it opened on — frame 12,
+and frames 6–21, of `mock_video`'s own 30. Name others with `frame=<n>` or `snippet=<lo>-<hi>`; both
+are held to the loaded video's own bounds, so an index past the end lands on its last frame rather
+than off it.
 
 **A long recording**: `mock_video_long` (optionally `=N` for the duration in seconds, default 14400,
 4 hours) previews the sliding-window timeline a recording past half an hour gets (see
@@ -54,9 +79,10 @@ spreading across it.
 
 **Remote listing**: `remote_listing=N` fakes the EMBER browse pane's dataset/video listing with `N`
 fake video files spread across a handful of fake datasets, bypassing the real bucket listing and
-manifest reads, and switches straight to the Browse EMBER pane so the fake listing is what shows. It
-is read-only in spirit: the fake video URLs resolve nowhere real, so clicking a row shows the ordinary
-"cannot be opened" refusal.
+manifest reads, and switches straight to the Browse EMBER pane so the fake listing is what shows. Each
+video's own path is BIDS-entity-shaped (`sub-01/ses-01/…`, and up), the same structure a real
+dandiset's own asset paths carry. It is read-only in spirit: the fake video URLs resolve nowhere real,
+so clicking a row shows the ordinary "cannot be opened" refusal.
 
 ## Expected console noise
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "clip-extractor",
-  "version": "1.0.0",
+  "version": "1.1.0",
   "description": "A single-page video player for selecting a frame range and extracting it as an upload-ready payload, built on sleap-io.js and ffmpeg.wasm.",
   "type": "module",
   "license": "MIT",
@@ -24,12 +24,14 @@
     "@ffmpeg/util": "^0.12.2",
     "@talmolab/sleap-io.js": "^0.5.9",
     "h5wasm": "^0.10.3",
+    "hash-wasm": "^4.12.0",
     "mediabunny": "^1.52.3",
     "spark-md5": "^3.0.2"
   },
   "devDependencies": {
     "@chromatic-com/playwright": "^0.14.11",
     "@eslint/js": "^9.39.5",
+    "@ffmpeg/core": "^0.12.10",
     "@playwright/test": "^1.48.0",
     "@storybook/html-vite": "^10.5.3",
     "@types/node": "^22.9.0",

src/lib/archives.ts

@@ -189,6 +189,33 @@ export async function fetchDandisetName(dandiset: ArchiveDandiset, signal?: Abor
 }
 
 /** A video file in the archive, ready to be streamed. */
+/** Where in the archive a video came from — enough to name it again later without re-reading a
+ * manifest. Built by {@link archiveSourceOf} from a listing entry, and recorded in a delivery's
+ * derivatives `SourceDatasets` (see lib/provenance.ts's `buildSourceDatasetEntry`). */
+export interface ArchiveSource {
+  dandisetId: string;
+  /** The asset's own path within that dandiset, e.g. `sub-1/mice.mp4`. */
+  path: string;
+  /** The id of the blob its bytes are stored as, read off the bucket URL those bytes stream from
+   * (`…/blobs/<x>/<y>/<blob id>`). Null when that URL is not shaped that way, or is not known at all
+   * — an embargoed asset's is signed on demand rather than listed (see lib/embargoed.ts) — since a
+   * blob id guessed off some other URL shape would be worse than none. */
+  blobId: string | null;
+}
+
+/** The blob id a DANDI bucket URL ends in, or null for a URL not shaped like one. Held to a UUID so
+ * a differently-shaped URL yields nothing rather than whatever its last segment happened to be. */
+export function blobIdFromBucketUrl(url: string | null): string | null {
+  if (!url) return null;
+  const last = url.split("?")[0].split("/").filter(Boolean).pop() ?? "";
+  return /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(last) ? last : null;
+}
+
+/** Where one listed video came from, in the shape a delivery records it. */
+export function archiveSourceOf(video: ArchiveVideo): ArchiveSource {
+  return { dandisetId: video.dandisetId, path: video.path, blobId: blobIdFromBucketUrl(video.streamUrl) };
+}
+
 export interface ArchiveVideo {
   dandisetId: string;
   /** Path within the dataset, e.g. `sub-1/mice.mp4`. */