feat(yjs): collaborative editing framework (base)#2785
Draft
JammingBen wants to merge 29 commits into
Draft
Conversation
JammingBen
force-pushed
the
feat/realtime-collaboration
branch
3 times, most recently
from
132aa9a to
035fead
Compare
…p apps Brings the realtime-collaboration PoC from opencloud-eu/web-extensions into the canonical web repo: - New `@opencloud-eu/web-pkg` component family at `src/components/Collaborative/`: `CollaborativeWrapper.vue` + `CollaborativeAdapter` type contract. Reads `useAuthStore` / `useConfigStore` via the existing composables barrel. - web-pkg deps gain `@hocuspocus/provider`, `yjs`, `y-protocols`, `semver` (+ `@types/semver` devDep). Editor-binding deps (`y-codemirror.next`, `@tiptap/y-tiptap`, `@tiptap/markdown`, `@tiptap/extension-collaboration`, `@codemirror/*`) stay with the consuming apps — web-pkg only ships the editor-agnostic plumbing. - Two new apps: `web-app-codemirror` + `web-app-tiptap`. Both are thin App.vue wrappers around `CollaborativeWrapper`, no own build config (web's central vite picks them up via the `web-app-*` convention). Adapter + editor component live alongside App.vue. - `realtimeUrl` prop is now three-state on the wrapper: `string` for an explicit URL, `null` for forced local-only mode, `undefined` (default) to derive from `configStore.serverUrl` plus the `/realtime` convention. Means collab is on out-of-the-box once the sidecar runs on the OC host. A first-class `options.realtimeUrl` field in OC's web config schema is a separate follow-up. - Tiptap StarterKit uses the v3 `undoRedo: false` option name (was `history: false` in v2) — clears the lingering `Partial<StarterKitOptions>` type error and keeps yUndoPlugin from `@tiptap/y-tiptap` as the collab-aware undo manager. - Registers codemirror + tiptap in `dev/docker/opencloud.web.config.json` `apps[]`. - Adds `REALTIME_COLLAB_MIGRATION.md` documenting the multi-phase plan (phases 0-1-2 done; 2.5, 3, 4, 4.5, 5 pending).
`CollaborativeAdapter.serialize` now takes a second `context: unknown` arg.
Editor components expose context via `defineExpose({ getAdapterContext() })`
and the wrapper grabs it through a template ref. Adapters cast `context`
to their expected shape and treat absence as "fall back to Y.Doc-only
serialisation".
TiptapEditor exposes `{ editor: editor.value }`. The markdown adapter now
calls `live.getMarkdown()` on the bound editor on each debounced serialise
instead of spawning a headless `new Editor({...})` (StarterKit + Markdown +
Collaboration setup) per keystroke. Dominant cost during interactive
editing, will matter a lot more once the text-editor refactor (Phase 4)
brings 4 content-type strategies × many extensions online.
CodeMirror returns no context. Its adapter's `Y.Text.toString()` is
already cheap and ignores the new arg.
Backwards-compatible signature change — `unknown` is the cleanest type
because a typed `editor: TiptapEditor` argument would make no sense for
the CodeMirror adapter or any future non-Tiptap adapter.
Headless-editor fallback inside the Tiptap adapter stays for the case
where no live editor is bound (e.g. stale-recovery on a peer that holds
the doc but never mounted a UI).
Brings the Hocuspocus collaboration sidecar from web-extensions into web as a new service. Same Dockerfile + patches + server.js as the PoC: - Node 22 image with `@hocuspocus/server` 4.0.0 patched to expose a `beforeHandleAwareness` callback (anti-spoof identity stamp on every inbound awareness update — server overwrites `user` with the authenticated identity) - Reuses the existing `/realtime` path-prefix on the `opencloud` Traefik entrypoint, so apps reach the sidecar at `wss://host.docker.internal:9200/realtime` (matches the wrapper's `serverUrl + /realtime` derive convention) - SQLite persistence on a named `hocuspocus-data` volume - `DEV_FAKE_TOKEN` is set so the integration vitest suite can bypass real OIDC tokens against the sidecar The sidecar's `OPENCLOUD_URL` env still points at `https://host.docker.internal:9200` because OC sits behind the same Traefik. `NODE_TLS_REJECT_UNAUTHORIZED=0` is the dev-cert workaround, gone in prod. Two follow-ups (tracked in REALTIME_COLLAB_MIGRATION.md): - Switch the etag probe in `server.js` from WebDAV HEAD to Graph `/items/{id}` for consistency with the existing Graph permissions call (investigate the "personal-drive 400" mentioned in the WebDAV-fallback comment). - Reject folder ids in `onAuthenticate` (currently we'd silently start a collab room for a folder, which has no file content to save).
Multiple editor apps (codemirror, tiptap, eventually the refactored text-editor) each pull yjs + y-prosemirror via their own dep tree. Without dedupe each app bundle ships its own Yjs instance, and at runtime the browser sees the "Yjs was already imported" warning plus broken `instanceof Y.Doc` / pluginKey identity checks across module boundaries. In the web-extensions PoC this manifested as `Cannot read properties of undefined (reading 'doc')` inside y-prosemirror's `createDecorations` when the cursor plugin (registered with one ySyncPluginKey instance) tried to read state attached by the Collaboration extension (registered with a different ySyncPluginKey instance from a different y-prosemirror copy). Solved upstream by switching the cursor wiring to y-tiptap's yCursorPlugin so both ends share a pluginKey, but the bundle-level duplication still slows page load and trips the constructor warning. Adding the deps to vite's `resolve.dedupe` forces a single instance per shared module across the entire app bundle. The Tiptap core / pm entries cover ProseMirror pluginKey identity checks across the various @tiptap/extension-* packages.
…eWrapper
Wires the existing text-editor through the realtime API. Every editor
instance now goes through a Y.Doc — single-user sessions still work via
the wrapper's local mode (standalone Awareness, immediate hydrate, no
provider).
Approach: extend the existing `useTextEditor` composable rather than
fork the whole text-editor app. The single new parameter `ydoc` is
optional and backwards compatible — the three other callers
(web-app-app-store AppDetails, web-app-files ListHeader and SpaceHeader
readme rendering) keep working without changes.
In collab mode `useTextEditor`:
- pushes `Collaboration.configure({ document: ydoc, field: 'default' })`
onto the strategy's extension list
- skips the initial `content` option (Collaboration paints from the
Y.Doc instead, which is hydrated by the wrapper)
- suppresses the `modelValue` → `setContent` watch (CRDT is the source
of truth — round-tripping would clobber peer edits)
All three Tiptap-based strategies (markdown / html / tiptap-json) flip
`StarterKit.configure({ undoRedo: false })` so the collab-aware
`yUndoPlugin` from `@tiptap/y-tiptap` (which the Collaboration extension
brings in) can take over without conflict. The read-only callers don't
use undo so the change is neutral for them. The plain-text strategy
already uses a minimal Document+Paragraph+Text+HardBreak set with no
StarterKit, no change needed.
text-editor app:
- New `src/adapters/textEditorAdapter.ts` — bridges any
`ContentTypeStrategy` to a `CollaborativeAdapter`. Hydration spawns a
detached Tiptap editor with the strategy's extensions plus
`Collaboration` to materialise content into the Y.Doc; serialisation
prefers the live editor surfaced via the wrapper's
`getAdapterContext()` channel (avoids per-keystroke headless spawn)
and falls back to a headless instance only when no UI is bound.
- New `src/TextEditorBinding.vue` — thin editor component the
CollaborativeWrapper mounts. Receives ydoc / awareness / provider /
isReadOnly from the wrapper and reads `contentType` via
provide/inject from App.vue (keeps the wrapper's editor-prop
signature free of text-editor specifics). Exposes
`getAdapterContext()` for the wrapper.
- `src/App.vue` is now a thin shell around `<CollaborativeWrapper>`,
same pattern as web-app-codemirror / web-app-tiptap.
- Direct deps for the type-only and runtime imports the new wiring
introduces (@hocuspocus/provider, @tiptap/{core,extension-collaboration,vue-3},
yjs, y-protocols).
`useContentStrategy` and the `ContentTypeStrategy` type are now part of
the public `@opencloud-eu/web-pkg/editor` exports so consumer apps can
build their own adapters.
`@tiptap/extension-collaboration` lands as a web-pkg direct dep (the
composable now imports it).
Unit test stubs the CollaborativeWrapper since the test can't mount a
real HocuspocusProvider chain; verifies App.vue still renders the
`.oc-text-editor` shell via the stub.
…t-editor
Different editor apps (codemirror, tiptap, text-editor) bind to
incompatible Y.Doc schemas — codemirror writes to a Y.Text, the
Tiptap-based ones to a Y.XmlFragment with different extension sets.
Sharing a Y.Doc room across them was producing broken hydrations and
app-version handshake lock-outs (each app has its own pkg.version and
they were trampling each other's `_oc_meta.appVersion`).
Wrapper gets an optional `documentPrefix` prop; the sessionKey /
provider URL now uses `${prefix}::${fileId}`. Each app's App.vue
passes its own application id (`codemirror`, `tiptap`, `text-editor`).
Same .md file opened in two different editors lands in two different
Hocuspocus rooms.
Sidecar `parseDocumentId` strips the `<scope>::` prefix before the
Graph permissions probe so ACL still targets the raw file id.
Cursor markers in text-editor: `useTextEditor` accepts an optional
`awareness` and registers a custom yCollaborationCursor extension
wired to `@tiptap/y-tiptap`'s yCursorPlugin (same pattern used in
web-app-tiptap, for the same reason — the upstream
`@tiptap/extension-collaboration-cursor@3.0.0` still imports from
unforked `y-prosemirror`, shipping a different ySyncPluginKey and
crashing on first paint). Cursor builder emits the canonical
`.collaboration-cursor__caret` / `__label` DOM, web-pkg ships matching
unscoped styles under `editor/styles/collab-cursor.css` so the caret
+ name label render correctly across every consumer (the bug the user
just hit — without the styles the default block-level <div> rendered
the name as a full-width background bar across the editor line).
`@tiptap/y-tiptap` promoted from transitive to direct web-pkg dep.
TextEditorBinding forwards awareness from the wrapper into the
composable.
Codemirror + tiptap e2e (9/9) stay green; the per-app prefix change
is invisible to them because they were already the sole occupants of
their old rooms.
Notes for later (see REALTIME_COLLAB_MIGRATION.md):
- Cross-app collab room shape revisit (per-app room vs per-app
fragment with shared awareness).
- text-editor read-only-fallback UX should be more obvious — a
reload-prompt banner is currently a single muted line in the
status strip that's easy to miss.
`useTextEditor` was unconditionally focusing the editor on mount when not readonly. With a bound Awareness this immediately publishes a selection at (0, 0) to every peer in the room — they see a phantom caret of ours sitting at the top of the doc before we've actually clicked into it. Gate the auto-focus on `!options.ydoc` so the single-user behaviour is unchanged and the collab path waits for a real user-driven selection before awareness emits. Note: this potentially regresses a11y for collab editors (keyboard / screen-reader users won't land in the editor on open). Tracked as a follow-up in REALTIME_COLLAB_MIGRATION.md.
`useTextEditor`'s onMounted previously did `focus()` whenever the editor wasn't read-only. That implicit policy collided with two things at once: 1. In collab mode it published a phantom caret at (0, 0) to every peer in the room before the local user had clicked into the editor. 2. The caller's intent was ambiguous — useTextEditor was making the UX decision for the consumer (when to put the cursor in the editor) instead of just building one. The composable now only mounts the editor and wires its transaction listeners. The auto-focus is gone entirely; every current consumer either renders read-only previews (web-app-app-store AppDetails, web-app-files ListHeader, SpaceHeader README rendering) and never wanted focus to begin with, or relies on user-driven click-to-edit (text-editor through CollaborativeWrapper). If a future caller wants focus-on-mount, they can call `instance.focus()` themselves. Replaces the `!options.ydoc` guard added in the previous commit, which was a confusing way to express "skip the focus that we now don't do at all anymore". All 5/5 codemirror + 4/4 tiptap e2e stay green; vue-tsc clean.
AppWrapper keeps `currentETag` private, set only when it loads or
saves itself. With realtime collaboration that's not enough: when
peer A in a Y.Doc room saves the file, peer B already has the new
content via CRDT sync — but B's AppWrapper.currentETag is stale.
B's next Ctrl+S / Save action / autosave PUT carries B's stale
`previousEntityTag` and the server 412s with a "file changed outside
this window" error. The user sees a popup for a conflict that
isn't really a conflict (content matches, etag doesn't).
Adds a Vue provide/inject contract on AppWrapper:
// AppTemplates/types.ts
export const appWrapperEtagSyncKey: InjectionKey<{
setCurrentETag(etag: string): void
}>
CollaborativeWrapper injects it (null when used standalone, no
regression) and pushes every `_oc_meta.etag` change through it. The
etag mirror watch already fires on both own-saves and peer-saves
(own writes go resource.etag → _oc_meta.etag, peer writes arrive
through CRDT). The setter is idempotent so the own-save round-trip
is a no-op.
Net effect: in a multi-peer text-editor session, anyone in the room
can save without colliding on the etag — the wrapper keeps everyone's
private AppWrapper.currentETag aligned with reality.
Doesn't address external writes (a non-collab sync client or another
editor app writing the file mid-session) — those bypass the CRDT, the
sidecar has to poll or be pushed. Tracked separately.
…ppWrapper The etag-sync inject in Phase 4.5 forwarded every `_oc_meta.etag` change to AppWrapper. That included the value the sidecar's SQLite extension applies during the first onSynced — which can be OLD if the file was modified externally between sessions or if the persistence just lagged behind. Sequence that broke a user's own first save: 1. AppWrapper loaded the file → currentETag = current native (NEW) 2. CollaborativeWrapper connected → onSynced fires 3. Persisted Y.Doc state arrives, including meta.etag = OLD and meta.isStale = true (sidecar's onLoadDocument flagged the drift) 4. metaObserver fires for both keys synchronously 5. My etag handler forwarded OLD → AppWrapper.currentETag = OLD 6. recoverFromStaleState (async, runs later) wipes + rehydrates + sets meta.etag back to NEW, but at this point the user has already clicked Save → PUT with previousEntityTag = OLD → server has NEW → 412 "file updated outside this window" Gate the forward on `meta.isStale !== true`. While stale-recovery is pending, the persisted etag is known-bad — let recovery clear the flag and write the native etag, then the next observer fire forwards the correct value. Doesn't affect the peer-save path (their writes don't carry isStale) or the own-save mirror (we never set isStale ourselves).
…aved etags" Revert "docs(realtime-collab): mark Phase 4.5 done" Revert "fix(realtime-collab): don't forward stale etag from initial sync to AppWrapper" The etag-sync-via-CRDT approach turned out brittle in practice. Pushing peer-saved etags through the inject + metaObserver path crossed too many race windows (warm-room persisted state, sidecar SQLite cruft, etag format differences between OC's endpoints) and produced false 412s on the user's OWN save in their OWN window. Reverting to a clean slate. A simpler refetch-on-412 path inside AppWrapper.saveFileTask follows in the next commit: when a save 412s, re-fetch the file, compare content, retry the save with the fresh etag if content matches (typical collab case — Y.Doc is already synced, only the etag tracking was stale). Avoids the wrapper having to second-guess what's in `_oc_meta` and when.
When putFileContents returns 412 or 409, the user's previousEntityTag no longer matches the server. In a collaborative session this usually means another peer just saved; the editor's Y.Doc state already has their edits propagated via CRDT, so the right response is to refetch the file, grab the fresh etag, and retry the save with our combined content. Falls back to the existing conflict popup only when the refetch or retry itself fails. Three outcomes: - Server content equals what we tried to save: the etag was stale but there is no real conflict. Reconcile silently, update currentETag and serverContent, no user-visible noise. - Server content differs (typical collab case after a peer save): retry the PUT with the fresh etag. Publishes our state on top. - Refetch or retry also errors: show the original conflict popup so the user can recover by copying their changes. Replaces the etag-sync-via-CRDT approach of the reverted Phase 4.5 patch; reaching across the wrapper into AppWrapper's private state turned out too brittle in practice.
Web's central vite.config bundles every web-app-* in a single build against one pnpm-resolved tree, so yjs / y-prosemirror / @tiptap/* are naturally deduplicated at build time. The dedupe entries added during the migration came from the web-extensions PoC where each app was an independent Module Federation remote and the duplication was real. External, federated apps shipped via extension-sdk would still re-bundle their own copies. That case needs a separate decision (externalizing the shared deps in the SDK vs forcing all collab to go through this wrapper); noted in REALTIME_COLLAB_MIGRATION.md. vue3-gettext stays in dedupe because it was there before this PR.
When peer A saves, peer B should not be prompted to save the same content and should not 412 on its next own save. Both fall out of one mechanism: the etag-mirror watch tags its meta-write with a `LOCAL_SAVE_ORIGIN` transaction origin, the meta-observer fans out to AppWrapper only when the change came in via CRDT (origin != local). Two emits, same shape as `update:currentContent`: - `update:serverContent` carries the freshly-saved content snapshot. AppWrapper sets `serverContent.value = value`, so `isDirty` (currentContent !== serverContent) flips false and the unsaved-changes modal stops firing on navigate. - `update:etag` carries the new etag. AppWrapper sets `currentETag.value = value`, so the next PUT's `If-Match` is current and we skip the 412 -> refetch -> retry recovery path. Also: AppWrapper's `saveFileTask` now updates the local `resource` ref after each successful save (happy path + 412-reconcile + 412-retry). Previously only `serverContent`, `currentETag` and the resourcesStore were touched -- `resource.value` stayed stale, so the etag-mirror watch in CollaborativeWrapper (keyed on `props.resource.etag`) never fired for the saver and the fan-out never reached the peer. Both emits self-heal: a wrong etag 412s, refetch+retry fixes it; a wrong serverContent snapshot gets corrected by the next currentContent emit. No race-prone gates required.
13 specs covering local + collab mode, hydration gating, debounced emit, internal-origin filtering, etag mirror + the Y.Doc-rebuild regression, and lifecycle teardown. HocuspocusProvider is mocked so the suite stays hermetic; useAuthStore / useConfigStore are mocked so we don't need pinia. A tiny inline adapter mimics codemirror's Y.Text-on-'content' shape -- the wrapper is adapter-agnostic, so any concrete implementation exercises the same contract. Also: update the migration plan to mark Phase 4.5 done with a note on the two actual mechanisms (refetch+retry on 412/409 + the new _oc_meta fan-out of `update:serverContent` / `update:etag`) that replaced the original etag-sync inject design.
Ports the 5 Playwright specs from the web-extensions PoC to web's cucumber suite plus a new cross-peer fan-out scenario covering the Phase 4.5 mechanism. 7 scenarios, 87 steps, ~21s run. Features: - codemirror-open: open + status + content, multi-file navigation - codemirror-save-back: type + Ctrl+S persists via WebDAV - codemirror-multi-user: shared file, peer caret on line N labelled by the server-stamped name, CRDT typing propagation - tiptap-empty-file: empty .md opens cleanly and accepts input - tiptap-open: Markdown rendered as rich text (headings, marks) - cross-peer-fan-out: A saves, B's next save uses the fresh etag (covers the new update:serverContent + update:etag emits) Shared infrastructure (small, only what the suite needs): - `support/objects/app-files/utils/collab.ts`: status-strip + per- editor content selectors, codemirror line + remote-caret helpers. - `cucumber/steps/ui/collaboration.ts`: realtime status check, content assertions, typing, Ctrl+S save, remote-caret-with-label assertion, WebDAV content assertion. - `support/api/davSpaces/getFileContentInPersonalSpace`: new helper for the post-save WebDAV round-trip assertion. Plumbing: - `cucumber/steps/ui/resources.ts`: extend the existing `opens file X via Y using the context menu` step's allowed viewers with `code-mirror` and `tiptap` (kebab-case of `appInfo.name`). - `cucumber.mjs`: scope the cucumber import glob to `tests/e2e/cucumber/**` and `tests/e2e/support/**` so a leftover playwright-mf scratch dir outside those paths doesn't break the runner.
Unifies dev and CI behind a single routing model: OC's reverse proxy WebSocket-upgrades incoming /realtime requests and forwards them to the hocuspocus sidecar via an `additional_policies` entry, the same pattern opencloud-music uses. Drops the Traefik labels on the dev hocuspocus service; CI now has parity without needing a Traefik sidecar of its own. Dev: - `dev/docker/opencloud/proxy.yaml`: add the /realtime route (alongside the existing radicale routes). - `docker-compose.yml`: remove the hocuspocus Traefik labels + depends_on, keep the service on the traefik network so OC can reach it by name. CI: - `tests/woodpecker/proxy.yaml`: new file, /realtime route only. - `.woodpecker.star`: cp proxy.yaml into OC's config dir during init, define `hocuspocusService()` (plain HTTP node service, no TLS), wire it into a new `collab` e2e suite that runs alongside OC. - `tests/woodpecker/config-opencloud.json`: register codemirror + tiptap so OC loads them. - Doc the unified routing in REALTIME_COLLAB_MIGRATION.md Phase 5. Verified locally: full 7/7 cucumber collab suite still green after removing the Traefik labels; OC's proxy handles the WS upgrade transparently.
CI surfaced two gaps from the migration: - Prettier formatting drift across 9 files (touched during the PoC port but not run through the project's prettier config). Reformat in place; no behavioural change. - vue-tsc TS7011 on the text-editor app stub: the inline strategy mock returned `() => []` for `extensions` / `editorActionGroups`, which TS infers as `() => any[]` and rejects under the strict config. Add explicit `: unknown[]` return annotations -- the shape doesn't matter to this test (it only asserts the wrapper mounts), it just needs to satisfy the interface. Local: prettier clean, `pnpm check:types` green, the affected unit specs (CollaborativeWrapper.spec.ts + app.spec.ts) still pass.
Adds a third collaborative editor app (alongside web-app-codemirror
and web-app-tiptap) backed by Excalidraw + y-excalidraw. Validates
the CollaborativeWrapper / CollaborativeAdapter contract against a
fundamentally different shape: Y.Array<Y.Map> with fractional-
indexing keys (vs Y.Text for codemirror and Y.XmlFragment for tiptap),
React-only library mounted via vanilla createRoot inside a Vue shell.
The wrapper itself, CollaborativeAdapter interface, types.ts, and
hocuspocus integration stay untouched. The app slots in additively.
Package layout (~420 LOC src + 70 LOC cucumber):
- src/index.ts: OC app registration + newFileMenu entry
- src/App.vue: CollaborativeWrapper shell + emit forwarders
- src/ExcalidrawEditor.vue: Vue shell mounting React via createRoot,
re-rendering on ydoc/awareness identity change (so file navigation
without app teardown still works). veaury was tried first but
v2.6.x crashes on React 19 inside __veauryMountReactComponent__.
- src/react_app/ExcalidrawCanvas.tsx: Excalidraw + y-excalidraw's
ExcalidrawBinding (element-level CRDT, awareness wiring, optional
Y.UndoManager). Test-only window.__excalidrawAPI hook for e2e to
read scene state without canvas DOM introspection.
- src/adapters/excalidrawAdapter.ts: CollaborativeAdapter impl —
hydrate JSON into Y.Array via y-excalidraw helpers + fractional-
indexing positions; serialize back to standard .excalidraw JSON
(interoperable with excalidraw.com, Obsidian plugin, mschneider82's
standalone, etc.).
Build wiring:
- vite.config.ts: @vitejs/plugin-react scoped to the package's tsx
files (Vue + React coexist cleanly, no extension overlap); react /
react-dom dedupe + alias to the package's installed copy so rolldown
resolves tunnel-rat/zustand's react peer; custom plugin to mirror
upstream @excalidraw/excalidraw/dist/prod/{fonts,locales,data} into
dist/excalidraw-assets/ so EXCALIDRAW_ASSET_PATH (set via
new URL('../excalidraw-assets/', import.meta.url) in the React canvas)
resolves correctly under any OC subpath deployment. vite-plugin-
static-copy's stripBase + rename flatten nested dirs when used with
**/* globs, so the mirror runs via writeBundle.
- Root tsconfig.json: jsx: react-jsx
- Root package.json: react + react-dom hoisted so pnpm's deep peer-
resolved dirs are reachable from the rolldown bundle.
OC app registration in dev/docker/opencloud.web.config.json and
tests/woodpecker/config-opencloud.json (apps[] += 'excalidraw').
Cucumber coverage (3 scenarios, all green):
- excalidraw-open: blank + pre-seeded scene hydrates from file
- excalidraw-multi-user: shape created by Alice via the imperative
API propagates to Brian's scene through the y-excalidraw binding
New cucumber steps (in collaboration.ts) that work against any
Excalidraw consumer: "should see the excalidraw canvas mounted",
"should see (N|at least N) elements in the excalidraw scene",
"adds a rectangle to the excalidraw scene via the API". The last
uses the window.__excalidrawAPI test hook because canvas pointer
interaction is brittle in Playwright.
Notes for later (informed by audit of y-excalidraw's source and
the Excalidraw CSP situation):
- y-excalidraw is element-level CRDT (whole-element overwrite on
concurrent same-element edits) by design — matches Excalidraw's
own cloud and the team's blog post on why field-level is hard.
- Excalidraw's font fallback chain always ends with esm.sh, so a
CSP that blocks esm.sh is fine as long as our self-hosted primary
URLs serve correctly. With the asset mirror in place the fallback
is never fetched.
- mschneider82/opencloud-excalidraw was reviewed as a reference for
the React-mount + UIOptions pattern. Our package is from-scratch
on our framework; will attribute mschneider in the PR description
and ask if more is wanted.
Stale-state detection used to be hocuspocus-only: its onLoadDocument
hook compared the persisted Y.Doc's etag against a fresh PROPFIND and
set _oc_meta.isStale = true when they drifted. Relay-only backends
(opencloud-yjs) cannot do this because they hold no persisted state
and never PROPFIND.
Moves the comparison into the wrapper's onProviderSynced: after sync,
if _oc_meta.etag (synced via CRDT from whichever peer joined first) is
non-empty and differs from props.resource.etag (just refetched by
AppWrapper), set isStale + nativeEtag exactly as the hocuspocus hook
would have. The existing metaObserver then fires recoverFromStaleState
with the existing election + rehydrate flow unchanged.
Fully drop-in compatible with hocuspocus: the server-side check still
fires earlier and an existing guard (if meta.get('isStale')) skips the
client-side branch. Verified by running the full cucumber suite against
both backends — 10/10 scenarios pass on each.
The two paths use byte-identical fields and comparison logic, so the
client check never produces a spurious trigger in hocuspocus mode.
…lite
@hocuspocus/server 4.1.0 ships the `beforeHandleAwareness` hook
natively (the very hook our patch added against 4.0.0), so:
- bump the dependency to ^4.1.0
- drop dev/docker/hocuspocus/patches/ entirely
- drop the Dockerfile's `apk add patch` + patch-apply step
- keep server.js's extension-hook on the payload-object form
(`async beforeHandleAwareness({ states, context, connection })`).
v4.1.0's TS types make the split explicit:
`Document.callbacks.beforeHandleAwareness` is positional
`(document, states, origin)` at the internal doc-level wire, but
extensions registered on the Server still receive the unified
payload object, the same contract our 4.0.0 patch already
exposed.
Same commit also drops the SQLite persistence layer. The relay-only
spike validated that file-backed collab (hydrate from
`currentContent`, no server-side Y.Doc snapshot) is sufficient for
our workloads. Carrying SQLite forward on hocuspocus added a
mount, a volume, an extension dep, and a stale-state code path that
we no longer rely on (the equivalent check moved to
CollaborativeWrapper.onProviderSynced).
- drop `@hocuspocus/extension-sqlite` dep
- drop the SQLite extension wiring in server.js
- drop DB_PATH env + the hocuspocus-data volume in docker-compose
- keep `onLoadDocument` commented out as scaffolding: if a future
persistence layer (extension-sqlite, redis, ...) is reintroduced
the cold-load etag-drift probe slots straight back in
Full cucumber collab suite (10/10) passes on the unpatched 4.1.0
build with the client-side stale-state path doing the work.
… config The CollaborativeWrapper only hydrates content from onProviderSynced, so an editor whose realtime URL is derived but unreachable (no sidecar) mounts empty. Deriving the URL by default meant every text file opened without a running hocuspocus showed blank content, in CI and in any sidecar-less deployment. Make collab opt-in instead: the editor apps now pass realtimeUrl null unless a realtimeUrl is set in their app config, so they hydrate locally by default. Internal apps only receive a config object through external_apps, so codemirror / tiptap / excalidraw move there with their realtimeUrl. The e2e config keeps text-editor local (no sidecar in those suites) and points the collab demonstrators at wss://opencloud:9200/realtime; the dev config wires all four to the dev sidecar. In local mode the wrapper now hydrates immediately instead of waiting on the 150ms peer-election timer. That wait only matters with a provider (to avoid two peers double-seeding); locally it just delayed first paint and lost the race against consumers that read editor content right after mount.
JammingBen
force-pushed
the
feat/realtime-collaboration
branch
from
035fead to
e1fbb9f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Subset of #2755 without the example apps we don't want to have in our monorepo.