|
| 1 | +# Typed host-callbacks across every platform |
| 2 | + |
| 3 | +Status: proposed |
| 4 | +Scope: `truapi-platform`, `truapi-codegen`, `truapi-server` (wasm + native), `@parity/truapi-host-wasm`, dotli, the iOS/Android host packages. |
| 5 | + |
| 6 | +## Problem |
| 7 | + |
| 8 | +A host (dotli web shell, iOS, Android) implements a *typed* capability surface, but the |
| 9 | +core delivers callback payloads as **SCALE-encoded bytes** at every binding boundary, so a |
| 10 | +bytes→typed adapter is wedged into each host: |
| 11 | + |
| 12 | +- **Web:** `WasmPlatform` (in `truapi-server/src/wasm.rs`) holds the typed Rust request, |
| 13 | + `.encode()`s it to a `Uint8Array`, and calls the JS callback. `createWasmRawCallbacks` |
| 14 | + (`js/packages/truapi-host-wasm/src/typed-callbacks.ts`, ~257 lines, hand-written) then |
| 15 | + SCALE-*decodes* it straight back into the typed object the host wanted. dotli wraps its |
| 16 | + typed handlers in `createWasmRawCallbacks(...)` at `bridge.ts:610`. |
| 17 | +- **Native:** the `#[uniffi::export(callback_interface)] trait HostCallbacks` |
| 18 | + (`truapi-server/src/native.rs`) takes `Vec<u8>` for every rich payload; `CallbackPlatform` |
| 19 | + re-encodes the typed value to bytes. Swift/Kotlin receive opaque `Data`/`ByteArray` and |
| 20 | + forward them to a WebView running `@parity/truapi` just to decode them. |
| 21 | + |
| 22 | +So a value the core already holds as a typed Rust struct is encoded, shipped as bytes, and |
| 23 | +decoded again on the far side — once per platform, in hand-written glue. The decode is the |
| 24 | +only reason a native host needs a SCALE codec at all. |
| 25 | + |
| 26 | +This is pure overhead: the wire protocol that *must* be SCALE is **product ↔ core**, not |
| 27 | +**core ↔ host**. The callback boundary is internal and free to carry native types. |
| 28 | + |
| 29 | +## Goal |
| 30 | + |
| 31 | +Each binding layer presents the `truapi-platform` traits as **native typed callbacks**, and |
| 32 | +the host implements them directly. No `createWasmRawCallbacks`, no dotli bridge wrap, no |
| 33 | +`Data`/`ByteArray` forwarding, no host-side SCALE codec. `truapi-codegen` owns the typed |
| 34 | +surface and the per-platform marshaling so none of it is hand-maintained. |
| 35 | + |
| 36 | +Non-goal: changing the **product ↔ core wire**. That stays SCALE, versioned, and |
| 37 | +language-agnostic (it is the sandboxed-dapp contract). Only the **core ↔ host callback** |
| 38 | +representation changes. |
| 39 | + |
| 40 | +## Principle: one typed source, projected per platform |
| 41 | + |
| 42 | +`truapi-platform`'s traits are the single typed source of truth. Each platform's binding |
| 43 | +generator projects them into native types from the same Rust definitions: |
| 44 | + |
| 45 | +``` |
| 46 | + truapi-platform traits (typed Rust: HostDevicePermissionRequest, ...) |
| 47 | + │ |
| 48 | + ┌─────────────────────────┼─────────────────────────┐ |
| 49 | + ▼ ▼ ▼ |
| 50 | + wasm-bindgen UniFFI truapi-codegen |
| 51 | + typed JsValue uniffi::Record/Enum generated TS interface |
| 52 | + (web host) (Swift / Kotlin host) (host author types) |
| 53 | +``` |
| 54 | + |
| 55 | +The product↔core wire stays SCALE on both transports (MessagePort/WebSocket frames). |
| 56 | + |
| 57 | +## Callback inventory and depth of change |
| 58 | + |
| 59 | +Three classes, from the platform-trait audit (`truapi-platform/src/lib.rs`): |
| 60 | + |
| 61 | +| Class | Callbacks | Platform trait today | Change needed | |
| 62 | +|-------|-----------|----------------------|---------------| |
| 63 | +| **Already typed in Rust** | `device_permission`, `remote_permission`, `push_notification`, `feature_supported`, `subscribe_theme`, `auth_state_changed`, `cancel_notification` | typed (`HostDevicePermissionRequest`, …) | **binding layer only** — stop re-encoding to bytes; project the typed value | |
| 64 | +| **Opaque even in Rust** | `confirm_*` family (sign payload, sign raw, create tx, account alias, resource allocation) | `review: Vec<u8>` | **deeper** — thread the typed review struct (`HostSignPayloadData`, `ProductAccountTxPayload`, `AllocatableResource`, …) through the platform trait + core runtime | |
| 65 | +| **Genuinely opaque** | storage read/write/clear, session read/write/clear, preimage submit/lookup, chain genesis hash | `Vec<u8>` / `String` | **none** — raw bytes the host stores or echoes, never renders. Stay bytes. | |
| 66 | + |
| 67 | +All rich payload types derive only `Encode`/`Decode` with no unbounded generics, so they map |
| 68 | +cleanly to `uniffi::Record`/`uniffi::Enum` and to the existing TS shapes (verified against |
| 69 | +`v01/permissions.rs`, `v01/notifications.rs`, `v01/signing.rs`, `v01/transaction.rs`, |
| 70 | +`v01/resource_allocation.rs`). |
| 71 | + |
| 72 | +## Web (wasm-bindgen) |
| 73 | + |
| 74 | +**Decision — keep SCALE on the wasm callback boundary; codegen *emits* the JS decode |
| 75 | +adapter.** The JS side already carries the SCALE codec: the product↔core wire needs it, and |
| 76 | +`@parity/truapi` already ships codegen-emitted `.dec`/`.enc` for exactly these `v01` types. |
| 77 | +The callback payloads *are* those wire types, so a thin generated decoder (`.dec` → typed |
| 78 | +handler → encode result) reuses tested infrastructure, leaves `wasm.rs` shipping |
| 79 | +`request.encode()` essentially unchanged, and introduces no second representation of any type. |
| 80 | + |
| 81 | +This is "move today's hand-written `typed-callbacks.ts` into `truapi-codegen`." Rejected |
| 82 | +alternative — Rust→`JsValue` converters (generalizing the hand-rolled `auth_state_to_js`): |
| 83 | +that builds a parallel conversion layer in `wasm.rs` that must shape-match the TS types, more |
| 84 | +code and more drift surface, for a purely cosmetic "no bytes on the boundary" win. Reusing the |
| 85 | +codec the JS client already has is cleaner and lower-risk. |
| 86 | + |
| 87 | +`createWasmProvider` applies the generated adapter *internally*, so the host passes its typed |
| 88 | +`HostCallbacks` object directly — `createWasmRawCallbacks` and the dotli `bridge.ts:610` wrap |
| 89 | +are both deleted; `WasmRawCallbacks` becomes an internal generated type. Subscriptions |
| 90 | +(`subscribe_theme`, `subscribe_session_store`, `lookup_preimage`) keep the `sendItem` push |
| 91 | +model — that plumbing (`driveResultStream`, `JsSubscriptionStream`) is genuine runtime |
| 92 | +support, not an adapter, and stays as a small hand-written module the generated code calls. |
| 93 | + |
| 94 | +The asymmetry with native is deliberate and principled: **web reuses SCALE because JS has the |
| 95 | +codec; native uses typed mirrors because Swift/Kotlin do not.** Same goal (host implements |
| 96 | +typed callbacks, no hand-rolled glue), realized with each platform's cheapest mechanism. |
| 97 | + |
| 98 | +### Symmetric SCALE boundary (enables a uniform generated adapter) |
| 99 | + |
| 100 | +Today the WASM raw boundary is *asymmetric*: the request crosses as SCALE bytes but the |
| 101 | +response comes back as a raw scalar (`invoke_bool` → `bool`, `invoke_u32` → `number`), and the |
| 102 | +core re-wraps it (`HostDevicePermissionResponse { granted }`). Reproducing that in codegen |
| 103 | +would require the generator to know each response type's single field (`.granted`, `.id`) — |
| 104 | +but those response types live in `@parity/truapi`, not in the platform crate the generator |
| 105 | +parses, so it cannot see them. |
| 106 | + |
| 107 | +Make the boundary **symmetric**: every raw callback exchanges the SCALE-encoded `ok` type in |
| 108 | +both directions (`bool` and structs alike encode via SCALE; `()` carries nothing; streams |
| 109 | +carry encoded items). `wasm.rs` decodes the response bytes into the typed `ok`. The generated |
| 110 | +adapter then has one uniform shape, derivable purely from the trait signature: |
| 111 | + |
| 112 | +```ts |
| 113 | +devicePermission: async (payload) => |
| 114 | + HostDevicePermissionResponse.enc(await host.devicePermission(HostDevicePermissionRequest.dec(payload))), |
| 115 | +``` |
| 116 | + |
| 117 | +Cost: `wasm.rs`'s `invoke_*` helpers and the `WasmRawCallbacks` return types change to |
| 118 | +bytes. That is the e2e-risk surface, so it lands and is verified before `confirm_*`/native. |
| 119 | + |
| 120 | +### confirm_* union review types |
| 121 | + |
| 122 | +`confirm_sign_payload` and `confirm_sign_raw` are each invoked from **two** runtime sites with |
| 123 | +**different** v01 types (standard `HostSignPayloadRequest` and |
| 124 | +`HostSignPayloadWithLegacyAccountRequest`), unified today only because both are erased to |
| 125 | +`Vec<u8>`. To type them, introduce a review enum per confirm method, e.g. |
| 126 | + |
| 127 | +```rust |
| 128 | +pub enum SignPayloadReview { Standard(HostSignPayloadRequest), LegacyAccount(HostSignPayloadWithLegacyAccountRequest) } |
| 129 | +``` |
| 130 | + |
| 131 | +The runtime wraps `inner` in the appropriate variant instead of `inner.encode()`; the |
| 132 | +`UserConfirmation` trait takes the review type; `wasm.rs`/`native.rs` project it. The single- |
| 133 | +site confirms (`confirm_create_transaction` → `ProductAccountTxPayload`, |
| 134 | +`confirm_resource_allocation`, `confirm_account_alias`) take their v01 type directly. |
| 135 | + |
| 136 | +## Native (UniFFI) |
| 137 | + |
| 138 | +Generalize the existing `AuthState`/`SessionUiInfo`/`HostTheme` mirror pattern in `native.rs` |
| 139 | +to every rich payload: a `uniffi::Record`/`uniffi::Enum` mirror of the `v01` type plus an |
| 140 | +`impl From`, used directly in the `HostCallbacks` trait signature in place of `Vec<u8>`. |
| 141 | +`CallbackPlatform` converts the typed platform value into the mirror instead of `.encode()`. |
| 142 | +UniFFI then generates idiomatic Swift enums/structs and Kotlin data classes; the |
| 143 | +`HostBridge` protocol/interface becomes typed and the `Data`/`ByteArray` forwarding is |
| 144 | +removed. The Swift/Kotlin `HostCallbackAdapter` pass-throughs become typed pass-throughs. |
| 145 | + |
| 146 | +**Decision — generate the mirrors:** `truapi-codegen` emits the `uniffi::Record`/`Enum` |
| 147 | +mirror types + `From` impls into `truapi-server`, so `native.rs` no longer hand-maintains |
| 148 | +them. (If emitting Rust into another crate proves awkward in step 1, the fallback is to keep |
| 149 | +the mirrors hand-written in `native.rs` following the proven existing pattern — this does not |
| 150 | +block the web path or the definition of done.) |
| 151 | + |
| 152 | +## Codegen changes (the heart of "codegen exposes rich callbacks per platform") |
| 153 | + |
| 154 | +`truapi-codegen` already parses `truapi-platform` rustdoc JSON and emits the typed TS |
| 155 | +`HostCallbacks` interface (`ts/host_callbacks.rs`, `platform.rs`). Extend it to also emit: |
| 156 | + |
| 157 | +1. **TS interface**: the typed `HostCallbacks` interface (already emitted); unchanged. |
| 158 | +2. **TS adapter (web)**: the SCALE-decode adapter (today's hand-written `typed-callbacks.ts`), |
| 159 | + emitted from the trait surface — `.dec` the payload, call the typed handler, encode the |
| 160 | + result. Consumed internally by `createWasmProvider`. |
| 161 | +3. **native mirrors**: `uniffi::Record`/`Enum` + `From` for each rich callback type, consumed |
| 162 | + by `native.rs`. |
| 163 | + |
| 164 | +Swift/Kotlin types themselves remain UniFFI-generated (not codegen) — UniFFI already projects |
| 165 | +the mirror types into both languages. |
| 166 | + |
| 167 | +## What stays SCALE |
| 168 | + |
| 169 | +- product ↔ core wire frames (MessagePort/WebSocket) — unchanged. |
| 170 | +- the genuinely-opaque callback payloads (storage/session/preimage bytes, genesis hash). |
| 171 | + |
| 172 | +## PR distribution (force-push authorized) |
| 173 | + |
| 174 | +| Change | Lands in | |
| 175 | +|--------|----------| |
| 176 | +| platform trait: typed `confirm_*` reviews | PR1 `01-core-runtime` | |
| 177 | +| codegen: emit wasm converters + native mirrors | PR1 | |
| 178 | +| `truapi-server` wasm.rs + native.rs typed marshaling | PR1 | |
| 179 | +| `@parity/truapi-host-wasm`: delete `createWasmRawCallbacks`, collapse `WasmRawCallbacks` | PR1 | |
| 180 | +| regenerated TS + generated Rust | PR1 | |
| 181 | +| dotli: drop the adapter wrap, typed handlers (submodule commit + pointer bump) | PR1 | |
| 182 | +| iOS/Android `HostBridge`: typed protocol/interface | PR2 `02-mobile-bindings` | |
| 183 | +| this design note | PR3 `03-docs` | |
| 184 | + |
| 185 | +## Verification / definition of done |
| 186 | + |
| 187 | +`make e2e-dotli` exercises only the **web** path (dotli web shell + playground in a browser, |
| 188 | +core as WASM). So: |
| 189 | + |
| 190 | +- **Web path must work end-to-end** — `make e2e-dotli` green (diagnosis flow: auth, |
| 191 | + permissions, session, notifications, theme, storage). |
| 192 | +- **Native path must compile and bind** — `cargo build --workspace`, `make uniffi`, and the |
| 193 | + host-packages Android/iOS assembles in CI. Its runtime isn't covered by `make e2e-dotli`. |
| 194 | +- Full Rust suite (fmt/clippy/test), TS package tests, playground build, and the codegen |
| 195 | + golden tests all green. |
| 196 | + |
| 197 | +## Risks |
| 198 | + |
| 199 | +- **Shape match (web):** the generated Rust→JS converter must produce exactly what the |
| 200 | + generated TS type expects. Mitigated by emitting both from codegen and covering them with |
| 201 | + the existing golden tests. |
| 202 | +- **`confirm_*` depth:** typing the reviews touches the core runtime that currently passes |
| 203 | + encoded review bytes. If this balloons, the reviews can ship typed in a follow-up while the |
| 204 | + already-typed callbacks land first; the web definition of done does not require the |
| 205 | + `confirm_*` reviews to be typed (dotli's confirm UI keeps working on the existing payload). |
| 206 | +- **dotli is a second repo:** the host-side change is a dotli commit + submodule bump. The |
| 207 | + truapi-side typed surface must land first (or together) so dotli builds against it. |
0 commit comments