Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 33 additions & 16 deletions docs/qt-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ Use an authorized test account with no production secrets in reports. Sign in in
not put NVIDIA credentials, refresh tokens, signing keys, or notarization passwords in command
arguments, logs, issue trackers, or acceptance artifacts.

The presenter remains out of process. The current Qt launch contract uses a paired native top-level
window aligned to the shell's stream region on Windows, X11, Wayland and macOS.
The surface contract carries a window handle, host-local and screen geometry, visibility, and scale,
but it does not establish cross-process Qt texture embedding. Foreign children and paired windows
cannot be covered reliably by ordinary Qt Quick items, so the shell hides the presenter before it
shows a QML menu, stats panel, reconnect screen, or error screen. The result is Qt-owned UI with a
temporarily suspended video surface, not a composited overlay over live video. No matrix row may
claim single-window composition or zero-copy into the Qt scene graph without a separate implementation
and measurement evidence.

## Performance evidence

Run the release package on the agreed baseline iGPU with its native display backend. Close frame
Expand All @@ -45,32 +55,38 @@ latency, frame intervals, missed-frame ratio, budgets, and the final pass/fail r
requires `pass: true` for both reports. The hardware flag rejects offscreen/minimal platforms,
software/null renderers, missing screens, and workloads that do not receive the requested physical
dimensions. It also rejects the test-only refresh-rate override, so release evidence always uses
the display-reported rate.
the display-reported rate. This measures the Qt shell workload; it does not prove stream-window
embedding, native decoder throughput or a zero-copy handoff into Qt.

## Authorized stream evidence

For every matrix row, launch a real account-owned title and keep the session active for at least
ten minutes. Exercise the following without restarting the app:

1. Complete device login, account switching, subscription and region refresh.
2. Create a session, pass queue/ads if present, and reach native first-frame playback.
3. Open and close every guide page over live video. Confirm video remains correctly ordered and
controller input transfers atomically between the shell and game.
2. Create a session, pass queue/ads if present, reach native NVST first-frame playback, and confirm
the live evidence reports `stream.transport: "nvst"`.
3. Open and close every guide and stats page while video is live. Confirm the presenter hides before
QML appears, returns after QML closes, never leaves a stale native handle, and transfers controller
input atomically. Record every platform as paired-window behavior rather than claiming a composited
live-video overlay.
4. Exercise keyboard, relative mouse, and every connected controller. Validate neutral controller
state after overlay entry, reconnect, pause, and resume.
5. Test window resize, fullscreen, display migration, the display's highest supported refresh
rate, and VRR/HDR only where the machine advertises them.
6. Enable voice-activity microphone mode before a new WebRTC session, grant the OS permission, and
confirm upstream audio at the remote endpoint. Record an explicit not-supported result for NVST
or hardware without an input device; do not reinterpret it as a pass.
6. Load a profile that previously selected WebRTC or another legacy transport and confirm settings,
session creation, streamer status and exported evidence all resolve it to NVST. If a persisted
microphone mode is armed, confirm the runtime reports upstream audio as unavailable without
changing transport; microphone audio is not a release gate for the NVST-only client.
7. Capture a screenshot, start and stop a source-stream Matroska recording, play the resulting
media, verify the generated thumbnail, and reveal both files through the Media screen.
8. Rebind and exercise all eight stream shortcuts. Confirm stats, pointer lock and fullscreen act
inside the native window; microphone, screenshot, recording and stop reach the shell exactly
once; and anti-AFK produces an F13 pulse after four minutes without leaking the key into the game.
9. Select a non-default microphone where available, verify live mute/unmute, enable the anti-AFK
indicator/reminder and session clock, then confirm the post-session report reflects elapsed time,
backend, first-frame latency, recovery/error counters and diagnostics navigation.
8. Rebind and exercise all eight stream shortcuts. Confirm stats and fullscreen reach Qt exactly once,
pointer lock remains native, microphone reports unavailable, screenshot, recording and stop reach
the shell exactly once, and anti-AFK produces an F13 pulse after four minutes without leaking the
key into the game.
9. Enable the anti-AFK indicator/reminder and session clock, then confirm the post-session report
reflects NVST transport, elapsed time, backend, first-frame latency, recovery/error counters and
diagnostics navigation.
10. Exercise favorites, entitlement-filtered aspect ratio/resolution/FPS choices, keyboard layout,
game language, console-friendly launch and in-game-settings persistence on a title that advertises
the corresponding NVIDIA feature.
Expand All @@ -79,7 +95,7 @@ ten minutes. Exercise the following without restarting the app:
12. Export both the redacted diagnostic report and **live evidence** from the Diagnostics screen
after the run. The live export is direct machine-readable JSON and must report
`observedPass: true`; it includes hashed screenshot/recording/thumbnail metadata and bounded
transport, first-frame, input ownership, guide, microphone, recovery and error checks without
NVST transport, first-frame, input ownership, guide, recovery and error checks without
exposing a local path, account, token, session identifier or endpoint.

Retain the two performance JSON files, redacted diagnostic export, screenshot, recording, package
Expand All @@ -90,8 +106,9 @@ artifacts is not proof of the gate.
Copy [`qt-acceptance-attestations.example.json`](qt-acceptance-attestations.example.json) for the
matrix row. Use exactly one of `windows-x64`, `windows-arm64`, `macos-apple-silicon`, `macos-intel`,
`linux-x64-x11`, `linux-x64-wayland` or `linux-arm64-native`. Leave a check `false` until it was
actually exercised. `hdr` and `vrr` accept `passed` or an evidence-backed `not-supported`; upstream
microphone audio does not have a not-supported exception. Declare every required release artifact,
actually exercised. `hdr` and `vrr` accept `passed` or an evidence-backed `not-supported`.
`microphoneUpstream`, if present in an older attestation template, is compatibility metadata and is
not a required NVST gate. Declare every required release artifact,
its byte size and SHA-256, and set the signing/update booleans only after the platform commands below
have succeeded.

Expand Down
74 changes: 53 additions & 21 deletions docs/qt-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,19 +118,38 @@ out of process so decoder or driver failures cannot take down the shell.

### 6. Native streaming

- [x] Replace Electron-specific surface ownership with a shell-neutral external-window contract.
- [x] Implement native frame presentation and shell/streamer window ordering for Windows, macOS, X11 and Wayland, with controller ownership transferred atomically while QML guide overlays are active. Cross-OS live-stream proof remains an acceptance gate.
- [x] Preserve out-of-process lifecycle, protocol-v4 health checks and restart isolation.
- [x] Preserve hardware decode selection and safe fallback behavior for every negotiable codec. Codec-specific H.264/H.265/AV1 WebRTC answers, strict automatic/hardware/software selection, prelaunch capability probing, Windows class-separated Media Foundation hardware and system-software probing/fallback, Linux bundled-FFmpeg fallback, and independently probed macOS VideoToolbox H.264/HEVC/AV1 are implemented. Windows HEVC/AV1 software availability follows registered D3D11-aware software MFTs instead of being mixed into the hardware result. macOS guarantees H.264 through OpenH264 fallback; HEVC/AV1 are offered only when the corresponding VideoToolbox hardware probe succeeds, and failures disable that codec instead of silently negotiating an unavailable software path.
- [x] Replace Electron-specific surface ownership with a shell-neutral external-window contract. The
core preserves `windowHandle`, `rect`, `screenRect`, visibility and scale updates and always starts
the out-of-process presenter with `OPENNOW_NATIVE_EXTERNAL_RENDERER=1`.
- [x] Implement native frame presentation and shell/streamer window ordering for Windows, macOS,
X11 and Wayland, with controller ownership transferred atomically while QML guide overlays are
active. The current external-presenter launch contract uses paired native top-level windows on all
four targets; none is a texture embedded in the Qt scene graph. Because ordinary QML cannot reliably
cover these native surfaces, the typed controller hides presentation for QML menus, stats, reconnect
and error states, then restores it from fresh host geometry. Cross-OS live-stream ordering proof
remains an acceptance gate, and this implementation does not claim a live-video QML overlay or
cross-platform zero-copy.
- [x] Preserve out-of-process lifecycle, protocol-v5 health checks and restart isolation.
- [x] Preserve hardware decode selection and safe fallback behavior for every negotiable codec. NVST
H.264/H.265/AV1 profiles, strict automatic/hardware/software selection, prelaunch capability
probing, Windows class-separated Media Foundation hardware and system-software probing/fallback,
Linux bundled-FFmpeg fallback, and independently probed macOS VideoToolbox H.264/HEVC/AV1 are
implemented. Windows HEVC/AV1 software availability follows registered D3D11-aware software MFTs
instead of being mixed into the hardware result. macOS guarantees H.264 through OpenH264 fallback;
HEVC/AV1 are offered only when the corresponding VideoToolbox hardware probe succeeds, and
failures disable that codec instead of silently negotiating an unavailable software path.
- [x] Route keyboard, mouse and up to four gamepads directly to the streamer while shell overlays are closed.
- [x] Transfer input ownership atomically to the shell for overlays and send neutral controller state on pause.
- [x] Integrate stream statistics, next-session bitrate, recording and Cloud G-Sync quick controls.
- [x] Add bounded SDL microphone capture, voice-activity Opus encoding and negotiated WebRTC upstream audio; classic NVST and push-to-talk fail closed.
- [x] Enumerate microphone capture devices, fall back safely from stale migrated device identifiers,
and support live mute/unmute for microphone-armed WebRTC sessions.
The configurable stats shortcut is forwarded as `shortcut-action: toggle-stats`; Qt owns the
overlay and renders core telemetry instead of asking the native presenter to draw it.
- [x] Fail closed when a persisted microphone mode is selected: upstream microphone audio is not
implemented by the NVST runtime, and diagnostics report it as unavailable rather than selecting a
WebRTC media session.
- [x] Apply all eight configurable native shortcuts, including pointer lock, recording, screenshot,
microphone, stop and real four-minute anti-AFK F13 pulses across WebRTC and NVST input transports.
- [ ] Validate microphone permissions and live upstream audio with an authorized account on Windows, macOS and Linux.
microphone, stop and real four-minute anti-AFK F13 pulses. Stats and fullscreen are forwarded to
Qt instead of mutating native presentation; pointer lock remains native. The microphone shortcut
retains settings compatibility but reports NVST upstream audio as unavailable.
- [ ] Validate HDR, high-refresh, VRR, resize, fullscreen and display migration.
- [ ] Validate screenshots and source-stream recording with an authorized live session on each supported OS.

Expand All @@ -141,8 +160,8 @@ out of process so decoder or driver failures cannot take down the shell.
- [x] Port proxy-aware HTTP, client identity, endpoint and error handling.
- [x] Port authentication, catalog, subscriptions and region discovery.
- [x] Port account connections.
- [x] Port fresh-session lifecycle and WebRTC signaling coordination.
- [x] Port active-session claim, recovery and classic NVST RTSP negotiation.
- [x] Port fresh-session lifecycle and CloudMatch coordination for NVST-only media sessions.
- [x] Port active-session claim, recovery and native-owned NVST RTSP negotiation.
- [x] Port updater discovery/channels, media library, diagnostics, Discord and opt-in telemetry.
- [x] No temporary Node service was introduced.

Expand Down Expand Up @@ -187,13 +206,12 @@ storage migration and controller-only operation are all removal gates.
The 2026-08-28 Linux checkpoint has a warning-fatal suite covering 48 shell,
controller, localization, native recording, route, overlay, performance-harness and lifecycle
tests; the complete four-way run takes about 19 seconds after making unused Qt Multimedia thumbnail
decoding lazy. The Rust application core has 59 unit tests, the acceptance verifier has four, the
decoding lazy. The Rust application core has 74 unit tests, the acceptance verifier has four, the
update-manifest generator has two, and the legacy boundary has three machine-readable contract tests
(68 total).
The native streamer workspace has 246 passing default-feature
platform/core/protocol/transport tests plus doc tests. Its system-FFmpeg Linux
configuration has 247 passing tests and one intentionally ignored live-hardware
probe, including linked H.264/H.265/AV1 software-decoder verification. Both Rust
(83 total).
The native streamer workspace's platform/core/protocol/transport tests and doc tests pass on the
development host. The system-FFmpeg Linux configuration retains one intentionally ignored
live-hardware probe, including linked H.264/H.265/AV1 software-decoder verification. Both Rust
workspaces pass clippy with warnings denied.

The macOS VideoToolbox crate passes `x86_64-apple-darwin` and `aarch64-apple-darwin` Rust type
Expand All @@ -209,8 +227,8 @@ launch passed while explicitly using the packaged `opennow-core` and
metadata, icon, license and generated third-party notices. The exact local artifact has SHA-256
`e1d076182530a3e128f18708371bdbe6c936cdc9fd6dbcaad826121e13cfa1bf`.

The native WebRTC answer now advertises exactly the CloudMatch codec configured
in the active decoder. A protocol-v4 child-process probe applies the selected
The native NVST launch context now carries exactly the CloudMatch codec configured
for the active decoder. A protocol-v5 child-process probe applies the selected
decoder policy before CloudMatch allocates a session, including automatic H.264;
explicit HEVC/AV1 sessions remain available only where the selected native backend
honestly reports them. CPack packages the Qt shell, Rust core, native streamer,
Expand All @@ -219,7 +237,7 @@ builds the distributable Linux streamer with bundled FFmpeg and smoke-tests the
checksum-pinned x64 AppImage.

The extracted DEB starts the Qt shell against its packaged core and streamer.
Its live protocol-v4 probe reports the bundled H.264/H.265/AV1 codecs, applies an
Its live protocol-v5 probe reports the bundled H.264/H.265/AV1 codecs, applies an
explicit unavailable V4L2 policy as zero available codecs, and rejects
`session.create` with `streamer_codec_unavailable` before any provider request.

Expand All @@ -231,13 +249,27 @@ remains intact.

The Diagnostics screen now exports both a human-readable redacted report and a direct
machine-readable live-acceptance manifest. The latter hashes the screenshot, recording and
thumbnail, records ten-minute streaming, first-frame, guide/input ownership, surface, microphone,
thumbnail, records ten-minute NVST streaming, first-frame, guide/input ownership, surface,
recording, recovery and error checks, and excludes session/process identifiers, endpoints,
executable paths and local media paths. The packaged `opennow-acceptance-verify` tool combines that
manifest with both hardware performance reports, the explicit manual-attestation template and all
required platform packages; it fails closed on any false/mismatched gate and emits a path-free
verification result.

The NVST-only core no longer contains its former browser WebSocket/SDP/ICE media fallback or the
associated Tungstenite dependency. Persisted `transportMode` remains part of the settings contract
for rollback compatibility, but every legacy value normalizes to `nvst` before session creation or
streamer launch. NVIDIA still requires some protocol labels whose names contain `WEBRTC`: device
authorization and browser-style region discovery retain the `nv-client-streamer: WEBRTC` identity.
The native streamer also retains DTLS/SCTP-named bundle, input and control structures because NVST
audio, input and RTCP use that encrypted bundle. Those names are wire compatibility, not a second
media transport.

No cross-process zero-copy path into the Qt scene graph is implemented or claimed. Hardware decode
and native presentation can still avoid software decode, but acceptance must treat that separately
from Qt texture sharing. macOS and Wayland acceptance therefore verifies the supported two-window
ordering and input-ownership model rather than requiring single-window embedding.

The exact live-account, hardware, signing and rollout procedure is maintained in
[`docs/qt-acceptance.md`](qt-acceptance.md). It defines which artifacts constitute proof, so an
offscreen pass or an operator-only checklist cannot accidentally close a release gate.
Expand Down
12 changes: 11 additions & 1 deletion locales/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -2590,6 +2590,16 @@
"s_qt_runtime_219": "HANDING OVER THE STREAM",
"s_qt_runtime_220": "Your rig is live. OpenNOW is starting the native media runtime.",
"s_qt_runtime_221": "Session menu",
"s_qt_runtime_222": "Stream stats"
"s_qt_runtime_222": "Stream stats",
"s_qt_runtime_223": "Fullscreen on",
"s_qt_runtime_224": "Fullscreen off",
"s_qt_runtime_225": "Go fullscreen",
"s_qt_runtime_226": "The Qt stream surface uses the current display",
"s_qt_runtime_227": "Microphone upstream is unavailable for NVST sessions",
"s_qt_runtime_228": "STARTING VIDEO",
"s_qt_runtime_229": "Fullscreen",
"s_qt_runtime_230": "Shows live stream statistics",
"s_qt_runtime_231": "Starts the next session with compact stats",
"s_qt_runtime_232": "Stream controls are available once the session is live."
}
}
Loading
Loading