Skip to content

gfx: caching + offscreen infrastructure (pre-scene, from #2109) - #2119

Open
jcelerier wants to merge 7 commits into
split/gfx-fixes-and-formatsfrom
planc/gfx-infra
Open

gfx: caching + offscreen infrastructure (pre-scene, from #2109)#2119
jcelerier wants to merge 7 commits into
split/gfx-fixes-and-formatsfrom
planc/gfx-infra

Conversation

@jcelerier

@jcelerier jcelerier commented Jul 14, 2026

Copy link
Copy Markdown
Member

Extracts the self-contained gfx infrastructure from #2109 that compiles against master independently of the coupled "scene concept" rework. Landing these first shrinks the eventual scene-rework PR and makes it reviewable.

These are additive: new files, wired into CMake, not yet referenced by any consumer — the scene rework (a later PR) wires them up. They are included here so that PR is smaller and each piece can be reviewed on its own.

Stacked on #2117 (test infrastructure); will retarget to master once #2117 merges.

Commits

  • gfx: add AssetTable + TextureLoader — a shared cache for decoded image/texture assets (Gfx/AssetTable.*, Gfx/Graph/TextureLoader.*). Depends only on QImage/QRhi/ossia::hash_map.
  • gfx: make staged-but-never-acquired AssetTable entries evictable (follow-up to the above).
  • gfx: add OffscreenDevice + RhiPreviewWidget + a Metal buffer-copy backend — offscreen QRhi rendering + a preview widget, using the existing Graph/GfxContext/BackgroundNode APIs.

Deferred to the scene-rework PR (Plan D)

  • The GPU memory arena (GpuResourceRegistry + OffsetAllocator submodule) — hard-depends on SceneGPUState.hpp, which the scene concept introduces.
  • The pipeline-cache — only a minimal QRhi load/store is separable; its async-flush hook is entangled with the RenderList rebuild, so the whole piece lands with the scene rework.

Validation

  • Builds cleanly against master (all plugins).
  • ctest: 17/17 (no regression; the additions are not yet on any exercised path).

🤖 Generated with Claude Code

@jcelerier jcelerier mentioned this pull request Jul 14, 2026
Open
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
tests(#2119): AssetTable unit test + hardware-probe harness
08516ad 
test_unit_gfx_assettable: AssetTable CPU logic + TextureLoader upload paths
on QRhi's Null backend (no GPU needed).
tests/hardware/: capability-probe framework (AJA/DeckLink/Magewell/PipeWire/
display) + ScoreHardwareTests.cmake — gated, run only on capture rigs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
gfx(#2120): fix 5 libisf importer bugs + render/scene test harnesses
5c79ed7 
FIX-ISF (libisf/src/isf.cpp — applies on top of this branch's ISF/CSF
feature rework, hence #2120 not #2119):
 - write_isf: seekp(0,end)-before-tellp corrupted the emitted descriptor
 - use-after-move on m_sourceVertex -> guard with .empty()
 - parse_shadertoy_json: GLSL45 preludes + iMouse vec4(0,0,0,0)
 - glsl_sandbox: duplicate-TIME uniform guard
 - replace_identifier: word-boundary match

Tests: test_unit_isf_importers (all 5 importer branches); golden-image
render regression (16 pinned JS-corpus cases, llvmpipe, SSIM/PSNR);
timeline-driven scenario ramp; gfx resource/leak soak (DISABLED — guards
a still-unfixed render-clock teardown UAF); live-edit churn corpus.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
gfx(#2121): fix NV12 + 10-bit encoder readback + encoder/video tests
bc681f8 
FIX-ENC:
 - NV12.hpp: UV plane encodes into an R8 target (16-byte-tight rows), was
   mis-sized producing garbage chroma on readback
 - YUVPlanar.hpp: p420_10 -> RGBA8-packed 16-bit LE on the Qt<6.10 GL path

Tests: test_unit_video_pixel_format (pixfmt plane math + avutil mapping);
test_integration_encoder_matrix (every encoder on a real QRhi backend,
byte-checked); test_regression_offscreen_teardown (full-app offscreen render
+ /stop + /exit exits 0 — guards 1228382 #2119 + 32ad555 #2121 teardown
UAFs); VideoDecoderTester + video-decoder-sweep.sh; EncoderTester ctest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
gfx(#2119): fix off-screen default position of the Text node
6470247 
An unconfigured Text process rendered nothing: the UBO default position was
{0.5,0.5} (TextNode.hpp), a clip-space offset added to the text quad in the
vertex shader, which shoved the default text ~half a screen up+right — off the
top edge — until Position was manually set. Default to {0,0} (screen centre).

Found + verified by the P3-text render harness (untouched Text now renders
1508 visible px; the harness's default-visibility check is a hard assertion).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
gfx(#2120): fix 5 libisf importer bugs + render/scene test harnesses
3d4ffb7 
FIX-ISF (libisf/src/isf.cpp — applies on top of this branch's ISF/CSF
feature rework, hence #2120 not #2119):
 - write_isf: seekp(0,end)-before-tellp corrupted the emitted descriptor
 - use-after-move on m_sourceVertex -> guard with .empty()
 - parse_shadertoy_json: GLSL45 preludes + iMouse vec4(0,0,0,0)
 - glsl_sandbox: duplicate-TIME uniform guard
 - replace_identifier: word-boundary match

Tests: test_unit_isf_importers (all 5 importer branches); golden-image
render regression (16 pinned JS-corpus cases, llvmpipe, SSIM/PSNR);
timeline-driven scenario ramp; gfx resource/leak soak (DISABLED — guards
a still-unfixed render-clock teardown UAF); live-edit churn corpus.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
tests(#2120): Text-node render validation harness (P3-text)
019649f 
Headless llvmpipe golden+value harness for the Gfx::Text process: grabs the
untouched-defaults frame (hard off-screen-default visibility assertion — guards
the #2119 default-position fix), then live-edits text/font/size/position/scale/
color over OSC /script and asserts pixel VALUES per case (coverage, bbox
ordering across sizes, centroid movement, channel dominance, blank empty
string, unicode/CJK/tofu/2000-char) + golden refs (compare.py strict).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
jcelerier added a commit that referenced this pull request Jul 17, 2026
@jcelerier @claude
gfx(#2121): fix NV12 + 10-bit encoder readback + encoder/video tests
002fa02 
FIX-ENC:
 - NV12.hpp: UV plane encodes into an R8 target (16-byte-tight rows), was
   mis-sized producing garbage chroma on readback
 - YUVPlanar.hpp: p420_10 -> RGBA8-packed 16-bit LE on the Qt<6.10 GL path

Tests: test_unit_video_pixel_format (pixfmt plane math + avutil mapping);
test_integration_encoder_matrix (every encoder on a real QRhi backend,
byte-checked); test_regression_offscreen_teardown (full-app offscreen render
+ /stop + /exit exits 0 — guards 1228382 #2119 + 32ad555 #2121 teardown
UAFs); VideoDecoderTester + video-decoder-sweep.sh; EncoderTester ctest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
@jcelerier
jcelerier changed the base branch from tests/infrastructure to master July 18, 2026 13:07
@jcelerier
jcelerier changed the base branch from master to planb/app-robustness July 18, 2026 13:37
@jcelerier
jcelerier force-pushed the planc/gfx-infra branch 2 times, most recently from 387b649 to 65ff611 Compare July 19, 2026 03:14
@jcelerier
jcelerier changed the base branch from planb/app-robustness to master July 19, 2026 21:59
@jcelerier
jcelerier force-pushed the planc/gfx-infra branch 6 times, most recently from 2b108de to 51e0e1b Compare July 25, 2026 11:53
@coveralls

coveralls commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown

Coverage Status

Coverage is 23.495% — planc/gfx-infra into split/gfx-fixes-and-formats. No base build found for split/gfx-fixes-and-formats.

@jcelerier
jcelerier changed the base branch from master to split/gfx-fixes-and-formats July 29, 2026 18:21
@jcelerier
jcelerier force-pushed the planc/gfx-infra branch 5 times, most recently from 3536464 to 3e157ee Compare July 31, 2026 00:17
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from 2f72efa to c6b09de Compare August 17, 2026 02:51
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from c6b09de to 5552446 Compare August 17, 2026 13:30
@jcelerier
jcelerier force-pushed the planc/gfx-infra branch 2 times, most recently from 07f5ae3 to c8df254 Compare August 17, 2026 13:51
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch 2 times, most recently from 6c3686a to f5e872d Compare August 17, 2026 19:04
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from f5e872d to ee75df3 Compare August 18, 2026 00:44
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from ee75df3 to 7bc0e11 Compare August 18, 2026 01:30
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from 7bc0e11 to 525b3ff Compare August 19, 2026 01:44
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from 525b3ff to d427269 Compare August 19, 2026 02:03
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from d427269 to b612cfa Compare August 19, 2026 03:14
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from b612cfa to b5eebc2 Compare August 19, 2026 03:35
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from b5eebc2 to ca1a743 Compare August 19, 2026 06:19
@jcelerier
jcelerier force-pushed the split/gfx-fixes-and-formats branch from ca1a743 to a6fb926 Compare August 19, 2026 12:20
jcelerier and others added 7 commits August 23, 2026 18:43
@jcelerier
gfx: add AssetTable + TextureLoader for shared decoded-asset cache
dfca620
@jcelerier
gfx: add OffscreenDevice + RhiPreviewWidget + Metal buffer-copy backend
ca3b966
@jcelerier
gfx: clear image inputs with no incoming edge each frame
0697b07
@jcelerier @claude
tests: AssetTable unit test + hardware-probe harness
f708604 
test_unit_gfx_assettable: AssetTable CPU logic + TextureLoader upload paths
on QRhi's Null backend (no GPU needed).
tests/hardware/: capability-probe framework (AJA/DeckLink/Magewell/PipeWire/
display) + ScoreHardwareTests.cmake — gated, run only on capture rigs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
@jcelerier @claude
gfx: fix off-screen default position of the Text node
f0b814e 
An unconfigured Text process rendered nothing: the UBO default position was
{0.5,0.5} (TextNode.hpp), a clip-space offset added to the text quad in the
vertex shader, which shoved the default text ~half a screen up+right — off the
top edge — until Position was manually set. Default to {0,0} (screen centre).

Found + verified by the text-render harness (untouched Text now renders
1508 visible px; the harness's default-visibility check is a hard assertion).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
@jcelerier @claude
gfx: extract the render-clock abstraction
f0f4101 
Replace the timer->set<OutputNode*> map with RenderClock: one TimerClock per
distinct manualRenderingRate owning the coalesced set of outputs it drives, and
a DisplayVSyncClock wrapping the swap-chain vsync callback. Behaviour is
unchanged; the clocks are torn down before the timer pool is rebuilt so their
dtors release the shared timers back to a still-live pool.

The teardown paths added earlier in this branch already unregister outputs from
the render clocks, but the abstraction they use landed further up the series;
this brings it down to where it is first needed. Dropping an output's edges on
teardown needs Graph::removeNodeAndEdges, which is part of the incremental
render-list rework, so that call moves to the commit that introduces it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
@jcelerier @claude
gfx: drive rendering off a timer on Wayland instead of the vsync loop
d7d89fa 
In VSync mode the on-screen render loop re-drives itself solely through
QWindow::requestUpdate(). On xcb that is an unconditional timer, so the loop
self-heals. On Wayland requestUpdate() is frame-callback gated: if the
compositor withholds a frame callback past QT_WAYLAND_FRAME_CALLBACK_TIMEOUT
(default 100ms) -- occluded window, blanked/DPMS output, another surface on
top, or heavy per-frame work -- Qt marks the window inexposed and the loop,
having no other driver, produces no frames. The render itself is fine: an
offscreen readback of the same frame returns correct pixels, so the symptom
is a black on-screen window despite correct rendering.

Confirmed by flipping only the VSync setting on a KDE-Wayland session, and by
a minimal QRhi repro whose requestUpdate loop dies under a forced 1ms
frame-callback timeout while a timer-driven loop is unaffected.

Gate the vsync loop off when the Qt platform is Wayland so it takes the timer
path. The compositor supplies frame pacing there, so no tearing results from
dropping swap-chain vsync. xcb, eglfs and the embedded backends are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jcelerier @coveralls