Skip to content

Fix GB200 gpu_roce decoding-server validation issues - #683

Merged
vedika-saravanan merged 6 commits into
NVIDIA:mainfrom
vedika-saravanan:relaybp-gpuroce-gb200-fixes
Jul 14, 2026
Merged

Fix GB200 gpu_roce decoding-server validation issues#683
vedika-saravanan merged 6 commits into
NVIDIA:mainfrom
vedika-saravanan:relaybp-gpuroce-gb200-fixes

Conversation

@vedika-saravanan

@vedika-saravanan vedika-saravanan commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Description

Follow-up fixes on top of the HSB/FPGA gpu_roce decoding-server path, found during end-to-end GB200 validation of RelayBP / nv-qldpc-decoder with the FPGA emulator.

Validated after rebase with an 85-shot / 425-frame gpu_roce RelayBP run against the FPGA emulator, using 512 playback windows. The emulator processed all windows successfully with 425 responses, 0 send errors, and 0 timeouts.

Changes:

  • Pin decoding-server graph capture to the decoder cuda_device_id, and reconcile it with the transitional HOLOLINK_GPU_ID setting.
  • Validate the server-side gpu_roce DOCA ring allocation against host page-size alignment, and make the HSB test harness choose an aligned server ring depth while keeping playback window capacity explicit.
  • Link optional private decoder_rpc_dispatch.cu support into decoding_server when present; public builds remain unaffected.
  • Avoid duplicate public/proprietary cudevice whole-archive linkage in decoding_server.
  • Make the gpu_roce link check verify symbol/link availability without constructing a transceiver or selecting GPU 0.
  • Fix HSB playback verification to wait for the expected ILA sample count instead of stabilization, and fail clearly if the capture times out before enough samples are collected.
  • Add separate HSB harness controls for server gpu_roce ring pages and FPGA/emulator playback window pages.

Note: I left this local intentionally because the HOLOLINK_GPU_ID path is transitional and PR692 removes it. Sharing the env parser now would add churn and increase conflicts for code that is about to disappear.

Runtime / performance impact

na

Self-review checklist

Please confirm each item before requesting review. Check [x] or strike
through and explain.

Before requesting review

  • I reviewed my own full diff in GitHub or my editor.
  • PR is in Draft if it is not yet ready for review.
  • Temporary / debugging changes have been removed.
  • Local test logs reviewed; no unexplained warnings or errors.
  • CI logs reviewed; no unexplained warnings or errors.
  • Full CI has been run.

Scope and size

  • PR is under ~1000 lines, or an exception is justified in the description.
  • Refactoring-only changes are isolated in their own PR(s).
  • No existing tests were disabled or modified just to make this PR pass
    (if so, an issue has been raised).

Tests

  • New functionality has new tests.
  • Tests fail if the new functionality is broken (including crashes), not
    just when it is missing.
  • Negative tests added where exceptions are expected.
  • Truth data added where simple EXPECT_* / assert checks are
    insufficient for algorithmic correctness.
  • CI runtime impact considered; team notified if significant.

Documentation

  • Public-facing APIs have Doxygen docs.
  • User-visible behavior changes have public docs, or a follow-up is
    tracked.

Code style

  • Naming follows the existing convention (snake_case vs camelCase) for
    the area being modified.

Dependencies

  • No new third-party dependencies, or the team has been notified and
    OSRB tickets filed.

@vedika-saravanan vedika-saravanan changed the title Relaybp gpuroce gb200 fixes GB200 fixes for HSB/FPGA RelayBP gpu_roce decoding-server path Jul 13, 2026
@vedika-saravanan
vedika-saravanan force-pushed the relaybp-gpuroce-gb200-fixes branch from 017229c to 8e5e9f7 Compare July 14, 2026 13:54
@copy-pr-bot

copy-pr-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…ge alignment, private dispatch symbols

Signed-off-by: vedika-saravanan <vsaravanan@nvidia.com>
@vedika-saravanan
vedika-saravanan force-pushed the relaybp-gpuroce-gb200-fixes branch from 8e5e9f7 to 75780c3 Compare July 14, 2026 13:56
Signed-off-by: vedika-saravanan <vsaravanan@nvidia.com>
@vedika-saravanan
vedika-saravanan force-pushed the relaybp-gpuroce-gb200-fixes branch from 75780c3 to 3de6452 Compare July 14, 2026 14:04
@vedika-saravanan vedika-saravanan changed the title GB200 fixes for HSB/FPGA RelayBP gpu_roce decoding-server path Fix GB200 gpu_roce decoding-server validation issues Jul 14, 2026
@vedika-saravanan
vedika-saravanan force-pushed the relaybp-gpuroce-gb200-fixes branch 2 times, most recently from a92b03c to fc0ceaf Compare July 14, 2026 16:15
@vedika-saravanan
vedika-saravanan marked this pull request as ready for review July 14, 2026 16:29
Signed-off-by: vedika-saravanan <vsaravanan@nvidia.com>
Comment thread libs/qec/lib/realtime/decoding-server-cqr/DecodingSession.cpp Outdated
Comment thread libs/qec/lib/realtime/CMakeLists.txt Outdated
Signed-off-by: vedika-saravanan <vsaravanan@nvidia.com>
Comment thread libs/qec/lib/realtime/decoding-server-cqr/DecodingSession.cpp Outdated
Comment thread libs/qec/tools/decoding-server/CMakeLists.txt
Signed-off-by: vedika-saravanan <vsaravanan@nvidia.com>

@bmhowe23 bmhowe23 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once Chuck's threads are marked as resolved.

@vedika-saravanan
vedika-saravanan merged commit 2cdc9df into NVIDIA:main Jul 14, 2026
24 checks passed
bmhowe23 added a commit to bmhowe23/cudaqx that referenced this pull request Jul 15, 2026
…ridge branch

Main landed the squashed PR 670 plus NVIDIA#656/NVIDIA#673/NVIDIA#674/NVIDIA#678/NVIDIA#680/NVIDIA#683/NVIDIA#688/
NVIDIA#690/NVIDIA#691/NVIDIA#692.  Resolution keeps this branch's design and ports main's
content into it:

- Naming: main's GpuRoce{Transceiver,Factory,LinkCheck} map 1:1 onto this
  branch's DeviceGraph* files (the de-transport-ing follow-up requested in
  the PR 670 review).  Main's post-review fixes are ported into the
  renamed files: ring-size overflow + host-page-size alignment validation,
  gpu_id resolved from the decoder's cuda_device_id instead of an env var,
  the factory taking pinned_cuda_device, and the linkcheck exercising the
  new factory signature.
- Schema: per-decoder 'dispatch: host|device_graph' plus the top-level
  transport section stay; main's per-decoder 'transport:' key does not
  return.  cuda_device_id is adopted as a per-decoder placement knob
  (config struct + YAML trait), and the hsb script now injects
  'dispatch: device_graph' + cuda_device_id and drives the server with
  QEC_DEVICE_GRAPH_* env and the device_graph READY sentinel.
- Features adopted from main: decoder CUDA-device pinning (worker-thread
  pin via promise, graph-capture pin, hardware_guards.h), per-session
  decode counters + print_session_stats + QEC_DECODING_SERVER_STATS
  server-side stats printing, DecodingServer ctor exception safety,
  virtualized realtime decoder API (NVIDIA#674), CMP0126 fix, CUDA::cudart on
  the core lib, and the cudevice-archive dedup in the server link.
- DecodingSession combines main's set_graph_capture_device with this
  branch's reserved-SMs decode-graph capture
  (QEC_DEVICE_GRAPH_RESERVED_SMS).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ben Howe <bhowe@nvidia.com>
anjbur added a commit to anjbur/cudaqx that referenced this pull request Jul 16, 2026
)"

This reverts commit 2cdc9df.

Signed-off-by: Angela Burton <angelab@nvidia.com>
melody-ren added a commit that referenced this pull request Jul 17, 2026
…698)

## Background
- #690 introduced the cuda_device_id placement knob for GPU decoders.
- #678 made worker threads and the inproc graph capture honor the pinned
device, and pinned decoder construction.
- #683 closed the remaining gap: the gpu_roce decoding-server capture
path now pins too.

Each fix added a device pin to one more path via its own ad-hoc helper.
The "which device" decision ended up duplicated across dispatch,
capture, the worker, construction, and the gpu_roce transport. And that
is the motivation for this PR.

## What this PR does
Basically, every path should be following a single source of truth. 

Consolidate all of it to: the decoder's `cuda_device_id`, resolved one
way (`decode_device_for`) and applied through two sanctioned wrappers —
`pin_decode_device` (dispatch/decode) and `capture_graph_pinned` (graph
capture) — in `hardware_guards.h.` Every path now derives its device
from that one field; none selects a device by any other means. Unpinned
(< 0) keeps existing semantics: dispatch no-ops, capture defaults to
device 0.

Signed-off-by: Melody Ren <melodyr@nvidia.com>
cketcham2333 added a commit that referenced this pull request Jul 24, 2026
…r decoder, mixed host + device_graph dispatch (#682)

Built on the companion cuda-quantum PR
NVIDIA/cuda-quantum#4915, **now merged
upstream** (squash `2a7911f`, 2026-07-23); `.cudaq_version` pins that
commit, so this branch builds against stock `NVIDIA/cuda-quantum` main.

## Summary

This PR re-architects the realtime decoding server around the CUDA-Q
bridge-provider boundary. The server no longer contains any transport
code: the wire is named in the deployment YAML (or a `--transport`
fallback), loaded at runtime as a `libcudaq-realtime-bridge-<name>.so`
provider, and every decoder gets its own ring buffer and its own
consumer -- a host dispatcher thread for `dispatch: host` decoders, or
the GPU device-graph scheduler for `dispatch: device_graph` decoders,
both side by side in one server process. A partner transport library
drops in as a single `.so` with zero decoding-server changes.

The guiding litmus test for the layering: cudaq-realtime and its
transport providers never say "QEC" or "decoder"; the QEC library code
never names a wire (UDP, RoCE, hololink). Wire names appear only where
deployments are described: YAML configs, launch scripts, and tests.

## Before (main prior to #670; #670 landed the first step)

```
┌──────────────────────────────────────────────────────────────────────────┐
│                     decoding_server (pre-refactor)                       │
│  compiled-in wire branches, one per transport:                           │
│   #ifdef udp path      #ifdef cpu_roce path        gpu_roce fork         │
│   udp wrapper calls    RendezvousInfo + hsb_fpga   GpuRoceTransceiver =  │
│   wired directly       QP handshakes INLINE        hololink bring-up     │
│                        (byte-exact copy of the     FUSED with the QEC    │
│                        CpuRoceChannel structs)     dispatch engine       │
│  hand-rolled std::thread around cudaq_host_ring_dispatch_loop            │
│  ONE ring buffer + ONE dispatch loop shared by ALL decoders              │
│  two transport knobs that had to agree (--transport + per-decoder YAML)  │
└──────────────────────────────────────────────────────────────────────────┘
      │ link-time: udp/cpu_roce wrappers; DOCA + hololink + HSB + ibverbs
      ▼ adding a wire = editing this repo; no partner drop-in possible
```

- The server had per-transport code paths: udp and cpu_roce wired
directly against transport wrapper headers, and a separate gpu_roce fork
that linked hololink/DOCA/HSB/ibverbs into the QEC libraries
(`DT_NEEDED` on the server binary). Adding a wire meant editing the
server.
- Transport selection was two knobs that had to agree (`--transport` on
the CLI AND a per-decoder YAML key), with silent misconfiguration when
they disagreed.
- One ring buffer served all decoders: every RPC funneled through a
single dispatcher, and per-decoder endpoints (the topology the product
needs: caller routes with `device_id == decoder_id`) were impossible.
- The GPU device-graph path (`gpu_roce`) was hololink-only, named after
one wire, and could not coexist with host-dispatch decoders in the same
process.
- The CPU-RoCE rendezvous / HSB-FPGA QP handshake lived in the server
itself instead of behind the transport boundary.
- #670 (now on main) took the first step -- the gpu_roce engine moved
out of the core library behind a weak factory -- but the component still
links hololink/DOCA/HSB directly, serves exactly one decoder, is named
for one wire, and the per-decoder `transport:` key remains the selection
knob.

## After

```
┌──────────────────────────────────────────────────────────────────────────┐
│                    decoding_server (transport-blind)                     │
│  YAML `dispatch:` picks the ENGINE     YAML `transport:` picks the WIRE  │
│   (per decoder)                        (per deployment; --transport is   │
│                                         a fallback, conflict = error)    │
│  ONE RING PER DECODER, each with its own consumer:                       │
│   host ─► dispatcher object over        <name>  ► libcudaq-realtime-     │
│           its provider ring (4869 API)            bridge-<name>.so       │
│   device_graph ─► DeviceGraphRing-      /path.so ► partner library,      │
│           Consumer (GPU scheduler)                verbatim, zero changes │
│  geometry + READY ring tokens derived FROM the providers (v2 queries)    │
└──────────────────────────────────────────────────────────────────────────┘
     │ links                            │ links (weak factory, WHOLE_ARCHIVE)
┌────────────────────┐   ┌────────────────────────────────────────────────┐
│ CQR plugin         │   │ DecodingServer core · device-graph component:  │
│ (HOST_CALL table)  │   │ DeviceGraphTransceiver = dispatch ENGINE only  │
└────────────────────┘   └────────────────────────────────────────────────┘
     │ links (the ONLY realtime link dependency)
┌──────────────────────────────────────────────────────────────────────────┐
│  libcudaq-realtime.so — bridge loader (iface v2) + dispatcher object     │
└──────────────────────────────────────────────────────────────────────────┘
     ◌ dlopen ─────────────── runtime plug-in seam ────────────── ◌ dlopen
┌────────────┬──────────────────┬─────────────────┬────────────────────────┐
│ bridge-udp │ bridge-cpu-roce  │ bridge-hololink │ partner transport .so  │
│ .so        │ .so (rendezvous  │ .so (built on   │ (out of tree, ~9 C     │
│            │ + hsb_fpga)      │ the HSB rig)    │ functions)             │
└────────────┴──────────────────┴─────────────────┴────────────────────────┘
```

- **Bridge-provider-only server.** `decoding_server` speaks only YAML +
the `cudaq_bridge_*` C API. Provider libraries resolve by name next to
the cudaq-realtime install (`QEC_BRIDGE_PROVIDER_DIR`) or verbatim by
path (partner drop-in). The QEC libraries link no transport libraries;
the rendezvous/hsb_fpga handshake moved down into the cpu_roce provider
(companion cuda-quantum PR).
- **Engine vs. wire, each named once.** Per-decoder YAML `dispatch:
host|device_graph` picks HOW RPCs execute; the top-level, shape-keyed
`transport:` section (`provider`, `args`, plus a `device_graph:`
override for the rings that must be GPU-pollable) picks the wire for the
whole deployment. `--transport` is a fallback for configs that
intentionally leave the wire unspecified (one YAML reused across wires,
selected per launch); a YAML that names a provider combined with an
explicit `--transport` is a startup error, never a silent precedence
decision. All pre-release aliases (`transport:` as a per-decoder key,
`--transport=gpu_roce`, `HOLOLINK_*` env fallbacks) are removed -- this
component is new this cycle, so there is no compatibility surface to
keep.
- **One ring per decoder.** The server opens one bridge instance + one
ring + one consumer per YAML decoder; ring geometry and endpoint
identity come from the provider's v2 queries instead of re-parsed CLI.
Readiness publishes every endpoint in one line
(`QEC_DECODING_SERVER_READY port=<p0> ... ring<id>=<port>`) before any
blocking connect, and shutdown reports per-ring traffic
(`QEC_DECODING_SERVER_RING decoder=<id> dispatched=<n>`). Callers route
with `cudaq::device_call(decoder_id, ...)` and per-device endpoint args
(`udp-port.<id>=<port>`).
- **Mixed dispatch in one process.** `DeviceGraphRingConsumer` runs the
CUDA-Q device-graph scheduler (self-relaunching GPU dispatch graph + the
decoder's captured decode graph) as a ring consumer over any
GPU-pollable ring -- hololink DOCA rings on a rig, or pinned+mapped udp
rings (`--pinned-rings`) on any CUDA box. `GpuRoceTransceiver` was
renamed to what it actually is (`DeviceGraphTransceiver`, the dispatch
engine, transport-blind) and now delegates to the ring consumer; the
standalone all-device_graph path remains for the HSB flow.
- **Device-graph wedge root-caused and fixed.** The scheduler deadlock
after the first decode trigger was cooperative-launch co-residency
starvation: the decode graph was captured sized for every SM and could
never become co-resident with the persistent dispatch graph, so the
device-side fire queued forever at `grid.sync()`. `DecodingSession` now
captures with `reserved_sms=1` (`QEC_DEVICE_GRAPH_RESERVED_SMS` to raise
it on rigs where hololink RX/TX kernels are also resident). The same bug
class affects host-path GPU-cooperative decodes when a scheduler is
resident -- documented, with the mixed-deployment guidance of a CPU
decoder on host rings until the plugin plumbs reservation into its host
path.
- **Works with the session stats from main.**
`QEC_DECODING_SERVER_STATS=1` prints per-decoder counters in the mixed
server too; they are host-session counters, so a `device_graph` ring
legitimately reports `decodes=0` -- its execution evidence is the
trigger diagnostics (`fires == tail_relaunches`) and the per-ring
`dispatched=` line.
- **Tests and config schema.** New coverage: transport-section
parse/round-trip with a mixed host+device_graph decoder set, a
two-process decode where the YAML alone names the wire, CLI/YAML
conflict rejection, a per-decoder-rings two-process test asserting
traffic on each ring, and
`app_examples.surface_code-4-yaml-mixed-dispatch` -- the flagship mixed
flow as a gated ctest (registered only when the server links the
device-graph component; skips without a GPU or the nv-qldpc plugin)
asserting scheduler health via the new trigger diagnostics (rc=0, fires
== tail_relaunches > 0). The generated JSON config schema
(`decoder_config_json_schema()`) advertises the new per-decoder
`dispatch:` key and the top-level `transport:` section (the removed
per-decoder `transport:` key is gone from the schema too), with a python
`jsonschema` test that validates a populated `dispatch`/`transport`
document against it and rejects the removed key. The python config
bindings expose the new fields as well -- `decoder_config.dispatch`,
`multi_decoder_config.transport`, and the `decoder_dispatch` /
`transport_config` / `transport_shape_override` types.

## Sample configurations

Three representative deployment shapes (YAML config + exact launch line
+ matching caller config):

**Two host decoders, one udp ring per decoder** -- the YAML says nothing
about the wire, so `--transport` (default udp) selects it per launch;
the READY line publishes every ring's endpoint.

```yaml
decoders:
  - id: 0
    type: pymatching
    # block_size / syndrome_size / H_sparse / ...
  - id: 1
    type: pymatching
    # ...
```

```
decoding_server --config=decoders.yaml
# QEC_DECODING_SERVER_READY port=<P0> transport=udp ring0=<P0> ring1=<P1>
```

Caller routes per decoder with device-scoped endpoint args:
`--cudaq-device-call=udp udp-host=127.0.0.1 udp-port=<P0>
udp-port.1=<P1>`.

**Mixed dispatch (host CPU decoder + GPU device-graph decoder), runs on
any CUDA box** -- the wire lives in the YAML transport section; the
shape-keyed override adds `--pinned-rings` only to the device_graph ring
so the GPU scheduler can poll it. No `--transport` on the command line
(combining both is a startup error). This is exactly what the new gated
ctest runs.

```yaml
transport:
  provider: udp
  device_graph:
    args: [--pinned-rings]
decoders:
  - id: 0
    type: multi_error_lut
    # ...
  - id: 1
    type: nv-qldpc-decoder
    dispatch: device_graph
    # ...
```

```
decoding_server --config=decoders.yaml
```

**Partner transport drop-in** -- an out-of-tree provider library is
named by path, verbatim; its args are forwarded untouched. Zero
decoding-server changes.

```yaml
transport:
  provider: /opt/partner/libpartner_bridge.so
  args: [--lane=3]
decoders:
  - id: 0
    type: pymatching
    # ...
```

```
decoding_server --config=decoders.yaml
```

## Validation

- Two-process suite 6/6 (udp; includes per-decoder rings and the
YAML-transport-section tests), decoding-server core + YAML suites,
per-decoder-rings app example.
- Re-validated after merging main with #670 landed
(#656/#673/#674/#678/#680/#683/#690/#691/#692): two-process suite,
DecoderYAMLTest 18/18, per-decoder-rings + mixed-dispatch app tests,
python surface_code-1, and the full mixed E2E all green against an
nv-qldpc plugin rebuilt for #674's virtualized decoder API (an ABI break
for out-of-tree plugin builds).
- Full mixed E2E on a WSL2 laptop, two-process, wire named only by the
YAML: multi_error_lut on a host ring + nv-qldpc RelayBP behind the GPU
device-graph scheduler on a pinned-udp ring -- decode graph fired for
every window (trigger rc=0, fires == tail_relaunches), correct
corrections, both rings dispatched, clean teardown. The HSB rig is no
longer required to exercise the device-graph dispatch path; rig runs
remain the validation for the hololink and cpu_roce RDMA data planes.

## Breaking changes vs. main

All of these surfaces are new this release cycle, so no deprecation
aliases are kept:

- The per-decoder `transport:` YAML key and the `DecoderTransport` enum
are replaced by per-decoder `dispatch: host|device_graph` plus the
top-level `transport:` section. A config using the old key fails to
parse loudly. `cuda_device_id` (#690) is kept unchanged and now also
places the device_graph rings and scheduler.
- The device-graph transceiver's environment surface is
`QEC_DEVICE_GRAPH_*` (was `HOLOLINK_*`), and `--transport=gpu_roce` no
longer exists: device_graph is a dispatch shape named in the YAML;
`--transport` only ever names a wire provider. The HSB test script is
updated to match (injects `dispatch: device_graph` + `cuda_device_id`,
sets `QEC_DEVICE_GRAPH_*`, waits for `READY device_graph`).
- `CpuRoceTransceiver` (the always-throwing placeholder) is removed;
cpu_roce is served by its bridge provider.

## Dependencies

- **Built on the companion cuda-quantum PR
[#4915](NVIDIA/cuda-quantum#4915 ([realtime]
pluggable transport providers / bridge interface v2 + per-device
device_call sessions), **merged upstream 2026-07-23** (squash `2a7911f`)
and itself built on cuda-quantum #4869. `.cudaq_version` pins that
squash commit, so this branch builds against stock main. This PR
consumes `cudaq_bridge_create_from_library`, the v2 endpoint/geometry
queries, per-device sessions and device-scoped channel args, the udp
provider's `--pinned-rings`, and `cudaq_dispatch_get_trigger_debug`.
- **Built on main with #670 landed** (origin/main is merged into this
branch). Relative to landed #670, this PR completes the follow-up its
review called for -- removing the server's transport awareness:
`GpuRoce{Transceiver,Factory,LinkCheck}` become `DeviceGraph*` (same
weak-factory / optional-component structure, now transport-blind), while
#670's post-review hardening (ring-size and host-page-alignment
validation, `cuda_device_id`-driven GPU placement threaded through the
factory) is preserved in the renamed files.
- The nv-qldpc decoder plugin and the proprietary cudevice archive are
consumed as prebuilt binaries
(`CUDAQ_QEC_REALTIME_CUDEVICE_PROPRIETARY_ARCHIVE`); builds without them
still compile and the device-graph paths fail at runtime with a clear
not-linked error, and the gated ctest is simply not registered.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Ben Howe <bhowe@nvidia.com>
Signed-off-by: Chuck Ketcham <cketcham@nvidia.com>
Co-authored-by: Chuck Ketcham <cketcham@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
cketcham2333 added a commit that referenced this pull request Aug 3, 2026
…er from an FPGA or a QPU kernel (#703)

## What

Adds a shipped example
(`docs/sphinx/examples/qec/realtime_decoding_demo/`) that
compiles against the installed SDK — installed headers/libs only — and
exercises
the delivered `decoding_server` end-to-end, plus its docs/CI
registration and an
FPGA RX-ring sizing fix in the unittests driver.

The example source is the refactored `surface_code-1.cpp` from #685 (new
syndrome
format, `decoder_context_from_memory_circuit`, seeded runs), copied with
three
adaptations: the header comment, the usage line, and a true-width
`--save_syndrome`
capture so the FPGA playback tool replays exact per-round measurement
counts.

## The example

One source, built two ways: a generator (decoder config + FPGA syndrome
file) and
the lowered `-cqr` kernel (`-frealtime-lowering -DQEC_APP_CQR`). One
script, two
syndrome sources, three decoders (`pymatching`, `multi_error_lut`,
`nv-qldpc-decoder`):

- `--source qpu-kernel` — the lowered kernel streams syndromes to the
server over
  UDP (no hardware; `QEC_DECODING_SERVER_PORT` routes the device_calls).
- `--source fpga` — the delivered playback tool streams syndromes from a
real
  FPGA over RoCE (`cpu_roce` / `gpu_roce`).

Runs are seeded (`--seed`, default 42), so results are reproducible.
Pass/fail
matches the existing surface-code tests: decoder-failure greps, the
completion
proof, a `num_shots/50` residual logical-error ceiling, an in-process
dispatch
count of 0 (the two-process test's proof the decode stayed in the
server), and a
`num_shots*(num_rounds+3)` server dispatch floor.

## CI

`lib_qec.yaml` builds the example against the installed SDK (enforcing
the
installed-headers-only rule) and runs a seeded 200-shot qpu-kernel
pymatching
decode over UDP loopback on the CPU job — a real decode, no hardware,
and
deterministic pass/fail.

## FPGA ring fix (`hsb_fpga_decoding_server_test.sh`)

Fixes a #683 regression: the playback ring modulus (`--num-pages 512`)
no longer
matches the cpu_roce server's 64-slot RX ring, starving the test after
~10 shots.
Both transports now run a ring equal to the HSB QP's 64-entry receive
WQE depth
(deeper rings drop frames), with a fail-fast preflight of DOCA's
host-page
alignment on 16K/64K-page hosts.

---------

Signed-off-by: Ben Howe <bhowe@nvidia.com>
Signed-off-by: Chuck Ketcham <cketcham@nvidia.com>
Co-authored-by: Ben Howe <bhowe@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants