Skip to content

Use yaml as the single source of truth for gpu_roce parameters - #789

Open
melody-ren wants to merge 9 commits into
NVIDIA:mainfrom
melody-ren:melodyr/yaml-single-source-of-truth-hsb-vars
Open

Use yaml as the single source of truth for gpu_roce parameters#789
melody-ren wants to merge 9 commits into
NVIDIA:mainfrom
melody-ren:melodyr/yaml-single-source-of-truth-hsb-vars

Conversation

@melody-ren

@melody-ren melody-ren commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Make YAML the single source of truth for GPU RoCE and HSB/FPGA transport configuration.

Previously, the same settings could reach the decoding server through YAML, command-line arguments, or QEC_DEVICE_GRAPH_* environment variables. This created ambiguous precedence and made the effective runtime configuration difficult to reproduce.

What changed

  • Require YAML transport configuration for:
    • all gpu_roce dispatch paths;
    • all device_graph dispatch paths.
  • Remove GPU RoCE and device-graph configuration through environment variables.
  • Reject conflicting transport arguments, including --gpu; decoder.cuda_device_id is now authoritative.
  • Pass the resolved YAML configuration directly into the device-graph transceiver.
  • Update the HSB FPGA demo and test helpers to:
    • copy the source configuration into a temporary launch YAML;
    • add runtime-discovered endpoint details to that YAML;
    • leave the source YAML unchanged;
    • launch the server using only the resulting YAML configuration.
  • Use the same 128-byte-aligned GPU RoCE page stride for allocation validation, playback, and the provider.
  • Parse sparse decoder matrices through CUDA-QX's canonical YAML parser instead of manually scraping YAML syntax. This supports both flow-style and block-style YAML produced by PyYAML.

Host-only, non-GPU-RoCE configurations retain the existing CLI fallback when no provider is specified in YAML.

Testing

Local container verification:

  • Built the affected realtime configuration and YAML test targets.
  • Passed:
    • DecoderYAMLTest.RejectsParserErrors
    • DecoderYAMLTest.AcceptsBlockStyleSparseMatrices
    • DecoderYAMLTest.TransportSectionAndMixedDispatch
  • Ran formatting and git diff --check.

GB200 hardware verification using the private NV-QLDPC plugin and proprietary cudevice archive:

  • extra/hsb-fpga-server/gpu_roce — PASS
  • extra/gpu-roce-qldpc-bridge — PASS
  • examples/fpga/device-graph/nv-qldpc-decoder — PASS

Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
Signed-off-by: Melody Ren <melodyr@nvidia.com>
@melody-ren
melody-ren marked this pull request as ready for review August 15, 2026 05:33
@melody-ren

Copy link
Copy Markdown
Collaborator Author

Verified on GB200 using Chuck's handy CI script

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant