Skip to content

Add cuda.coop TopK with Numba-CUDA-MLIR - #11512

Open
tpn wants to merge 2 commits into
codex/cuda-coop-numba-radix-rank-sortfrom
codex/cuda-coop-numba-topk
Open

tpn wants to merge 2 commits into
codex/cuda-coop-numba-radix-rank-sortfrom
codex/cuda-coop-numba-topk

Conversation

@tpn

@tpn tpn commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Why this is needed

A kernel often needs only the smallest or largest keys in its block's tile.
This adds common and Numba-CUDA-MLIR TopK operations without requiring a
full sort:

selected_keys, selected_indices = coop.topk_max_pairs(
    block, keys, indices, k=8, valid_items=count
)

Behavior

  • Add min/max keys and pairs for one-dimensional blocks, with numeric
    payloads and static or runtime k and valid_items controls.
    The core participation contract records that k and supplied
    valid_items must be uniform across the block.
  • Preserve inputs and key/value association. Only the first
    min(k, valid_items) blocked output positions are defined. The selection
    is unordered, and ties have no ordering or selection guarantee.
  • Support automatic scratch or explicit TempStorage, with the existing
    reuse-synchronization contract. The qualified API also accepts local arrays.
  • Validate counts before narrowing to CUB's integer ABI, with host,
    compiler, GPU, and isolated runtime-trap coverage.

The provider isolates the bundled private cub::detail::block_topk
implementation behind one compatibility shim. It uses the current CUB
signature; bit-range controls and Warp TopK are not exposed.

Stacked on #11511. User guides, expanded API documentation, and
executable examples are added in downstream #11320.

Validation

A fresh installed wheel at the final stack tip passed 751 host/packaging
checks on each of Python 3.10 and 3.14, plus 54 focused TopK and backend
unit/compiler checks with GPUs hidden. Six host regressions reproduced
the missing uniform-count preconditions before the correction.

The family previously passed 56 GPU tests on CUDA 13.3 with
Numba-CUDA-MLIR 0.5.2. The uniform-count correction changes declarative
planner metadata; provider and compiler lowering code are unchanged.
Source, wheel, installed module, and bundled-header bytes were compared.
Wheel validation and changed-file pre-commit passed.

@copy-pr-bot

copy-pr-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-project-automation github-project-automation Bot moved this to Todo in CCCL Sep 18, 2026
@cccl-authenticator-app cccl-authenticator-app Bot moved this from Todo to In Progress in CCCL Sep 18, 2026
@tpn
tpn marked this pull request as ready for review September 18, 2026 23:34
@tpn
tpn requested a review from a team as a code owner September 18, 2026 23:34
@tpn
tpn requested review from rwgk and a lite review from Copilot and removed request for a team and Copilot September 18, 2026 23:34
@cccl-authenticator-app cccl-authenticator-app Bot moved this from In Progress to In Review in CCCL Sep 18, 2026
@tpn

tpn commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

Please review this PR against its stacked base, focusing on correctness,
compiler integration, synchronization, input/output contracts, and meaningful
regressions. TopK is based on Radix Rank/Sort (#11511), which follows Merge Sort (#11510).

Expanded API docstrings, user guides, and runnable documentation examples are
in the downstream docs PR #11320; launch-bounds inference follows in #11392.
Consider those layers when assessing apparent omissions, and distinguish an
actual defect in this PR from an intentional downstream addition. Keep optional
refactoring and unrelated features separate from actionable findings.

@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

@tpn important: I will review #11512 against its stacked base. I will focus on correctness, compiler integration, synchronization, input/output contracts, and regressions. I will treat the documented downstream additions as out of scope unless they cause an implementation defect.

✅ Action performed

Full review finished.

@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: NVIDIA/cccl/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b531b36f-f5bb-453e-a49a-bf2216c09f79

📥 Commits

Reviewing files that changed from the base of the PR and between ff7a94c and 14e0f01.

📒 Files selected for processing (2)
  • python/cuda_coop/cuda/coop/_core/group/topk.py
  • python/cuda_coop/tests/contracts/core/test_core_group_topk.py

Included review availability: Your plan provides up to 12 included reviews per hour; 5 remain after this review.


📝 Summary

Summary by CodeRabbit

  • New Features

    • Added block-level Top-K selection for minimum or maximum keys.
    • Added matching key-value pair selection for minimum or maximum results.
    • Supports partial tiles, configurable selection counts, and optional temporary storage.
    • Available through portable and Numba-MLIR public APIs.
    • Results preserve input data and return an unsorted selected prefix.
  • Bug Fixes

    • Added validation for selection controls, data types, dimensions, and key-value compatibility.

Walkthrough

Changes

The pull request adds block TopK selection for minimum and maximum keys or key-value pairs. It adds CUB-backed planning and lowering, public exports, typed declarations, runtime controls, temporary storage support, and compilation, runtime, contract, packaging, and typing tests.

Block TopK

Layer / File(s) Summary
Block TopK contracts
python/cuda_coop/cuda/coop/_core/block/topk.py, python/cuda_coop/cuda/coop/_core/block/__init__.py, python/cuda_coop/cuda/coop/_core/__init__.py
Adds BlockTopKCoop, BlockTopKSpec, and make_block_topk_spec. The implementation validates counts, block shape, selection mode, bindings, and tile-size limits.
Group TopK planning
python/cuda_coop/cuda/coop/_core/group/topk.py, python/cuda_coop/cuda/coop/_core/group/__init__.py, python/cuda_coop/tests/contracts/core/test_core_group_topk.py
Adds group semantics and planning for complete block groups. The planner creates result contracts, classifies static and runtime controls, and targets CUB block TopK.
Public TopK APIs
python/cuda_coop/cuda/coop/_core/api/*, python/cuda_coop/cuda/coop/__init__.pyi, python/cuda_coop/cuda/coop/numba_mlir/*
Exports typed and runtime APIs for minimum and maximum key-only and key-value TopK operations.
Compiler and CUB lowering
python/cuda_coop/cuda/coop/numba_mlir/_compiler/_group_topk.py, python/cuda_coop/cuda/coop/numba_mlir/_lowering/_topk.py, python/cuda_coop/cuda/coop/numba_mlir/_group_topk.py
Adds payload inference, control validation, temporary-storage planning, output construction, rewrite registration, and CUB lowering factories.
Behavior validation
python/cuda_coop/tests/backends/numba_mlir/{compile,runtime,unit}/*, python/cuda_coop/tests/contracts/core/test_core_block_topk.py
Tests compilation, selection results, partial and empty prefixes, input preservation, ties, signed zero, invalid controls, synchronization, chaining, and unsupported planning cases.
Packaging and typing surfaces
python/cuda_coop/tests/packaging/*, python/cuda_coop/tests/backends/numba_mlir/unit/test_public_foundation.py
Checks wheel contents, public callable identity, valid typing calls, and invalid TopK argument combinations.

Priority: ➖ Normal

Change: Feature

Merge Risk: ⚪ Minimal · up to 14e0f

The added TopK planning contract coverage matches the recorded uniform participation requirements. No actionable merge risk remains in the reviewed changes.


Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
python/cuda_coop/cuda/coop/numba_mlir/_compiler/_group_topk.py (1)

52-55: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

suggestion: guard the dtype during payload inference.

_infer_payload runs before _lower_topk. If both dtype lookups remain None, validation can fail during inference. Add the diagnostic before _validate_common_numeric_dtype:

        if dtype is None:
            raise GroupRewriteError(f"topk could not infer {name} dtype")

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: NVIDIA/cccl/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c07daef7-5afc-4974-a0f2-6b9d84983760

📥 Commits

Reviewing files that changed from the base of the PR and between ec442e5 and 8e1bbbc.

📒 Files selected for processing (25)
  • python/cuda_coop/cuda/coop/__init__.pyi
  • python/cuda_coop/cuda/coop/_core/__init__.py
  • python/cuda_coop/cuda/coop/_core/api/__init__.py
  • python/cuda_coop/cuda/coop/_core/api/__init__.pyi
  • python/cuda_coop/cuda/coop/_core/api/topk.py
  • python/cuda_coop/cuda/coop/_core/api/topk.pyi
  • python/cuda_coop/cuda/coop/_core/block/__init__.py
  • python/cuda_coop/cuda/coop/_core/block/topk.py
  • python/cuda_coop/cuda/coop/_core/group/__init__.py
  • python/cuda_coop/cuda/coop/_core/group/topk.py
  • python/cuda_coop/cuda/coop/numba_mlir/__init__.py
  • python/cuda_coop/cuda/coop/numba_mlir/__init__.pyi
  • python/cuda_coop/cuda/coop/numba_mlir/_compiler/_group_topk.py
  • python/cuda_coop/cuda/coop/numba_mlir/_group_topk.py
  • python/cuda_coop/cuda/coop/numba_mlir/_group_topk.pyi
  • python/cuda_coop/cuda/coop/numba_mlir/_lowering/_topk.py
  • python/cuda_coop/tests/backends/numba_mlir/compile/test_topk_compile.py
  • python/cuda_coop/tests/backends/numba_mlir/runtime/test_topk.py
  • python/cuda_coop/tests/backends/numba_mlir/unit/test_public_foundation.py
  • python/cuda_coop/tests/backends/numba_mlir/unit/test_topk_planning.py
  • python/cuda_coop/tests/contracts/core/test_core_block_topk.py
  • python/cuda_coop/tests/packaging/test_wheel.py
  • python/cuda_coop/tests/packaging/typing/invalid_consumer.py
  • python/cuda_coop/tests/packaging/typing/numba_consumer.py
  • python/cuda_coop/tests/packaging/typing/portable_consumer.py

Included review availability: Your plan provides up to 12 included reviews per hour; 9 remain after this review.

@tpn
tpn force-pushed the codex/cuda-coop-numba-topk branch from 8e1bbbc to ff7a94c Compare September 18, 2026 23:58
Copilot AI lite review requested due to automatic review settings September 18, 2026 23:58
@tpn
tpn force-pushed the codex/cuda-coop-numba-radix-rank-sort branch from ec442e5 to a987aae Compare September 18, 2026 23:58
@tpn

tpn commented Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

This PR was restacked over the read-only input validation correction in Merge Sort #11510. Its own primitive implementation is unchanged; preserve the dispositions from the completed full review.

Please review the current head incrementally. Expanded documentation remains in #11320, followed by launch-bounds inference in #11392. Focus on credible correctness or contract findings within this PR.

@coderabbitai

coderabbitai Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

important: I will review the current head incrementally. I will preserve the completed review dispositions and limit new findings to credible correctness or contract regressions in this PR.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical contract and moderate packaging and validation issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity · 2 Medium severity

Open (3)
What changed in this PR

Adds block TopK key and key-pair operations with CUB and Numba-CUDA-MLIR integration, validation, typing, and GPU coverage.

Changes:

  • Adds min/max TopK APIs and backend lowering.
  • Adds planning, storage, and count validation.
  • Expands tests, exports, typing, and wheel checks.

Final findings include one critical uniformity-contract issue and moderate issues involving temp_storage validation and independent packaging allowlists.

File Summary
python/​cuda_coop/​tests/​packaging/​typing/​portable_consumer.py Portable TopK typing coverage.
python/​cuda_coop/​tests/​packaging/​typing/​numba_consumer.py Qualified TopK typing coverage.
python/​cuda_coop/​tests/​packaging/​typing/​invalid_consumer.py Invalid-call diagnostics.
python/​cuda_coop/​tests/​packaging/​test_wheel.py Wheel membership and header checks.
python/​cuda_coop/​tests/​contracts/​core/​test_core_block_topk.py Core TopK contract tests.
python/​cuda_coop/​tests/​backends/​numba_mlir/​unit/​test_topk_planning.py Planning validation tests.
python/​cuda_coop/​tests/​backends/​numba_mlir/​unit/​test_public_foundation.py Public operation registration checks.
python/​cuda_coop/​tests/​backends/​numba_mlir/​runtime/​test_topk.py GPU runtime behavior tests.
python/​cuda_coop/​tests/​backends/​numba_mlir/​compile/​test_topk_compile.py Provider compilation tests.
python/​cuda_coop/​cuda/​coop/​numba_mlir/​_lowering/​_topk.py TopK provider factories.
python/​cuda_coop/​cuda/​coop/​numba_mlir/​_group_topk.pyi Qualified TopK type declarations.
python/​cuda_coop/​cuda/​coop/​numba_mlir/​_group_topk.py Qualified TopK API.
python/​cuda_coop/​cuda/​coop/​numba_mlir/​_compiler/​_group_topk.py TopK planning and lowering.
python/​cuda_coop/​cuda/​coop/​numba_mlir/​__init__.pyi Qualified export typing.
python/​cuda_coop/​cuda/​coop/​numba_mlir/​__init__.py Qualified runtime exports.
python/​cuda_coop/​cuda/​coop/​_core/​group/​topk.py Group planning semantics.
python/​cuda_coop/​cuda/​coop/​_core/​group/​__init__.py Group semantic exports.
python/​cuda_coop/​cuda/​coop/​_core/​block/​topk.py CUB TopK specification and validation.
python/​cuda_coop/​cuda/​coop/​_core/​block/​__init__.py Block specification exports.
python/​cuda_coop/​cuda/​coop/​_core/​api/​topk.pyi Portable TopK declarations.
python/​cuda_coop/​cuda/​coop/​_core/​api/​topk.py Portable TopK API.
python/​cuda_coop/​cuda/​coop/​_core/​api/​__init__.pyi API type exports.
python/​cuda_coop/​cuda/​coop/​_core/​api/​__init__.py API runtime exports.
python/​cuda_coop/​cuda/​coop/​_core/​__init__.py Core symbol exports.
python/​cuda_coop/​cuda/​coop/​__init__.pyi Public TopK exports.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread python/cuda_coop/cuda/coop/_core/group/topk.py
Comment thread python/cuda_coop/tests/packaging/test_wheel.py
Comment thread python/cuda_coop/tests/packaging/test_wheel.py
tpn added 2 commits September 18, 2026 17:14
Select minimum or maximum keys and pairs with blocked per-thread
results. Preserve inputs, validate wide count controls, and support
caller-managed temporary storage. Isolate the private CUB TopK
dependency in the family provider.

Signed-off-by: Trent Nelson <trent@trent.me>
Record dtypes inferred from indexed ThreadData writes on their original
constructors so copied results remain typed in subsequent group calls.

Exercise core and qualified key/pair chains with irregular block sizes
and add positive and negative public typing consumer coverage.

Signed-off-by: Trent Nelson <trent@trent.me>
@tpn
tpn force-pushed the codex/cuda-coop-numba-topk branch from ff7a94c to 14e0f01 Compare September 19, 2026 00:15
@tpn

tpn commented Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review the current head incrementally. The only change since your completed review is the TopK participation contract recording uniform k and supplied valid_items, with six host regressions. This records a caller precondition; it does not add runtime uniformity enforcement. The fix is folded into the original TopK commit, with the dtype follow-up retained.

Focus on credible correctness and contract regressions in this PR relative to #11511. Downstream #11320 remains documentation, and #11392 handles launch bounds. The two packaging comments have been addressed with evidence of the existing real-wheel CI checks.

@coderabbitai

coderabbitai Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

@tpn important: I will review the current head incrementally. I will focus on credible correctness and contract regressions relative to #11511. I will treat documentation and launch-bounds work as out of scope. I will preserve the prior packaging dispositions.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-actions

Copy link
Copy Markdown
Contributor

🥳 CI Workflow Results

🟩 Finished in 29m 16s: Pass: 100%/8 | Total: 56m 13s | Max: 18m 12s

See results here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

2 participants