CK BLAS backend returns uninitialized memory for fp16 mm on gfx9 (dispatch_half_gemm body is #if 0)

[lohiaj]

torch.backends.cuda.preferred_blas_library("ck") silently returns wrong results for fp16 torch.mm on gfx950. No error, no NaN, no warning. bf16 on the same path is correct.

Repro

import torch
a = torch.randn(1024, 1024, device='cuda', dtype=torch.float16)
b = torch.randn(1024, 1024, device='cuda', dtype=torch.float16)
ref = a.float() @ b.float()
torch.backends.cuda.preferred_blas_library("ck")
c = torch.mm(a, b); torch.cuda.synchronize()
print((c.float() - ref).abs().max())   # 475.25

Observed

shape	max_err vs fp32 ref	nonzero	run-to-run
128^3	549.5	87.7%	same
512^3	109.9	100%	differs
1024^3	475.3	100%	same
4096^3	830.1	87.5%	same

bf16 on the identical harness is correct at every size (max_err 0.125 / 0.25 / 0.496 / 1.0), so the test is sound and the fault is specific to fp16.

The 512^3 case differs between two consecutive calls, and two sizes contain ~12% exact zeros, both consistent with an output buffer that is never written. Timing agrees: the fp16 path measures ~2.4M TFLOP/s, roughly 1000x peak, because no kernel is launched.

Root cause

aten/src/ATen/native/hip/ck_gemm_half.hip, still present on main:

void dispatch_half_gemm(CUDABLAS_GEMM_ARGTYPES(at::Half)) {
#if 0
  ... 284 lines ...
#endif
}   // empty body

and the dispatcher:

if (isGPUArch(wmma_archs))     dispatch_half_gemm_wmma(...);  // gfx11/12, implemented
else if (isGPUArch({"gfx9"}))  dispatch_half_gemm(...);       // gfx950 lands here, no-op
else TORCH_CHECK(false, "gemm_internal_ck<at::Half> unsupported gfx arch");

gfx950 matches "gfx9" and calls the empty function, so the output tensor is never written. ck_gemm_bfloat16.hip has no #if 0, which is why bf16 works.

Composable Kernel itself is not involved here. No CK kernel is reached, because no call is made.

This looks like a leftover from pytorch#143971, which enabled the fp16 CK path for gfx11 WMMA while the gfx9 body stayed disabled and the dispatcher kept routing to it.

Suggested fix

Minimal: route gfx9 to the existing TORCH_CHECK(false, ...) so an unsupported configuration fails loudly instead of returning uninitialized memory. Restoring the disabled implementation is the larger fix, but the silent-wrong-answer behaviour should not survive either way.

Environment

torch 2.15.0.dev20260815+rocm7.2, ROCm 7.2.1 (HIP 7.2.53211), MI355X (gfx950), USE_ROCM_CK_GEMM enabled.

Note

The file lives upstream in pytorch/pytorch and is unchanged on origin/main there, so a fix will need to land upstream. Filed here for ROCm triage. No existing issue found in either repo for ck_gemm_half or dispatch_half_gemm.

cc @geozhai