Research/bf16 phase0 review - #129
Closed
DeanTMaxim wants to merge 203 commits into
Closed
Conversation
…ications ContractionAlgebra + Representation ABCs (boundary encode/decode) activated via set_contractor(algebra=...); routed in-source by cons._algebraic_base_contraction (no monkey-patch). Two reference applications in applications/: tropical (max-plus ground-state energy, configuration recovery, degeneracy counting; arXiv:2008.06888) and complex<bfloat16> pair-algebra (4M real bf16 matmul; boundary encode/decode keeps the pair axis off tn.Node, dodging the axis==edge wall).
…ing concatenation
- Replace prefer_einsum: bool on ContractionAlgebra ABC with get_contractor_kwargs()
method (default {}), keeping the ABC clean of cotengra-specific flags
- ComplexPairAlgebra overrides to return {'prefer_einsum': True}
- cons.py unpacks via **alg.get_contractor_kwargs()
- De-hardcode float32/complex64 in _pair_to_complex to use cons.rdtypestr/dtypestr
- Remove personal pythonpath config from pyproject.toml and example docstring
- Add unit test for get_contractor_kwargs default behavior
- Tighten test_tropical_public_api_surface to direct import assertions
- Merge base.py into contraction_algebra.py (122 lines total → single module) - Remove subpackage directory (__init__.py + base.py → contraction_algebra.py) - Fix deep imports in cons.py and test_contraction_algebra.py
- Tighten GHZ-8 test tolerance (5e-2 → 1.5e-2)
- Replace fake_node string with real tn.Node in merge_single_gates test
- Fix 'Opt-in' comment → 'ALGEBRAIC EXECUTION PATH'
- Document thread-safety limitation of _aux_outputs_store
- Explain be/kbe naming and einsum diagonal safety
- Inline _expand_to_layout_pair (single call-site)
- Fix CHANGELOG: set_contractor → set_contraction_algebra
- Revert unrelated black version pin in requirements-dev.txt
- Add _stash_aux_outputs({}) to legacy contraction path
…ebra - _contraction_algebra defaults to None; _standard singleton for fallback - _ns_alg check uses is not None (not isinstance) - _algebraic_base_contraction uses alg or _standard - Zero overhead for default users: legacy path runs with no diff - Cleaner branch, better backward compat and reviewer friendliness
…ified / E1 per-backend)
…test(bf16): native single-operand einsum on GPU; chore: pylint import order
tc-ng's pytorch backend creates tensors with no device= (defaults to CPU), so the
'GPU' benchmark silently ran on CPU: peak_alloc_bytes=0 and constant peak_smi
(CUDA context baseline) across all dtypes/n. Probe confirmed convert_to_tensor and
state() both reported device=cpu, 0 GPU bytes allocated.
Add _setup_gpu_device(name, gpu): torch.set_default_device(f'cuda:{gpu}') for
pytorch routes all tensor creation to CUDA with zero tc code changes; jax/tf/cupy
reach the GPU on their own (jax via XLA, tf via visible GPU + memory growth set in
_disable_prealloc, cupy GPU-native), so they are no-ops. Wired into worker and
micro_worker after set_backend.
Also fix the latent CPU assumption in contract() and micro_worker: np.asarray on a
CUDA tensor raises ('can't convert cuda:0 device type tensor to numpy'), so use
be.numpy() (cpu().numpy()) for host conversion. Exposed by the device fix.
Found during L3 Task 10 on RTX 5070 Ti (Blackwell sm_120).
…m_120) Smoke: 3/3 pytorch GPU tests pass. K3: native bf16 Tensor-Core GEMM confirmed via torch.profiler (cutlass_80_tensorop_bf16_s16816gemm kernel; 48 vs 12 TFLOPS, 4.06x over fp32) — substitutes for ncu (unavailable in the tcng env / WSL). Micro 4096^2 complex-bf16 matmul 0.0575 s on GPU (5.9x over CPU). E2e n=14..22 across ghz/brickwork/qaoa-ising; supplementary e2e_pytorch_bign.csv shows the memory inversion (bf16 ~1.5x complex64 peak, OOMs sooner) at ghz n=24..28.
Smoke: 3/3 jax GPU tests pass (jax 0.6.2, CudaDevice(id=0), default backend gpu). K3: StableHLO dump (results/jax_hlo.txt) shows the 4M complex-bf16 contraction lowers to 4 bf16 stablehlo.dot_general ops -> bf16 cuBLAS Tensor Core. Micro 4096^2 matmul 0.052 s on GPU. Important jax nuance: DEFAULT matmul precision (precision=[DEFAULT,DEFAULT] in the HLO) already permits bf16/TF32 Tensor-Core accumulation for complex64 too, so the complex64 baseline is itself reduced-precision (ghz amplitude 0.70703125 = bf16(1/sqrt2)), making bf16 accuracy identical (ghz max_abs_err=0.0) for 4 dots vs the 2 a native complex matmul uses -> strictly more work. brickwork/qaoa show 7e-5..4e-4 (paths not bit-equal). E2e n=14..22 ghz/brickwork/qaoa-ising: bf16 ~0.4-0.7x complex64 speed, ~1.05x peak_smi -> confirms pytorch's inverted-premise finding (no contraction-level win; XLA does NOT fuse away the 4-matmul overhead). peak_alloc null for jax (no max_memory_allocated). Big-n omitted: XLA JIT compile of large pair network + cotengra pathfinding made a single n=28 trial run >7 min (killed); pytorch big-n already shows the memory inversion.
re + 1j*im fails on tensorflow eager (TypeError: cannot convert 1j to EagerTensor of dtype float) — numpy/jax/torch/cupy accept a Python 1j on a float tensor, TF does not. Use be.i(dtype) (1j as a backend tensor, implemented by all 5 backends) so the complex recombine works uniformly. Behaviour identical for numpy/jax/torch/cupy; unblocks TF. Found during L3 Task 11 GPU testing on RTX 5070 Ti (TF 2.21). Verify: TF round-trip [1+2j,3-1j] correct; GPU smoke 18 passed / 3 skipped (cupy), no regression.
… (sm_120) Smoke: 3/3 tensorflow GPU tests pass (after the _pair_to_complex fix in b36d18e). TF 2.21 GPU on sm_120: no prebuilt Blackwell kernels -> PTX JIT (CUDA module cache makes small kernels fast: ~6s/trial incl. import). Two env fixes needed and documented: (a) TF needs the nvidia-*-cu12 wheel libs on LD_LIBRARY_PATH (set in the run harness); (b) the tensorflow[cuda] install had left nvidia/nvjitlink/lib empty -> force-reinstalled nvidia-nvjitlink-cu12==12.8.93 to restore libnvJitLink.so.12. torch/jax verified intact. Micro 4096^2 bf16 matmul 0.0455 s. E2e n=14..22: bf16 0.26-0.47x complex64 speed, mem ratio 0.85-1.08 (no win), accuracy 7.5e-5..4.3e-4 (true fp32 complex64 baseline, unlike jax's DEFAULT-precision degeneracy). Confirms the inverted-premise finding on a 3rd backend.
…st host-convert) Three cupy gaps found during L3 Task 12 GPU testing on RTX 5070 Ti (cupy 14.1.1): 1. _bf16_dtype cupy branch returned cupy.bfloat16, which does not exist -> use ml_dtypes.bfloat16 (cupy accepts it natively; astype + bf16 cuBLAS GEMM both work, same resolution as numpy). 2. tensornetwork's cupy backend has not implemented einsum (NotImplementedError), so _einsum_single_operand_half routed cupy to cupy.einsum directly (cupy.einsum exists, runs bf16 on cuBLAS; np.diagonal on cupy also works as an alternative). 3. GPU smoke test build() used np.asarray(c.state()), which fails on cupy (cupy arrays need explicit .get()) -> use tc.backend.numpy() uniformly (matches the harness). Verify: full GPU smoke + numpy kernel 21 passed across numpy/jax/pytorch/tensorflow/cupy.
Smoke: 3/3 cupy GPU tests pass (after the cupy compat fixes in 7dc31d7). cupy 14.1.1, bf16 via ml_dtypes.bfloat16, cuBLAS GEMM. Micro 4096^2 bf16 matmul 0.0367 s (leanest of the four backends). E2e n=14..22: bf16 0.53-0.94x complex64 speed, mem ratio 0.85-1.07 (no win), accuracy 7.5e-5..4.3e-4. Confirms the inverted-premise finding on all four tc GPU backends (pytorch/jax/tensorflow/cupy).
…cing, all inputs in X''') Y''' binds Git tree X''' (9c27636) to 5 evidence file hashes. validate_review_subject=True: every manifest input is retrievable from X''' via git show (F6a clean snapshot). dirty_worktree=False; subject_commit != aggregation source_commit (fc35d75) -> non-self-referencing.
…content hashes; re-verify region judgment + strict bool; reject unknown self-report + cross-path attempted
….json (non-self-referencing X''''); subject X''''
…lf-referencing, dirty=False)
…ectness (3 seeds)
…avoidance test assertions
…fused_full_anchor_run=true)
…ute MEASURED PASS verdict (review finding tensorcircuit#1)
…s, real cancellation_v2 measured)
… binding requires hash+verdict allowlist; aggregate strict source=measured; run_context provenance (reviewer B findings)
…OLICY_NOT_ACCEPTED -> continuous local-gate, in-repo, freeze manifest, deprecated max_rel)
…_ACCEPTED v2 -> metric bool/real distinction, NEW fields no overloading, SHA-1 git identity, kernel-variant-bound freeze manifest)
…etrics_dual_gate + apply_policy_region_fused + NEW full_anchor_correctness fields + c2.py reads new fields (17 mutation tests)
…v3 dual-gate fields (worst_local_scaled_max, global_rel_l2)
…te gonogo_test to expect honest UNKNOWN/CONFLICT state
…_ACCEPTED v3 -> fix self-reference, placeholder identities, per-cell/summary schema split, freeze-then-measure flow)
…gative-value fail-open + v4 independent summary schema (reviewer B v3)
…endent worst-global/worst-local, any_nan_inf)
…R_ACCURACY_v5 (B POLICY_ACCEPTED 30a0048)
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This draft PR publishes the current BF16 Phase 0 GPU research package for TC-NG review. It is an evidence and methodology review point, not a request to merge a production BF16 API or to authorize Phase 1.
This work is stacked on #125. Until that dependency lands, GitHub may also show the underlying contraction-algebra commits in this PR's comparison.
Current decision
phase0_completion = INCONCLUSIVEphase1_authorization = NOT_AUTHORIZEDVIABLEroutes: 0Key measured findings
region_fused/directis NOT_VIABLE.Reviewer entry points
phase0_reviewer_briefing.md— project context, evidence interpretation, and review checklist.phase0_full_report.md— complete technical report and artifact index.region_fused_v5_research_report.md— focused 18-cell accuracy result.region_prototype_accuracy.csv— raw 18-cell measurements.Scope boundaries
Review requested
Please review the research methodology, evidence provenance, numerical policy application, and the interpretation of the current Phase 0 boundary. Follow-up implementation and cleanup can be proposed separately after the evidence is understood.