Skip to content

test GB CI for Oceananigans bump#2595

Open
Pangoraw wants to merge 20 commits into
mainfrom
Pangoraw-patch-2
Open

test GB CI for Oceananigans bump#2595
Pangoraw wants to merge 20 commits into
mainfrom
Pangoraw-patch-2

Conversation

@Pangoraw

@Pangoraw Pangoraw commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@Pangoraw Pangoraw changed the title test GB CI for Oceananigans bumo test GB CI for Oceananigans bump Jul 2, 2026
Pangoraw and others added 5 commits July 2, 2026 12:57
Applies the fix from openxla/xla#45444 (spmd-enzyme: Be more
conservative when walking constant DUS chain) on top of the pinned
XLA commit. ProcessUpdatePieceExtractOperand's constant-DUS unwind
loop was unconditionally descending into a nested DynamicUpdateSlice's
update operand even when that update only partially covered the
piece, discarding the real data in the DUS's base operand. This is
the root cause traced for the sharded V=0 correctness bug (V = integral
v dz coming out zero under Y-sharding of the NormalFlow barotropic
velocity field's halo-fill pad).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 31.07%. Comparing base (36b215f) to head (fe01978).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2595      +/-   ##
==========================================
+ Coverage   28.68%   31.07%   +2.38%     
==========================================
  Files         225      225              
  Lines       44480    44512      +32     
==========================================
+ Hits        12760    13833    +1073     
+ Misses      31720    30679    -1041     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Pangoraw

Pangoraw commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author
    metrics = Metrics(
        flops = 0.11671181737040262,  # [flops / (peak flops * program time)], capped at 1.0
        bandwidth_utils = [0.5740172565663073, 0.0, 0.0],
        uncapped_flops = 0.11671181737040262, 
        raw_time = 23.26307480s,  # Raw time in seconds
        raw_flops = 8.46749149821572e14,  # Total FLOPs performed
        bf16_flops = 8.46749149821572e14,  # Total FLOPs Normalized to the bf16 (default) devices peak bandwidth
        raw_bytes_accessed = [2.0767217595944e13, 0.0, 0.0],
        occurrences = 0,
        raw_flops_rate = 3.639884912468974e13,  # Raw FLOPs rate in FLOPs/seconds
        bf16_flops_rate = 3.639884912468974e13,  # BF16 FLOPs rate in FLOPs/seconds
    )

it's green again ✔️

@wsmoses

wsmoses commented Jul 15, 2026

Copy link
Copy Markdown
Member

what's the error for f64?

@Pangoraw

Copy link
Copy Markdown
Collaborator Author

f64 GPU was failing because of some CUDA error on the runner, it's good as well now.
TPU still suffers from the initialization problem that openxla/xla#45444 ought to fix 🤔

@wsmoses

wsmoses commented Jul 20, 2026

Copy link
Copy Markdown
Member

@Pangoraw I started a rerun [just merged a later libtpu to reactant, that contains its own xla and needs to be updated to accomodate updated xla]

@wsmoses

wsmoses commented Jul 20, 2026

Copy link
Copy Markdown
Member

@Pangoraw correctness here is restored tpu, seemingly?

@Pangoraw

Pangoraw commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator Author

Correctness looks good, flops-wise not so much on tpu?

    metrics = Metrics(
        flops = 0.0006474319815451959,  # [flops / (peak flops * program time)], capped at 1.0
        bandwidth_utils = [0.804190223014755, 0.015916926572879797, 0.011573493297569806],
        uncapped_flops = 0.0006474319815451959, 
        raw_time = 27.28843157s,  # Raw time in seconds
        raw_flops = 1.672573072726e13,  # Total FLOPs performed
        bf16_flops = 1.672573072726e13,  # Total FLOPs Normalized to the bf16 (default) devices peak bandwidth
        raw_bytes_accessed = [3.5945906937972e13, 1.0118574451204e13, 5.093594964828e12],
        occurrences = 0,
        raw_flops_rate = 6.12923856928837e11,  # Raw FLOPs rate in FLOPs/seconds
        bf16_flops_rate = 6.12923856928837e11,  # BF16 FLOPs rate in FLOPs/seconds
    )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants