Skip to content

Fix Gemma remat static argument for dropout - #1021

Open
primorLee wants to merge 1 commit into
Physical-Intelligence:mainfrom
primorLee:fix/gemma-remat-static-arg
Open

Fix Gemma remat static argument for dropout#1021
primorLee wants to merge 1 commit into
Physical-Intelligence:mainfrom
primorLee:fix/gemma-remat-static-arg

Conversation

@primorLee

Copy link
Copy Markdown

Summary

  • correct the lifted nn.remat static argument index for Gemma blocks
  • add a regression test that initializes the scanned Gemma module with dropout enabled

Why

Flax Linen's lifted checkpoint transform interprets static_argnums against the original module method signature, including self, and then shifts the indices internally. Block.__call__ therefore needs index 6 for deterministic.

With index 5, adarms_cond is selected instead and deterministic remains traced. Enabling Gemma dropout then reaches Flax's Python boolean branch with a traced value and raises TracerBoolConversionError during model initialization.

The regression test uses the real gemma.Module remat + scan + dropout path with a one-layer dummy config and a reduced vocabulary for speed.

Fixes #987

Testing

  • pytest -q src/openpi/models/gemma_test.py (1 passed)
  • pre-commit run --files src/openpi/models/gemma.py src/openpi/models/gemma_test.py

@primorLee
primorLee requested a review from kvablack as a code owner August 22, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

In the setup function inside the Module class in gemma.py, should the static_argnums parameter in nn.remat be set to (5,) or (6,)?

1 participant