Experts trainability: bit-exact recompute-in-backward + ExpertsNbit (8/16-bit storage)#1

Merged

pjordanandrsn merged 2 commits into

feature/experts-4bitfrom

feature/experts-4bit-training

Jul 3, 2026

Commits on Jul 3, 2026

feat: generalize Experts4bit to ExpertsNbit with 8-bit and 16-bit storage

pjordanandrsn
and
claude
committed
mem: re-dequantize frozen expert weights in backward instead of saving them

pjordanandrsn
and
claude
committed