feat(integrations): expert_offload — single-GPU CPU offload of frozen 4-bit MoE experts#3797

Open

pjordanandrsn wants to merge 5 commits into

axolotl-ai-cloud:mainfrom

pjordanandrsn:expert-offload-integration

Commits on Jul 13, 2026

feat(integrations): expert_offload — single-GPU CPU offload of frozen 4-bit MoE experts

pjordanandrsn
and
claude
committed
review: schema-native validation; fail loudly when nothing to offload

pjordanandrsn
and
claude
committed
feat(expert_offload): plain-DDP multi-GPU support

pjordanandrsn
and
claude
committed
fix(expert_offload): support grouped 3D expert stacks (quantize_moe_experts)

pjordanandrsn
and
claude
committed
fix(expert_offload): strip bnb parametrize-cache hooks — checkpoint early-stop leaks the global cache and retains every dequantized expert (pool x4 bytes)
pjordanandrsn
committed