Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs

Ai2 released Olmo-core 3, a redesigned open-source MoE training system that scales to trillion-parameter sizes. It switches from FSDP to a DDP-based approach, with a 47B-parameter MoE reaching a throughput of 52 on 8 NVIDIA B300 GPUs.