Decoupled DiLoCo: A new frontier for resilient, distributed AI training

Google DeepMind released the Decoupled DiLoCo distributed training architecture, which splits large model training into asynchronously communicating compute islands, isolating hardware failures and sharply reducing bandwidth requirements.