Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

In TRL v1.14, Hugging Face enabled AsyncGRPOTrainer to support LoRA training, syncing only a few MB of adapters to vLLM. A Storage Bucket mount shares files between the trainer and two vLLM replicas, paired with a proxy that adds auth headers, routes requests by KV prefix, and broadcasts adapter loads—no NCCL cross-machine communication needed, cutting 500 steps from 3 hours 27 minutes to 53 minutes.