Same Cluster, 33 Points More Utilization: What Changed Was the Order

Hugging Face built a constraint-aware GPU allocator and benchmarked it against a FIFO scheduler across seven scenarios. On identical hardware and workloads, GPU utilization rose by up to 33 percentage points (for example, from 53.6% to 87.0% for an 8-GPU training-heavy workload), priority-weighted output increased in every scenario by up to 105%, averaging 52%. The key change is allocating real-time inference demand across time steps along its curve and placing batch-like tasks by priority across the whole scheduling horizon, rather than assigning by arrival order.