LiveBench · Language and instructions
LiveBench · An equal-weight average of Language and IF, at 50% each.
What it measures and how
Language and IF scores are calculated separately, averaged at 50% each, then rounded to one decimal. This differs from selecting either category alone on the official site. Language measures comprehension; IF measures instruction and formatting constraints, not fiction quality.
How this evidence is used
Retains 3% of the overall writing-and-expression budget. Excluded from the literary-writing category.
Evaluation results
Fixed rules select one representative configuration per public model. Anonymous test models are omitted. Original ranks are retained, with up to 30 models per evaluation.
| Source rank | Source model name | Raw result | Representative configuration |
|---|---|---|---|
| 1 | gemini-3.8-flash-highGoogle | 84.6% | High reasoning |
| 2 | claude-fable-5-max-effortAnthropic | 83.2% | Max reasoning |
| 3 | gemini-3.7-flash-highGoogle | 82.7% | High reasoning |
| 4 | gpt-6-astra-maxOpenAI | 82.5% | Max reasoning |
| 5 | gemini-3.1-pro-preview-highGoogle | 82.2% | High reasoning |
| 6 | gpt-6.1-sol-maxopenai | 82.1% | Max reasoning |
| 7 | claude-fable-5-1-max-effortAnthropic | 81.2% | Max reasoning |
| 8 | muse-spark-1.3-xhighMeta | 80.4% | xHigh reasoning |
| 9 | gemini-3.5-flash-highGoogle | 80.1% | High reasoning |
| 11 | gpt-5.6-sol-maxOpenAI | 79.8% | Max reasoning |
| 12 | gemini-3.6-flash-highGoogle | 79.6% | High reasoning |
| 13 | gpt-5.5-xhighOpenAI | 79.0% | xHigh reasoning |
| 14 | kimi-k3Moonshot AI | 78.4% | Source default |
| 15 | muse-wonder-1.0-max— | 78.0% | Max reasoning |
| 16 | grok-4.6xAI | 77.8% | Source default |
| 17 | grok-4.7-xhighxAI | 77.7% | xHigh reasoning |
| 18 | smaug-agenticAbacus.AI | 77.7% | Source default |
| 19 | grok-4.5xAI | 77.2% | Source default |
| 21 | gpt-6-sol-maxOpenAI | 76.9% | Max reasoning |
| 22 | qwen3.7-maxAlibaba | 76.9% | Source default |
| 23 | qwen3.8-maxAlibaba | 76.9% | Source default |
| 24 | muse-spark-1.2-xhighMeta | 76.4% | xHigh reasoning |
| 25 | gpt-5.4-xhighOpenAI | 76.4% | xHigh reasoning |
| 27 | claude-opus-5-max-effortAnthropic | 76.2% | Max reasoning |
| 28 | claude-opus-5-5-max-effortAnthropic | 76.0% | Max reasoning |
| 29 | qwen3.8-flash-nextAlibaba | 75.9% | Source default |
| 30 | claude-opus-4-8-max-effortAnthropic | 75.8% | Max reasoning |
| 31 | deepseek-v4-flash-vision-expDeepSeek | 75.7% | Source default |
| 32 | deepseek-v4.1-flash-maxDeepSeek | 75.6% | Source default |
| 33 | smaug-miniAbacus.AI | 75.5% | Source default |
Evaluation limits & attribution
The average measures combined performance, not Language alone. Official category and fine-tuned-model filters change scores and rankings. Organizer Abacus.AI also develops Smaug; exclusive results need independent corroboration.
Data license: Apache 2.0
LiveBench publishes the components. AIHOT averages the two categories equally and ranks them. This average and the AIHOT consensus index use different scales and cannot be added directly.