Skip to content
Back to the leaderboard

How rankings work

Understand the evidence and methods behind rankings from multiple public evaluations.

0—100

Shared evidence, a clear order.

Combine public evaluations into a complete ranking while minimizing conflict with known results. The overall board and four category boards each display at most 30 models.

The consensus index scales differences in support for the original ranking to 0–100 for comparison. It is neither accuracy nor a percentage capability gap. Near-equal support can produce tied display scores; full evidence still determines the order.

  1. 01

    First, establish model identity

    Normalize model names and versions across boards. For models with different reasoning levels, select one representative configuration using a predefined rule, without cherry-picking the best score. Exclude anonymous test aliases and runs that mix other models into the task. Officially public preview versions are eligible.

  2. 02

    Compare models that were actually evaluated

    Use measured results from the same evaluation. When both results have published uncertainty, small differences within that uncertainty are treated closer to a tie. An unmeasured model contributes no comparison.

  3. 03

    Set evidence weights before combining results

    Each evaluation receives a predefined budget. The overall board requires at least three evaluation organizations, three evidence families and coverage of three capability areas. Repeated collection or display slices from the same source do not create extra voting weight.

  4. 04

    Find the complete ranking with the fewest conflicts

    Evaluations can disagree. Select the order that minimizes net evidence weight opposed to it, then calculate the displayed index separately. Missing results are not filled with zeros, and unpublished scores are not estimated.

A BALANCED VIEW

Composite tests, human blind votes and specialist evaluations.

Composite evaluations receive 30%, human blind votes 10%, and specialist evaluations 60% in total. New evaluations are checked for overlap before receiving a share of the relevant budget. Missing shares are not reassigned to other evaluations.

  • Broad evaluations30%AA Index
  • Human preferences10%Arena Text · Arena Creative Writing
  • Coding and design12%Arena WebDev · DeepSWE v1.1 · TapTap Maker · LiveBench · Coding
  • Writing and expression9%LiveBench · Language and instructions · Creative Writing v3 · Longform Writing
  • Mathematics and reasoning12%LiveBench · Reasoning and mathematics · FrontierMath v2 · Tiers 1–3 · FrontierMath v2 · Tier 4 · Chess Puzzles · Mystery Game Puzzles
  • Knowledge and facts6%SimpleQA Verified · GPQA Diamond
  • Visual understanding6%Arena Vision
  • Tools and office work6%APEX-Agents 1.1 · τ³-Banking
  • Chinese and multilingual ability6%AA Chinese / Multilingual
  • Professional tasks3%Vals Finance Agent

Coding, reasoning, knowledge and professional work use actual evaluations, with category scores calculated independently. Vision and multilingual evidence remain in the overall board. Renaming categories does not add votes for the same result. Overall scores are not arithmetic averages of category scores. Categories usually require two valid evaluations and five comparable models. Knowledge currently uses two Epoch evaluations and discloses the single-organization limitation. Creative preference and web-development evidence remain in the overall board; separate aesthetics and writing boards are not currently published.

You may also want to know

Why can a model rank with relatively few evaluations?
Evaluation count and capability are different. The overall board requires evidence from at least three organizations across capabilities. Most categories require two organizations; knowledge uses two evaluations from one organization and discloses that limitation. Missing scores are not zeros, but missing evidence can still introduce bias.
What does “evidence-sensitive” mean?
This label appears when removing an evaluation or organization, shifting an individual weight by 20%, or changing uncertainty handling produces a rank range of at least three places, causes ineligibility in some scenarios, or leaves comparisons unfinished. The scenario range is not a 95% confidence interval and excludes unknown results.
Does a higher-ranked model always win head to head?
No. A may beat B, B beat C, and C beat A. A complete ranking balances such conflicts, so nonadjacent ranks may differ from an isolated comparison. Mathematical optimality means the least total conflict under the current rules, not proof of a universal capability order for real work.
Why might rankings differ from my experience?
The board summarizes capability supported by public evaluations. Experience also depends on product version, reasoning level, tools and stability on long tasks. New evaluations are used to reassess rankings. When scores are close, a difference of one or two places should not be treated as a clear capability gap.
When will a new model appear?
NexGateHub checks upstream results four times daily. New models can be ranked only after evaluators publish results. Page refreshes, price updates and our collection time do not count as new evaluations.
What happens when a source board is temporarily unavailable?
Valid source snapshots remain usable. Within the same evaluation version, temporarily missing rows may carry forward verified records from the past seven days. Valid results still present on public boards are not removed merely because their scores have not changed. Official withdrawals, corrections and version changes invalidate or replace records; historical best scores are not retained. If a full calculation fails, the last valid board and its original timestamp remain.
Can composite indexes double-count specialist evaluations?
We check overlap using public task sets and methodology, and cap the combined share of related sources. Unconfirmed correlations remain a limitation. The AA index currently receives 30%. Arena overall text and creative preference share the existing 10% human-preference budget, at 5% each. Their overlapping votes remain one evidence family rather than being presented as independent evaluators.
Do price or speed affect ranks?
Neither affects the ranking. Prices describe API usage costs per million tokens, with official provider links. They do not represent subscription fees.
Calculation details and current version

Method version: 2026.09-public-consensus-v15. The method uses weighted incomplete Kemeny ranking. Net support M is aggregated for each pair of jointly evaluated models. The objective minimizes total net support reversed by the ranking. Integer optimization reports an optimal status and objective bounds; only fully validated solutions are published.

When both results have explicit standard errors, net support is 2Φ(score difference / combined standard error) − 1, assuming zero covariance by default. Other comparisons use only the observed direction. Unknown uncertainty is not zero uncertainty; treating small differences ordinally remains a limitation. Voting budgets apply to potential model pairs. Sources covering more models occupy more comparison positions, so nominal budgets are not exact percentages of their contribution to final ranks.

The displayed index preserves the original order. Each neighboring pair is reversed while other models may reorder, and the minimum additional reversed net support is calculated. These nonnegative support gaps are accumulated along the original ranking, then mapped to 0–100 with a sigmoid against a fixed reference group. Equally optimal alternative orders retain a zero gap. Scores display one decimal without an artificial minimum gap. The index does not reorder models. Eligibility, reference models and evidence can still change its scale, and score differences are not real capability distances. Equal-cost solutions use a fixed model-ID order. Integer optimization uses HiGHS 1.15.3; the normal CDF for uncertainty conversion uses the same algorithm as SciPy norm.cdf. Sources, protocols, eligibility and each calculation's inputs are versioned. Disconnected evidence components are not given a falsely precise shared order.

Fixed reference models: claude-fable-5、gpt-5-6-sol、kimi-k-3、qwen-3-8-max、gpt-5-4、claude-opus-4-8、claude-sonnet-5、grok-4-5、gemini-3-5-flash、glm-5-2、claude-sonnet-4-6、qwen-3-7-max、kimi-k-2-6、deepseek-v-4-pro、qwen-3-6-plus、minimax-m-3、gpt-5-4-mini、grok-4-3. Reference models set the index scale without prescribing any provider's rank. Indexes from different categories are not directly comparable.

Explore every evaluation →