Mercor APEX-Agents 1.1
Mercor / Professional work · Long investment-banking, consulting, and legal tasks assess whether a model can assist with an entire job.
What it measures and how
Fixes dataset 1.1 and official Loop. Uses Pass@1 and its corresponding uncertainty, selecting tiers by predetermined rules. Mean Score is average rubric completion, not task completion.
How this evidence is used
Uses Pass@1 in official Loop 1.1, receiving 4% of the tools-and-office budget. Another 2% goes to banking under common conditions.
Evaluation results
Fixed rules select one representative configuration per public model. Anonymous test models are omitted. Original ranks are retained, with up to 30 models per evaluation.
| Source rank | Source model name | Raw result | Representative configuration |
|---|---|---|---|
| 1 | gemini-4-argongoogle | 82.4% | High reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 2 | claude-sonnet-5-5anthropic | 75.5% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 3 | claude-opus-5-5Anthropic | 73.5% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 4 | claude-fable-5.1Anthropic | 68.6% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 5 | gemini-3.7-flashGoogle | 67.8% | High reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 6 | claude-opus-5Anthropic | 65.8% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 7 | grok-4-6-xhighxAI | 65.3% | xHigh reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 8 | gpt-6-astra-maxOpenAI | 64.7% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 9 | gemini-3.8-flashGoogle | 64.3% | High reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 10 | claude-fable-5Anthropic | 63.6% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 11 | qwen-3-8-max-xhighAlibaba | 63.3% | xHigh reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 12 | gpt-6-1-solopenai | 60.0% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 15 | gpt-5.6-terra-maxOpenAI | 58.2% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 16 | mimo-v2.6-flash-rlXiaomi | 57.4% | Full system · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 17 | glm-5-3Z.ai | 56.6% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 18 | grok-4-5xAI | 56.2% | High reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 19 | claude-haiku-5-5— | 56.1% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 20 | deepseek-v4-flashDeepSeek | 55.3% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 21 | gpt-5.5-xhighOpenAI | 55.1% | xHigh reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 22 | grok-4.7xAI | 54.6% | xHigh reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 23 | claude-sonnet-5-maxAnthropic | 54.5% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 24 | mimo-v2.6-proXiaomi | 54.4% | Full system · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 25 | gpt-6-solOpenAI | 54.3% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 26 | glm-5-3-flashZ.ai | 52.8% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 28 | responses/gpt-5.4OpenAI | 52.4% | xHigh reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 29 | gpt-5-6-sol-max-proOpenAI | 51.5% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 30 | claude-opus-4-7Anthropic | 49.2% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 31 | claude-opus-4-8Anthropic | 48.9% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 32 | kimi-k3Moonshot AI | 47.8% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
| 33 | muse-spark-1-3-maxMeta | 47.6% | Max reasoning · loop_truncated_tools_agent · apex-agents:v1.1:loop_truncated_tools_agent |
Evaluation limits & attribution
Limited to investment banking, consulting, and law. The source omits per-model evaluation dates; reasons for a few missing runs are unpublished. This is not reliability across all office work.
Data license: Official results are public. Code and data licenses do not establish leaderboard redistribution rights.
Results are published by Mercor. Original scores and the NexGateHub consensus index use different scales and cannot be added directly.