MCP Atlas
Scale AI / Professional work · Uses real MCP services to retrieve information and operate documents and databases across applications.
What it measures and how
Fixes the April 2026 protocol and 100-tool-call budget. Full 1,000-task and public 500-task sets stay separate. Only task pass rate is used, without double counting criterion coverage. Judges, retries, and tools must be fixed per batch.
How this evidence is used
Under review: could add cross-application tool evidence. Stable exports, actual evaluation batches, and redisplay terms are unconfirmed. Source listing only; no automatic collection or scoring.
Evaluation limits & attribution
Old method text and live results may differ. Public task code is not a live-results export. Scale's other leaderboards share the same evaluator and add no independent organization.
Data license: Official environment code is MIT. It does not automatically license live page results; redisplay terms remain unresolved.
Results are published by Scale AI. Original scores and the NexGateHub consensus index use different scales and cannot be added directly.