Skip to content
Evaluation sources

SWE-rebench

Nebius / Coding · Continually gathers real repository issues to assess repair across programming languages.

Official evaluation
Across NexGateHub AI News Observing
Evidence budgetNot scored
Upstream data timeUnverified
Last successful syncCollection has not started

What it measures and how

Fixes task windows, ReAct environment, and text/tools protocol. Uses mean resolution rates and standard errors across repeated runs. Complete agent products and base models remain distinct.

How this evidence is used

Candidate under review: real repository tasks could add evidence. Verifiable run dates, stable result access, and redisplay terms are pending. No automatic collection or scoring has begun.

Evaluation limits & attribution

Rolling questions, environments, and task windows affect comparability. Recent website maintenance does not mean all models were retested.

Data license: The official Hugging Face CC BY 4.0 dataset contains tasks. Stable exports and redisplay terms for final webpage results remain unconfirmed.

Results are published by Nebius. Original scores and the NexGateHub consensus index use different scales and cannot be added directly.