Google has upgraded its Gemini-based Geospatial Reasoning agent into an experimental system capable of multi-step reasoning across planetary-scale data and autonomously training custom prediction models, and released a technical report introducing two new benchmarks, GeoKTools and GeoExpertQueries.
Asana uses GPT-6 Astra in Codex, cutting the cost of its browser agent 76x and making it 5x faster in testing, so it can offer customers more capable models.
GitHub has released ReviewBench, an open offline benchmark for evaluating AI code review agents. The benchmark is modeled on the distribution of 103.9 million GitHub pull requests.
Microsoft and Hugging Face released the ThinkingBox agent sandbox and ThinkingBox-Bench benchmark, with 507 stateful business workflows and 20 runs per task, using executable checks to evaluate end-state database state and side effects, now open on Hugging Face via OpenEnv.
Why it matters: The article uses executable backend-state checks to distinguish surface success from real outcomes, and provides consistency and cost data for each model, a directly useful reference for evaluating agents.
OpenAI has launched MentalHealthBench, an expert-informed benchmark for evaluating how effective and safe AI responses are in real-world mental health conversations.
The UK AI Security Institute (AISI) has begun using EvalEval's infrastructure to publicly share evaluation results, including verified results for five benchmarks—HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro and Terminal-Bench 2.0—as well as two cyber evaluations, Cyber CTFs and The Last Ones.
AMD has released a white paper evaluating 6th Gen AMD EPYC 9006 "Venice" server CPUs across general-purpose, enterprise, cloud-native, AI and HPC workloads.
NVIDIA Vera Rubin NVL72 made its debut in MLPerf Inference v6.1 preview submissions, delivering up to 3.7x the throughput of GB300 NVL72 on Qwen3-VL and 2.5x on DeepSeek-R1.
Hugging Face has launched BenchMIRT, a multidimensional IRT-based method that audits what capabilities LLM benchmarks actually measure at the level of individual prompts.
Voice Arena and Hugging Face released four Monsoon evaluation sets for the Open ASR Leaderboard, covering Indian English and Hindi, with Hindi the first non-European language on the multilingual leaderboard.
Why it matters: The original details the Monsoon sets' split structure, speaker coverage and regional analysis examples, helping readers understand sources of model differences beyond aggregate WER.
Microsoft Research introduced the MindTopo benchmark, evaluating multimodal large models' ability to reason about and plan with topological concepts such as connectivity, closure, ordering, separation, and knots. Testing shows models perform markedly better on static image recognition than on interactive planning tasks, and both fall far short of human level, with errors mostly involving lost structural relations during planning or physically infeasible actions. The benchmark aims to provide a controlled diagnostic tool for research on agents in robotics and interactive environments.
Hume released the Real World VoiceEQ benchmark to evaluate the human quality of voice AI interactions, covering ASR, TTS, S2S, and speech understanding, assessing over 40 leading closed- and open-source voice models across 15+ dimensions and 60+ metrics.
Hugging Face launched the open-source benchmark ScarfBench (Self-Contained Application Refactoring Benchmark), which evaluates AI agents on enterprise Java cross-framework migration tasks across the Spring, Jakarta EE and Quarkus ecosystems.
Every Eval Ever (EEE) now interoperates with Hugging Face Community Evals, supporting cross-posting of eval results with backlinks to the full EEE records.
In the paper 'Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation,' Google Research studies the trade-off in AI evaluation between the number of annotations per item and the number of items annotated, and open-sources a simulator based on real datasets.