Three editors at The Verge spent a week trying five laptops with Nvidia's RTX Spark N1X chip, including the Surface Laptop Ultra, Asus ProArt P16, Lenovo Yoga 9n and HP OmniBook Ultra, priced from $2,599.99 to $6,999.99.
Google DeepMind's Gemini 4 Argon scored 97.5% on ARC-AGI-3 using the Provider Adapter Harness, at a cost of $4.
Why it matters: The original piece compiles evaluation data from both ARC-AGI-3 and Epoch ECI, and points out that scores depend heavily on the test environment, offering verifiable details for cross-model comparisons.
Researchers introduced TheoPhysEval, a benchmark of routine theoretical physics research problems, and OpenAI's GPT-6-Astra (running in the Codex environment) solved 49 of 50 problems without human hints.
The Verge's hands-on with the first RTX Spark laptops details the adaptation work Microsoft and Nvidia have done in Windows. An Arm CPU plus Blackwell GPU and unified memory can run large models locally, and Windows offers a Get Started two-step agent installation experience with launch support for Hermes, Perplexity, and OpenClaw.
Google has upgraded its Gemini-based Geospatial Reasoning agent into an experimental system capable of multi-step reasoning across planetary-scale data and autonomously training custom prediction models, and released a technical report introducing two new benchmarks, GeoKTools and GeoExpertQueries.
Artemis Security has launched Orion-1, a security-focused model that can autonomously process log data from identity services, cloud, endpoints, SaaS and network devices, chain signals to determine attacks and suggest responses. It scored above 80 on all five tasks of its in-house DGR benchmark, including 80.8 on attack reconstruction, ahead of the next-best GPT-6 Sol at 58.3.
The Verge reviewed the $349 camera-free Ray-Ban Meta Audio smart glasses, arguing that dropping the camera makes them more comfortable to wear in public and that they look like ordinary glasses. They currently run the weaker Muse Spark model, which handles short tasks like schedule lookups, reminders, and walking navigation, but complex requests often fail, and Meta's Muse AI won't arrive until later this year. They are essentially good open-ear headphones in glasses form, but the AI experience isn't yet worth the price.
AI startup Elorian and Scale AI released the visual intuition reasoning benchmark Humanity's Sixth Sense (HSS), containing 522 questions, 288 images and 234 short videos (17.6 hours of video in total).
Vals AI tested GPT-6 Sol and Claude Opus 5.5 in single-agent versus team setups on Vibe Code Bench, finding teams cost 1.8x to 5.1x as much as single agents, and of four comparisons only GPT-6 Sol at the medium reasoning tier improved 7.3 points with statistical significance.
Asana uses GPT-6 Astra in Codex, cutting the cost of its browser agent 76x and making it 5x faster in testing, so it can offer customers more capable models.
The Verge tested the AI agent Instinct, whose company reached a $10 billion valuation in late September; it has no app or monthly fee, requires an invite or waitlist, and works via iMessage, WhatsApp or SMS, and email.
Arena has completed a $200 million Series B at a $3.1 billion valuation, led by Lightspeed Venture Partners and Khosla Ventures with participation from a16z and others; the company previously raised a $150 million Series A in January at a $1.7 billion post-money valuation and announced $100 million in annualized revenue in June.
GitHub has released ReviewBench, an open offline benchmark for evaluating AI code review agents. The benchmark is modeled on the distribution of 103.9 million GitHub pull requests.
Simon Willison tested Mistral Large 4 with an SVG generation prompt about an armadillo in fishnet stockings jaywalking on Mars, comparing its output with claude-opus-5.5, gpt-6.1-sol, and gemini-3.8-flash, with each model using its default reasoning level.
A benchmark tested whether a local Qwen3.8-27B-Q4_K_M.gguf model can produce exact results of positive integer addition using only English words, with 5,070 no-reasoning cases and 169 reasoning-enabled control cases. With reasoning off, numerical accuracy was only 23.57% — 97.04% for 1-3 digit problems but 6.44% for 10-13 digits, with 96.17% format compliance; with reasoning on, it answered 167 of 169 attempts correctly.
Microsoft and Hugging Face released the ThinkingBox agent sandbox and ThinkingBox-Bench benchmark, with 507 stateful business workflows and 20 runs per task, using executable checks to evaluate end-state database state and side effects, now open on Hugging Face via OpenEnv.
Why it matters: The article uses executable backend-state checks to distinguish surface success from real outcomes, and provides consistency and cost data for each model, a directly useful reference for evaluating agents.
OpenAI released GPT-6.1 Sol, priced at $2/$10 per million input/output tokens, ranking 5th on the Agent Arena, 39% cheaper than GPT-6 Sol with a score 1.52 points higher, and 81% cheaper than Astra.
Mercor's research had 12 licensed accountants with an average of five and a half years of experience complete simplified tasks from the APEX Accounting Benchmark; results showed today's AI models outperform accountants on speed, accuracy, and cost, whereas 18 months ago the best models scored about 37% below the accountants' average.
OpenAI has launched MentalHealthBench, an expert-informed benchmark for evaluating how effective and safe AI responses are in real-world mental health conversations.
The UK AI Security Institute (AISI) has begun using EvalEval's infrastructure to publicly share evaluation results, including verified results for five benchmarks—HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro and Terminal-Bench 2.0—as well as two cyber evaluations, Cyber CTFs and The Last Ones.
AMD has released a white paper evaluating 6th Gen AMD EPYC 9006 "Venice" server CPUs across general-purpose, enterprise, cloud-native, AI and HPC workloads.
NVIDIA Vera Rubin NVL72 made its debut in MLPerf Inference v6.1 preview submissions, delivering up to 3.7x the throughput of GB300 NVL72 on Qwen3-VL and 2.5x on DeepSeek-R1.
Hugging Face has launched BenchMIRT, a multidimensional IRT-based method that audits what capabilities LLM benchmarks actually measure at the level of individual prompts.
Voice Arena and Hugging Face released four Monsoon evaluation sets for the Open ASR Leaderboard, covering Indian English and Hindi, with Hindi the first non-European language on the multilingual leaderboard.
Why it matters: The original details the Monsoon sets' split structure, speaker coverage and regional analysis examples, helping readers understand sources of model differences beyond aggregate WER.
DiG-bench (Discovery in Games) has been released, a benchmark of 70 games for testing AI's ability to autonomously discover hidden rules through exploration; 21 of the games are publicly available.
Microsoft Research introduced the MindTopo benchmark, evaluating multimodal large models' ability to reason about and plan with topological concepts such as connectivity, closure, ordering, separation, and knots. Testing shows models perform markedly better on static image recognition than on interactive planning tasks, and both fall far short of human level, with errors mostly involving lost structural relations during planning or physically infeasible actions. The benchmark aims to provide a controlled diagnostic tool for research on agents in robotics and interactive environments.
Hume released the Real World VoiceEQ benchmark to evaluate the human quality of voice AI interactions, covering ASR, TTS, S2S, and speech understanding, assessing over 40 leading closed- and open-source voice models across 15+ dimensions and 60+ metrics.
Hugging Face launched the open-source benchmark ScarfBench (Self-Contained Application Refactoring Benchmark), which evaluates AI agents on enterprise Java cross-framework migration tasks across the Spring, Jakarta EE and Quarkus ecosystems.
Every Eval Ever (EEE) now interoperates with Hugging Face Community Evals, supporting cross-posting of eval results with backlinks to the full EEE records.
In the paper 'Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation,' Google Research studies the trade-off in AI evaluation between the number of annotations per item and the number of items annotated, and open-sources a simulator based on real datasets.