Piloting the world's first double-blind AI evaluations

Google DeepMind launched what it says is the world's first double-blind evaluation for proprietary frontier models, confining external evaluations to cryptographically secured isolated environments to prevent benchmark contamination from models seeing test questions in advance. In partnership with the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons, it tested a Gemini Flash Lite model with confidential benchmarks to improve the credibility and integrity of evaluations.