#Hugging Face

Today 0 items

Oct 4Sun

Sunday

06:56
Hugging Face BlogSelectedAI score6969
The Agent Said It Was Done. The Database Disagreed.
Microsoft and Hugging Face released the ThinkingBox agent sandbox and ThinkingBox-Bench benchmark, with 507 stateful business workflows and 20 runs per task, using executable checks to evaluate end-state database state and side effects, now open on Hugging Face via OpenEnv.
Research #Benchmarks #Agent #Microsoft
Why it matters: The article uses executable backend-state checks to distinguish surface success from real outcomes, and provides consistency and cost data for each model, a directly useful reference for evaluating agents.

Sep 29Tue

Tuesday

21:07
Hugging Face BlogAI score4949
Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents
A Hugging Face team has proposed ProvenanceGuard, a source-aware factuality verification layer for MCP agents, which checks after generation whether the sources supporting each claim match the sources the answer states or implies, identifying cross-source confusion.
Research #Research #Agent #RAG

Aug 28Fri

Friday

08:00
Hugging Face BlogSelectedAI score6262
The Open ASR Leaderboard Adds Its First Global South Language
Voice Arena and Hugging Face released four Monsoon evaluation sets for the Open ASR Leaderboard, covering Indian English and Hindi, with Hindi the first non-European language on the multilingual leaderboard.
Research #Product updates #Benchmarks #Open-source ecosystem
Why it matters: The original details the Monsoon sets' split structure, speaker coverage and regional analysis examples, helping readers understand sources of model differences beyond aggregate WER.

Aug 21Fri

Friday

08:00
Hugging Face BlogSelectedAI score6262
Measuring benchmark optimization in speech recognition
Hugging Face released research using three types of tests — consistency disagreement probes, digit silence retrieval, and spelling switches — to evaluate benchmark optimization (benchmaxxing) behavior in 11 open-source ASR models.
Research #Research #Voice #Hugging Face
Why it matters: The article quantifies benchmark optimization in speech recognition with three probes and provides a model list, ratio data, and reusable detection scripts, helping readers avoid the pitfall of selecting models by WER alone.

Aug 10Mon

Monday

18:05
Hugging Face BlogAI score4848
Making Knowledge Distillation Cheap Enough to Run at Scale
Hugging Face has published a paper that enables offline distillation by caching the teacher's top-100 logits, and introduces a fused chunked KL loss that avoids generating full vocabulary-by-sequence-length matrices.
Research #Research #Practical guides #Open-source ecosystem

Jul 15Wed

Wednesday

08:00
Hugging Face BlogAI score5656
Introducing Real World VoiceEQ: Measuring the human quality of voice AI
Hume released the Real World VoiceEQ benchmark to evaluate the human quality of voice AI interactions, covering ASR, TTS, S2S, and speech understanding, assessing over 40 leading closed- and open-source voice models across 15+ dimensions and 60+ metrics.
Research #Benchmarks #Voice #Hugging Face

Jul 1Wed

Wednesday

02:32
Hugging Face BlogAI score4545
ScarfBench: Benchmarking AI Agents for Enterprise Java Framework Migration
Hugging Face launched the open-source benchmark ScarfBench (Self-Contained Application Refactoring Benchmark), which evaluates AI agents on enterprise Java cross-framework migration tasks across the Spring, Jakarta EE and Quarkus ecosystems.
Research #Research #Benchmarks #Coding