Skip to content

Speech and Audio

Advances in AI speech and audio: models and products for speech synthesis, real-time conversation, music generation, and audio understanding.

11 selected items Related topicsMultimodalAI VideoProduct Updates

Latest selected stories

Items 1–11 of 11 items
Sep 23Wed
  1. Google DeepMind64

    Gemini 3.8 text-to-speech says hello

    On September 23, Google DeepMind released two text-to-speech models, Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS, which support customizing character voices via natural-language prompts in over 100 languages, cloning voices from 30-second audio samples, directing delivery and dialect changes line by line, and natively generating two-speaker scenarios.

    Why it matters: The official announcement details both TTS models' capabilities, benchmark results, and availability, so readers can judge how voice creation workflows will change.

Sep 16Wed
  1. Google DeepMind73

    Introducing Gemini 3.8 Live and 3.8 Live Extended Thinking

    Google DeepMind has released Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, two real-time voice conversation models targeting cost efficiency at scale and high-complexity multi-step reasoning tasks, respectively.

    Why it matters: The official release lays out how the two product lines divide capabilities, with multiple benchmark figures that help readers weigh trade-offs when choosing models for voice agent scenarios.

Sep 10Thu
Aug 27Thu
  1. Google DeepMind64

    Intelligent transcription with Gemini 3.5 Transcribe

    Google DeepMind released Gemini 3.5 Transcribe, positioned as a more accurate real-time speech transcription model with intelligent interaction. The model converts raw audio directly into formatted text, automatically removes filler words and self-corrections, supports custom vocabularies, over 85 languages, up to three speaker diarization, and can call other Gemini models via function calling.

    Why it matters: The official release details the model's capabilities in noisy environments, custom vocabularies and multi-speaker scenarios, plus API access, letting readers evaluate whether to migrate their transcription workflows.

Aug 21Fri
  1. Hugging Face Blog62

    Measuring benchmark optimization in speech recognition

    Hugging Face released research using three types of tests — consistency disagreement probes, digit silence retrieval, and spelling switches — to evaluate benchmark optimization (benchmaxxing) behavior in 11 open-source ASR models.

    Why it matters: The article quantifies benchmark optimization in speech recognition with three probes and provides a model list, ratio data, and reusable detection scripts, helping readers avoid the pitfall of selecting models by WER alone.

Jul 1Wed
  1. Hugging Face Blog61

    Hugging Face and Cerebras bring Gemma 4 to real-time voice AI

    Hugging Face and Cerebras jointly demonstrated a real-time speech-to-speech architecture: voice input is transcribed by Nvidia's Parakeet, reasoning is handled by Google DeepMind's Gemma 4 31B running on Cerebras, and speech output is synthesized with Alibaba's Qwen3TTS.

    Why it matters: The article lays out a complete open-source speech-to-speech stack with components at each layer, so developers can build or swap parts of their own real-time voice AI pipeline.

Jun 9Tue
  1. Google DeepMind78

    Fluid, natural voice translation with Gemini 3.5 Live Translate

    Google DeepMind released the audio model Gemini 3.5 Live Translate, offering near-real-time speech-to-speech translation across 70+ languages with automatic language detection. It continuously generates translations that preserve the speaker's tone, rhythm, and pitch, lagging the speaker by only a few seconds.

    Why it matters: The official announcement details the model's capabilities, access options, and product integrations, helping readers judge how it changes real-time speech translation workflows.

Apr 16Thu
  1. Google DeepMind60

    Gemini 3.1 Flash TTS: the next generation of expressive AI speech

    Google DeepMind released the text-to-speech model Gemini 3.1 Flash TTS, which scores an Elo of 1,211 on the Artificial Analysis TTS leaderboard and supports 70+ languages and native multi-speaker dialogue.

    Why it matters: The official post introduces audio tags and AI Studio scene control among other new capabilities, showing readers how expressive control of TTS speech works in practice.

Mar 24Tue
  1. Mistral AI65

    Speaking of Voxtral

    Mistral AI released its first text-to-speech model, Voxtral TTS, with 4B parameters supporting 9 languages including English, French and German, positioned for low-latency speech generation. In human evaluations its naturalness beats ElevenLabs Flash v2.5 and matches ElevenLabs v3; latency is 70ms for a 10-second sample plus 500 characters, with RTF around 9.7x.

    Why it matters: The original gives key details on latency, pricing, human evaluation comparisons and the open license, helping readers assess its fit for voice agent scenarios.

Feb 5Thu
  1. Mistral AI65

    Voxtral transcribes at the speed of sound.

    Mistral AI has released Voxtral Transcribe 2, comprising two speech transcription models, Voxtral Mini Transcribe V2 and Voxtral Realtime, supporting 13 languages, speaker diarization, and word-level timestamps.

    Why it matters: The official post provides latency, error rate, and pricing data for both transcription models and releases the Realtime weights, so readers can compare against their current speech solutions.

Jul 15Tue
  1. Mistral AI70

    Voxtral

    Mistral released Voxtral, an open-source speech understanding model available in a 24B Voxtral Small and a 3B Voxtral Mini, both under the Apache 2.0 license and callable via its API at prices as low as $0.001 per minute.

    Why it matters: Official benchmarks and API pricing for two open-source speech understanding model sizes let readers assess whether Voxtral can replace their current ASR solutions.