DeepL wants real-time translation to finally capture sarcasm and mood shifts
DeepL wants real-time translation to finally capture sarcasm and mood shifts
DeepL launched its new Aure model, which carries a speaker's tone, emotion, emphasis and speaking pace into the target language, replacing the previous approach that relied on static voice samples captured at the start of a session and could not detect emotional shifts or sarcasm. Aure continuously analyzes translations and original audio during conversations, was trained by separating timbre, speech melody and content for independent transfer, and uses reinforcement learning to remove accents. It runs automatically in DeepL Voice's online meetings, face-to-face conversations and DeepL API.
DeepL Voice's real-time translations are about to sound a lot more natural. The company's new Aure model carries over tone, emotion, emphasis, and speaking pace from the original speaker's voice into the target language. Until now, DeepL Voice generated translated speech from a static voice sample captured at the start of a conversation. That meant it couldn't pick up on mood shifts or sarcasm. Aure works differently, continuously analyzing both the translated text and the original audio as the conversation unfolds.
During training, DeepL separates voice identity, speech melody, and content so each component can be transferred into the new language independently. Reinforcement learning keeps accents out of the output. Aure runs automatically in DeepL Voice for Online Meetings, In-Person Conversations, and the DeepL API. At launch, the model supports English, German, Spanish, French, Italian, Japanese, Portuguese, and Korean.
DeepL first rolled out voice preservation in September. In May, the company cut about 250 jobs to restructure around smaller, AI-powered teams as part of what it called becoming an "AI-native" company. DeepL also acquired the team behind audio streaming specialist Mixhalo to bolster its real-time speech translation. Block is chasing a similar goal of capturing meaning beyond words. The company recently unveiled Dessa, a speech model designed to detect emotions and sarcasm directly from audio.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Source: The Decoder · the-decoder.com