Skip to content

#多模态

Today 0 items
Oct 16Fri
  1. Hugging Face Blog81

    Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta

    Meta has released Muse Spark Open, an open-weight multimodal mixture-of-experts model with roughly 816B total parameters and about 42B activated per token. It supports text, image, audio and video inputs, offers a context window of up to 1M tokens, and was trained for agentic coding.

    Why it matters: The official blog details the model's scale and architecture plus a full deployment path from llama.cpp to TRL fine-tuning, letting readers evaluate the cost of putting it into production.

Oct 15Thu
  1. NVIDIA Blog71

    Open From Data Center to Desktop: NVIDIA Expands Nemotron Family With Highest Efficiency Super VL and Bolt Models

    NVIDIA has released the open-source 120-billion-parameter multimodal model Nemotron 3.5 Super VL, plus two on-device models — Nemotron 3.5 Bolt 74B-A9B 24GB (already available) and Bolt E4B 4GB (available October 23) — covering agent deployment from data centers to local hardware.

    Why it matters: NVIDIA discloses parameters, quantization and deployment tiers for the three open Nemotron models, so readers can pick the enterprise, workstation or 4GB on-device option that fits.

  2. Hugging Face Blog64

    Apertus 1.5 Release Notes

    Project Apertus has released Apertus 1.5, adding image and audio understanding and an optional thinking mode to its 8B and 70B models via continued pretraining, and extending the context window to 262,144 tokens — 4x that of Apertus 1.0.

    Why it matters: The official release notes fully document the training pipeline, open data and intermediate checkpoints, letting readers trace how the multimodal and reasoning capabilities were built.

  4. Hugging Face Blog58

    HiRO-ACEv2: Simulating extreme weather at 3 km resolution for a fraction of the cost

    Ai2 released HiRO-ACEv2; the new downscaling model HiROv2 can predict multiple high-resolution fields—rainfall, wind, and pressure—simultaneously, and with coarsened X-SHiELD data as input it matches the high-resolution statistics of even one-in-a-million extreme events to within 3%, generating structurally realistic hurricanes and mountain downslope windstorms.

Oct 14Wed
  1. Google DeepMind83

    Gemini 4 Argon: Built to solve your hardest problems

    Google DeepMind released Gemini 4 Argon, available worldwide to developers, enterprises, scientists and consumers via the Gemini API, AI Studio, Antigravity and the Gemini app, priced at $2 per million input tokens and $10 per million output tokens, with cached input tokens at a 95% discount.

    Why it matters: The official release includes API pricing, engineering and science use cases and safety data, letting readers assess its real value for coding and agent scenarios.

Oct 8Thu
Oct 7Wed
  2. Hugging Face Blog43

    Falcon-Emirati: When an LLM Learns the Dialect, the Culture, and the Nuance

    The co-publishers with Hugging Face released Falcon-Emirati-7B, fine-tuned from the Falcon-H1-Arabic 7B variant to understand and generate Emirati Arabic dialect. The model uses a hybrid Mamba and Transformer attention architecture, trained on Emirati dialect web data, Emirati-culture MSA data, and synthetic data constrained by lexicon and style rules, with a context window supporting up to 128K/256K tokens.

  3. Mistral AI75

    Introducing Mistral Large 4

    Mistral released the public preview of Mistral Large 4 (ML4), a natively multimodal model with 1 trillion parameters and 49 billion activated parameters, with weights to be available for download by the end of the month.

    Why it matters: The official release covers parameter counts, benchmark scores, training infrastructure and the open-weights timeline, letting readers assess its real standing in the open-source camp.

  4. Google DeepMind68

    EmbeddingGemma 2: an open, lightweight multimodal embedding model

    Google DeepMind released EmbeddingGemma 2, a 740M-parameter model built on the Gemma 4 architecture under the Apache 2.0 license, mapping text, code, images, video and audio into a shared embedding space.

    Why it matters: The official release provides concrete metrics such as parameter count, memory footprint and benchmark scores, helping readers judge whether it suits on-device retrieval scenarios.

Oct 1Thu
Sep 30Wed
  1. Hugging Face Blog63

    Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning

    Hugging Face has launched the Open TTS Leaderboard, evaluating open-source TTS models with objective metrics including Qwen3 ASR WER/CER, RTFx and TTFA on H200, and WavLM speaker similarity, cutting per-model evaluation time from about two weeks to hours.

    Why it matters: The post explains how objective metrics shrink TTS evaluation from weeks to hours, and why mainstream arenas structurally underrepresent open-source models.

Sep 29Tue
  1. Microsoft Research62

    Introducing Quine: An AI research system designed for the complexity of biology

    Microsoft Research launched Quine, a research system combining a multimodal world model of biology with an interactive framework connecting scientific tools, literature, and researchers. With the Broad Institute, it applied the system to pancreatic ductal adenocarcinoma (PDAC) research, predicting and ranking compounds that could drive tumor cell state transitions; wet-lab validation confirmed top-ranked candidates, and narrowing the compound pool to validation candidates took just one weekend.

    Why it matters: Microsoft's official introduction covers Quine's architecture and real PDAC cancer research results, showing readers how AI world models can compress wet-lab iteration cycles.

Sep 28Mon
Sep 26Sat
Sep 25Fri
  1. Google Research54

    Automating coherent long-form video generation

    Google Research unveiled its AI video co-director research, building a multi-agent orchestration layer based on Gemini and Veo with four frameworks — Co-Director, CANVAS, A²RD and VQQA — that reformulates long-form video generation as a global optimization and world-state tracking problem.

Sep 24Thu
  1. Hugging Face Blog60

    Accelerating vision-language models with LFM2.5-VL-DSpark

    Liquid AI has released experimental DSpark draft models for the vision-language model LFM2.5-VL-3B, using speculative decoding to trade for faster inference without changing output quality — decoding speeds up to 3.13x on-device and 2.66x on H100, with end-to-end gains up to 2.62x and 2.27x respectively.

    Why it matters: The article details a speculative decoding acceleration scheme for vision-language models with concrete speedups and memory overhead, plus integration guidance for llama.cpp, MLX-VLM, and SGLang.

Sep 23Wed
Sep 18Fri
  1. Google Research47

    The future of practice: Enabling teachers to create learning interactives with generative UI

    Google Research launched a new research experiment where teachers can use generative UI (GenUI) to dynamically generate customized, guided interactive learning simulations based on their teaching goals and curriculum. Built on active learning principles from the LearnLM education model family, the release includes over 30 AI-generated, teacher-reviewed English STEM learning interactives covering physics, chemistry, biology and math for middle and high school.

Sep 16Wed
Sep 3Thu
  1. Hugging Face Blog52

    NeoMME: an efficient Multimodal-native and Multilingual Encoder

    H Company released NeoMME, a multilingual multimodal encoder in 260M and 800M sizes. A single bidirectional Transformer processes both text tokens and 32×32 image patches, trained from scratch with a masked discrete diffusion objective, with no reliance on a pretrained vision tower or a causal language model. Both versions are open-sourced under Apache 2.0.

Sep 2Wed
  1. Google DeepMind70

    Introducing agentic video understanding with Gemini

    Google DeepMind released agentic video understanding for Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite, replacing static 1 FPS processing with dynamic searching and inspection of video segments; official benchmarks show up to 66% lower cost, up to 88% fewer tokens, and up to 7% higher accuracy.

    Why it matters: The official release gives concrete cost, token and accuracy figures plus how to enable it, letting developers directly assess switching their existing video processing pipelines.

Sep 1Tue
Aug 28Fri
  1. Google Research50

    Planetary prediction engine: Automating global models via Earth AI

    Google Earth AI has introduced an experimental research capability called the Planetary Prediction Engine (PPE), which autonomously performs the full geospatial prediction workflow—data discovery, cleaning, feature engineering, model training, and evaluation—starting from natural language queries, compressing work that previously took weeks of manual data engineering down to a few minutes.

  2. Google DeepMind60

    Gemini Omni 1.1 Flash lets you build with more control

    On August 27, Google DeepMind released Gemini Omni 1.1 Flash, bringing a suite of creative controls for generative video aimed at developers, available via the Gemini API in Google AI Studio.

    Why it matters: The official release provides specific parameters for new capabilities like scene extension, keyframe interpolation and 4K upscaling, along with access channels, so developers can evaluate adopting it in generative video workflows.

Aug 26Wed
Aug 21Fri
  2. Microsoft Research50

    Broadening access to Skala creates a faster path to predictive DFT

    Microsoft Research released Skala 1.1, trained on 2.5 times the data of the previous version, ranking first in 32 of the 55 categories of the GMTKN55 benchmark with a weighted average error of 2.8 kcal/mol at compute cost comparable to a meta-GGA functional. Skala is now available in CP2K and being integrated into Psi4, FHI-aims, ORCA, and VASP, alongside a dynamic benchmark tracking each version's computational performance.

Aug 17Mon
Aug 13Thu
  2. Microsoft Research44

    MindTopo reveals VLMs’ spatial reasoning abilities

    Microsoft Research introduced the MindTopo benchmark, evaluating multimodal large models' ability to reason about and plan with topological concepts such as connectivity, closure, ordering, separation, and knots. Testing shows models perform markedly better on static image recognition than on interactive planning tasks, and both fall far short of human level, with errors mostly involving lost structural relations during planning or physically infeasible actions. The benchmark aims to provide a controlled diagnostic tool for research on agents in robotics and interactive environments.

Aug 12Wed
  1. Google DeepMind74

    Putting sign language AI into users’ hands

    Google DeepMind released a large-scale multilingual sign language-to-text (SL2T) translation model, bringing new sign language input features to deaf and hard-of-hearing users, first supporting American Sign Language (ASL) to English in Gboard and Live Transcribe on Pixel 11, with more devices and languages to follow.

    Why it matters: The article gives concrete details on training scale, benchmark scores, and privacy design, letting readers understand the real technical path to deploying sign language translation.

  2. Google Research65

    Advancing AMIE towards expert-level audio-visual clinical consultations

    Google Research has launched AMIE (Video), built on Gemini and Project Astra, which can conduct real-time synchronized clinical video consultations, perceive non-verbal cues, and guide virtual physical exams.

    Why it matters: The article covers the architecture, evaluation scale, and quantitative comparisons against primary care physicians (PCPs), while candidly noting limitations like simulated scenarios, helping readers understand the practical boundaries of audio-visual clinical AI.