On-Device AI
All topicsAI running on phones, PCs, and edge devices: advances in small models, local inference, and edge chips.
13 selected items Related topicsDeployment EngineeringOpen-Source EcosystemModel Releases
Technology timeline
News collected over the past year · Latest 6 entries · Collection time is used when the source has no publication date
Key developments
- NVIDIA RTX Spark laptops officially go on sale
- NVIDIA releases Nemotron 3.5 Super VL and Bolt edge models
- Liquid AI Releases On-Device Inference Model LFM2.5-24B-A2B
- NVIDIA RTX Spark Software Ecosystem Ready
- NVIDIA releases RTX Spark and DGX Station for Windows
- Google Releases EmbeddingGemma 2 Open-Source Embedding Model
Latest selected stories
Items 1–13 of 13 itemsNVIDIA BlogSelectedAI score7171
NVIDIA BlogSelectedAI score7171 Open From Data Center to Desktop: NVIDIA Expands Nemotron Family With Highest Efficiency Super VL and Bolt Models
NVIDIA has released the open-source 120-billion-parameter multimodal model Nemotron 3.5 Super VL, plus two on-device models — Nemotron 3.5 Bolt 74B-A9B 24GB (already available) and Bolt E4B 4GB (available October 23) — covering agent deployment from data centers to local hardware.
Why it matters: NVIDIA discloses parameters, quantization and deployment tiers for the three open Nemotron models, so readers can pick the enterprise, workstation or 4GB on-device option that fits.
Hugging Face BlogSelectedAI score6767 Run fast, private agents on-device with LFM2.5-24B-A2B
Liquid AI released LFM2.5-24B-A2B, a reasoning model with 24B total parameters and only 2B activated per token, aimed at fast private on-device agents, published with open weights on Hugging Face.
Why it matters: The release includes full benchmark comparisons, inference speed and a five-stage training pipeline, so readers can judge whether it fits local agent workflows.
NVIDIA BlogSelectedAI score6868 All Systems Go: Global PC App Ecosystem Ready for NVIDIA RTX Spark
NVIDIA announced that RTX Spark laptops go on sale October 16, with up to 128GB unified memory and 1 petaflop of AI compute, alongside multiple software updates released with ecosystem partners.
Why it matters: The official roundup covers the software ecosystem around the RTX Spark launch, with specific updates and timing for local agent, creative, and developer tools.
NVIDIA BlogSelectedAI score7272 NVIDIA, Microsoft Kick Off a New Beginning for Windows PCs With RTX Spark and AI Agents
NVIDIA and Microsoft announced at the Windows AI and Surface event in San Francisco that they are jointly building agent software and hardware for Windows PCs.
Why it matters: The article details the specs and launch timeline for RTX Spark and DGX Station for Windows, helping readers assess how realistic running large models locally with resident agents will be.
Google DeepMindSelectedAI score6868 EmbeddingGemma 2: an open, lightweight multimodal embedding model
Google DeepMind released EmbeddingGemma 2, a 740M-parameter model built on the Gemma 4 architecture under the Apache 2.0 license, mapping text, code, images, video and audio into a shared embedding space.
Why it matters: The official release provides concrete metrics such as parameter count, memory footprint and benchmark scores, helping readers judge whether it suits on-device retrieval scenarios.
Hugging Face BlogSelectedAI score6060 Accelerating vision-language models with LFM2.5-VL-DSpark
Liquid AI has released experimental DSpark draft models for the vision-language model LFM2.5-VL-3B, using speculative decoding to trade for faster inference without changing output quality — decoding speeds up to 3.13x on-device and 2.66x on H100, with end-to-end gains up to 2.62x and 2.27x respectively.
Why it matters: The article details a speculative decoding acceleration scheme for vision-language models with concrete speedups and memory overhead, plus integration guidance for llama.cpp, MLX-VLM, and SGLang.
Hugging Face BlogSelectedAI score7070 Transformers now runs llama.cpp quants
Hugging Face has announced that transformers can load GGUF models — passing a gguf_file to from_pretrained lets you run quantized checkpoints from the Hub on your local machine.
Why it matters: Hugging Face provides official usage of GGUF in transformers plus benchmarks against llama.cpp, helping readers pick a viable local inference path.
Hugging Face BlogSelectedAI score6666 Up to 3.2x Faster Inference with LFM2.5-DSpark
LiquidAI released DSpark draft model checkpoints for the LFM2.5 series' LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B, speeding up inference via speculative decoding without changing output quality; throughput on H100 improves up to 3.18x, and up to 2.87x on edge devices.
Why it matters: The official release provides measured speedup data for the DSpark speculative decoding draft models along with llama.cpp and SGLang usage, so edge deployers can compare and judge the benefits.
Google DeepMindSelectedAI score7676 Introducing Gemma 4 12B: a unified, encoder-free multimodal model
Google DeepMind released Gemma 4 12B, a unified, encoder-free multimodal model in which vision and audio inputs go directly into the LLM backbone, positioned between E4B and the 26B MoE.
Why it matters: The article details the encoder-free architecture, 16GB-memory runnability, and Apache 2.0 open sourcing, helping readers evaluate on-device deployment options.
Mistral AISelectedAI score6565 Voxtral transcribes at the speed of sound.
Mistral AI has released Voxtral Transcribe 2, comprising two speech transcription models, Voxtral Mini Transcribe V2 and Voxtral Realtime, supporting 13 languages, speaker diarization, and word-level timestamps.
Why it matters: The official post provides latency, error rate, and pricing data for both transcription models and releases the Realtime weights, so readers can compare against their current speech solutions.
Mistral AISelectedAI score7878 Introducing Mistral 3
Mistral has released its new Mistral 3 model series, including the MoE-architecture Mistral Large 3 (41B active, 675B total parameters) and Ministral 3 models in 3B, 8B, and 14B sizes, all open sourced under Apache 2.0.
Why it matters: The official announcement fully lists model specs, open source licenses, deployment options, and available platforms, so readers can assess feasibility of running them in their own environments.
Mistral AISelectedAI score6262 Mistral Small 3.1
Mistral AI released Mistral Small 3.1, which improves text performance over Mistral Small 3, adds multimodal understanding, extends the context window to 128k tokens, and runs at 150 tokens per second.
Why it matters: Coming from the official release, it includes benchmark comparisons, the open-source license, and hardware requirements, so readers can assess its fit for lightweight deployments.