Skip to content

#数据/训练

Today 0 items
Oct 16Fri
  1. Hugging Face Blog50

    Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline

    NVIDIA and IIT Madras's AI4Bharat released the Nemotron-pretraining-Indic-v1 pretraining corpus, roughly 2.8 trillion tokens covering all 22 scheduled languages of India, with 2,601.0B natural-script tokens and 224.6B romanized tokens (Gemma-4 tokenizer statistics); it will soon ship alongside an open-source, GPU-accelerated data generation pipeline.

Oct 13Tue
  1. Microsoft Research40

    Simulating student learning for better AI tutoring

    Microsoft Research released StudentSim, a research system for evaluating and improving AI tutoring tools before testing with real learners, training a personalized simulator for each student based on publicly available de-identified learner records. It also introduces the StudentSimEval benchmark, which evaluates along two dimensions, behavioral fidelity and instructional responsiveness, across three domains: chess, second-language English writing, and math, outperforming leading baselines. The system is for research-stage use only, is not a product, has not been deployed in classrooms, and is not intended to replace teachers.

Oct 7Wed
  1. Hugging Face Blog43

    Falcon-Emirati: When an LLM Learns the Dialect, the Culture, and the Nuance

    The co-publishers with Hugging Face released Falcon-Emirati-7B, fine-tuned from the Falcon-H1-Arabic 7B variant to understand and generate Emirati Arabic dialect. The model uses a hybrid Mamba and Transformer attention architecture, trained on Emirati dialect web data, Emirati-culture MSA data, and synthetic data constrained by lexicon and style rules, with a context window supporting up to 128K/256K tokens.

Oct 2Fri
  1. Google Research51

    Toward provably private learning from federated data

    Google has announced a next-generation federated learning system based on trusted execution environments (TEEs), providing remotely verifiable and auditable guarantees for data anonymization. Access policies are published to the public transparency log Rekor, the KMS and data processing binaries are reproducibly built from open source code, and encrypted training data is decrypted and processed only inside the TEE, with workload operators seeing only metrics and differentially private model weights.

  2. Hugging Face Blog39

    AutoSynthData: Generating Training Data for Enterprise Agents

    ServiceNow CoreAI launched AutoSynthData, which uses a target model's failures in an environment and successful trajectories from stronger teachers to automatically generate executable, realistic, and hard training tasks for enterprise agent post-training. Each task consists of a system specification, user prompt, and verifier, and must meet feasibility, realism, and difficulty requirements. The team demonstrated the pipeline with a public dataset from EnterpriseOps Gym and dynamically adjusted the curriculum as models improved.

Oct 1Thu
Sep 28Mon
Sep 26Sat
  1. AWS Machine Learning Blog31

    Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput

    AWS proposed an architecture based on Amazon EKS, EFA and DeepEP to accelerate large-scale reinforcement learning post-training of MoE models, delivering 40% higher throughput. The solution targets workloads like RLHF, PPO and GRPO, addressing resource balancing between rollout generation and policy training, high-throughput communication across hundreds of accelerators, and the dynamic all-to-all token routing overhead introduced by Expert Parallelism.

Sep 16Wed
Sep 11Fri
Sep 10Thu
Sep 3Thu
  1. Hugging Face Blog62

    Training a coding model to paint watercolours with TRL and OpenEnv

    A Hugging Face blog author used TRL and OpenEnv to openly reproduce Surya Narreddi's approach, training Qwen/Qwen3.5-35B-A3B to write p5.brush code that paints watercolours.

    Why it matters: The author has open-sourced the training recipe, environments, reference pool, and model, so readers can reproduce the entire pipeline with a single command and learn from the troubleshooting notes.

Aug 28Fri
  1. Google Research50

    Planetary prediction engine: Automating global models via Earth AI

    Google Earth AI has introduced an experimental research capability called the Planetary Prediction Engine (PPE), which autonomously performs the full geospatial prediction workflow—data discovery, cleaning, feature engineering, model training, and evaluation—starting from natural language queries, compressing work that previously took weeks of manual data engineering down to a few minutes.

Aug 27Thu
  1. Google Research47

    GlucoFM: Foundation model for continuous glucose monitoring

    Google unveils GlucoFM, a self-supervised foundation model with a two-stream design that separates slow glucose baseline trends from short-term deviations. Across four cohorts, seven clinical prediction tasks and 14 evaluations, its mean PR-AUC improves from 54.7 to 58.8 over the strongest baseline CGM baseline, leading on all diabetes risk and beta cell dysfunction evaluations, and achieves the lowest MAE in postprandial glucose response prediction.

Aug 25Tue
  1. Hugging Face Blog61

    Granite 4.2 LLMs: How They're Built

    IBM released the Granite 4.2 reasoning model family, comprising 3B, 8B and 30B dense decoder models built on the Granite 4.1 base (pretrained from scratch on about 15T tokens, with a five-stage strategy extending context to 512K), post-trained with SFT and multi-stage GRPO reinforcement learning, all open-sourced under the Apache 2.0 license.

    Why it matters: The official disclosure covers the full training pipeline from pretraining and SFT to multi-stage GRPO and agentic RL, transferable to similar model-building workflows.

Aug 10Mon
Jul 26Sun
  1. Berkeley AI Research46

    Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction

    Berkeley AI Research proposes the ABBEL framework, which replaces full interaction history with a natural-language belief state as the agent's working context and introduces belief grading to supervise summary content. On the CollabBench collaborative coding benchmark, ABBEL-rec-BG closes roughly 50% of the performance gap with full-context models, cuts training steps from 100 to 50, and uses significantly shorter peak context token lengths than the full-context setting.

Jul 9Thu
Jul 6Mon
  1. Hugging Face Blog47

    PRX Part 4: Our Data Strategy

    Hugging Face details the data pipeline for its PRX image generation model: mixing public and internal datasets for pretraining, using a VLM to regenerate long captions, then converting them into streamable training data. The text encoder has been switched to Qwen3-VL, with text latents computed online during training at a throughput cost of only about 3–4%. Images are uniformly encoded as JPEG quality 92, and the data is built with Lance and streamed for training via MDS (Mosaic Streaming).

Jun 30Tue
Jun 17Wed
Jun 11Thu
Jun 4Thu
May 16Sat
Apr 22Wed
Apr 16Thu
Apr 9Thu
Apr 1Wed
Mar 18Wed
  1. Mistral AI43

    Introducing Forge

    Mistral AI released Forge, letting enterprises train frontier-grade models on internal documents, codebases, and business data so models understand internal terminology, workflows, and compliance requirements. Forge supports pretraining, post-training, and reinforcement learning, works with dense and MoE architectures and multimodal inputs, and ships with data pipelines and Mistral AI's training methods, supporting continuous evaluation and iteration. ASML, Ericsson, and the European Space Agency, among others, have partnered with Mistral AI to train proprietary models.

Mar 12Thu
  1. Google Research62

    Introducing Groundsource: Turning news reports into data with Gemini

    Google Research launched Groundsource, using Gemini to extract structured disaster records from news reports in 80 languages, building an open dataset of 2.6 million urban flood events covering 150+ countries from 2000 to today.

    Why it matters: The original gives the full method and validation numbers for extracting disaster data from news with Gemini, letting readers assess its research value.

Jan 10Sat
  1. Berkeley AI Research45

    Information-Driven Design of Imaging Systems

    Berkeley AI Research proposes a NeurIPS 2025 framework that evaluates and optimizes imaging systems directly on information content, using a mutual information metric to unify factors like resolution and noise. The method needs only noise measurements and a noise model to estimate information, and provides an upper bound on true information. Across color photography, radio astronomy, lensless imaging, and microscopy, the metric predicts decoder performance and its optimized results match SOTA end-to-end methods with lower memory and compute and no task-specific decoder.

Nov 1Sat
  1. Berkeley AI Research46

    RL without TD learning

    Berkeley AI Research proposes an off-policy RL value-learning algorithm based on divide and conquer that no longer uses temporal difference (TD) learning, whose Bellman recursion error accumulation makes it hard to scale to long-horizon tasks.

Sep 1Mon
  1. Berkeley AI Research49

    What exactly does word2vec learn?

    Berkeley AI Research proposes a quantitative theory of the word2vec learning process: under several mild approximations, its learning problem reduces to unweighted least-squares matrix factorization, the gradient flow dynamics have a closed-form solution, and the final embeddings are equivalent to PCA on the target matrix M*.