Skip to content

Data and Training

The craft of training: dataset construction, synthetic data, pretraining and post-training methods, compute and training costs.

Technology timeline

News collected over the past year · Latest 1 entries · Collection time is used when the source has no publication date

Latest selected stories

Items 1–4 of 4 items
Oct 1Thu
  1. The Decoder78

    Meta dodges billions in US taxes by calling its AI data centers experiments

    Meta classified its AI data centers as "experimental models" and its Nvidia chips as experimental materials, using the federal R&D tax credit to save $3.9 billion in 2025, up from $2 billion the prior year and $700 million in 2023, making it the largest beneficiary of the credit among public companies.

    Why it matters: The piece gives specific tax-savings figures over time and SEC risk disclosures, letting readers see how AI infrastructure investment is monetized through the 1981 R&D tax credit.

Sep 3Thu
  1. Hugging Face Blog62

    Training a coding model to paint watercolours with TRL and OpenEnv

    A Hugging Face blog author used TRL and OpenEnv to openly reproduce Surya Narreddi's approach, training Qwen/Qwen3.5-35B-A3B to write p5.brush code that paints watercolours.

    Why it matters: The author has open-sourced the training recipe, environments, reference pool, and model, so readers can reproduce the entire pipeline with a single command and learn from the troubleshooting notes.

Aug 25Tue
  1. Hugging Face Blog61

    Granite 4.2 LLMs: How They're Built

    IBM released the Granite 4.2 reasoning model family, comprising 3B, 8B and 30B dense decoder models built on the Granite 4.1 base (pretrained from scratch on about 15T tokens, with a five-stage strategy extending context to 512K), post-trained with SFT and multi-stage GRPO reinforcement learning, all open-sourced under the Apache 2.0 license.

    Why it matters: The official disclosure covers the full training pipeline from pretraining and SFT to multi-stage GRPO and agentic RL, transferable to similar model-building workflows.

Mar 12Thu
  1. Google Research62

    Introducing Groundsource: Turning news reports into data with Gemini

    Google Research launched Groundsource, using Gemini to extract structured disaster records from news reports in 80 languages, building an open dataset of 2.6 million urban flood events covering 150+ countries from 2000 to today.

    Why it matters: The original gives the full method and validation numbers for extracting disaster data from news with Gemini, letting readers assess its research value.