They built a language model that has never read anything above fifth grade — then found they could not teach it more
Researchers at the Max Planck Institute for Intelligent Systems, the ELLIS Institute Tübingen and ETH Zürich released LittleLearner: language models of 0.6B, 1.3B and 5B parameters trained from scratch on an 88-billion-token corpus distilled from FineWeb-Edu through a five-stage filter aligned to US Common Core standards, with everything taught above Grade 5 removed. Each model has a matching control trained on the same pipeline without the filter. Their central finding is that scaling the model up, post-training with SFT and GRPO, and in-context learning all amplify what the curriculum already contained — but none of them meaningfully improved performance on knowledge outside it. The corpus and models are published as a sandbox for studying how models acquire knowledge.
Related stories
Twenty tries at the same task cut the best agent's score by a third.2026-10-06A model learned to write without the method that trains every AI.
2026-10-06The US got 16 countries to put AI at the centre of science.
2026-10-05Mathematicians cracked five open problems using an ordinary chat box.
2026-10-05Untuned models solved agent tasks their polished versions could not.
2026-10-04