Concert· Paris
CHINCHILLA
The "Chinchilla" paper (Hoffmann et al., 2022, DeepMind) introduced a compute-optimal 70B-parameter language model trained on 4x more data than models of similar size, showing that most large LLMs were significantly undertrained on data. Its key finding—that model size and training tokens should be scaled roughly proportionally—reshaped how subsequent language models were trained.