Definition

Emergent Abilities

Emergent abilities are capabilities that appear once a model reaches a certain scale without anyone training for them explicitly. Translation, code generation, mathematical reasoning, and few-shot learning all showed up as GPT models grew from millions to billions of parameters. The book treats this as the central surprise of the scaling era: predict the next token at sufficient scale and new skills keep appearing.