Definition

Transformer

The Transformer is the neural network architecture introduced by Google in 2017 that replaced recurrence with attention. It stacks self-attention, multi-head attention, positional encoding, feed-forward layers, and residual connections. Because every token is processed in parallel, it scales almost linearly with hardware and handles long-range dependencies directly. Nearly every major model, from GPT and Claude to Stable Diffusion and Sora, is built on it.