Definition
Decoder-Only Transformer
A decoder-only Transformer uses just the generating half of the original architecture, reading text left to right and predicting the next token. BERT took the encoder and became a reader; GPT took the decoder and became a writer. The training objective needs no labels, only raw text, so the whole internet becomes training data. This choice is why modern LLMs are general-purpose and scale so well.
Explained in
Chapter 3: Attention Is All You Need FreeThe paper behind every major AI model today. Google invented it. Then watched someone else ship it.
Related terms