Definition

Positional Encoding

Because self-attention processes all tokens simultaneously, a Transformer has no built-in sense of word order; "dog bites man" and "man bites dog" would look identical. Positional encoding fixes this by adding a unique mathematical fingerprint, built from sine and cosine waves at different frequencies, to each token's embedding before it enters the model. Position is supplied by math, not learned.