Definition

Embedding

An embedding is a vector of hundreds or thousands of numbers that represents a token's meaning as a position in high-dimensional space. Words with similar meanings land close together, which is why king minus man plus woman lands near queen. Models learn these positions during training. Embeddings also power RAG: document chunks and queries are embedded so similar content can be found by distance.