A component of transformer models that lets the model weigh the importance of different words relative to each other when processing text.
Attention is what allows a model to understand that 'it' in a sentence refers to a specific earlier noun, or that certain words matter more than others for a given task.
