An attention mechanism lets a neural network focus on the most relevant parts of its input when making a prediction. Instead of treating every pixel or feature equally, attention computes a weighted combination where important regions get higher weights. This idea first appeared in sequence-to-sequence models for machine translation and has since become central to computer vision.

Self-attention (the core of transformers) computes relationships between all pairs of input tokens. Given an image split into patches, self-attention lets each patch "look at" every other patch and decide which ones carry useful information. Multi-head attention runs several attention computations in parallel, each learning different types of relationships. One head might attend to spatial neighbors, another to semantically similar regions. Cross-attention connects two different inputs, like linking image features to text tokens in vision-language models.

In convolutional architectures, attention takes lighter forms: channel attention (SE-Net, ECA-Net) re-weights feature channels by importance, spatial attention (CBAM) highlights informative image regions, and deformable attention (Deformable DETR) learns to attend to sparse, task-relevant locations instead of every position. Vision Transformers, SAM, and D-FINE all rely heavily on attention.