An activation function sits between layers in a neural network and decides whether a neuron's output gets passed forward. Without it, stacking layers would just produce another linear equation, no matter how deep the network goes. The activation function introduces non-linearity, which is what lets neural networks learn complex patterns like edges, textures, and object shapes.

ReLU (Rectified Linear Unit) is the default choice in most vision models. It outputs zero for negative inputs and passes positive values through unchanged. Variants like Leaky ReLU allow a small gradient for negative values to avoid "dead neurons," while GELU (used in transformers like ViT) applies a smooth, probabilistic gate. Sigmoid and tanh still appear in specific roles: sigmoid for binary outputs, tanh for normalized ranges. But ReLU variants dominate hidden layers because they train faster and avoid vanishing gradient problems.

Choosing the right activation function affects training speed, gradient flow, and final accuracy. Most modern object detection and segmentation architectures (YOLO, EfficientNet, DETR) use SiLU/Swish or GELU in their building blocks.