Multimodal alignment is the process of training a model so that semantically related inputs from different modalities end up with similar representations. An image of a sunset and the phrase a sunset over the ocean should produce vectors that are close together in the model's embedding space. Without good alignment, a VLM may see the image correctly and understand the text correctly but fail to connect the two, leading to irrelevant answers, missed objects, and hallucinations.

Alignment is achieved through different training strategies. Contrastive alignment (CLIP, SigLIP) learns by matching image-text pairs from large web-crawled datasets. Generative alignment (BLIP-2, CoCa) learns by training the model to generate text that describes the image. Projection alignment (LLaVA, PaliGemma) trains a small projection layer to map image encoder outputs into the LLM's token space, typically on image-caption pairs. Most VLMs use a multi-stage approach: first align representations through contrastive or captioning pre-training, then fine-tune for instruction-following. Poor alignment manifests as hallucination (the model generates plausible text that doesn't match the image) or visual neglect (the model ignores the image and answers from language priors alone).

Alignment quality determines real-world VLM reliability. When evaluating a VLM for production use, testing alignment on your domain data is essential. A model aligned on web images may perform poorly on medical scans, satellite imagery, or manufacturing inspection photos. Domain-specific fine-tuning partially re-aligns the model to new visual distributions.