A multimodal embedding is a numerical vector that represents data from any modality (image, text, audio) in a single shared space. In this space, an image of a golden retriever and the text a golden retriever playing in a park have similar vectors, even though one is pixels and the other is words. This shared representation is what makes cross-modal search, retrieval, and matching possible.

CLIP and SigLIP produce the most widely used multimodal embeddings. Both use contrastive learning to train paired encoders: the image encoder and text encoder are trained so that matching pairs have high cosine similarity and non-matching pairs have low similarity. The resulting embedding vectors (typically 512 or 768 dimensions) can be stored in vector databases (Pinecone, Weaviate, Milvus) for fast similarity search across millions of items. OpenCLIP provides open-source implementations with various backbone sizes. For domain-specific applications, fine-tuning the encoders on domain data improves retrieval accuracy.

Multimodal embeddings power visual search engines, content recommendation systems, and duplicate detection in large datasets. Upload a photo and the system finds similar images in milliseconds. They also enable data curation (checking if image embeddings match assigned labels), zero-shot classification, and retrieval-augmented generation where a VLM retrieves relevant images before answering a question. Any workflow that matches images against text relies on multimodal embeddings.