Vector Embeddings

Vector embeddings are fixed-length numeric representations that place items in a continuous space where distance approximates semantic similarity. Unlike one-hot or sparse encodings, embeddings are dense — typically 64 to 4096 floating-point dimensions — and are learned so that related items end up close under cosine similarity or Euclidean distance. Embeddings are produced by neural models trained on objectives that pull semantically related items together and push unrelated items apart. Word2vec, GloVe, and fastText popularized word embeddings; sentence and document embeddings now come from transformer encoders such as Sentence-BERT, E5, BGE, and Qwen3-Embedding. Image embeddings come from CNN or ViT backbones trained with contrastive or classification objectives; CLIP-style models produce shared text and image embeddings in a single space. Once items are embedded, many tasks reduce to nearest-neighbor queries: Semantic Search: Finding Content by Meaning Instead of Keywords finds documents matching a query's meaning, recommendation surfaces users or products with similar embedding vectors, deduplication clusters near-duplicate items, and RAG (Retrieval-Augmented Generation): How LLMs Access External Knowledge feeds an LLM the embedding-matched chunks for a question. Because exact search scales poorly in high dimensions, embeddings are usually indexed with approximate nearest neighbor structures such as HNSW or product quantization inside a vector database.