What are embeddings? How text vectors power search and RAG

Key takeaways

• An embedding is a list of floating-point numbers (a vector) that represents the meaning of a piece of text, so texts with similar meaning get vectors that sit close together.
• Cosine similarity, the measure OpenAI and Ollama both recommend, scores how closely two vectors point in the same direction. For unit-length vectors it is simply the dot product.
• Common models output vectors of 384 to 3,072 dimensions. OpenAI's text-embedding-3-small costs $0.02 per million tokens; open models such as EmbeddingGemma and all-MiniLM-L6-v2 run free on your own hardware.
• Embeddings power semantic search, recommendations, clustering and the retrieval step of RAG. Always index and query with the same model.
• No single model wins everywhere: the MTEB benchmark paper found no method dominated across all of its 8 task types.

What are embeddings?

What are embeddings? They are numeric fingerprints of meaning: a model reads text and returns a fixed-length vector, often 768 or 1,536 numbers. OpenAI's documentation defines it as a vector of floating-point numbers where the distance between two vectors shows how related the texts are.

The point of turning words into numbers is that computers can compare numbers quickly. "How do I reset my password?" and "I forgot my login" share almost no words, yet a good embedding model places them near each other because they mean nearly the same thing. A keyword search would miss that match; an embedding search finds it.

This is why vector embeddings sit underneath most large language model (LLM) apps that work with your own data. Before a chatbot can answer from a company handbook, it has to find the right paragraphs, and it finds them by comparing embeddings.

A short history: from word vectors to sentence embeddings

The modern idea took off in January 2013, when Tomas Mikolov and colleagues at Google published the word2vec paper, "Efficient Estimation of Word Representations in Vector Space". Their models learned a vector for each word from a 1.6-billion-word dataset in less than a day of training, and the vectors captured both grammatical and semantic similarity between words.

Word vectors had a limit: each word got one vector no matter the context, so "bank" meant the same thing beside "river" and beside "loan". Transformer models fixed this by reading whole passages. Today's text embedding models produce one vector for an entire sentence, paragraph or document chunk, which is what search and RAG systems need.

How text becomes a vector

The pipeline inside an embedding model has three broad steps:

1. Tokenisation. The text is split into tokens, which are words or word pieces. Each model has a maximum input length. OpenAI's embedding models accept up to 8,192 tokens, while all-MiniLM-L6-v2 truncates input longer than 256 word pieces by default.
2. Encoding. A neural network, usually a transformer, processes the tokens and produces a vector for each one, informed by the tokens around it.
3. Pooling and normalisation. The token vectors are combined into one fixed-length vector for the whole input. Many providers then scale it to length 1. OpenAI says its embeddings are normalised to length 1, and Ollama's /api/embed endpoint returns L2-normalised vectors.

The model learns where to place texts during training. all-MiniLM-L6-v2, for example, was trained on about 1.17 billion sentence pairs, according to its model card. The training rewards the model for putting matching pairs (a question and its answer, a title and its article) close together and unrelated texts far apart.

Individual dimensions rarely mean anything a human can name. No single number stands for "about finance" or "positive tone". Meaning is spread across all of them, and only the position of the whole vector relative to other vectors carries information.

How cosine similarity works

Once texts are vectors, "how similar are these two texts?" becomes "how close are these two vectors?" The standard answer is cosine similarity: the cosine of the angle between the two vectors.

The formula divides the dot product of the vectors by the product of their lengths:

cosine(A, B) = (A · B) / (|A| × |B|)

The result runs from −1 to 1. A score near 1 means the vectors point the same way (very similar meaning); near 0 means they are unrelated. In practice, scores from text embedding models tend to sit in a narrower band, so the ranking matters more than the absolute number.

A toy example with three-dimensional vectors shows the arithmetic. Real models do the same thing with hundreds or thousands of dimensions.

Because OpenAI and Ollama already return unit-length vectors, both lengths equal 1, so cosine similarity becomes a plain dot product. OpenAI's guide notes this lets you compute it slightly faster, and that the choice of distance function usually makes little difference. Euclidean distance gives the same ranking as cosine similarity when vectors are normalised.

Embedding dimensions and cost compared

The table below compares widely used embedding models using figures from each provider's official documentation or model card.

Prices and specifications as listed on the providers' pages, checked 24 October 2026 (UTC). Google's documentation also describes a newer EmbeddingGemma 2 with about 740 million total parameters and an 8,192-token input limit; the Ollama library page for embeddinggemma lists the 300M, 2K-context version.

What the numbers mean in practice

Cost. At $0.02 per million tokens, embedding 10 million tokens of documents with text-embedding-3-small costs about $0.20; the same job with text-embedding-3-large costs about $1.30. OpenAI frames the small model as roughly 62,500 pages per dollar. Embedding is usually the cheap part of an LLM app.

Storage. Dimensions drive storage. A 1,536-dimension vector stored as 32-bit floats takes 6,144 bytes, so one million of them need about 6.1 GB before index overhead. At 3,072 dimensions that doubles to about 12.3 GB; at 384 dimensions it falls to about 1.5 GB.

Shortening. OpenAI's v3 models accept a dimensions parameter that trims vectors while keeping their meaning, and EmbeddingGemma supports Matryoshka truncation from 768 down to 128. Smaller vectors cut storage and speed up search at some cost in accuracy, a trade-off similar to the one described in our guide to LLM quantization.

Benchmarks. The Massive Text Embedding Benchmark (MTEB), introduced in 2022 by Niklas Muennighoff and colleagues, covers 8 task types, 58 datasets and 112 languages. Its authors found that no text embedding method dominated across all tasks, so test on your own data rather than picking the top leaderboard entry.

Where embeddings are used

• Semantic search: rank documents by similarity to a query instead of by shared keywords.
• Retrieval-augmented generation (RAG): find the passages most relevant to a question and pass them to an LLM as context. Our step-by-step guide shows how to build a RAG app in Python with Ollama and Chroma.
• Recommendations: suggest items whose vectors are close to the ones a user liked.
• Clustering and deduplication: group support tickets by topic or flag near-duplicate articles.
• Classification: use vectors as features for a simple classifier with few labelled examples.

Embeddings also help with a model's memory limit. Rather than pasting a whole document into a prompt and running into the context window, an app retrieves only the few chunks that matter.

Python example: semantic search with Ollama embeddings

This script runs fully on your own machine. It needs Ollama installed (see how to run a local LLM with Ollama), the model pulled with ollama pull embeddinggemma, and the Python client installed with pip install ollama.

import math
import ollama

docs = [
    "Reset your password from the account settings page.",
    "Our office is closed on public holidays.",
    "Invoices are emailed on the first day of each month.",
]
query = "I forgot my login details"

# One call embeds all documents plus the query
resp = ollama.embed(model="embeddinggemma", input=docs + [query])
vectors = resp["embeddings"]
doc_vecs, query_vec = vectors[:-1], vectors[-1]
print("dimensions:", len(query_vec))

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(y * y for y in b))
    return dot / (norm_a * norm_b)

scores = [(cosine(query_vec, v), text) for text, v in zip(docs, doc_vecs)]
for score, text in sorted(scores, reverse=True):
    print(f"{score:.3f}  {text}")

The password-reset sentence should rank first even though it shares no key words with the query. Exact scores vary by model version. To use OpenAI instead, swap the embedding call for the official client:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment
resp = client.embeddings.create(
    model="text-embedding-3-small",
    input=docs + [query],
    dimensions=512,  # optional: shorten from the default 1,536
)
vectors = [item.embedding for item in resp.data]

For more than a few thousand documents, store vectors in a vector database or index (Chroma, pgvector, FAISS and similar tools) rather than looping in Python.

Common mistakes to avoid

• Mixing models. Vectors from different models live in different spaces and cannot be compared. Ollama's documentation says to use the same embedding model for indexing and querying. Changing models means re-embedding everything.
• Ignoring the input limit. Text beyond a model's maximum is cut off, often silently. Split long documents into chunks first.
• Treating scores as probabilities. A cosine score of 0.8 is not "80% relevant". Compare scores within one model and set thresholds by testing.
• Skipping evaluation. Benchmark scores are averages over many datasets. A handful of real queries from your users is a better test.

What it means

Embeddings turn the fuzzy question "what is this text about?" into geometry that a computer can search in milliseconds. For anyone building with LLMs, they are the bridge between a general-purpose model and your own documents.

The practical choices are few. Pick a model that fits your budget, privacy needs and languages; a hosted API such as OpenAI's is cheap and simple, while local models through Ollama keep data on your own machine. Choose dimensions with storage in mind, compare vectors with cosine similarity, and test retrieval quality on your own queries before trusting any leaderboard.

Sources

• OpenAI: Vector embeddings guide
• OpenAI: API pricing
• Ollama documentation: Embeddings
• Ollama library: embeddinggemma
• Google AI for Developers: EmbeddingGemma
• Hugging Face: sentence-transformers/all-MiniLM-L6-v2 model card
• Muennighoff et al., MTEB: Massive Text Embedding Benchmark (arXiv:2210.07316)
• Mikolov et al., Efficient Estimation of Word Representations in Vector Space (arXiv:1301.3781)