Embeddings

An embedding model reads a text, such as a sentence, a paragraph or a search query, and outputs a fixed-length list of numbers called a vector. Texts with similar meaning get vectors that sit close together, and that closeness can be calculated. Whether two texts mean the same thing becomes a math problem.

Embeddings exist for words, sentences, whole passages and, increasingly, for images, code and mixed content. They are the basis of semantic search, meaning search by meaning rather than by exact wording, and they handle the search step in most RAG systems. An embedding model does not write text. It is a separate model that works alongside the language model that writes the answers.

Embeddings matter to executives because they decide what an AI assistant can find in the company's own knowledge base. Whatever the search misses, no language model can use. Two decisions have lasting effects: the chosen embedding model ties the entire search index to itself, and the vectors are sensitive data, because they reveal a good deal about the original text.

How an embedding is created.

An embedding model is a trained neural network that splits text into tokens and outputs a vector of fixed length. That length is called the dimension. Common models produce a few hundred to a few thousand numbers per vector; OpenAI lists 1,536 and 3,072 dimensions for its text-embedding-3-small and text-embedding-3-large models. What carries information is where a vector sits relative to others, not any single number.

That position is usually measured with cosine similarity, which compares the direction of two vectors. If they point in almost the same direction, the texts are related in content. Many providers return vectors normalized to length 1. In that case the simple dot product gives the same value as cosine similarity and is faster to compute.

This is how the search and the passages are represented as numeric vectors, so the system can calculate similarity between them. The result is a ranked list of the passages closest to the query. No fixed threshold marks a hit as correct; the cutoff and the number of hits passed along are set per application, based on examples.

From word vectors to whole passages.

The idea predates today's language models. In 2013 Tomas Mikolov and colleagues at Google introduced Word2Vec, and in 2014 Jeffrey Pennington, Richard Socher and Christopher Manning at Stanford University followed with GloVe. Both learn a fixed vector per word from how words co-occur in large bodies of text, so words used in similar contexts end up close together.

These word vectors had a weakness: a word always got the same vector, no matter which sentence it appeared in. “Bank” as in riverbank and “bank” as in financial institution could not be told apart. Models built on the transformer architecture take context into account. Sentence-BERT, published by Nils Reimers and Iryna Gurevych in 2019, made it practical to represent whole sentences and paragraphs as a single vector and compare them quickly.

Today's embedding models produce vectors for passages up to a set maximum length, measured in tokens. Some also handle images, code or mixed documents such as slides and screenshots. There are general-purpose models and models tuned for fields such as law, finance or software code. The choice should depend on a company's own material rather than on a general leaderboard.

Where embeddings work inside a company.

The best-known use is semantic search. Keyword search finds the same words; search based on embeddings also finds the same meaning in other words. A search for vacation cover will also find a passage about handovers during absences: different words, related content. For part numbers, names and legal sections, keyword search is often more reliable, because exact identifiers barely differ in meaning. Many systems combine both.

In RAG systems, embeddings handle the search step: they find the passages that a language model then turns into an answer.

Some tasks need no language model at all. Incoming requests can be assigned to a topic by similarity. Duplicate or near-identical documents stand out because their vectors sit close together. Large collections such as customer feedback or ticket histories can be grouped without anyone defining categories in advance. “Similar products” and “related articles” often rely on embeddings too. What all these uses have in common is that embeddings measure closeness. They do not deliver a decision or a verified statement.

The model becomes part of data management.

The query and the documents have to be converted by the same model, or their vectors cannot be compared. Vectors from different models generally do not match. The exception is a model family that a provider has explicitly designed to share one vector space, as Voyage AI has done with its Voyage 4 series. Switching embedding models therefore means, in most cases, recomputing the entire collection. With a large collection, that is a project of its own, including a test of whether search gets better or worse.

Every stored vector should therefore carry a record of which model and version produced it. If a provider retires a model, there needs to be a plan for recomputing.

Some models distinguish between queries and documents. Voyage AI, for example, uses a parameter to mark a text as a query or a document and prepends a different instruction for each. According to the provider, leaving it out lowers retrieval quality.

Preparation matters too. Every model has a maximum input length, and long documents are split into passages first (chunking). Passages that are too long dilute the meaning; short ones lose context.

Embeddings are sensitive data.

Vectors are often assumed to be anonymous because they are just numbers. A 2023 paper by John X. Morris and colleagues shows otherwise: their method, Vec2Text, recovered 92 percent of short 32-token texts word for word from their embeddings, including full names from clinical notes. The authors conclude that embeddings should be treated with the same precautions as the original text.

An index built from personnel files, contracts or customer data therefore needs the same protection as the documents themselves, including under data protection law such as the GDPR. Deletion obligations have to reach the search index as well; when a document is deleted, its vectors go with it.

The OWASP project lists vector and embedding weaknesses as a separate item (LLM08) in its 2025 top ten risks for applications built on language models. The risks it names include missing access control, data leaking between user groups, reconstructing text from vectors and tampered content in the index.

A cloud embedding service receives every text, which calls for the same review as any other third-party processing. Open models in a company's own data center are the alternative when text should not leave the building.

What a good implementation looks like.

A good implementation starts with a test list of real queries whose matching passages are known, and checks whether those passages show up among the top hits. Public comparisons such as the Massive Text Embedding Benchmark (MTEB, Niklas Muennighoff and colleagues, 2023) offer orientation. Even the original study found no method that came out ahead on every task. Whether a model handles industry jargon, internal abbreviations and product names well only shows up in an in-house test.

Other signs of quality: search combines meaning and keywords wherever exact identifiers matter. Outdated versions are removed or flagged before they go into the index, because an embedding has no expiration date. Every vector is linked to its model, version, source document and access rights. New and changed documents are recomputed through a set process.

On cost: embedding services are usually billed by the number of input tokens. Cost arises when the index is first built, every time documents change, and with every query. Shorter or compressed vectors save storage but can cost accuracy, and that belongs in the test as well.

Where the term stops working

Embeddings measure similarity, not truth. Similarity is not factual correctness, and an outdated passage is found just as easily as the current one; quality depends on the model and on how content is prepared, among other things. Embeddings are distinct from the vector database that stores and searches the vectors, from RAG, which passes retrieved passages to a language model, and from fine-tuning, which changes a model through further training.