Semantic search

A classic full-text search finds documents that contain the search terms. A semantic search also finds texts that say the same thing in different words. A search for “vacation coverage” can then surface a passage about handing over work during an absence. Today this usually relies on embeddings: a trained model turns the query and each text chunk into numeric vectors, and the search returns the chunks whose vectors sit closest to the query.

The term is not used consistently. It also covers older approaches built on ontologies and knowledge graphs, where relationships between concepts are defined by hand. Web search engines have interpreted meaning for years; in October 2019 Google announced that its BERT language model would help it better understand about one in ten English-language searches in the U.S. In business applications, semantic search today mostly means vector search over embeddings, which the research literature calls dense retrieval.

For companies, semantic search is the retrieval step behind many AI assistants. In RAG systems, it decides which passages a language model receives as its basis. If the search returns the wrong passage, the model writes a fluent but wrong answer. The quality of a knowledge assistant therefore depends to a large extent on the search.

How semantic search works.

The work starts before the first query. Documents are split into chunks, usually a few paragraphs long, because an embedding model only processes text up to a maximum length and shorter chunks match more precisely. The model computes a vector for each chunk. The vector, the text and a pointer to the source document go into an index, often a vector database.

At query time, the same model turns the question into a vector. The search finds the chunks whose vectors are nearest, usually measured by cosine similarity. On large collections it typically searches approximately so results come back fast.

Many systems then add a second stage, reranking. Another model reads the query and each candidate together and reorders the top results. It is more accurate but costlier, so it only sees the top candidates. At the end come a permissions filter and the display with a source link.

Queries and documents must be embedded with the same model. Switching embedding models means re-embedding the entire collection.

From matching words to matching meaning.

Most full-text search engines still rank results with BM25. Stephen Robertson and Hugo Zaragoza summarized the foundations of the method in 2009; it goes back to work from the 1970s and 1980s. BM25 weighs how often a search term appears in a document, how rare it is across the collection and how long the document is. If the word is not in the text, there is no match, even when the text describes exactly what the user wants.

Early semantic approaches tackled this with thesauri, ontologies and knowledge graphs. They are precise but have to be maintained by hand. The breakthrough for vector search came with language models built on the transformer architecture. In 2019, Nils Reimers and Iryna Gurevych introduced Sentence-BERT: finding the most similar pair among 10,000 sentences took about five seconds instead of roughly 65 hours with the original BERT.

In 2020, Vladimir Karpukhin and colleagues showed with Dense Passage Retrieval that a trained vector search found 9 to 19 percentage points more relevant passages in the top 20 results than a strong BM25 system on the question-answering datasets they tested.

Where keyword search still wins.

Semantic search measures similarity, and that is a drawback with exact identifiers. Part numbers, contract numbers, personal names, dates or rare technical terms barely differ in meaning from their neighbors. Order number 40-1187 can land close to 40-1178 in vector space. Microsoft’s Azure AI Search documentation explicitly names product codes, specialized jargon, dates and people’s names as cases where keyword search performs better because it finds exact matches.

Then there is transferability. The BEIR benchmark by Nandan Thakur and colleagues tested retrieval methods in 2021 on 18 datasets from different domains without tuning the models to them first. The result: BM25 proved a robust baseline, while dense retrieval models often underperformed outside their training domain. That is exactly the position of a company pointing an off-the-shelf model at its own documents.

Embedding models have improved since then. Whether a model handles a specific collection, internal abbreviations and industry terms well still only shows up in a test with real queries. Anyone relying solely on vector search should run that test before rollout, not after.

Hybrid search combines both methods.

In hybrid search, a full-text search and a vector search run in parallel, and their result lists are merged into one. Microsoft’s Azure AI Search works this way.

Merging is not trivial, because the scores from the two methods are not comparable. A BM25 score and a cosine similarity measure different things. A common solution is Reciprocal Rank Fusion, described in 2009 by Gordon Cormack, Charles Clarke and Stefan Büttcher. It looks only at where a document ranks in each list: the higher it appears across several lists, the higher it lands in the combined result. The authors used a constant of 60, which dampens the weight of the top positions.

A reranking step can follow. Microsoft calls its version the “semantic ranker,” which shows how loosely the term gets used in practice: sometimes it means vector search, sometimes reranking, sometimes the whole system.

For mixed collections of running text and fixed identifiers, hybrid search is usually the safer choice.

Similar does not mean correct or permitted.

Semantic search finds texts that resemble the query. It does not check whether they are correct, current or something the person asking is allowed to see. An outdated price list can rank as high as the current one, because both say almost the same thing. So outdated versions should be removed or flagged before indexing.

Permissions add a second problem: search actively surfaces content nobody would have found by browsing folders. OWASP lists vector and embedding weaknesses as a separate item in its 2025 list of risks for applications built on language models. It names inadequate access controls and information leaking between user groups that share a vector database. Its recommendations are fine-grained access controls, permission-aware vector stores and strict partitioning of datasets.

In practice, the permissions filter applies inside the search itself. Showing a result first and checking afterward whether it was allowed is too late once a language model has already processed the text. Embeddings computed from personal data need the same protection as the texts themselves.

What good implementation looks like.

A good implementation starts with a test list: real everyday queries where it is known which chunks contain the right answer. The measure is whether those chunks show up among the top results. It covers paraphrased questions and exact identifiers such as part numbers.

Every result shows its source so users can check it. Permissions apply in the search. New and changed documents enter the index through a fixed process, and deleted ones disappear along with their vectors. Each vector is stored with the model and version that produced it. Where exact identifiers matter, the search is hybrid.

On cost: embeddings are computed when the index is built, whenever a document changes and for every query; cloud services usually bill by processed tokens. Storage and compute for the index come on top, plus one reranking call per query if reranking is used. Often the bigger effort goes into maintaining the data itself: the search only finds what is current and stored in one findable place.

Where the term stops working

Semantic search measures similarity; it does not understand text the way a person does and checks neither accuracy nor currency. It is distinct from keyword search, which matches words; from hybrid search, which combines both methods; from RAG, which passes retrieved passages to a language model; and from query fan-out, which splits one question into many sub-searches. Semantic SEO is a marketing discipline, not a retrieval method.