Vector Database

A vector database is built for one job: given a query vector, find the entries closest to it. The vectors come from an embedding model that turns text, images or other content into lists of numbers. Alongside each vector, the database stores metadata such as source document, date or access rights, and usually the text passage itself. A query returns a ranked list of the passages that fit best in content.

The term covers two things: standalone systems built only for vectors, and vector search as a feature inside a broader database, for example in a relational, search or document database. Both solve the same problem; they differ in how they are run and how well they fit existing data infrastructure.

Executives tend to run into the term when a knowledge assistant is supposed to work with company documents and the question comes up which search the RAG system needs and whether that requires a new system. The answer depends on what users ask, the data volume and the requirements for access rights and deletion. How well the search performs depends more on the embedding model, how passages are cut and how content is maintained than on the choice of database.

How a vector database works at its core.

During setup, documents are split into passages, an embedding model computes a vector for each passage, and the database stores it along with its metadata. It also builds an index that makes later searches fast.

A query is converted by the same model, and the database returns the entries whose vectors sit closest, with their similarity scores. Common measures of closeness are cosine similarity, dot product and Euclidean distance; the choice has to match the embedding model.

A relational database, by contrast, checks exact conditions in rows and columns: customer number matches, date in range, status open. A vector database sorts by closeness and always returns the nearest entries, even if none of them really fits. A hit therefore only means that a passage is more similar to the query than the others. For master data and accounting, the conventional database remains the right tool. The vector database complements it wherever search is about meaning.

Why search is usually approximate.

The simplest search compares the query with every stored vector. That guarantees the most similar entries, but it gets slow with large collections. Vector databases therefore usually rely on approximate nearest neighbor (ANN) search. It searches only part of the collection and finds the most similar entries with high but not complete certainty.

A widely used method is HNSW (Hierarchical Navigable Small World) by Yury Malkov and Dmitry Yashunin, published in 2018 in IEEE Transactions on Pattern Analysis and Machine Intelligence. It links vectors into a multilayer graph that the search works through from coarse to fine. Other methods split the collection into clusters and search only the relevant ones (IVF), or compress the vectors. The Faiss library from Meta bundles many of these methods.

Every method trades accuracy for speed and memory. The share of the truly nearest entries that a search actually finds is called recall. The documentation for pgvector, the PostgreSQL extension, explicitly notes that results change once an approximate index is added. Small collections can often use exact search; for large ones, recall should be measured before launch.

A database of its own or a feature of an existing one.

Vector search runs either in a vector database of its own or as a feature inside a broader database. Examples of standalone vector databases are Pinecone, Weaviate, Qdrant, Milvus and Chroma. Many broader databases also offer vector search, such as PostgreSQL with the pgvector extension, search systems like Elasticsearch and OpenSearch, or document databases like MongoDB.

Using a feature in an existing database has a clear advantage: the vectors sit next to the rest of the data. Access rights, backups, monitoring and deletion run through channels the company already operates, and queries can combine similarity with ordinary conditions.

A dedicated vector database tends to pay off with very large collections, heavy load, many tenants or features an existing database lacks. In exchange, it adds another system to run and secure and, as a cloud service, its own data processing agreement.

The decision follows what is already in place more than the technology: a separate system makes sense once the existing setup hits its limits.

Full-text, vector or both.

Which search fits the RAG system depends on the questions people ask. Not every RAG system needs a vector database. Full-text search can also be a good fit: it finds passages with the same words and is strong on part numbers, contract numbers, product names and legal sections. A widely used scoring method here is BM25. Vector search finds passages with related meaning, for example when someone asks what gets reimbursed for a drive to a client and the document is titled “Travel expense policy.”

A hybrid method merges both result lists and can also be a good fit. A common way to do that is reciprocal rank fusion, described in 2009 by Gordon Cormack, Charles Clarke and Stefan Büttcher: each hit earns points based on its rank in each list, and the total sets the new order. Some systems then re-rank the top hits with a separate model.

A simple check helps: collect the most common real queries from future users and see whether they ask for exact terms or for meaning. If exact terms dominate, good full-text search is often enough. If free-form questions about inconsistently written documents dominate, vector search is needed. With a mix, the hybrid method is the safer choice.

Access rights and deletion belong in the database.

A vector database returns whatever is similar to a query and checks permissions only if explicitly set up to do so. The OWASP project names missing access control in vector stores as part of risk LLM08 (vector and embedding weaknesses) in its 2025 list for applications built on language models.

The usual solution is metadata filtering: each passage is tagged with department, tenant or permission group, and every query is limited to what the person may see. Order matters here. If the filter is applied only after the approximate search, too few matching hits, or none at all, may be left. The pgvector documentation describes this effect and offers iterative index scans that widen the search step by step; other systems filter during the search itself. Vendors should be able to explain how they handle this.

Deletion needs a set process too. When a document is deleted or changed, its vectors have to go as well or be recomputed. That applies to deletion obligations under the GDPR as much as to outdated rules. Because text can partly be reconstructed from embeddings, the index deserves the same protection as the source documents.

What a good implementation looks like.

A good implementation is measured against a test list of real queries whose right passages are known. It shows whether the search finds them, whether an approximate index loses hits, and whether changing the model, passage size or database improves anything.

In operation, a few points matter more than the product. Every vector records which embedding model produced it, because switching models generally means recomputing all of them. New and changed documents reach the index through a set process. Backup, restore and response times are tested and monitored.

Cost depends mainly on the number of vectors, their length and the number of queries. A vector with 1,024 dimensions stored as 32-bit floating-point numbers takes up about 4 kilobytes before the index and metadata are added. Techniques such as quantization store values with fewer bits; Voyage AI cites a quarter of the storage for 8-bit integers and one thirty-second for binary values. Lower precision can cost hits, which belongs in the test. And outdated versions in the index lead to outdated answers.

Where the term stops working

A vector database finds similar content, not verified answers. It always returns the nearest entries, even if none fits, and it finds an outdated version as reliably as the current one. Not every RAG system needs a vector database; full-text and hybrid methods can also be a good fit. It is distinct from embeddings, which produce the vectors, from RAG as the overall method of search plus answer, and from graph databases, which store relationships between objects.