What is a vector database?

A vector database answers one question, very fast

Give it a list of numbers and it returns the stored lists that are closest to it. That's the whole interface.

The lists are embeddings. An embedding model turns a passage, a product description or an image into a few hundred to a few thousand numbers, placed so that similar meanings land near each other, and what are embeddings in AI covers how. A vector database is where those lists live once you have a lot of them. It stores each one with an ID and some metadata, builds an index over them, and answers nearest neighbour search: "here is a query vector, give me the ten closest."

The hard part is the word fast. Comparing a query against a million stored vectors of 1,536 numbers each is 1.5 billion multiplications, and doing that on every keystroke doesn't scale. So vector databases use approximate nearest neighbour indexes. The most common is HNSW, a graph structure from a 2016 paper by Malkov and Yashunin that visits a few hundred candidates instead of all million and returns the right answer almost every time. That "almost" is the trade: a little recall for a thousandfold speedup.

Your existing database matches words; a vector database matches meaning

A relational database finds rows whose values match: WHERE customer_id = 4127. A full-text index finds documents containing words: refund AND policy. A vector database finds items whose meaning is near: the query "how do I get my money back" returns the passage titled "Returns and refunds" even though the two share no words.

The last row shows how blurred the category is. pgvector adds vector columns and an HNSW index to ordinary Postgres, and Elasticsearch has carried a vector field since version 7.0 in 2019 and HNSW search since 8.0 in 2022. A "vector database" is often a capability inside a database you already run rather than a new product, and the standalone products compete on scale and operational polish, not on doing something Postgres can't.

Chat-with-your-documents is a vector search with a model on the end

Retrieval-augmented generation, which what is RAG walks through, has a search step and a writing step. The vector database is the search step.

1. Split each document into passages of a few hundred to a thousand tokens.
2. Embed every passage and store the vectors, with the passage text and its source, in the vector database.
3. When a question arrives, embed it and ask the database for the nearest passages.
4. Hand those passages to the language model with the question, and let it write the answer.

Steps 1 and 2 happen once, when the documents are added. Steps 3 and 4 happen on every question. The language model never sees the whole collection; it sees the five or ten passages the database picked. That's why the answer is only as good as the search. If the nearest vector is the wrong passage, the model confidently answers the wrong question. That's a retrieval miss, not an invention, and what is an AI hallucination files it that way.

When you need one, and when sending the whole file is better

The decision is about size. A model can only consider its context window at once, and every token inside it is paid for on every turn. A vector database exists to put only the relevant slice in front of the model. For one document that fits, retrieval is a step you can skip; for a collection that doesn't, it's the only option.

Price the two designs on a 300,000-token report, several hundred pages, asked ten questions, retrieving five 1,000-token passages a question. Sending the whole file every time is 3 million input tokens. Retrieving five passages of 1,000 tokens per question is 50,000. At the list input rates in the AI model cost index, fetched from OpenRouter on 2026-08-20, the gap is 60x on every model. Prompt caching, where the provider offers it, bills a re-read of the same file at a tenth to a half of list price, which narrows the gap to roughly 6x to 30x. Retrieval still wins, by less.

The whole-file design buys something for that money when the file fits the budget. A 60-page contract, about 40,000 tokens, lets the model answer questions that need the entire document at once, such as "does anything here contradict clause 14". Retrieval can't ask that, because it never sees clause 14 and the contradiction in the same batch. One document, or a handful, goes in whole; a library goes through a vector database. Between the two, split the upload by topic so the model gets the section it needs.

Whizi sends the text itself, up to a 40,000-token turn

Whizi doesn't run a vector database for your chat. When you attach a file, the document upload path extracts the text and sends it to the model as prompt text, up to a flat 40,000-token budget per turn. A longer upload is cut, with a toast and an in-message marker saying so. Project files work the same way: the projects feature pins up to 10 files and rebuilds their text into the prompt on every message, capped at 32,000 characters per file and 120,000 for the whole project block.

The caps are the edge of the design. A pinned file past 32,000 characters is truncated, an attachment past the 40,000-token turn budget is cut, and a project is a folder, not a searchable library. The trade is deliberate. For a contract or a paper that fits the turn budget, having the text present beats having it searched. The check that matters, asking the model to quote the passage it used, works because the passage is there. For a thousand documents, a vector database is the right tool and a chat app is the wrong one.

Two habits carry over from the mechanism. Name the file in your question once several are attached, because the model chooses silently otherwise. And on any answer you'll act on, ask for the quote. A retrieval system can return the wrong passage; a whole-file system can answer from memory instead of the file. The quote catches both.