Collections, IDs, and Metadata

A vector database is more than vectors

A vector database is a tool built specifically to store vectors so they can later be searched by similarity, using the cosine-similarity idea from the previous module. In practice, a real collection stores much more than just the raw numbers: alongside each vector it also keeps a stable ID, the original document text the vector was computed from, and metadata - extra structured facts like the document's source, which customer it belongs to, a version number, or a timestamp. Giving each entry a stable ID is what makes updating it idempotent - a term meaning that repeating the same operation produces the same end result, rather than quietly piling up duplicate copies every time it's run again.

ChromaDB and Qdrant are two popular vector-database tools, and both offer a mode this platform relies on heavily: embedded mode. In this mode, Chroma's PersistentClient(path=...) and Qdrant's QdrantClient(path=...) run directly inside your own Python process, saving their data straight to local disk files, rather than requiring you to separately start and manage a background server process listening on a network port. That means no extra server memory budget, no port to configure, and one fewer moving part to keep track of - ideal for a lab environment or an early prototype.

Tip: Build each ID from something stable and reproducible, such as the source file's path combined with a chunk number, or a hash of the content itself. Avoid generating a fresh random universally unique identifier (UUID) for every insert - a new random UUID on every retry means every retry looks like a brand-new record to the database, which is exactly how silent duplication happens.