Ingest and Query

A safe ingestion loop

Ingestion is the job that puts source material into the vector store. Read a source, normalize it into a consistent form, split it into chunks, create an embedding for each chunk, then upsert its ID and metadata. Upsert means update an existing ID or insert it when missing. Record the source checksum and embedding contract. Later, embed the question with the same model and request the top-k matches.

Inspect returned distance, document text, and metadata. Ordering alone is not enough evidence that a result is relevant.

Scenario: A deploy reruns ingestion and doubles every document because it used add with new UUIDs. Stable IDs and upsert turn the job into a safe repeatable operation.