RAG

RAG for private documents: retrieval before generation

Design retrieval, evidence and evaluation as a system instead of treating RAG as “put PDFs in a vector database”.

Start with the questions

Before choosing embeddings or chunk size, collect the questions users will ask and identify the evidence that should answer them. Your test questions become the beginning of a golden set for retrieval and answer evaluation.

Chunk around meaning

Chunk size is a trade-off. Very small chunks can lose context; very large chunks dilute relevance and consume more model context. Preserve document structure, headings and metadata when they help retrieval.

Retrieval is not only vectors

Hybrid retrieval can combine lexical ranking with semantic vectors. Exact identifiers, names and legal phrases often benefit from lexical search. Reranking can improve the final ordering after broad retrieval.

Citations are part of the product

Return the evidence that supports an answer, not only the answer itself. The user should be able to inspect the source and distinguish retrieved facts from model interpretation.

Evaluate retrieval and answer separately

If the system answers badly, determine whether the right evidence was retrieved first. Measure retrieval recall, citation quality and answer correctness separately; otherwise you will tune the wrong component.