Start with the questions
Before choosing embeddings or chunk size, collect the questions users will ask and identify the evidence that should answer them. Your test questions become the beginning of a golden set for retrieval and answer evaluation.
Chunk around meaning
Chunk size is a trade-off. Very small chunks can lose context; very large chunks dilute relevance and consume more model context. Preserve document structure, headings and metadata when they help retrieval.
Retrieval is not only vectors
Hybrid retrieval can combine lexical ranking with semantic vectors. Exact identifiers, names and legal phrases often benefit from lexical search. Reranking can improve the final ordering after broad retrieval.
Citations are part of the product
Return the evidence that supports an answer, not only the answer itself. The user should be able to inspect the source and distinguish retrieved facts from model interpretation.
Evaluate retrieval and answer separately
If the system answers badly, determine whether the right evidence was retrieved first. Measure retrieval recall, citation quality and answer correctness separately; otherwise you will tune the wrong component.