RAG Quality: Retrieval First, Generation Second

When answers sound confident but conflict with your documents, the usual mistake is to tune the model before you measure retrieval. In most production RAG stacks, bad chunks or wrong ranks explain more failures than “the LLM forgot.”

Start with the evidence trail. Log the top k chunk IDs, scores, and short previews for every query (with redaction for PII). If the right passage never appears in those k results, no amount of prompting fixes that—you need better chunking, metadata filters, embeddings refresh, or hybrid sparse+dense search.

When retrieval looks sane and answers still drift, tighten the contract: require citations, forbid facts not present in retrieved text, and surface “not enough context” as an explicit outcome instead of guessing.

This keeps iteration honest: you upgrade ingestion and retrieval with numbers, and you treat generation as the last mile, not the first knob.