
ai
What a RAG evaluation harness actually needs to measure
Retrieval quality and answer quality fail in different ways, and a single score hides both. Measure the two stages separately, on a set you wrote by hand.
Engineering notes
Architecture decisions, tradeoffs, and things that broke — written by the people who build.

Retrieval quality and answer quality fail in different ways, and a single score hides both. Measure the two stages separately, on a set you wrote by hand.

Account structure, naming, state layout, network topology and identity. Everything else in a landing zone is reversible; these five are not, so decide them first.

The projects that survive contact with production are the ones whose failure mode is obvious. Pick a problem where being wrong is cheap and visible.