Four things get settled before anything gets built.

Retrieval, orchestration, and serving designed for the volume and latency the product will meet in production, not the volume of the demo. Most of what makes an AI product feel broken is not the model — it is retrieval returning plausible-but-wrong context, an agent looping with no stop condition, or a pipeline that falls over the first time real data arrives.

We decide between hosted APIs, self-hosted models, and hybrid setups on cost and latency numbers we measure, not on vendor defaults. Frequently the right answer is a smaller, cheaper, more reliable model than the newest release.

Ingestion, cleaning, and labeling — plus the ongoing process that keeps the data usable after launch. A retrieval system quietly degrades as a corpus grows. A nightly job silently drops half a day of records the first time a source system changes its schema without warning.

We design for data minimisation from the start: a system that does not need a field should not receive it. Where a pipeline can work on pseudonymised or aggregated data, that is what gets built, which is both better privacy practice and a smaller blast radius.

An evaluation harness that runs against real cases pulled from your own traffic — not a handful of examples chosen because they looked good. A model scoring 94% on an eval set can still fail a third of the time in production, and usually the model did not get worse: the eval set never resembled production traffic in the first place.

The harness is a deliverable, not an internal artifact. It ships with the system and keeps proving the system as your data shifts underneath it.

What the system does when a dependency is slow, wrong, or unavailable. What it will not attempt. What happens the moment it is uncertain, and who it hands to.

The failure mode specific to this class of system is confident wrongness — a wrong answer looks exactly like a right one until somebody checks. So it is instrumented as its own tracked event rather than lumped in with server errors, because it never surfaces as one.