Migrating a workload without a quality regression

The cheapest possible AI is one that produces the wrong answer. How to move a production workload and prove nothing got worse.

· 7 min read

Agree the bar before you touch anything

The failure mode in every cost-reduction project is that the target is a number on the invoice and quality is left as a feeling. Six weeks later nobody can say whether the system got worse, and the argument becomes political rather than empirical.

Write the bar down first. Accuracy on a held-out set, latency at the 95th percentile, refusal and escalation rates, whatever actually governs the business outcome. Those become constraints on the migration, not aspirations for it.

Shadow before you switch

The safest migration runs both paths at once. Real production traffic is served by the existing external API and simultaneously replayed against the new dedicated path, with outputs compared offline.

This costs a little more for a few weeks and removes almost all of the risk. By the time you cut over, you are not predicting that quality holds. You have several weeks of paired outputs on your own traffic showing that it does.

Move in stages, by workload

Nothing about this needs to be a single event. Workloads move one at a time, each measured against the path it replaces, each independently reversible.

  • Start with the highest-volume, lowest-variance workload: best economics, easiest evaluation.
  • Keep the external path warm until the new one has held the bar under real load.
  • Cut over a percentage of traffic first, not all of it.
  • Leave workloads that fail the bar exactly where they are.

Keep the exit open

Reducing dependency on one provider is not an improvement if it creates an equally rigid dependency somewhere else. The application should keep talking to a standard interface throughout, so that moving a workload back is a routing change rather than a project.

If a migration can only go one way, the savings are not worth what you gave up to get them.

The short version

Write the quality bar down, shadow production traffic against the new path, migrate one workload at a time, and keep the route back open.

Read next

Sizing capacity for spiky demand

Provisioned infrastructure has one genuine failure mode: a spike you did not size for. Here is how to design so it never becomes an outage.

Next step

Curious whether this applies to your workload?

Tell us how many people are in the firm and what kind of client work they handle. We will tell you what it would cost to give all of them a private workspace, before anyone talks about a contract.

Talk to Runvo

Tell us the size of the firm and what your client contracts require.