Start from utilisation, not from price
The instinct is to compare a per-token rate against an hourly instance rate. That comparison is meaningless without a third number: how much of that instance you will actually keep busy.
Dedicated capacity is cheap per request only when it is well utilised. A workload running at 15% utilisation on reserved hardware is usually more expensive than the same workload on a metered API, no matter how good the headline rate looks.
The four numbers that decide it
You can get a defensible answer from four figures, all of which are already in your logs.
- Sustained request rate: the median, not the peak.
- Peak-to-median ratio: how spiky the demand curve really is.
- Token shape: average input and output length per request.
- Quality floor: the smallest model that still clears your bar.
Peak shape matters more than volume
Two workloads with identical monthly volume can land on opposite sides of the line. A steady background batch job utilises reserved capacity beautifully. A workload that sits idle for twenty hours and then triples for four does not.
This is why the useful mitigation is rarely 'buy more capacity'. It is to keep an external API in place as an overflow path, size dedicated capacity to the sustained band, and let the spike spill over. The economics come from the base load; the resilience comes from the overflow.
The workloads that should not move
It is worth being explicit about the negative case. Low-volume workloads, highly variable ones, anything that genuinely needs the newest frontier model the week it ships, and anything where the quality bar cannot be met by a model you can run yourself. These belong on an external API, and moving them is value destruction dressed up as cost saving.
A migration that moves the right 60% of your spend and deliberately leaves the rest alone is a better outcome than one that moves everything.