The question auditors actually ask
It is rarely 'which model are you using'. It is 'where did this customer record go, who could read it, and how do you know'. Those are infrastructure questions, and a vendor’s terms of service is a weak answer to all three.
When inference runs on infrastructure inside your own account, the answers become the same answers you already give for your database and your application servers, because it is the same boundary, the same logging, the same retention policy.
What actually moves
Running privately does not mean building a model. It means the weights and the serving layer sit inside your boundary, and the request never crosses it.
- Inference runs in the region and account you nominate.
- Prompts and responses land in your logging stack, under your policy.
- Model versions are pinned, so nothing changes underneath an evidenced control.
- Access follows your existing identity and network controls.
Version pinning is the underrated part
Teams focus on residency and miss change control. On a managed API, the model behind a version string can be updated, deprecated or retired on the provider’s schedule, which may land in the middle of your audit cycle, after you have already evidenced how the system behaves.
Pinned weights on infrastructure you control remove that class of surprise entirely. You upgrade when you have re-tested, not when someone else ships.
What it does not solve
Private infrastructure is not a compliance programme. It does not decide your lawful basis, write your DPIA, or make a badly-scoped prompt safe. It removes one specific and awkward category of risk (third-party data transit at inference time) and leaves the rest of the work where it was.
Claiming more than that is how privacy engineering loses credibility with the people who have to sign the forms.