There's a version of the data sovereignty conversation that ends quickly. You ask the vendor where the data is hosted. They name a region. You note it in the procurement checklist. Everyone moves on, assuming the AI vendor risk assessment is completed.
That conversation is not wrong, exactly. Server location is a critical component of sovereign AI infrastructure; it's just nowhere near enough, and the gap between what most enterprises believe they've established and what they've actually established is where regulatory exposure quietly accumulates.
AI sovereignty isn't a single answer. It's (at least) six. Here are the questions worth asking, and what a defensible answer actually looks like for each.
1. Where does our data live at rest, and who can reach it from outside our jurisdiction?
This is the question most procurement processes ask, and it's a reasonable starting point. But the server location alone doesn't determine who has legal access to what's on it. A provider headquartered in the US can be compelled under the CLOUD Act to produce data held on European servers. The location of the infrastructure is necessary information, but it doesn’t tell the complete picture about AI data localization.
The question underneath the question is: under which jurisdiction's laws does your provider ultimately operate? And what does that mean for your data when legal process arrives?
2. Where does the retrieval layer run?
In the quest for AI data sovereignty, this one rarely makes it onto the checklist. It should.
The retrieval layer – the embeddings, the vector indexes, the semantic structures the AI uses to find relevant content – is not the same thing as the underlying data. It's a derivative of it. And it doesn't always sit in the same place. A deployment can have its source data cleanly in one region while the retrieval infrastructure that queries it operates somewhere else entirely.
That's an AI sovereignty gap most organizations won't discover unless they actively for it. A defensible answer confirms that indexes and embeddings reside in the same region as the underlying data, by design and by default.
3. Where does the model actually execute?
When a user submits a query, the model has to run somewhere. That somewhere has a jurisdiction. And depending on which model endpoint is in use – a third-party API, a shared cloud service, an in-region deployment – the inference location may be entirely different from the data location.
For most enterprise AI use cases, this is manageable with the right configuration. For central banks, supervisory authorities, and sovereign wealth environments, it can be a hard requirement. The question to ask is specific: where does inference happen, who controls that endpoint, and does the customer have the option to run the model within their own environment?
A vendor who can't answer this precisely is a vendor who hasn't thought through it.
4. What happens to the prompt?
Every query generates a prompt comprising the assembled context that gets sent to the model. That prompt typically contains retrieved content, user context, and configuration instructions. It may also contain sensitive enterprise data.
Include these questions in your AI vendor assessment: Where does that prompt go? Is it logged by the model provider? Does it leave the customer's environment? Is it used for model training downstream?
The answers will tell you whether data that never left your servers ends up, in practice, being processed under someone else's terms. A defensible answer specifies exactly what is transmitted, to whom, and under which contractual and technical controls.
5. Where do the logs go, and who controls them?
Audit and observability data is sovereignty-relevant in its own right. Logs capture what users asked, what the system retrieved, what the model generated, and what actions were taken. In aggregate, that's a detailed picture of your organization's operations, decisions, and knowledge base.
If that telemetry is routed to vendor-controlled infrastructure by default, the sovereignty question applies to the logs just as it applies to the source data. A defensible answer gives the customer control over where observability data is collected and retained – ideally routed to customer-controlled storage, not a shared observability platform the vendor manages.
6. Who holds the keys, and where are they located?
The final layer is the one that determines all the others. Administrative access and encryption key management define who can, in practice, reach anything in the deployment. A customer can have data residency in the right region, retrieval infrastructure correctly co-located, in-region inference, controlled prompts, and customer-managed logs, and still have a sovereignty gap if a vendor's administrative team operating under a different jurisdiction holds the keys.
The question is simple: who has administrative access to this environment, where are they located, and under which jurisdiction do they operate? The answer should be unambiguous. If it isn't, keep asking.
AI Sovereignty: What a Complete Answer Looks Like
Six questions, six layers. They don't align automatically, and a deployment that's sovereign at one layer can be exposed at another without anyone noticing until the moment it matters.
The vendors worth trusting on this topic will answer all six specifically, without redirecting to the server location question you already asked. They'll also be able to show you the architecture that makes each answer true without simply hiding behind an assertion.
Sovereignty isn't a checkbox. It's a configuration. And it has to be deliberate at every layer, or it isn't really there.
This is the second post in our series on what regulated enterprises should actually be asking when they evaluate an AI platform. The series begins with The Questions SOC 2 Doesn't Answer.
