Preparing Your Microsoft Environment for Enterprise AI

The first enterprise AI project in an organisation is usually described afterwards as having “run into unexpected problems.” The problems were rarely unexpected to anyone who had looked at the tenant. They were the estate’s existing debts — in identity, data, networking and governance — and the AI project was simply the first workload that could not route around them.

A knowledge assistant that must answer only from documents the asker is allowed to see is a very direct test of whether your permission model reflects reality. It turns out many do not.

What follows is the preparation that makes the difference, in the order it tends to matter.

Identity: the asker’s identity has to travel

Everything else depends on this. An AI application on Microsoft data must know who is asking, and that identity has to flow all the way through to retrieval — so that what comes back is what that person, and only that person, is permitted to read.

In practice this means the application authenticates users through Entra ID and acts on their behalf against data sources, rather than with a powerful identity of its own. Delegated permissions, not application permissions, wherever the source supports them. The convenient shortcut — one service principal that can read everything, with the application deciding what to show — moves the entire access-control model into your code, where it will be wrong.

The application’s own identity should be a managed identity: no client secret to store, rotate or leak. And the AI application is itself a sign-in target, so it belongs inside Conditional Access like any other, not exempted because it was new.

Data: your sharing model is your retrieval model

Retrieval-augmented systems inherit the permissions of the content they index. If the underlying sharing is wrong, the assistant faithfully reproduces the error, at conversational speed, to anyone who asks the right question.

Two problems surface immediately. Oversharing — SharePoint sites where “Everyone except external users” was the easy choice years ago, so documents that were technically open to the whole company but practically undiscoverable become instantly discoverable. And stale permissions — groups that reflect where people used to work.

Before indexing anything, know what is overshared and decide what to fix. Purview sensitivity labels give you a second lever: label what is confidential, then decide by label what AI may index at all. Some content should never be retrievable by a general assistant regardless of who is asking, and the label is the honest place to record that.

Data residency belongs here too. Know which regions your data may be processed in before choosing where the model runs, not after.

Network: the same landing zone, not a special one

AI workloads have a habit of being deployed outside the platform’s normal patterns, because they were a pilot, or a different team, or in a hurry.

Resist this. Azure OpenAI, AI Search and the storage behind them belong in the same landing zone model as everything else: private endpoints, no public network access, traffic through the hub, DNS resolving privately. A model endpoint with a public address and a key is exactly the kind of resource the security review later finds.

The practical wrinkle is regional availability. Model deployments and quotas are per region and change over time, and the region your landing zone prefers may not offer the model you want in the capacity you need. Discover this during design, not during the week you planned to go live.

Governance: decide who may build what

Without a governance decision, AI adoption in a Microsoft estate happens through whichever path has the fewest gates — usually a personal subscription, a developer’s own app registration and a SaaS tool somebody expensed.

Three decisions prevent most of this:

• A responsible-AI policy that is short enough to read. What use cases are permitted, what data may be used, what requires review. Two pages that engineers actually consult beat forty that legal approved.
• Named ownership per assistant. Every deployed AI application has a person accountable for its behaviour, its permissions and its cost — the same discipline you would apply to any application handling company data.
• A path for saying yes quickly. If the approved route takes six weeks, the unapproved route wins. Governance that works is mostly a fast lane with guardrails.

Add to this the operational reality that model versions are retired on a schedule. An assistant built against a specific model version is a dependency that will expire, and the estate needs a lifecycle process that upgrades and re-evaluates before that happens, not after.

Security controls specific to AI

The controls above are the ordinary platform ones applied consistently. A few are genuinely new.

Content filtering on model endpoints, configured deliberately rather than left on defaults you have not read. Logging of prompts and completions for investigation, designed with the privacy team, because these logs contain what your employees asked. And indirect prompt injection — content in your index that reads as an instruction — which is bounded in a retrieval-only system and considerably less bounded once the assistant has tools. The RAG and agents piece covers why that changes whose identity a tool runs as.

Cost: instrument the request, not the resource

The unit of AI spend is the request, and the resource-level tagging your FinOps practice relies on cannot see inside it. From the first deployment, log per request: which use case, which model, how many tokens in and out, how many calls. That is the difference between being able to state a cost per conversation and being able to state a monthly total nobody can act on. The AI FinOps article sets out why this becomes urgent with adoption rather than before it.

Evaluation: know what “working” means before you start

An assistant that answers plausibly is not the same as one that answers correctly, and the gap between the two is invisible without measurement.

Before anything goes to users, assemble a set of real questions with known-good answers and the sources they should come from. Run every change against it. Measure retrieval separately from generation, because when quality drops, the cause is usually that the right passage was never found — and tuning the prompt will not fix that.

This is also what lets you reduce cost with confidence: the cheaper configuration is acceptable if the evaluation set says so, and a guess otherwise.

People: the operating model is part of the environment

Somebody has to own retrieval quality when the index drifts. Somebody has to answer the question from finance about cost per conversation. Somebody has to decide whether the new model version behaves acceptably before the old one is retired.

None of these are roles most organisations have. They are not full-time roles either, at least initially. But they need names against them before launch, because the moment they are needed is the moment nobody has time to work out whose job it was.

A readiness checklist

Roughly in the order to tackle them:

1. Users sign in through Entra ID and the application acts on their behalf; the application itself uses a managed identity
2. Oversharing in SharePoint and M365 identified and remediated; sensitivity labels applied to what must never be indexed
3. AI resources deployed inside the landing zone with private endpoints and no public access; regional model availability confirmed
4. A short responsible-AI policy, a named owner per assistant, and a fast approval path
5. Content filtering configured and prompt/completion logging agreed with privacy
6. Per-request telemetry — use case, model, tokens, calls — from day one
7. An evaluation set with known-good answers, run on every change
8. Named owners for retrieval quality, cost and model lifecycle

The environment is the project

None of this is glamorous, and almost none of it involves a model. That is the point. The organisations whose AI programmes reach production are not the ones with the best prompts. They are the ones whose identity model was honest, whose data was shared as intended, whose platform patterns held for the new workload, and whose governance said yes quickly to the right things.

Prepare the environment and the AI work is an application project. Skip it and the AI work becomes an unplanned audit of everything underneath — conducted in public, on a deadline.