Cloud infrastructure after the Microsoft-OpenAI split

Cloud infrastructure after the Microsoft-OpenAI split

The split between Microsoft and OpenAI has forced a lot of teams to recheck where AI runs and who supplies it. I would treat that as a reminder to harden cloud setups before a partner shift does the work for you.

Assess the current setup

Start with facts. I inventory workloads, data flows, and dependencies. For each service, I ask where it runs, who owns the model or API, and what happens if that provider changes terms or access.

Steps I use:

1. Map every service to a business function and the cloud services it uses, such as compute, object storage, managed databases, GPU or TPU instances, and identity.
2. Mark dependencies on third-party AI models or APIs, plus the authentication method.
3. Classify each dependency by criticality: single point of failure, recoverable, or optional.

Keep the inventory simple. A CSV or YAML file is enough. Example columns: service, cloud provider, region, AI model provider, auth method, failover plan, RTO, RPO.

Separate infrastructure from AI providers

Do not bet everything on one vendor. I split platform providers from model providers. A cloud service can host several model providers, so keep the roles clear.

• Infrastructure provider for compute and storage.
• Model or API provider for pretrained models.
• Niche vendors for specialist models or tooling.

Check that you have access to your data and keys. If an AI partner can revoke access or change revenue share overnight, you need alternatives. Ask for SLAs and exportable model checkpoints where possible.

Build for movement

I keep cloud builds portable where I can.

• Use containerised inference with Docker and Kubernetes, or serverless wrappers, so models can move between clouds or on-prem.
• Use open formats such as ONNX and SavedModel where possible.
• Keep infrastructure-as-code modules per provider so equivalent builds can be spun up quickly.
• Use multiple regions and cross-region replication for stateful stores.

One approach is a primary inference cluster on hosted GPU instances and a smaller warm standby in another provider. If the primary provider restricts a model, traffic can move to the standby while the licensing mess gets sorted.

Tighten security

Cloud services and model APIs change. Security matters more, not less.

• Keep secrets in a vault with rotation and short TTLs.
• Use least privilege with IAM roles, not broad keys.
• Encrypt data at rest and in transit. Treat prompts with sensitive data as regulated.
• Log the important stuff and ship logs to an immutable store.

I run periodic breach drills. Simulate a credential revocation and check that keys can be rotated and deployments pushed back inside the RTO.

Monitor what matters

You cannot fix what you do not measure.

I watch:

• Latency and error rates per model and per provider.
• Cost per inference and cost per request.
• Model quality drift if I run my own fine-tuned models.

Set alert thresholds for cost spikes and error-rate increases. Automate traffic shaping so a small share of requests can go to candidate providers before a full cutover. Use canary deployments and rollbacks for model swaps.

Keep an eye on the market

I track releases and licensing changes from major players, and I test public models as they mature. That tells me whether a technical migration is realistic.

A useful habit is keeping three candidate model providers on a short list and running quarterly tests on cost, latency, and output quality. It keeps switching costs visible before they become a problem.

Match the setup to the job

AI choices should fit what actually matters. I ask whether the model cuts cost, speeds up work, or adds revenue. If the answer is thin, I keep the service simple.

• Critical revenue function: prioritise resilience and multi-provider redundancy.
• Internal productivity: use hosted cloud services with low ops overhead.
• Experimental features: use short-lived cloud resources or isolated accounts.

Watch contracts and cost

Contracts and cost are part of the technical picture. I treat licence terms and revenue share as hard constraints.

• Ask for exit clauses that let you export data and models.
• Budget for dual-run periods where both old and new providers run in parallel.
• Set billing alerts. They are dull, but they stop nasty surprises.

If the legal terms are unclear, slow the rollout until they are not. Migration under pressure is expensive and messy.

Keep provider contact live

I keep lines open with providers: named contacts, regular technical calls, and clear reporting on usage. Good relationships buy time during disputes. They are not a substitute for an SLA, though.

Set up a runbook for provider incidents with named contacts, escalation steps, and a cutover plan.

Leave room for the next change

I keep the architecture layered so a new model format, hardware accelerator, or edge inference option can slot in without a full rewrite.

• Use a simple API gateway so downstream code does not call providers directly.
• Keep training data and model artefacts in a neutral, exportable format.
• Automate tests against new hardware or model versions in CI.

Whenever a new provider or format is added, run a full regression of latency, cost, and output quality. Document the differences and update the runbooks.

Final takeaways
Treat cloud infrastructure as replaceable parts. Inventory everything. Isolate AI model dependencies. Automate deployments and secrets. Test failovers and billing scenarios before they bite. Make contractual exit paths explicit. Keep an eye on market shifts and run regular migration rehearsals. That way the Microsoft-OpenAI split is a planning problem, not a panic.