Cloud · Microsoft
Azure West US loses connectivity to the WAN after an over-scoped break-fix repair
- Started
- Jul 23, 2026, 14:44 UTC
- Resolved
- Jul 23, 2026, 19:41 UTC
- Duration
- 4h 57m (from provider timestamps)
- Affected
- App Service, Application Insights, Azure AD B2C, Azure AI Bot Service, Azure AI Search, Azure API Management, Azure Application Gateway, Azure Bastion, Azure Cosmos DB, Azure Data Explorer, Azure Database for PostgreSQL, Azure Databricks, Azure ExpressRoute, Azure Firewall, Azure Kubernetes Service, Azure Monitor, Azure Speech in Foundry Tools, Azure Virtual Desktop, Azure Virtual WAN, Azure VMware Solution, Azure VPN Gateway, Microsoft Sentinel, Power BI Embedded, Microsoft 365 (downstream, MO1437424)
- Cause
- Final reviewAn automatically processed break-fix repair on one optical device was expanded by a defect in Microsoft's blast radius analysis system to cover every optical device leaving one West US datacenter. The safety check validated each device on its own rather than the combined effect, so routes were withdrawn from all of them and traffic into and out of the region was cut. Automated rollback depended on the same lost connectivity and failed, and misleading WAN signals delayed correlation until a manual rollback.
- Was it an attack?
- Not stated
- State
- resolved
Anatomy
1. Trigger
Maintenance automation
2. Already broken
Latent bug, Missing validation
3. What made it spread
Control-plane dependency
4. Blast radius
single region: West US
5. Cause category
Software or automation
Labels follow the root-cause taxonomy. A person's change is recorded as the trigger and the missing safeguard as what was already broken; “human error” is never a cause here.
How the explanation changed
July 24, 2026 · Preliminary cause
“a bug in the request conversion system incorrectly marked additional network devices as part of the maintenance event”
BleepingComputer, Microsoft blames massive Microsoft 365 outage on maintenance bug (2026-07-24)
Date not shown on the source · Final review
“A defect in our blast radius analysis system incorrectly expanded the scope of the repair event to include all optical devices egressing a specific datacenter. The safety validation step, which is designed to confirm that at least one of the two redundant datacenter paths remains available, ran but incorrectly concluded the operation was safe.”
What the provider committed to change
- Fixed the blast radius analysis defect; input validation blocks break-fix on both diverse paths of a datacenter at once (completed)
- Scanned existing break-fix requests for similar conditions (completed)
- Safety checks block any maintenance affecting all diverse paths of a datacenter (completed)
- Record break-fix requests in the standard Azure change ledger (est. August 2026)
- Faster detection of abnormal datacenter inbound and outbound traffic (est. September 2026)
- Change tooling guardrails so multiple devices cannot be isolated at once (est. September 2026)
- Automated recovery fails faster and escalates sooner (est. October 2026)
- Automated rollback that works when its own connectivity is degraded (est. October 2026)
Post-incident reviews
Other Microsoft incidents
- Microsoft 365 global outage: latent networking gateway defect cascades across the fleet (EX1550543 / MO1550821), November 12, 2026
- Azure East US control plane failures, April 24, 2026
- Azure West US power event, February 7, 2026
- Azure Front Door and Azure CDN global connectivity and DNS outage, October 29, 2025
Sources
- Azure PIR ZJV6-SGG, Issues accessing resources in West US (provider, read November 17, 2026)
- BleepingComputer, Microsoft blames massive Microsoft 365 outage on maintenance bug (2026-07-24) (news, read November 17, 2026)
- Petri, West US Azure outage was caused by network maintenance error (2026-07-27) (news, read November 17, 2026)
- Network World, Microsoft explains why its West US Azure and cloud services failed (news, read November 17, 2026)
Last verified November 17, 2026. Spot an error? The methodology page says how corrections work.