Skip to content

Cloud · Microsoft

Azure West US loses connectivity to the WAN after an over-scoped break-fix repair

Started
Jul 23, 2026, 14:44 UTC
Resolved
Jul 23, 2026, 19:41 UTC
Duration
4h 57m (from provider timestamps)
Affected
App Service, Application Insights, Azure AD B2C, Azure AI Bot Service, Azure AI Search, Azure API Management, Azure Application Gateway, Azure Bastion, Azure Cosmos DB, Azure Data Explorer, Azure Database for PostgreSQL, Azure Databricks, Azure ExpressRoute, Azure Firewall, Azure Kubernetes Service, Azure Monitor, Azure Speech in Foundry Tools, Azure Virtual Desktop, Azure Virtual WAN, Azure VMware Solution, Azure VPN Gateway, Microsoft Sentinel, Power BI Embedded, Microsoft 365 (downstream, MO1437424)
Cause
Final reviewAn automatically processed break-fix repair on one optical device was expanded by a defect in Microsoft's blast radius analysis system to cover every optical device leaving one West US datacenter. The safety check validated each device on its own rather than the combined effect, so routes were withdrawn from all of them and traffic into and out of the region was cut. Automated rollback depended on the same lost connectivity and failed, and misleading WAN signals delayed correlation until a manual rollback.
Was it an attack?
Not stated
State
resolved

Anatomy

  1. 1. Trigger

    Maintenance automation

  2. 2. Already broken

    Latent bug, Missing validation

  3. 3. What made it spread

    Control-plane dependency

  4. 4. Blast radius

    single region: West US

  5. 5. Cause category

    Software or automation

Labels follow the root-cause taxonomy. A person's change is recorded as the trigger and the missing safeguard as what was already broken; “human error” is never a cause here.

How the explanation changed

  1. July 24, 2026 · Preliminary cause

    “a bug in the request conversion system incorrectly marked additional network devices as part of the maintenance event”

    BleepingComputer, Microsoft blames massive Microsoft 365 outage on maintenance bug (2026-07-24)

  2. Date not shown on the source · Final review

    “A defect in our blast radius analysis system incorrectly expanded the scope of the repair event to include all optical devices egressing a specific datacenter. The safety validation step, which is designed to confirm that at least one of the two redundant datacenter paths remains available, ran but incorrectly concluded the operation was safe.”

    Azure PIR ZJV6-SGG, Issues accessing resources in West US

What the provider committed to change

  • Fixed the blast radius analysis defect; input validation blocks break-fix on both diverse paths of a datacenter at once (completed)
  • Scanned existing break-fix requests for similar conditions (completed)
  • Safety checks block any maintenance affecting all diverse paths of a datacenter (completed)
  • Record break-fix requests in the standard Azure change ledger (est. August 2026)
  • Faster detection of abnormal datacenter inbound and outbound traffic (est. September 2026)
  • Change tooling guardrails so multiple devices cannot be isolated at once (est. September 2026)
  • Automated recovery fails faster and escalates sooner (est. October 2026)
  • Automated rollback that works when its own connectivity is degraded (est. October 2026)

Post-incident reviews

Sources

  1. Azure PIR ZJV6-SGG, Issues accessing resources in West US (provider, read November 17, 2026)
  2. BleepingComputer, Microsoft blames massive Microsoft 365 outage on maintenance bug (2026-07-24) (news, read November 17, 2026)
  3. Petri, West US Azure outage was caused by network maintenance error (2026-07-27) (news, read November 17, 2026)
  4. Network World, Microsoft explains why its West US Azure and cloud services failed (news, read November 17, 2026)

Last verified November 17, 2026. Spot an error? The methodology page says how corrections work.