Skip to content

Cloud · Microsoft

Azure East US control plane failures

Started
Apr 24, 2026, 11:30 UTC
Resolved
Apr 24, 2026, 23:22 UTC
Duration
11h 52m (from provider timestamps)
Affected
Azure Virtual Machines, Virtual Machine Scale Sets, Azure Kubernetes Service, Azure Virtual Desktop, Azure App Service, Azure Application Gateway, Azure Batch, Azure Cache for Redis, Azure Data Explorer, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Red Hat OpenShift, Azure Service Fabric, Azure VMware Solution, Oracle Database@Azure
Cause
Final reviewA recently deployed version of the Azure PubSub service, part of the networking control plane, carried a latent regression that caused lock contention and slowed replica builds in East US's large zones. Automatic and manual failover of the affected partition both failed, so provisioning, scaling and update operations failed. Staged rollbacks to the last known good version were slow, and impact moved across all three zones before it was mitigated.
Was it an attack?
Not stated
State
resolved

Anatomy

  1. 1. Trigger

    Code deployment

  2. 2. Already broken

    Latent bug, Failover failed

  3. 3. What made it spread

    Control-plane dependency

  4. 4. Blast radius

    single region: East US

  5. 5. Cause category

    Change or deployment

Labels follow the root-cause taxonomy. A person's change is recorded as the trigger and the missing safeguard as what was already broken; “human error” is never a cause here.

How the explanation changed

  1. Date not shown on the source · Final review

    “Our post incident investigations confirmed that a latent regression in a recently deployed version of the PubSub service exacerbated the replica build times for the large Availability Zones of this region. This regression also contributed to the failure of the failover process”

    Azure status history PIR, tracking ID 5GP8-W0G (served by the history page's results endpoint)

What the provider committed to change

  • Rolled back the PubSub update in other regions with large Availability Zones (completed)
  • Extend test coverage for the failure cases and load patterns involved (estimated June 2026)
  • Reduce partition blast radius by adjusting PubSub partition scale (estimated June 2026)
  • Enrich Network Resource Provider monitoring to auto-notify impacted subscriptions (estimated August 2026)
  • Ensure resource constraints do not delay rollback and recovery with co-located compute and data (estimated September 2026)
  • Improve AI-assisted incident communications (estimated September 2026)

Impact, as reported

Every figure is attributed to whoever stated it. Crowd reports are labelled user reports and never treated as users affected.

Post-incident reviews

Sources

  1. Azure status history PIR, tracking ID 5GP8-W0G (served by the history page's results endpoint) (provider, read November 17, 2026)
  2. Azure status history page (provider, read November 17, 2026)

Last verified November 17, 2026. Spot an error? The methodology page says how corrections work.