Cloud · Amazon Web Services
AWS us-east-1 outage from DynamoDB DNS automation race condition
- Started
- Oct 20, 2025, 06:48 UTC
- Resolved
- Oct 20, 2025, 21:20 UTC
- Duration
- 14h 32m (from provider timestamps)
- Affected
- Amazon DynamoDB, Amazon EC2, Network Load Balancer, AWS Lambda, Amazon ECS, Amazon EKS, AWS Fargate, Amazon Connect, AWS Support Center, other services dependent on DynamoDB
- Cause
- Final reviewA latent race condition between two DNS Enactors in DynamoDB's automated DNS management system left an incorrect empty DNS record for the regional DynamoDB endpoint. Systems that depended on DynamoDB, including EC2's droplet lease manager, failed. After DNS was restored, EC2 lease recovery collapsed under its backlog, network state propagation lagged, and NLB health check failures caused further connection errors.
- Was it an attack?
- No, per the provider or investigators
- State
- resolved
Anatomy
1. Trigger
Maintenance automation
2. Already broken
Race condition, Latent bug
3. What made it spread
DNS dependency, Control-plane dependency, Health-check cascade, Recovery backlog
4. Blast radius
single region: us-east-1 (N. Virginia)
5. Cause category
Software or automation
Labels follow the root-cause taxonomy. A person's change is recorded as the trigger and the missing safeguard as what was already broken; “human error” is never a cause here.
How the explanation changed
Date not shown on the source · Final review
“The root cause of this issue was a latent race condition in the DynamoDB DNS management system that resulted in an incorrect empty DNS record for the service’s regional endpoint (dynamodb.us-east-1.amazonaws.com) that the automation failed to repair.”
AWS: Summary of the Amazon DynamoDB Service Disruption in the Northern Virginia (US-EAST-1) Region
What the provider committed to change
- DynamoDB DNS Planner and DNS Enactor automation disabled worldwide
- Fix the race condition and add protections against applying incorrect DNS plans before re-enabling
- NLB velocity control limiting capacity a single NLB can remove on AZ failover
- Additional EC2 test suite exercising the DWFM recovery workflow
- EC2 data propagation throttling based on queue size
Post-incident reviews
Other Amazon Web Services incidents
Sources
- AWS: Summary of the Amazon DynamoDB Service Disruption in the Northern Virginia (US-EAST-1) Region (provider, read November 17, 2026)
Last verified November 17, 2026. Spot an error? The methodology page says how corrections work.