War damage to data centers exceeded what AWS services are designed to withstand.
Half a year after Iranian drone strikes knocked out multiple Amazon data centers, the US tech company has acknowledged the permanent loss of some customer data that was hosted in Bahrain and the United Arab Emirates.
Amazon Web Services was “unable to restore access to the resources and data” hosted in some of the war-damaged data centers, according to an AWS dashboard update posted on September 15. The development was first reported by Reuters and suggests that the Iranian strikes inflicted catastrophic damage on the data centers.
Customer data was irretrievably lost in one of three AWS availability zones in the United Arab Emirates region, specifically the mec1-az2 availability zone. Each availability zone is serviced by one or more Amazon data centers.
Fellow SRE here.
The number of times I’ve heard “we have replication” followed by “we’ve tested it…on paper” make me cry.
Trying to convince a CEO that the only way to know for sure that their backups work is to do an actual disaster test is harder than trying to convince my parents that I don’t need to eat more.