Lightning not kind to Amazon, Microsoft cloud centers
- By Kurt Mackie
- Aug 09, 2011
A lightning strike temporarily knocked out cloud data centers in Europe run by Amazon and Microsoft over the weekend.
The lightning caused a power outage at one of Amazon's "availability zones" in Dublin, affecting Amazon's Elastic Compute Cloud (EC2) service in Europe. Amazon indicated in its publicly accessible service health dashboard that the event took place at 10:41 a.m. PDT on Aug. 7. The lightning sparked an explosion and fire that also affected backup power systems.
"The transient electric deviation caused by the explosion was large enough that it propagated to a portion of the phase control system that synchronizes the backup generator plant, disabling some of them," the Amazon team wrote at 3:01 p.m. PDT. "Power sources must be phase-synchronized before they can be brought online to load. Bringing these generators online required manual synchronization."
Amazon cloud crash keeps Energy site offline
Lesson from Amazon’s cloud crash: Plan for the worst
By late in the day on Aug. 8, Amazon had completed copying over data from affected snapshots on its EC2 platform in Dublin, allowing customers to run a recovery tool. The team apparently was still working on "restoring connectivity to affected Single-AZ database instances" for its Amazon Relational Database Service in Ireland, according to the health dashboard.
Meanwhile, Microsoft simply reported on Aug. 7, via Twitter, that a "Europe data center power issue affects access to BPOS," referring customers to their private service health dashboards for updates. That this power outage was caused by the same lightning strikes in Dublin that affected Amazon's data center was explained in a Data Center Knowledge story. Microsoft referred customers to private dashboards used by Business Productivity Online Service customers for further information. The publicly accessible Microsoft online service notifications page described nothing about the Dublin data center problems.
Microsoft apparently restored services somewhat quickly after the lightning strike in Dublin, or within the day on Sunday. The last Twitter post about the Dublin BPOS service outage stated that "BPOS services are back online for EMEA customers."
Microsoft's press release (PDF) on the Dublin facility describes it as "the largest data center outside the United States." It was opened in July of 2009, and uses natural cooling except for "a few days per year." The data center uses half the power of comparable facilities, according to the press release.