Hotmail, indeed: Overheated servers take down Microsoft's e-mail service
Connecting state and local government leaders
A firmware upgrade at Microsoft's data center gave servers a fever, resulting in an outage for Hotmail and Outlook.com users that lasted, in some cases, up to 16 hours.
The importance of keeping data centers cool has been getting some attention lately, particularly in light of some innovative new ways of keeping servers running under optimal temperatures.
Microsoft this week experienced what can happen things get too hot. Users of its Hotmail and new Outlook.com e-mail services, along with its SkyDrive file-hosting service, were out of luck for 16 hours March 12-13 during a service disruption the company blamed on a hot data center.
The outage hit at 4:35 p.m. EDT March 12 after the company performed a routine firmware update in its data center facility, according to a blog post by Microsoft Vice President Arthur de Haan. Although the update had been done before without a hitch, de Haan wrote, it “failed in this specific instance in an unexpected way.”
The result: “a rapid and substantial temperature spike in the data center,” he wrote.
It got hot enough to trigger the company’s safeguards, which prevent access to mail boxes and deter any automatic failovers, for a large number of servers in that part of the company’s data center, which houses the infrastructure for Hotmail, Outlook.com and Sky Drive, he said. A data center team got to work on the problem, gradually restoring service, but full restoration took until 8:43 a.m. EDT March 13. De Haan said restoration, atypically, required human intervention in addition to infrastructure software, which is why it took so long.
The outage prompted a flurry of activity on Twitter, naturally, with some tweeters putting the blame at the company’s feet.
Microsoft actually has been one of the companies developing new ways of running cooler, more energy-efficient data centers, building a new outdoor data center in Virginia. It’s hard to say whether even new cooling techniques would have made a difference in this case, however, since it appears the firmware update gave the servers something of a fever.
The company is in the process of switching its Hotmail users to the new Outlook.com, which has a cleaner interface and more social media integration. The full switch will come this summer, though users can move to Outlook.com now.
Hotmail, which Microsoft bought in 1997, was the pioneering Web e-mail system and dominated the market for years, but steadily lost ground and was eventually overtaken in the United States by Gmail and Yahoo mail. (Hotmail is second to Gmail worldwide, and third behind Yahoo and Gmail among U.S. users.)
The move to Outlook.com, which of course shares the name of the dedicated e-mail client so many people use, could boost the company’s Web mail prospects, but Hotmail’s travails were not lost on Twitter users during the outage.
In his post, de Haan said the restoration has given the company an understanding of why the crash happened and the team was working to ensure it doesn’t happen again.