GCN LAB IMPRESSIONS

Do Comcast's Internet 'glitches' vindicate cloud skeptics?

Millions lost access in two disruptions that look suspiciously like hacks

A funny thing happened when I sat down to my weekly Lord of the Rings online game with friends last week – nothing. I was one of the millions of customers on the East Coast who got dropped by Comcast and could not get online until well after midnight, when most of my guild, being lazy hobbits, had already gone to bed.

I didn’t think too much of it until the same thing happened to millions of Comcast customers in Chicago one week later.

Comcast said the trouble was a glitch in its Domain Name System servers, though one has to wonder if hackers might be the real problem.

Most operations have backup DNS servers. Even the GCN lab has a backup DNS system, so one would suspect that Comcast, being in the business of Internet access, would also have backup. I doubt Comcast would admit if hackers were bringing down its networks across the country, but it’s not as if Comcast hasn’t been hacked before.

It just seems too suspicious that millions of people were knocked offline in different parts of the country just one week apart. That sounds more like an attack than a hiccup to me, especially since two outages together never happened before, at least not on such a large scale.

Certainly, Internet access is a target for hackers, who can cause quite a disruption. It’s one of the reasons I’m not as gung-ho about cloud computing as everyone else seems to be.

Federal, state and local governments seem to be tripping over themselves to get into the cloud; GCN has had many cloud application stories in recent years. But there is an inherent danger in this approach that the recent outages revealed. If all of your business and government files and programs exist “in the cloud,” what happens when Internet access goes down?

In the past, having problems with Internet access was annoying, but you could always work locally with your programs and files stored at your office. But with cloud computing, a link is being inserted into the chain between you and your data. And the recent attacks (okay, Comcast, let’s call them glitches – wink, wink) show that the chain, which is more of a lifeline, is far from secure. If you move vital applications and data to the could and your Internet access goes down, you’re sunk.

I’m not sure how many people panicked during the recent outages. On the East Coast, the outage happened on a Sunday night, so most business were closed. It was still enough to crash the Comcast help lines, which rang busy for hours. Had this happened during a work day, during working hours, it might have become a full-blown crisis.

The situation reminds me of one of the best episodes of "The Twilight Zone," “The Monsters Are Due on Maple Street,” in which seemingly normal people begin to go crazy when the electricity and modern conveniences, such as they were in 1960, were disrupted. The aliens didn’t have to do anything overtly against us. They flashed a couple lights and everyone turned on each other.

These days, we can plan for most disruptions, but not so much for Internet outages. Uninterrupted power supplies can protect homes and businesses from power outages. Backup hard drives can protect data in the event of a fire or flood. But few agencies at the federal level, and almost none at the state and local levels, have backup Internet access built into their buildings.

I doubt people would turn into shotgun-toting, stone-throwing mobs like they did on Maple Street in that “Twilight Zone” episode, but government would effectively be cut off from doing its job. And that’s why those links would make juicy targets for hackers: They are outside of the government’s control, sitting in private hands somewhere. That makes them less dangerous to attack, but no less debilitating to the government that relies on them.

I think the lesson is to not be so hasty moving into the cloud. Until you can guarantee the link between your agency and your data — and even if Comcast’s outages were glitches, the fact remains that millions of people lost access — you might be better off with local copies of important files and programs.

The cloud offers a lot of advantages, but only if you can safely get there. I suspect the monsters of Maple Street have already shown their presence in the recent outages. It might be good to make sure they don’t get invited over again, or that they can’t cause so much damage when they do come knocking.

 

Reader Comments

Mon, Dec 20, 2010 Mike D

I agree with Davis L, Comcast is first a media company that wants to provide content. Its secondary purpose is to provide the pipes to connect to that content. Connection to the internet as a whole is third at best and therefore low on the priority.Its been years since I have trusted the DNS servers of any ISP that I use and have deoeneded up open DNS. They provide a much better internet experience than any of the ISP do as access to the interent as a whole is always low on their list.

Mon, Dec 13, 2010

the government IT leadership is either corrupt, insane, or both -- if it is going to trust some business, corporation, etc to manage it's data using cloud computing. wikileaks has proven that the government has lost control of it's data. it has been compromised by contractors. the federal government should eliminate the majority of its contractors, and bring the IT units back into their organizations. reagan had alzheimers when he began his privitization crusade and only drove up the national debt and deficit paying much higher total cost of ownership. however, the biggest loss that continues to bleed badly is knowledge management and the government's "corporate knowledge"... all the smarts were given away starting in the 80's and continue on... the government will never operate effectively if they continue to make bad financial decisions are unable to audit the Defense Department (remember that goal? being able to do a financial audit of the DoD!).

Mon, Dec 13, 2010 Pat

Redundant links to the Internet are useful if the interruption is just due to a problem at one ISP, but what if the disruption is at the other end? If your hosting vendor is undergoing a DOS attack no amount of redundancy on your end will give you access to your cloud resources.

Mon, Dec 13, 2010 David L Washington, DC

"Comcast, being in the business of Internet access..." See, the problem with your argument is you are starting from a false premise. Comcast is not now, nor has it ever been in the business of providing Internet access. They are in the business of making money and providing content that they feel you want (whether or not you want it). Providing Internet access (in the form of allowing you to access the Internet over their pipes) is just a side benefit to you, the user. They don't care if you can't game, or access critical data. That is not their business model.

Mon, Dec 13, 2010 Mike Squires Bloomington, Indiana

I believe that the Comcast DNS problem was due to updates being applied to both the primary and backup DNS servers at the same time. Adding non-Comcast DNS servers to your list (in my case /etc/resolv.conf) resulted in the service being restored instantly. The only problem here is a management problem. I've started an OpenDNS subscription, so that won't happen again.

Show All Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above