FAA identifies computer error that caused delays

Flight delays persist despite resolution

A software problem that caused flight delays across the country today was not the first serious hiccup Federal Aviation Administration computer systems have suffered in recent years.

FAA officials said the problem had been resolved by mid-morning, after about four hours. Flight delays persisted in the afternoon, however. The glitch made it impossible for airlines to enter flight plan information into the National Airspace Data Interchange Network, according to the agency. FAA personnel entered the information manually, and the additional time needed for that led to delays.

However, the NADIN system was not the ultimate system at fault, said Paul Takemoto, an FAA spokesman. That was the Federal Telecommunications Infrastructure, which is the Internet protocol used for non-safety critical data transmissions. Tammy Jones, an FAA spokeswoman, said the problem is being attributed to a software configuration problem with a router at the Salt Lake City facility.

Takemoto said the FTI system is deployed nationwide in air traffic control facilities. FAA has had FTI for several years and hasn't had previous problems with it, he said.

However, FAA systems in general have been troublesome recently. An incident in August 2008 delayed flights around the country. FAA at the time said that was caused by a "database mismatch" that interfered with data transmission.

FAA officials said that today's problem didn’t affect radar coverage or communications with flights in progress, but according to the Wall Street Journal, the air-traffic controllers union said the FAA systems that provide information on weather and wind speeds at airports weren’t functioning.

According to the Government Accountability Office, the FAA’s systems are overburdened by too much demand. Even in the current recession, the air transportation system is straining to meet all the demands, leading to delays or cancellations in one of five airline flights from January to October 2009, according to an Oct. 28 report from GAO. The problems are expected to worsen as the economy recovers and travel increases.

To improve its capacity and efficiency while maintaining safety, the FAA is moving to the Next Generation Air Transportation System, but that won't be complete until 2025. The improvements include integrated systems and procedures, aircraft performance capabilities, satellite-based surveillance and navigation and network-centric operations. Some of the changes will be implemented between 2012 and 2018, and some stakeholders have urged even quicker deployments.

The NextGen Midterm Implementation Task Force issued a report in September that dealt with several areas, including runway access, congestion relief,  data communications and integrated air-traffic management.

However, the FAA faces several challenges in carrying out the NextGen objectives, including adjusting its culture and business practices, concluded the GAO. The main change is from a focus on system acquisition to an emphasis on integration and coordination, the report said.

While today's problem appeared to be a software issue and not an attack, it did provoke speculation. Vulnerability to cyber hackers has been a growing concern at the FAA, according to the Transportation Department’s Office of Inspector General. As the air traffic control system has become modernized with the use of Internet Protocols and commercial software, it is considered more at risk of hackers because of vulnerabilities in that protocol and software.

However, one expert who studies the motives and consequences of hacking and cyberattacks said the chances that today's incident was an attack are small.

“You have to look at how the attacker could gain” from the attack, said Scott Borg, chief economist for the U.S. Cyber Consequences Unit, an independent research institute. “The ones we watch for and that trigger alarms right away are the ones that someone can gain from.”

Criminals most commonly are seeking financial gain, ideologues seek to score points to sway public opinion, and militants and nation states would be interested in disrupting critical infrastructure and processes. The delay of airline flights because flight plans have to be manually entered is more of an inconvenience than a serious disruption of the nation’s air traffic system.

“This doesn’t accomplish any of those things,” Borg said. “It’s not going to cause airplanes to crash. It’s not going to cause big economic effects. You are not going to extort money out of the FAA. There is little for anyone to gain from it.”

In a May 4, 2009 audit, the inspector general’s staff was able to gain unauthorized access to the FAA’s systems. The inspector general concluded in the report that the FAA is not fully secured against unauthorized access and does not have effective intrusion-detection systems.

“In our opinion, unless effective action is taken quickly, it is likely to be a matter of when, not if, air-traffic control systems encounter attacks that do serious harm to air traffic control operations,” Rebecca Leng, assistant inspector general for financial and information technology audits, wrote in the May 4 report.

She cited the FAA’s intrusion detection system as inadequate, because it was deployed to only 11 of the hundreds of air-traffic control facilities, and said cyber incidents that were detected were not corrected in a timely manner.

“By exploiting these vulnerabilities, the public could gain unauthorized access to information stored on Web application computers,” the May 4 report said. “Further, through these vulnerabilities, internal FAA users (employees, contractors, industry partners, etc.) could gain unauthorized access to Air Traffic Control systems because the Web applications often act as front-end interfaces (providing front-door access) to ATC systems. In addition, these vulnerabilities could allow attackers to compromise FAA user computers by injecting malicious code onto the computers.”

During fiscal 2008, more than 800 cyber-incident alerts were issued to the Air Traffic Organization, which is responsible for Air Traffic Control operations.

According to the inspector general’s report, recent serious attacks have included:

  • In February 2009, hackers entered an FAA public-facing Web application computer and used it as to gain unauthorized access to personally identifiable information on 48,000 current and former FAA employees;
  • In 2008, hackers took control of FAA’s critical network servers, giving them the power to shut down the servers and cause disruptions to the network; and
  • In 2006, a viral Internet attack shut down a portion of the Air Traffic Control systems in Alaska.

Reader Comments

Tue, Nov 24, 2009

Finally....It was hard sitting back and watching all the mainstream news media go on and on about how there was an issue with the FAA's computer systems, then to have all the people in Congress that needed a little publicity jump on the bandwagon. And finally, an artical comes out that really describes what happened, a contractor owned and operated Cisco 8850 router located in the Saltlake City NADIN facility took down the entire IP network that the FAA uses to exchange flight plans between facilities. Having to sit back and listen to the "blame game" from those that know nothing about how the FAA's systems are built is a hard pill to swallow, especially if you know what the issues really are. A big THANK YOU should be given to all the FAA personnel that minimized the effects of this outage by manually entering flight plans.

Fri, Nov 20, 2009

It appears FAA air traffic control data has been acceptably classified as non-safety critical. (Where is the outcry regarding this classification?) It seems the focus should be the pursuit of a data classification that mandates use of a transmission protocol that simultaneously incorporates a minimum level of safeguards against such occurrences. The best an individual could expect would be ONLY a simular occurence of the referenced February 2009 hack or the referenced 2008 hack without securing a data classification change and implementing the associated safeguards inherent with the change in classification.

Fri, Nov 20, 2009

"However, the NADIN system was not the ultimate system at fault, said Paul Takemoto, an FAA spokesman. That was the Federal Telecommunications Infrastructure, ...". No, no, no, of course not. That and "computer error". That chain of blame isn't going to stop until they go to the chip fabricating plant in Malaysia that made the resistors for the router, and nail the janitor.

Fri, Nov 20, 2009 Buddy Somewhere in the USA

Sounds like a Government Citrix configuration issue! Since at least one huge Government Activity insist most things now got to be access REMOTELY for some reason the regular DESKTOP not good enough! (and yet I have no idea what the issue was)

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above