A cautionary tale of IT and OT dependencies
Connecting state and local government leaders
An inspector general’s report identified problems at NASA that are no doubt familiar to IT managers at large organizations working keep IT and operational technology systems secure.
In a modern-day cautionary tale, NASA discovered that even small changes to security software can cause big problems for operational controls technology and even endanger critical equipment and agency missions.
According to a Feb. 8 report from the space agency's inspector general, a security patch that shut down monitoring equipment in a large NASA engineering oven resulted in a fire that destroyed spacecraft hardware inside it. Because the computer reboot to accommodate the software upgrade also crippled fire alarm activation, the fire in the oven wasn't discovered for three and a half hours.
When vulnerability scanning was used to identify software flaws, physical equipment failed, resulting in a communications blackout with an Earth science spacecraft during an orbital pass. As a result, the pass was rendered unusable and data could not be collected until the next pass.
Yet another incident involved a computer problem in a climate system in which a safety feature was disabled, causing the heat in a data center to jump 50 degrees in minutes, resulting in a shutdown of the center.
Like many large enterprises, NASA has been automating many of its isolated, manually controlled technologies in favor of more sophisticated and interconnected IT equipment. But the agency's approach to integrating cyber, IT and physical systems is still a work in progress, and gaps in standards, training and security best practices must be remediated, according to the OIG.
Knowing which IT systems incorporate OT components is especially critical for NASA, it said, because applying traditional IT security practices to OT systems can cause underlying systems to malfunction.
According to the OIG, 65 percent of the agency's critical infrastructure -- including environmental monitoring and control systems for heating, cooling, ventilation, power, rocket propulsion testing systems, spacecraft and aircraft command and control systems -- are managed and supported by OT, or hybrid OT/IT systems, yet NASA has not developed a centralized inventory of OT systems or established a standard protocol to protect systems that contain OT components.
The report's list of NASA's OT shortcomings is probably familiar to IT managers at large organizations working to get a handle on the increasing number of internet-connected, or interconnected, devices and systems that were previously manually operated.
In reply comments, NASA outlined plans to inventory areas of technological interdependency and to come up with practices to guard against problems like those described in the IG report. It also said it will define and segment operational technology and industrial control systems from agency IT.
NASA expects improvements to be in place by Oct. 1, 2018.
A longer version of this story was first posted to FCW, a sister site to GCN.