Big data takes on disaster coordination

Big data takes on disaster coordination

Big data analysis has become a useful tool in tackling complex problems that until recently might have been all but unsolvable.  Coordinating disaster response, for example, is a multi-level effort that runs from the federal government all the way down to the local level. 

When Hurricane Sandy slammed into the East Coast in 2012, decisions on to how to mitigate the more than $68 billion in damages and nearly 300 lives lost were affected by the specific geography, infrastructure and populations with which various emergency response organizations worked.

“The ability to quickly identify and tap the experience of mission partners through a digital library would have been invaluable during Hurricane Sandy,” said Cherrie Black, former bureau chief of infrastructure protection at the New Jersey Office of Homeland Security and Preparedness. Likewise, access to risk analysis and lessons learned from events with similar characterizes and similar scope would have helped, she said.

But there was just too much information to sift through: 100,000 facility records, 20,000 documents, almost 50,000 images and 5,000 web pages.

That sounded like a big data problem to some of the creative minds at the Idaho National Laboratory,   who worked to normalize the data sources into what  became known as the Infrastructure Protection Digital Library.

That searchable resource gives government leaders – at all levels access to emergency planning and risk management information that can be used to improve emergency planning, but it faced some challenges along the way.  Users can filter results by geographic area, type of emergency and even security clearance.

One of the bigger problems dealt with categorizing information without a human interface and integrating access controls based on information sensitivity.  Developers turned to dynamic clustering to solve this issue, which applied user permissions based on content and metadata.  Eliminating pre-defined schema and enabling content to grow organically fixed the issue of incorporating information from several sources into a single unified view of structured and unstructured content.

The Infrastructure Protection Digital Library serves as a invaluable tool for emergency coordinators and planners, providing a solution that had been previously unavailable.  INL expects to expand the application to include predictability as threats, facilities and demands grow across all levels of government.   

About the Author

Mark Pomerleau is a former editorial fellow with GCN and Defense Systems.


  • Records management: Look beyond the NARA mandates

    Pandemic tests electronic records management

    Between the rush enable more virtual collaboration, stalled digitization of archived records and managing records that reside in datasets, records management executives are sorting through new challenges.

  • boy learning at home (Travelpixs/

    Tucson’s community wireless bridges the digital divide

    The city built cell sites at government-owned facilities such as fire departments and libraries that were already connected to Tucson’s existing fiber backbone.

Stay Connected