Internaut | Managing a flood of new data

Government information technology managers are constantly inundated with new types of data arriving from an ever-increasing number of sources. It's their job to figure what's worth keeping
from each data stream, how to store it, how to access it and how to make the data available to a wide variety of applications.

One type of information already having an impact is data generated by various civil engineering projects. That includes information from road and bridge sensors, water level sensors,
smart lighting controls for buildings or public spaces, and even citywide networks of traffic controls, highway signs and monitors
along fences and borders.

To understand that growing data flow and associated issues, let’s start with the sensors themselves. They are often
transducers. A transducer typically measures energy produced by pressure, movement or heat then converts that energy into something
else, such as an electrical impulse that can be recorded as data. New types of sensors implanted in bridges can measure the movement
of girders or plates, metal corrosion, and other types of wear. A local system usually collects sensor impulses and converts that
information into a specific type of data that can then be sent to a computer.

Those systems produce a variety of data types, many of which are proprietary. But a standard called Transducer Markup Language is
becoming increasingly common. It can be used to create a type of XML document that describes data produced by a transducer in a
standardized way and includes metadata that describes the system producing the data, the date and time it was collected, and how
multiple devices relate to one another within a network or via Global Positioning System coordinates.

Marport Canada Inc.’s SmartBridge is one of the leading systems for that type of data collection.

Of the nation's nearly 500,000 bridges, the Federal Highway Administration cataloged 25.8 percent as structurally deficient or
functionally obsolete as of 2006. That doesn't mean they are heading for collapse, but it does mean they need monitoring.
Traditionally, that has meant periodic visual inspections. But as the 2007 collapse of the I-35W Mississippi River bridge in
Minneapolis showed, visual inspections might not be enough.

The replacement bridge built in Minneapolis contained hundreds of special sensors, many cast right into the concrete. The
University Of Minnesota and the Minnesota Department of Transportation monitor the data those sensors collect.

Realizing that there will be a growing demand for such systems, researchers at Clarkson University have developed a prototype
bridge sensor that doesn't need a battery. It powers itself via the vibrations of a typical bridge, similar to those flashlights that
you charge by cranking or shaking.

Right now, transportation-related sensors are leading the way. But other agencies will soon notice their own flood of sensor data.
The Agriculture Department will see more data from crop and livestock sensors. The Energy Department will see more information
on energy consumption and how weather and cost affect it.
Meanwhile, the Homeland Security Department is already dealing with data from border sensors and video surveillance systems.

Government data-center and network managers can prepare for the flood of sensor data by asking the following questions when systems
are installed:

• Who is ultimately responsible for the sensor network? Who will
  maintain it, and who is responsible for troubleshooting it if the
  flow of data is interrupted?
  
• How will the collected data be moved from the remote site to a
  receiving facility? Via a government network? University network?
  Leased lines? If so, leased by whom?
  
• Will the sensor data simply pass through a network on its way
  to a specific end user (researcher, highway manager, etc.)? Will
  that end user be solely responsible for collecting and storing the
  data, or will government IT managers also be responsible for data
  collection, database development, backups and more?
  
• Is there a service-level agreement associated with the data
  collection, perhaps covering how often data will be updated, how
  long it will be stored and how accurately the information will be
  represented?
  
• Is any sort of data conversion necessary for multiple
  applications to access and integrate the data?
  

Finally, civil engineering projects are producing more than just
sensor data these days. As design technologies and site mapping
improve, they create new data files for the government to manage,
some of which are quite large.

In addition to computer-aided design files, site plans,
elevation information and environmental impact reports, a solution
called building information modeling creates 3-D datasets that
allow users to navigation through all the data associated with a
large engineering project, including the management of a
facility’s entire life cycle.

Clearly, civil engineering projects will continue to spawn a new
flood of data types and system requirements. Being prepared and
asking the right questions are essential for anyone who is expected
to handle that flood.