As networking makes all types of data readily available to warfighters, technical planners are continuing to enhance techniques to make sure warfighters can find the files they need.
As networking makes all sorts of data readily available to warfighters, technical planners are continuing to enhance techniques to make sure warfighters can find the files they need. Standards for tagging and data management provide structure for searches, while improved storage techniques help make those searches faster.
The military’s imagery users are benefiting from commercial satellite technologies such as Google Earth and the movie industry’s need to file digital images so they can be accessed and reused. Military agencies are also developing their own techniques for data management. Using the best of both worlds brings benefits to warfighters.
“There are some really exciting things going on, especially in metadata tags for imagery,” said Jacqueline Knudson, data strategy lead on the technical director's staff at the Defense Information Systems Agency's Program Executive Office for Command and Control Capabilities. The National Geospatial-Intelligence Agency "has done a lot of exceptional work in terms of putting structure behind the ways data is handled and getting it implemented across the full infrastructure so warfighters can see real benefits.”
It’s no secret that the volume of data is soaring as the military deploys more satellites, unmanned aerial vehicles and land-based sensors. Military users are also making greater use of satellite images taken by commercial suppliers such as DigitalGlobe and GeoEye. That’s a rapidly growing market.
“Over the last five years, the market has grown steadily at 20 percent compounded annually, growing to a global total of $1.1 billion in 2009. U.S. defense and military organizations consume roughly two thirds of that,” said Adam Keith, director of earth observation at Euroconsult. Growth in that area will continue at nearly 20 percent over the next few years, he added.
The rising volume of imagery and other files has prompted some concern that warfighters will be overloaded with data or spend too much time searching for files. But observers agree that having access to a broad range of data in real time over networks is far better than waiting for aging images that were carried on DVDs or sent via slow, bandwidth-constrained links.
“Those issues still exist to a degree, but I think everyone would agree that too much data is a better problem than too little,” said Jack Hild, vice president of U.S. defense strategy at DigitalGlobe.
He said commercial tools augment standards created by industry and military planners. “Tools like Google Earth and Bing allow you to see what you have better than ever before and with temporal tools you can easily view back in time,” Hild said.
Finding common ground
Standards are one critical data management tool used to create an infrastructure that makes files easy to access. Many industry groups such as the Open Geospatial Consortium (OGC) and government-related standards bodies such as the Geospatial Intelligence Standards Working Group are writing a multitude of documents that ensure that data gathered by many sources can be shared across multiple networks.
Knudson said she reviews about 80 standards every quarter to make sure the goals of the committees and Defense Department users are closely aligned. She said using improved data identification standards are helping users find new ways to use files shared over networks.
“We started working seriously on tagging about 10 years ago, and we’re starting to see a tipping point now, where people are starting to put things together in ways it wasn’t designed to be used,” Knudson said. “People see the benefits of tagging and standards, and they start doing more with it.”
Commercial companies that provide a growing portion of DOD’s geospatial imagery are equally bullish on the role of standards. “Industry and governments have been working under OGC's leadership for years to implement interoperability standards, and now I don't think there is a credible tool or major dataset on the market that isn't OGC-compliant,” Hild said. “These folks are the unsung heroes in today's geospatial industry.”
Using common terms makes it possible to use images in many different ways. When everyone uses the same standards, military agencies can easily share their images with the Homeland Security Department or with international groups that respond to floods, hurricanes or other disasters.
“We want to go through and make sure that as information comes in, we disseminate it in a timely manner — that is, it has the accuracy requirements that our customer needs,” said Barry Barlow, director of acquisition at NGA. “And we send it out broadly to people whether they’re in the national community, the homeland community, the defense community. We want to make sure that the geoint gets out to the people that need it.”
Keep it down
Another strategic data management problem is keeping file sizes down. Smaller files take less space on disks and help improve search times and less bandwidth is needed when less data is sent over networks.
Engineers have developed a number of techniques to reduce size to improve storage and communication. For example, when maps change, only the modifications are updated.
“The way we’ve done our standards, when there are changes to a map, we only forward those changes, not the whole map,” Knudson said.
She said the biggest problem for mapping is not the size of the image. Instead, the issue is scaling the information so it’s relevant to the environment, where small screens such as tablet computers are seeing increased use.
As satellites and other data gatherers create more video, the value of this imagery often declines as it gets older. Unless there’s a need to keep every second of old video for comparison, it can be erased.
“I'm intrigued by what some refer to as elegant aging,” Hild said. “Unimportant video is systematically reduced over time, 30 frames per second is reduced to 15 frames per second after three months, halved again after three more months and so on until you've achieved orders of magnitude of reductions. Add data compression and you can dramatically decrease storage.”
He said avoiding redundancy is an important factor for still images. Warfighters don’t want to get search results that have 20 links to the same thing, and content providers don't want to store redundant images.
“We continue to look for storage improvements across our combined infrastructure," Hild said. "Simple process changes that eliminate storing images multiple times can go a long way to reduce storage requirements.”
Location is another changing aspect of storage. Technical strategists are developing techniques that ensure that data is stored in locations that make it readily accessible for warfighters.
“We’re looking at many things in storage, things like cloud computing and content staging,” Knudson said. “In places like Afghanistan, we want to forward stage the data that’s used most often.”
NEXT STORY: Windows Server 8 will have a tint of Azure