5 open-source technologies powering the Army's battlefield cloud
Connecting state and local government leaders
The tactical intelligence network in Afghanistan has essentially unlimited capacity.
Cloud computing has, without a doubt, forever changed military intelligence, according to Army and contractor executives working on the Distributed Common Ground System-Army Standard Cloud (DSC) in Afghanistan.
More on the Army's DCGS-A
How troops in Afghanistan get a clear view of intell
The Army’s DSC, the first tactical deployed cloud in a war zone, gives soldiers in Afghanistan full access to intelligence and fast response to queries. Read more.
“From a technology perspective, when you are dealing with data processing and data management and data fusion, having the idea where you can almost have almost unlimited processing power, unlimited memory, unlimited storage is a paradigm that software engineers have not had before,” said Shane Miller, a principal at Booz Allen Hamilton, which is the DCGS-A lead contractor.
The cloud computing approach furnishes the massive storage capacity for the upwards of 75 million records created since the beginning of Operation Enduring Freedom in Afghanistan and the incoming data from 600 source feeds. To build DSC, the Army relied heavily on open source platforms and software available in the commercial and government sectors, thereby avoiding costly licensing fees and keeping down development costs in a highly constrained budget environment.
DSC’s infrastructure consists of 228 servers, more than 1,800 CPUs, 100 dual six-core processors operating at 2.80 GHz and 128 dual quad-core processors running at 2.93 GHz, according to the Army. The infrastructure requires 13.92 terabytes of RAM and 1.032 TB of hard drive storage.
The Army, which served as the systems integrator for the project, owns not only open-framework platforms used on the project, but also a large portion of the software, Miller said.
For DSC, the Army tapped five key open-source technologies — Hadoop, Hadoop Core, Accumulo, Condor and SolrCloud.
“Hadoop is an open-source computing system that a lot of cloud technologists have been looking at,” said Col. Charles Wells, DCGS-A project manager.
Hadoop, from the Apache Software Foundation project, is frequently associated with big data for good reason. DSC developers employed Hadoop Distributed File System, which is an open-source version of Google File System, to give them fast and reliable analysis of both structured and unstructured data and to scale to the exabyte level.
The developers employed the Hadoop Core parallelization infrastructure for a comparable open-source version of the MapReduce programming model that parallelizes tasks across a large number of nodes.
The open-source tool Accumulo gave developers a robust, high-peformance data storage and retrieval system. Using Accumulo, they were able to leverage open-source technology for DSC for storing structured data at the petabyte level.
Condor is an open-source management infrastructure platform. The developers chose Condor because it offers a high-throughput computing environment and gave their cloud the elasticity it required.
DSC’s Internet-like search results capability, which gives answers to queries in a matter of seconds, was made possible by SolrCloud, a distributed index management capability capable of supporting high-speed searches.
Deploying a cloud computing environment using COTS hardware and open-source software platforms to drive the processing and storage power needed to make the Army’s intelligence cloud an effective, high-performance system is something about which the service is very proud, Wells said.
NEXT STORY: After Sandy hit, Coast Guard comm got ... better