Army finds its own way to build a big data system
Connecting state and local government leaders
The Enterprise Management Decision Support System draws on 3,500 database systems to create a picture of military readiness.
The Army is harnessing big data to help commanders answer questions that were previously unsolvable by pulling relevant information from thousands of databases into a holistic common operating picture, specifically focused on readiness and resources.
The Army Enterprise Management Decision Support System reaches out across 3,500 different database systems, drawing data from the most relevant systems into one place, according to Lt. Colonel Bobby Saxon, division chief and program director with the Army EMDS Program Office.
So far this analysis is being accomplished without using some of the well-known big data tools such as the Apache Hadoop and Pig platforms, mainly because the Army EMDS Program Office’s environment did not fit into the typical big data scenario, Saxon said.
While a typical organization might have 40 terabytes of data that analysts need to gather to parse and understand, this was not Saxon’s scenario. Instead, the Army needed to put data into one place so senior leadership could make more informed decisions. “So our choices were built around that,” Saxon said, during a presentation Oct. 24 at an FCW Executive Briefing on big data analytics held in Washington, D.C.
A variety of tools have been deployed to connect the data and assemble it so that it can be accessed and viewed via dashboards, including Oracle’s WebLogic Server, identity management, business intelligence and Endeca Discovery software; Composite Software’s data virtualization technology (now a part of Cisco); the Apache Tomcat application server; and Internet Explorer 9 and Firefox Web browsers.
“As we have grown to understand our data and the ability to utilize big data, we have started to open up the door and look at more vendors,” Saxon said.
The Army’s need for real-time data is an ongoing challenge, but one that is complicated by the complexity of pulling data from multiple sources across the Army, Reserve and National Guard.
“We live in a world of near real-time most of the time,” Saxon said. Often the EMDS team cannot get access to all of the real-time data across disparate systems and will have to pull data together overnight from a lot of these systems. As a result, senior management is looking at yesterday’s picture of the Army.
Army Chief of Staff Gen. Raymond Odierno and former Vice Chief of Staff Gen. Lloyd Austin (now commander of U.S. Central Command) initially wanted EMDS to provide real-time information, until they understood how difficult it is to get that information. Real-time readiness information on 10,000-plus Army units would require someone at each unit to enter data into a computer every time something changed. That is not what the Army needs at this time, Saxon noted. What the president and chief of staff need to know is, if there is an incident today, what forces are ready, he said.
Through EMDS’ dashboards, senior management has a common picture of equipment, personnel, readiness and training activities. For instance, they’ll know if the 48th Brigade out of Georgia is about to deploy or the latest status of the 82nd Airborne. Prior to EMDS they would have had to pull that information from multiple sources on their own.
The Army is dealing with a growing trove of structured and unstructured data and Saxon’s team is trying to figure out how to get access to unstructured big data, which can be difficult because of the way security is set up in many systems. “We live in a world of PowerPoint,” he said, adding that much of that unstructured data has become authoritative information that needs to be in EMDS.
“We hope chaos is becoming order,” with the deployment of EMDS, Saxon said. The system can also provide trending information for predictive analysis. Odierno wants to predict the readiness of the Army two to 10 years down the road. “In order to do that, we have to be able to answer questions that were previously beyond our reach,” Saxon said.
NEXT STORY: NIH backs 3 projects to create medical co-robots