Sherlock detects air traffic in near-real time
- By Matt Leonard
- Aug 18, 2016
As a plane takes off from Charlotte, N.C., the local air traffic control system monitors it until handing off to the air traffic controllers along the flight’s next leg. But now, thanks to upgrades in its Sherlock Air Traffic Management Data Warehouse, NASA will be able to monitor the entire flight in near-real time so it can more quickly analyze the entire U.S. air traffic system.
Until recently, NASA researchers and its partners could only see data from 77 individual FAA air traffic facilities, which made tracking a flight across the county difficult and slow. For example, the data from the air traffic facility at the Charlotte airport would show the surrounding 60-70 miles, but data for the remainder of the flight would be in a different file, generated by the next control center to monitor the flight.
The fragmented data also made it difficult for air traffic management researchers to quickly perform large-scale analyses of the U.S. air traffic system, to find where improvements will have the most benefit in terms of environment, safety and efficiency.
With algorithms provided by the ATAC Corporation of Santa Clara, Calif., researchers now get all the info in one place. Michelle Eshow, NASA’s lead for real-time air traffic management automation software development for the Aviation Systems Division, said ATAC’s algorithms are able to combine the data from the 77 FAA facilities into one nationwide, near-real-time file on air traffic data.
“Believe it or not, that’s a really hard thing to do and something NASA had not done before,” Eshow said.
NASA receives the data from FAA via a secure connection that the two agencies have had for years, Eshow said. They are working on getting the data in XML format, but for now it is being sent in different forms of binary, which is decoded by software when it reaches NASA. The software then sends it on to a Linux file system.
The front end of the Sherlock system is built on a variety of open source and commercial products. GeoServer is an open source tool that is used to visualize geospatial data. Oracle’s Apex handles basic search and download functions within the database. The heavy lifting of big data management is handled by Hadoop.
Sherlock contains over 36 terabytes of raw data, with more collected daily. The merged flight data will soon be available for all of the archived files going back to 2011, which will allow researchers to study many billions of flight tracks to better identify trends in the national traffic picture -- which in turn will help predict future traffic.
Editor's note: This article was changed Aug. 18 to correct the source of the algorithms. It's the ATAC Corporation of Santa Clara, Calif.
Matt Leonard is a reporter/producer at GCN.
Before joining GCN, Leonard worked as a local reporter for The Smithfield Times in southeastern Virginia. In his time there he wrote about town council meetings, local crime and what to do if a beaver dam floods your back yard. Over the last few years, he has spent time at The Commonwealth Times, The Denver Post and WTVR-CBS 6. He is a graduate of Virginia Commonwealth University, where he received the faculty award for print and online journalism.
Leonard can be contacted at firstname.lastname@example.org or follow him on Twitter @Matt_Lnrd.
Click here for previous articles by Leonard.