The Data Deluge: Understanding How to Visualize and Manage Data in the Transportation Industry

When you hear that the Federal Aviation Administration (FAA) tracks every single flight, airport, control tower, and runway in the country, you may not realize that air traffic personnel are monitoring about 5,000 flights and 2.5 million passengers at about any given moment. And starting in the beginning of August, FAA starts its preparations for the enormous rush and biggest travel season of the year -- Thanksgiving. FAA puts a lot of data behind analyzing how to add more flights and passengers, all while maintaining world's safest airspace system.

Flight data must be combined with weather, schedule, operations, and air traffic data from several other sources in an incredibly complex conversion, extraction, and integration effort to help support this effort. 

Behind this sophisticated and carefully orchestrated effort to assemble required data, lies a significant amount of work put in by computer scientists, database professionals, data scientists, and subject matter experts. Such work includes development of complex algorithms, calculations, lookups, visualizations, and other technical elements that help deliver actionable insights to the FAA leadership, operations management, and the airlines.

It is also imperative that the analysts work with the FAA to determine critical variables and information that must be extracted from the raw data and visualize them for easy interpretation. It is also important to know that having data does not mean it is ready for analysis or visualization. 

Ankush Karnik is one of the analysts who makes sense of large data sets by bridging gap between computer scientists and operational analysts. A project manager at Volanno, a woman-owned small business consulting firm based in Washington DC, he counts over fifteen years of experience in the ground transportation industry (at New Jersey Transit), finance (at Citibank) and has leveraged his skills in different industry sectors and is now working closely with the FAA to help derive actionable insights from aviation data. 

Ankush delivered a talk at St. John’s University last Monday, November 19, 2018 based on his recent publication in The Journal of Air Traffic Control (Summer 2018) that won 2^ndplace publishers award from the Air Traffic Controllers Association (ATCA). 

In his presentation, Ankush recognized that while data is everywhere and in abundance, it must be synthesized, mined, and analyzed carefully to be converted into action. More importantly, he explained that “It is also imperative that the analysis results be presented in a succinct, readable, and interpretable format, thereby allowing users to be more inquiring and enabling them to delve into advanced analyses and reasoning.”

He cautioned that while self-service data analyses platforms continue to proliferate, it is important to adhere to the best practices and fundamentals for preparing, organizing, and analyzing data. Unprepared data frequently hinders analyses and provides uninterpretable results. 

Attended by more than sixty students and faculty in the Master of Data Mining and Predictive Analytics and the Master in Homeland Security and Criminal Justice Leadership, the talk discussed data analytics in practice, using live use-cases from the aviation industry, focusing on leading edge predictive and prescriptive analytics, as well as online and offline analytics.

The one key lesson: just because you can easily aggregate data, blend data, or create complex visualizations using available tools, it does not mean that you should. Ankush reminded us that sound data analytics starts with hypothesis and question formulation. Dumping everything in a data warehouse and expecting to automatically get insights is still a far-fetched, and a dangerous dream.