analytics (Wright Studio/


3 data strategies to help crackdown on internal corruption

In late 2017, the public’s trust in the Massachusetts State Police was rocked when an internal audit revealed discrepancies between overtime earnings paid and actual hours worked for several state troopers. Another hit came in August of that year when payroll records showed a number of troopers for the Massachusetts Environmental Police took overtime assignments and off-duty details in the middle of the workday and scheduled their normal state work around more profitable side gigs. The discovery led to two guilty pleas of embezzlement and dozens of investigations and prosecutions.

These recent scandals are evidence of the struggle government organizations have had in managing and accurately reporting basic human resources data like employee hours, payroll and travel information. More importantly, they shed light on a troubling reality for administrations everywhere: Without proper data intelligence strategies in place, a data analytics oversight could open the door for internal issues even among the most respected institutions. 

Governmental agencies must implement intuitive data intelligence strategies, so users (regardless of skill level or industry) can collect information accurately, access and share it easily and crunch numbers efficiently. Only then can the foundation for accountability truly be built.

Below are three data intelligence tips and best practices for setting the stage for a transparent workforce.

Step #1: Ensure data accuracy and reliability with data preparation and lineage tracking

Trust is a key element for any reliable report. When working with numbers, employees must be confident that the information came from a trustworthy source and has been prepped correctly. Without this assurance, agencies don’t have the necessary support for proper reporting. To achieve accuracy, employers should look for data intelligence solutions that can extract information from unstructured formats, like PDFs. This will help cut down not only on the time and labor spent on tedious tasks like manual data entry, but it will also reduce errors stemming from human input.

Once data is extracted, agencies should add an extra layer of security by using a platform that tracks data lineage, showing  where the information originated, who manipulated it and how it was altered. By having a comprehensive history, users won’t need to blindly trust the data, they’ll be equipped with the information to know that it is suitable. More importantly, if analysts spot an irregularity, they’ll have the tools to know who entered it and the evidence to show how it was altered. In this way, data extraction and lineage tracking features not only provide support for dependable information, but also offer the means for positive change. 

Step #2: Create a culture of collaboration through data sharing and socialization

Hidden information has the potential to be false information. When organizations fail to develop data intelligence strategies that make information accessible and socialization easy, they unknowingly open the door to errors stemming from lack of review and supervision.

Rather than encouraging employees to harbor and edit numbers on their private desktops, employers should consider a cloud-based data analytics platform with a data marketplace or warehouse that allows users to store and share information easily. In addition to the simple point of access, the platform should come with a means to collaborate on datasets through socialization features commonly found on social media sites, such as liking, commenting and sharing. Intuitive features like these will give organizations visibility into datasets and build accountability into the process as a result.

Additionally, employers may find that a culture of data sharing and socialization may have the added benefit of enabling operational efficiency and empowering a shared vision of overall organizational goals. When all analysts access a data marketplace that provides the context and means of communication and can see how their actions contribute to the “big picture,” employers may find that workers will be more likely to think outside their department and be more willing to provide much needed insight and perspective for cross-departmental collaboration.

Step #3: Enable data literacy within the entire organization

More data eyes mean more supervision. To develop a truth-driven agency, employers should look for ways to help their workers -- regardless of skill or department -- contribute to the entire agency's wealth of data knowledge. One of the ways this can be achieved is to invest in a data analytics platform that has machine learning features built into the software.

Innovations in machine learning are drastically shortening the data analytics learning curve by providing suggested actions based on organizational behavior. For example, if the software sees reoccurring actions done to a dataset, it will recommend the same action in similar cases. Now analytics knowledge no longer has to be isolated to few data scientists, but can be open to everyone. This level of cultural data literacy will not only provide widespread awareness on how accurate info can benefit an entity, but it will also enable the culture of collaboration and productivity that most groups are trying to achieve.

Respected institutions are facing scandals of internal corruption stemming from lack of trustworthiness, reliability and collaboration. But with some of the newest innovations in data analytics, government agencies can do more than waiting for the negative press: They can be proactive in cracking down on corruption.

One of the ways they can start is by revising their data intelligence strategies so that they ensure accuracy, create a culture of accountability through collaboration and enable data literacy. New features like data prep, lineage tracking, data marketplaces, data socialization and machine learning recommendations will not only help lay the groundwork for much needed visibility and accountability in the analytics process, but also empower agencies with the tools for positive change.

About the Author

Jon Pilkington is chief product officer at Datawatch.


  • automated processes (Nikolay Klimenko/

    How the Army’s DORA bot cuts manual work for contracting professionals

    Thanks to robotic process automation, the time it takes Army contracting professionals to determine whether prospective vendors should receive a contract has been cut from an hour to just five minutes.

  • Russia prying into state, local networks

    A Russian state-sponsored advanced persistent threat actor targeting state, local, territorial and tribal government networks exfiltrated data from at least two victims.

Stay Connected