AI gears (Zapp2Photo/


How AI and machine learning can drive government efficiency

Government digital transformation is about the application of IT to drive better outcomes, from improved public health to more effective cyber defense strategies. It includes the move to web-based and mobile technologies, the use of open standards and open source software and the adoption of cloud architecture principles such as elasticity, easily provisioned storage and computing -- all in the context of enhanced data security. However, a quickly emerging part of the digital transformation strategy for national security and public safety will be the application of advanced mathematics to reduce the cost, resources, time and imprecision of current methods of information processing and exploitation.

Data science, artificial intelligence and machine learning

For a few years, U.S. cabinet departments, the intelligence community and government scientific institutions have been hiring data scientists -- people with a combination of skills in statistics, information science and varying degrees of programming. Data scientists apply a range of statistical and other mathematical techniques to business- and mission-critical problems. They fill positions that in previous decades were designated for business intelligence and reporting experts, operations research/information science professionals and analysts.

Although the job can be rewarding, in reality, data scientists spend upwards of 80 percent of their time manipulating and aggregating data from different systems just to get it into a form that can be used with the advanced tools they have developed. There’s just too much money going into the ‘plumbing’ rather than the mission. Integrating these data silos and getting the data into shape so that data scientists can work with representative slices to create, test and field algorithms remains a significant impediment to using artificial intelligence and machine learning to drive transformation.

AI speaks to getting machines, built on computers, to think. Machine learning and a related field known as deep learning are methods of getting machines to teach themselves to think. But before we reach the objective of truly sentient machines, AI can help with any number of manual, mundane, detailed, discrete and deceptively complicated  processes.

AI and machine learning applications can reduce the human-factors burden for the government across national security and public safety, policy making, financial services, entitlements and benefits or infrastructure by:

  • Automating detection, tipping and cueing of patterns and anomalies and determining whether they are threats or opportunities.
  • Classifying, labeling and/or tagging entities and events as well as discovering the non-obvious relationships between them.
  • Aiding in decision-making and operations.

Today, the action is around the mathematicians and data scientists who are creating the algorithms and frameworks and the new infrastructure to support them. However, just as in the past when computers were used only by specialists who could code machine language, today nearly anyone with a phone or laptop is part of the transition to AI and machine learning.

AI and machine learning represent incredible opportunities to enrich the operational data on which government runs. This transformation signals a number of implications for government:

  • AI and machine learning will be at the center of initiatives to modernize agency systems and business processes, driving the need to break down data silos.
  • The human-computer interaction will drive enhanced feedback, more-granular data access controls and greater security, creating a closed loop for continuous improvement of algorithms and techniques.

  • Design, development, testing and application of algorithms must be done in a ‘fail fast’ and iterative experimental environment while not triggering complete ‘rip and replace’ of legacy systems.

  • Agencies will need a database management system, such as an enterprise data hub, that can take advantage of new technologies. Systems should reduce data wrangling time and complexity so the insights derived from AI and machine learning can be easily operationalized.

AI and machine learning could be the most effective way that government can thread the needle between ensuring that costs are contained while meeting quickly evolving threats.

About the Author

Billy Sokol is the CTO, Global Public Sector, at MarkLogic.


  • Records management: Look beyond the NARA mandates

    Pandemic tests electronic records management

    Between the rush enable more virtual collaboration, stalled digitization of archived records and managing records that reside in datasets, records management executives are sorting through new challenges.

  • boy learning at home (Travelpixs/

    Tucson’s community wireless bridges the digital divide

    The city built cell sites at government-owned facilities such as fire departments and libraries that were already connected to Tucson’s existing fiber backbone.

Stay Connected