Data sharing in the public sector: authorized, secure and in real time
- By Ravi Shankar
- Jul 02, 2019
On a global level, governments are instituting data-sharing initiatives to gain more value through enhanced collaboration while maintaining the highest possible security and access control.
To improve data-sharing processes, IT managers must understand and respect the potentially differing needs and restrictions among agencies sending and receiving data. For example, a local motor vehicle department office might need to access FBI files to run a background check on a driver’s license applicant. However, the FBI might have restrictions on what the DMV can do with the data -- it might not be able to store the requested data or only for a limited amount of time. In other cases, certain datasets may not be available at all.
Depending on the methods used for transferring data, an agency might not receive data in its desired format, in which case transformations would be necessary. Such transformations add an additional step to the process which, in turn, inhibits the agency's ability to use the information immediately. In addition to different formats, requested data may reside in multiple different files, which will require the receiving agency to manually integrate the files in order to gain a unified view of the information.
Enter data virtualization
To overcome these data integration challenges, agencies are turning to data virtualization. Rather than facilitating the transfer of data from one agency to another, or the integration of data from numerous different files collected from different agencies, data virtualization provides secure, authorized views into different datasets, leaving the data in its original location. This means that there is no need for the receiving agency to physically store any more information than is strictly necessary.
Data virtualization acts as a layer between agencies, facilitating real-time, authorized data access. As such, the data virtualization layer itself holds no data; instead, it contains the meta-data required for accessing the available data, including the appropriate controls for which individual or group is authorized to view which dataset.
In building each requested data view, data virtualization takes care of all of the necessary transformations on the fly, automatically “translating” the data for the receiving agency. For the recipient, this means that regardless of how the source data is formatted, the received data is available in wide variety of formats. Beyond the basic SQL used by most databases, such formats include web services or XML over SOAP, formats optimized for delivering information to applications, dashboards or websites.
Data virtualization seamlessly blends views of data from multiple sources, including those from one or multiple agencies and presents them to the receiving agency as though they are one unified view drawn from a single source. This is powerful on its own, but data virtualization goes one step further: On the receiving agency side, stakeholders can easily use the data virtualization layer to explore the origins of each of the individual components of the data, including full lineage, relationships and associations. These features are critical in providing for real-time data access and the ability to track each element to its source and owner.
Data virtualization acts as a central data access layer between agencies, providing search engine-like capabilities, which means agencies no longer have to hunt down, request, wait for and transform data. Rather, they simply have to search for it.
More importantly, all security and access protocols can be controlled in the data virtualization layer simultaneously. This ability means that agencies can establish a powerful and extremely granular authorization layer specifying which group, role or individual is authorized to view what data, down to the table, the row and even the specific cell. Fields that cannot be viewed by a particular individual can be removed or masked at the agency’s discretion.
Beyond data sharing, there are two other ways in which data virtualization can benefit agencies:
- Logical data warehousing. Using data virtualization, agencies can facilitate the gathering of large amounts of historical data for analysis about their activities, so that they can improve their processes over time. Just as a data virtualization layer can be established between agencies, a data virtualization layer can also sit above one agency’s multiple data sources, including one or more data warehouses. In doing so, the agencies can establish a logical data warehouse, which is achieved when data is connected logically rather than stored physically together.
- Data services. Data virtualization can publish data views as data services, which plug directly into web or mobile applications. This enables agencies to build data-rich, citizen-facing applications with much less effort and technical know-how than is normally required.
The sharing economy
The agility, security and data abstraction features inherent in data virtualization make it a logical choice for many government agencies. As authorized, secure, real-time data sharing is soon to become imperative, data virtualization can better connect agencies in a simple, immediate and powerful way, while ensuring proper governance and security measures remain intact.
Ravi Shankar is senior vice president and chief marketing officer at Denodo.