4 principles of simplified data management
Connecting state and local government leaders
It may seem that government database managers must collect large amounts of data and store it for many years, but this process is inefficient, unnecessary and results in escalating costs and headaches.
Confucius wrote, “Life is really simple, but we insist on making it complicated,” a sentiment that could be applied to the way government agencies manage data. For while it may seem a fait accompli that database managers must collect large amounts of data and store it for many years, this process is actually terribly inefficient, unnecessary and results in escalating costs and headaches.
If you’re a government database administrator, embracing these four principles will help alleviate your pain and simplify your processes.
Not everything is worth collecting
There’s an infinite amount of data that can be collected, but that doesn’t mean it should be.
The key is to gain a deep understanding of what’s important for your agency at this particular time so you’re only collecting what’s necessary. Ask youself, “What are our current priorities, and what type of data do we need to collect to support those priorities?” Other types of information may not be as critical to current needs and can be set aside. This division of data eliminates a lot of white noise and makes data processing more efficient.
There is no decision making technology for this; rather, it requires insight into the overall goals of the agency. You must become just as proficient at interacting with other teams, including those who set the overall direction for the agency, as you are with managing data.
Finally, it’s also helpful to remember not to get too bogged down in intricate calculations when it comes to data analysis. Often simple algebra will help you sort and filter data correctly. There’s no need to panic; you’ve got this.
Alerts and notifications: Choose wisely
The rule of simplicity also applies when setting up automated alert software. While it might be tempting to cover all bases by setting up alerts for every event, that practice can lead to alert overload. Soon, you’ll be like the townspeople in the story of the boy who cried wolf – ignoring the real threat when it eventually comes knocking at your door.
It’s better to set up a simpler system consisting of both alerts and notifications. Understand that there are distinct differences between the two. Alerts let you know of something that needs to be taken care of immediately – a potential security breach, for example. Notifications are less timely. They might inform you of performance issues impacting response time, which is not something to be ignored, but not an emergency either. This process gets you the information you need, when you need it, without overwhelming you.
Show appropriate respect for older data
There’s no hard and fast rule about how much data to keep or how long to keep it – and that poses problems. Because while many have likely heard the unwritten “7 year rule,” which dictates that all data should be kept for that period of time, it’s also likely that most agencies keep data for much longer than that, even if they’re not required to.
Instead of archiving, database administrators should consider using a tiered storage approach. Tiered solutions help enhance efficiencies by prioritizing data, separating new or more important data from that which is older or less useful. For example, tier 1 storage could be used to house newer data that might need to be more accessible or retrieved more quickly. Simultaneously, less valuable or older data should be placed in tier 2 or tier 3 storage sets, which are generally slower, so the data will take more time to retrieve.
Again, it’s all about simplification – separating what you need from what you do not, in the process making your database efforts run more efficiently.
Know what you don’t know
External audits are never fun, but they can be eye opening and critically important, especially for managers who are drowning in data.
Every database manager should consider hiring outside data analysts to provide an unbiased report of the agency’s data management practices. Many of these analysts are architects and modelers; they live knee-deep in data and can provide a good overview of the types of data a federal manager needs to collect, whether or not it already exists and what to do with it. They’re the kings of data simplification, and their expertise can be invaluable.
Along with the other aforementioned principles, they can help focus your data management efforts – and simplify your life in a way that would make Confucius proud.