Should your legacy data move with you to the cloud?
Connecting state and local government leaders
By analyzing data before moving to the cloud, agencies can not only reduce their cloud storage costs but also enable a leaner, more agile organization.
There’s something about computers that makes it easy to hang onto every version of every document and every email ever sent or received. However, there are a few changes agencies can make that force users to revisit their data management practices. Moving data to the cloud and leveraging a software-as-a-service model is one of them.
Following the federal adoption of the cloud-first policy and as part of the 25 Point Implementation Plan to Reform Federal Information Technology Management, more federal organizations are moving to the cloud. Before moving, however, executives must make some important decisions about migrating their agency's data, considering storage and transfer costs, data integrity and security, as well as Freedom of Information Act requirements and other policies and regulations.
Here are a few questions to help agencies decide how to handle their legacy data when moving to the cloud:
What data do I have and why do I have it?
Before migrating data to the cloud, agencies must establish a process requiring users to review and delete data that’s no longer needed. Whether it’s structured or unstructured data, it’s taking up space. Organizations should consider how often the data is used, how it’s secured, what regulations influence how it’s stored and whether it’s vital for business operations.
Legacy data can present additional challenges. As agencies move to the cloud, in many cases the software they’ll use is either a newer version of a legacy application or a different application altogether, which means the organization must either modernize the existing legacy application or map data from the legacy system to a new application in the cloud. This analysis can be intensive, but it is necessary to help leadership determine the value of the legacy data and whether it’s best to update and migrate or leave the data on-premise.
Without procedures to rationalize legacy data, the amount of storage consumed grows exponentially over time. For agencies with their own data centers, that’s a costly practice; and for those moving to the cloud, a lack of data management policies can prove wildly expensive.
What am I required to keep and for how long?
Some internal factors may influence data retention requirements, but the primary federal requirements originate from the FOIA. To remain compliant, it’s critical that federal agencies effectively manage their data.
For some files, that might mean destroying them after one year; for others, it might mean transferring them to the National Archives and Records Administration after 30 years. When in doubt about what to keep and for how long, it’s best to reference the administrative instructions for records and information management, such as the AI-15.
FOIA requirements raise an important question, however: Is data in a cloud environment subject to a FOIA request? According to NARA, federal agencies are responsible for managing their records in accordance with NARA statues including the Federal Records Act (44 U.S.C. Chapters 21, 29, 31,33) and NARA regulations (36 CFR Chapter XII Subchapter B), regardless of the cloud service and deployment models adopted.
To assist federal agencies with the challenges of moving to the cloud while also remaining compliant with regulations, NARA has developed several recommendations and even a clause that can be tailored to fit the planned type of service and specific agency records management needs.
Somewhat ironically, many agencies are moving their FOIA systems to the cloud, including the Department of Housing and Urban Development, which resulted in improved system performance and a reduced backlog.
How will the legacy data be used once it’s migrated?
Before transporting data from on-premise data centers to the cloud and paying to keep it there, it’s important to understand how the data will be used. In scenarios where the data isn’t directly feeding applications or processes or being consumed by the organization’s user base directly, it may not need to go to the cloud at all. Conversely, data that is used regularly, such as the data used in customer relationship management tools, should be migrated to be used by a SaaS alternative tool.
How will data be moved to the cloud?
The major differences between transfer options are security, convenience and cost. Most cloud service providers offer a cloud storage transfer service and a variety of transfer applications to match an agency's existing data size and network speed. Pricing models generally include egress and operation charges, insertion, transfer and early deletion charges, among others. For agencies with smaller data sets, these models may work well. Although less convenient and subject to more risk, when faced with really large amounts of data or network bandwidth limitations, organizations also have the option of physically moving the data. In either case, extensive testing is required to ensure the data is transferred successfully. Securing the data during the transfer is the biggest challenge.
How should I enforce data management policies?
Organizations with well-established and dutifully enforced data management policies and practices may find migration to the cloud simpler and more straightforward than organizations without good data management practices. After migration, following data hygiene best practices such as limiting mailbox size, restricting folder creation on shared drives and limiting file types on shared drives (e.g., banning .jpg or .png files) can help organizations maintain data targets.
Moving to the cloud is complex for a number of reasons; legacy data being one of the biggest. Developing a plan to effectively handle an agency's legacy data is critical for a successful transition from on-premise databases to the cloud. Thoughtful and strategic planning and ample testing are necessary to ensure that the agency remains operational and secure, risks are mitigated and users can continue to execute the mission with minimal disruption.
As long as it’s part of the original migration plan, it’s okay to operate both on-premise and in the cloud on a temporary basis. Operating in a dual state for a prolonged period of time, however, will quickly prove to be more costly, and make it tougher to declare cloud success. By analyzing data before moving to the cloud, agencies can not only reduce their cloud storage costs but also enable a leaner, more agile organization.