Million-message mailboxes complicate EPA's move to Office 365

The sheer size of legacy mailboxes at the Environmental Protection Agency — some of them with more than a million messages in them — threatened the agency’s transition to Office 365, Microsoft’s cloud-based collaboration and communication service.

EPA officials discovered that hitch as they worked with Lockheed Martin and Microsoft to move 25,000 employee mailboxes to Microsoft Office 365 for Government, a multitenant service that stores U.S. government data in a segregated community cloud and includes e-mail, calendars, scheduling and collaboration tools for internal and external use.

“EPA wanted an extremely aggressive migration schedule,” said Lynn Singleton, director of environment services for Lockheed Martin. “I don’t know if they understood what they were really asking for when it came to the amount of work that had to occur over the relatively short amount of time,” Singleton said.

EPA officials brought in Lockheed Martin in September 2012 and wanted the primary migration to be completed by early 2013. Migration work was completed over the Presidents’ Day holiday weekend in February, Singleton said. The move to Office 365 is expected to save the EPA approximately $12 million over the four-year contract period.

EPA managers had done a lot of preliminary work to prepare for the migration, performing outreach and communication efforts to make the employees aware of the transition and dealing with governance issues. But an unexpected hurdle was the size of many of the mailboxes that needed to be transitioned, which  forced EPA and Lockheed to rethink how to approach the migration midway through the project.

For 15 years, EPA employees had used IBM Lotus Notes as an enterprise e-mail system. The agency didn’t have a size limit on mailboxes to trigger purging or archiving of information. Some of the larger mailboxes had over a million objects in them. In all, the mailboxes contained a very large universe of information, representing about 90 terabytes of data, Singleton said.

Initially, EPA wanted to transition all of the mailboxes along with their information to the cloud. “But the throughput requirements of that, just the physical constraints, were such that we could not do it,” Singleton said. If the team proceeded as planned there would have been network latency problems.  Plus, the team determined that they could not handle the quantity of information with the migration tool they had selected.
“So we did a number of things to try to get our arms around the magnitude of the migration,” Singleton said.

The Lockheed team broke up the mailboxes into size categories — small, medium, large and enormous — and began to clean them out and archive material. There were about 700 to 1,200 “enormous” mailboxes, and some mailboxes could not be migrated because they were corrupted due to large attachments or other issues. So the team weeded out mailboxes with large attachments and archived legacy information that was older than a year.

In addition to e-mail, the migration involved other mail-related components such as the movement of calendars, rooms and resources, groups, mail and databases as well as about 4,500 BlackBerry cell phones. Each of these components had to be treated as a separate entity by the migration tool, Singleton said. The team used a migration tool from Quest Software, which is now a part of Dell.

“You want to be sure that your calendar is up to date, so calendaring is another big issue.” Moving e-mail groups adds more complexity to the equation because of “nesting,” or groups within groups.  “So we had 16,800 groups to migrate,” Singleton said. 
“We did a lot of piloting,” he said. EPA’s Office of Environmental Information agreed to serve as a test case, giving Lockheed the opportunity to move small groups of users in a test environment.

The testing prepared Lockheed to begin what Singleton calls “differential migration,” in which the IT team took a large number of mailboxes — initially around 12,000 — and migrated them into the multitenant cloud with 15 days of mail. The team then moved the remaining mailboxes with 15 days of e-mail without turning the service on. This allowed the team to check the mailboxes for problems before the big migration over the Presidents’ Day weekend.

This approach cut down on the quantity of mail that had to be migrated and the “pre-touching of things made the main migration more predictable,” Singleton said. “That was responsible in large part for our success.”

For agencies with aggressive schedules to transition e-mail to the cloud, Singleton advises managers to be aware that the actual migration might be far more complex than originally envisioned. Do not view the move to cloud computing as just a commodity bid, he said. Take into account that there are related business practices, engineering and technical work and processes that must be addressed for a successful migration. Often, agency managers don’t realize the amount of work involved in getting ready for the transition to the cloud, he noted.