How one Texas town built IT resilience
- By Benny Yazdanpanahi
- Oct 04, 2019
For cities and towns across America, recent ransomware attacks are a stark wakeup call signaling the importance of mitigating risk, strengthening data protection and bolstering IT resilience.
IT disruptions can be caused by man-made events such as ransomware attacks, unplanned events such as natural disasters or unforeseen consequences of new technologies, services, changes or errors. Studies suggest technology-related disruptions are quite common. A recent IDC survey found that 91% of senior-level IT professionals have experienced tech-related disruptions in the past two years.
These disruptions can have an especially negative impact on cities where IT enables mission-critical public-safety services that protect thousands of citizens in emergencies and ensure the water is clean, electricity is working and roads are safe.
For Tyler, Texas, our IT disruption was caused by an error we had with a software driver. The glitch knocked out critical IT applications for several days and temporarily interrupted public-safety services. The seriousness of the event prompted us to immediately reevaluate our business practices and rethink how to better protect the city’s data and applications to ensure citizens' safety.
We determined we had to strengthen and modernize our capabilities to ensure critical operations are continuously available for the city’s public-safety and public-works organizations, which more than 100,000 residents rely upon every day.
Previously, we used hypervisor technology and had conducted nightly backups across our three data centers. But after the software glitch, we realized this approach simply wasn’t going to be enough to protect the city’s infrastructure going forward. We set out to modernize our disaster recovery technology and processes and ensure our systems have the highest resiliency and availability possible.
After evaluating different approaches, we implemented solutions from VMware and Zerto for scalability, ease of use and reliability.
With this combined approach, we protect 300 virtual machines (VMs), with 50 of those running critical applications that support emergency medical services, 911 and the police and fire departments. The technology allows us to keep data running and replicated across multiple locations and ensures we can recover data quickly and easily. It also enables us to keep systems available to support the city’s public safety and public works in the event of any type of disaster -- whether it’s a natural, technology-related or man-made.
With the modernized solution, our IT team can now quickly spin up or repair VMs and replicate and restore data and applications within seconds of a failure or event. And as an added bonus, it simplifies preparation needed for annual security audits by the Criminal Justice Information Systems. With the improved scalability, speed and reliability, the new approach has created more efficiencies for us, making it possible to innovate with secure multiclouds for city services and expand our IT services to nearby towns.
With the confidence we’ve gained from stronger IT resilience, our team can focus on better serving the citizens and employees of Tyler. And after 18 years of keeping the city’s IT up and running, I can think about the next chapter and focus on our organization’s goals over the next five to 10 years.
Key technology considerations
Other city and state organizations that want to advance and strengthen their IT resilience strategies in the way that we have should carefully evaluate technology offerings.
Modern disaster recovery solutions that involve continuous data protection technology continuously capture and protect data. This capability can significantly move a local government organization beyond the limitations of today’s legacy backup -- which currently leaves major gaps between snapshots -- and offers the ability to recover all data from just seconds before systems were disrupted. As cloud strategies mature, workload mobility capabilities and multicloud agility also allow data and applications to be moved between on-premises and any cloud for optimized recovery.
After all, just as city populations grow, IT organizations must evolve and grow with them. Remaining resilient with modern, cloud-based disaster recovery technologies will position city and state agencies well for future opportunities and unexpected disruptions.
Benny Yazdanpanahi is the CIO, City of Tyler, Texas.