17 million cyberattacks a day: How Oklahoma stops 99% of them

Oklahoma’s government networks come under attack from cybercriminals about 17 million times a day, creating a lot of potential headaches for the state’s IT security staff.

But Oklahoma is in a far better position to handle those threats than it was four years ago when the COVID-19 pandemic forced most Americans, including the state’s 30,000-plus workforce, to work from home 

Suddenly, the state’s networks were more vulnerable than ever as those 30,000 employees signed on to work remotely each day using a VPN, a connection between a computer and a remote server that can be exploited by hackers.

Remote work also taxed the state’s aging tech infrastructure, prompting leaders to start a path to modernization. They embraced a zero-trust security framework, which grants users and devices access only to parts of the network that are essential to their task. The approach is designed to protect users, applications, infrastructure and data, whether systems reside in an agency data center, in the cloud or in a hybrid environment.

Today, when an employee signs on, they encounter more robust multi-factor authentication, underpinned by new security software that has been rolled out to more than 100 of the state’s 180 agencies. Consolidating under one security approach has saved the state $875 million.

It is all part of an effort that has been underway for more than a decade to modernize the state’s legacy infrastructure. Chief Information Security Officer Michael Toland recently sat down with Route Fifty to discuss that modernization effort.

This interview has been edited for clarity and brevity.

Route Fifty: Take me back and give me a bit of a history lesson, if you would, about what the IT infrastructure in the state was like before you guys started this modernization journey.

Michael Toland: The state had the same problem everybody had when COVID hit, which is, “How do we send people home and keep them working?” A lot of the common threads were classic VPN solutions that simply couldn't carry the number of connections and had difficulties getting application experiences that were good enough [for employees] to actually be effective in the job…. Before, you might start up your computer, log into the VPN client, or in the case of certain VPN clients, maybe start it two or three times [before] you've got your access. Now you’re remotely into a system where you open an application and the experience is very hit or miss. You might authenticate three or four times before you do a task, so you've got high security friction. You might get kicked off in the middle of a transaction and have to start all over again. It wasn't working, and so we chose to use Zscaler. We put the process of authenticating to the trusted network right there at the initial Windows login.

Route Fifty: What does this new regime look like?

Toland: The experience now is you open up your state-owned computer that has all the agents on it, and when you log in, you're logged into the computer and the network, you can immediately access the apps with the same user experience anywhere in the world. The only variable I can't control is the bandwidth available to you wherever you're sitting. But we even have a solution for that: We have monitoring in place so that if a person calls me, [we] can take a look and say, 'Okay, your processor’s good, memory’s good, your bandwidth is low and that's where the latency is coming in.’ We have the ability to know what's going on there. 

We're not there yet—there are certain applications that are not yet protected. There are legacy applications that simply do not lend themselves to zero-trust architecture. As they are being phased out and their replacements are being written, we just get better and better.

 "I don't want to say COVID was good, but there was a silver lining to COVID," said Toland. "When you have no choice but to change, some of the arguments around optionality go away." 

Route Fifty: Was it hard to transition to this kind of new zero-trust architecture away from VPNs and the old way you did it? How did you go about transitioning your employees?

Toland: I don't want to say COVID was good, but there was a silver lining to COVID, as there was no choice. The old method simply didn't work. It's always hard to change things. But I feel like it actually made it a little bit easier. When you have no choice but to change, some of the arguments around optionality go away. Three years before or three years after, it might have been a little harder, it might have been more difficult to sell people on the idea, but I feel like it worked out well, just based on the timing and the need to move. What people saw was a win right upfront, and that good experience was enough that the little bit of friction that goes into the zero-trust model wasn't felt as strongly as it might have otherwise been.

Route Fifty: Those 17 million attacks each day on your networks sound scary too.

Toland: It is a terrifying number. However, with the security platforms in place, [more than] 99% of them never make it to the eyes of an engineer. We stop them at the edge, or we know about an IP address because of threat intelligence feeds from this data, so we just block them at the firewall. We still track that because it's worth knowing. 

The state is a big target. There's a lot of data here. We have data that falls under just about every regulatory framework that you care to mention. From a cybercriminal’s point of view, this is a pretty tempting target. If you want something to steal, odds are we have it.

Route Fifty: You mentioned that some applications maybe haven't lent themselves to this transition to zero trust. What kind of things? Are you transitioning legacy stuff and that sort of thing? What's the status of all that?

Toland: What is modern today will be legacy tomorrow, so it's a never-ending churn. It's the same things that everybody has: old mainframe applications that are 40 years old but still work. Is it really worth $3 million or whatever it's going to cost to write a new one, implement it, train everybody up and go through the lows of the new system blues as you find and fix workflow or application problems? Quite frankly, in a lot of cases, the answer to that is no. It's old, but it works. 

Other things where we feel a little bit more pain are applications written on technologies that are no longer supported. There are some database-driven apps where the databases won't support a more modern version of Oracle or SQL or whatever. Those are hurting us a little bit more because the vulnerabilities are stacking up and they're not being patched, so we're attacking those first. 

We try to triage in order of severity. It's the same problem everybody in our business has. There's tech debt, there always will be, it needs to be modernized, but there's only so many dollars to go around. We just attack them as best we can. What we've chosen to do is when we identify an application that is particularly vulnerable, but it's not yet in process for modernization, we just find technologies to put around it. We do our virtual moats, put up our virtual walls and do our best to protect those systems so they don't end up being our next security event.

"What is modern today will be legacy tomorrow," Toland said, "so it's a never-ending churn."

Route Fifty: Is it hard to build a moat or wall around a 40-year-old application?

Toland: Everything is hard to one extent or another. But there's a lot of technology out there, there's a lot of good people out there that know how to do some pretty creative and cool things. I'm very privileged to work with a lot of them. We use additional firewalls, load balancers, we put the system behind another layer of protection to make it just that much harder to get to it, and we monitor them more closely. It's not perfect. Nothing ever is. At the end of the day, it all comes down to a risk versus reward discussion. There's always a certain amount of risk that you inherently have to accept.

Route Fifty: What effects are you seeing from the zero-trust architecture that you have in place, say in terms of employee experience?

Toland: I think it's improved the employee experience. It does add a bit of security friction, where you can't just stay logged into an application forever, you have to reauthorize periodically. That can be frustrating for people, and I empathize because I growl at my keyboard when I have to do it. That's a fact of life, it affects everybody and I'm looking forward to the days when the technology matures even beyond that, which I obviously feel it will. 

Going passwordless will help as more technologies are able to support better authentication methods, so you don't necessarily have to remember 17 passwords or whatever the number is these days. By and large, I think it's actually improved the experience, because people are able to just work and maybe not worry so much about getting kicked off a VPN. The platform we're using that is facilitating zero trust has eliminated that. While we have added a little bit of security friction, I feel like we've taken more away, and so it's been a net benefit to our users.