R Fink | The Packet Rat: Data snafu in Alaska leads to a paper chase
packet rat illustration by Michael J. Bechetti
The Rat has had a lot of bad days lately'an infected tooth, a root canal, a stomach virus and the flu, all one after another. Talk about March Madness. But there's at least one guy to whom the whimpering whiskered one can concede the title of Worst IT Day Ever. And that guy works for the State of Alaska's Department of Revenue.
Maybe you missed the story of the hapless technician who accidentally erased the accounting data for the $38 billion Permanent Fund Dividend, Alaska's oil-funded handout to its residents'by reformatting the hard drive the database was on. And, oh, yes, during the same routine maintenance, the backup disk was reformatted too.
And the tape backups? They were corrupted. The only remaining source of data was the original paperwork submitted by Alaska citizens.
The wirebiter winced when he read of the data disaster. Of course, he was in the endodontist's chair at the time, reading the article on his wireless handheld while waiting for the anesthetic to take effect.
Now, all of this (the Alaskan episode, not the root canal) happened back in July of 2006. Fortunately, there were only 600,000 or so payments to be made at the end of the fiscal year, so a month's worth of 70 people working mega-overtime through August're-scanning every paper document and linking them to the appropriate database record'was enough to fix the problem. And most of the checks got sent out on time. Hardly anyone noticed anything out of the ordinary range of government efficiency.
In fact, the Rat guesses that the whole episode actually improved the quality of the database. Oh, an added benefit of all this: they now have a tape backup system that works.
All was happy in Revenue-ville. No harm, no foul. According to an AP report, the technician kept his job, and, as the former revenue commissioner of Alaska, Bill Corbus, was quoted as saying, 'There was no witch hunt.'
'But I sure bet there was discussion of tarring and feathering,' the Rat tried to mumble as the endodontist found yet another root to canal. 'What's that?' replied the endodontist. 'No, lidocaine shouldn't taste like herring.'
An added benefit of all of this: they now have a tape backup system that works.
While there was no harm, there was certainly overtime, and consulting from Microsoft Corp. and Dell Inc. to tell the Revenuers that, yes, their data had passed into the great beyond. That bill came to $220,700. The whole episode probably never would have made the news if the Department of Revenue didn't have to ask the legislature for permission to use a little bit of the Permanent Fund to pay for the error.
Of course, $220,700 seems a small price to pay for a cautionary tale you can hold over the heads of agency sysadmins everywhere: 'Make sure your backups are verified, or you could end up chained to your desk for the whole summer, rekeying data from paper!'
'Just like a root canal is a small price to pay for an example to one's offspring of why you need to floss, brush and visit the dentist regularly,' the Rat tried to moan.
'Don't worry,' said the endodontist. 'You won't feel a thing ... until later. Later you might be a bit sore.'The Packet Rat once managed networks but now spends his time ferreting out bad packets in cyberspace. E-mail him at firstname.lastname@example.org.