Don’t Assume Anything

I’ll preface this post with a note that the story itself is not really work- or DBA-related, but the lessons learned certainly are. I consider myself fairly conscientious when it comes to internet security and backing up my most important files (pictures of kids, music, etc), and I am diligent about taking at least a weekly backup of those files so that if (when) something catastrophic happens, I am ready. Once again, as my short life as a DBA has proven, theory and practicality rarely meet.

Monday night, I needed a stress reliever. My family and I recently moved to Ottawa from Wichita and it has been a five-month-long ordeal. I used to be an avid gamer, but with changing jobs, selling the house, moving, family issues, etc., I haven’t had a lot of time to kick back and relax. I decided to visit one of my favorite websites, www.armchairgeneral.com, to see if they had any good reviews of new games to play. One in particular caught my eye, called “Mount and Blade”. Looks like an interesting twist on your standard medieval-based RPG — the combat is in first person. The game is still in beta, but can be downloaded by anyone wanting to play it. So I downloaded it from a link on the game’s website that pointed me to CNet.

Not the wisest decision in my life, as about 10 minutes later my virus scan software (AVG) warned of viruses and Trojan horses. I quarantined everything that it found, but it wasn’t enough — I was officially infected for the first time in four years. I tried for an hour that night to undo the damage, but didn’t have much success. This bug was nasty — it even went so far as to detect that I tried to go back to a previous System Restore Point and it erased them. My colleagues at Pythian took an immediate interest in my dilemma and suggested several helpful tools (like Process Explorer) that I also tried with limited success.

I didn’t mind spending some time seeing if I could easily get the virus removed as I had an ace up my sleeve: I had taken a full system backup on Sunday. Take that, you nasty virus! With a feeling of having been just a bit smarter than the schmuck who wrote the virus, I put aside my troubleshooting efforts on Wednesday night and decided to restore the backup. When I originally set up my automated system backups, I rigorously tested it to ensure that it could be performed in a crisis, with no issues. This is where my breakdown in thought occurred — I had made some bad assumptions (a no-no for any DBA):

1. The system backups were originally set up on a different PC than the one I have now.
2. The system backups were only about 25GB, whereas the size of my current system is a bloated 182GB.
3. I hadn’t tested the backup in nearly three years as I didn’t want to overwrite my current system in case the backup didn’t work.

I proceeded with restoring the backup after creating a Linux-based boot disk with the restore application loaded on it. At 32% complete, the backup hung, and I eventually had to hard-power off the PC to end it. When I booted up and went back into the application, it could no longer find a valid backup. So much for having a full system backup. Fortunately there was a partition on the hard drive that would assist you with restoring to the original factory settings, and I was able to rebuild my system in about four hours. I also had backups of everything in “My Documents” on another external hard drive that was also backed up on Sunday, so I didn’t lose my most important files.

The moral of this story: just don’t assume that because you’ve had backups and you tested them a while back that they are still going to work. Changes in hardware, architecture, size of the backups, etc., can all have unintended consequences on the restorability of your systems. I just hope that if you ever find yourself in a situation such as mine, be it at home or at work, that you are as fortunate as I was.