No Time Like the Present

I’m not one to panic over non-events. I wouldn’t survive very long in my career if I were. Besides, I always have plenty of real emergencies to worry about. But when even the New York Times tells us to prepare for the end of the world, which may or may not arrive in a few days, I start to worry. I even found a nice website to tell me if the apocalypse happened yet.

Like any self-respecting DBA, I can’t prepare for anything without concrete requirements. “Apocalypse” and “End of the World” are rather vague descriptions and don’t give me a good idea of what I’m preparing for. I asked some people what they expect the end of the world to be like and got the following list: earthquakes, war, shortage of natual resources, chaos, stock market crash, epidemic, and satanic hosts of demons.

If it weren’t for the demons, I’d say the apocalypse already arrived, and there is nothing more to worry about.

But fact remains that most Americans, and DBAs among them, are woefully unprepared for an apocalypse. Here at Pythian, we want to help out customers and friends to prepare for the apocalypse in the hope that we still have customers and friends next week.

Prepare your database for the apocalypse with these 10 easy tips:

• Best advice for the new era is from our Oracle ACE, OCM, and RMAN expert, Yuri Velikanov, who said: “RMAN DROP DATABASE should make all the preparations much trouble-less”. Indeed – No database, no problems!

• If you are somewhat reluctant to actually delete your database, you can try a slightly less drastic approach. Take backups a few hours before the end of the world, and then shut down. In other words, treat the end of the world like you would any other risky maintenance, such as applying a patchset. I’m sure you’ll find that there are many similarities between the events.
  Of course, the interesting question is where to store the backup. If the world is about to end, the only safe place for your backup is in orbit. Launch your backup satellite before it’s too late.

• Over years of working in IT, we noticed a certain interesting correlation: Database problems typically happen after DBAs do something to the database. Deployments, upgrades, space allocations, all sorts of routine maintenance actions have some probability of ending in tears. If no one touches the database, it will typically keep on happily functioning for years.
  To survive the end of the world without a hitch, you should prevent your DBA team from touching your database. We are not suggesting going as far as firing them (although we will be happy to hire them at Pythian if you would), but consider sending them on early new-year vacation somewhere remote and possibly safe from trouble. Hawaii should be wonderful this at time of year, but I’m not sure about volcanic activity.

• On the topic of DBAs, you should definitely prepare for the event that some of your DBAs will not survive the apocalypse. Place the job ads today, or better yet, sign a flexible contract with your friendly remote-DBA provider.

• Why stop at just signing a contract with a friendly remote-DBA provider? It’s time to spend your budget like there’s no tomorrow. Personal Exadata X3-8 for the entire DBA staff perhaps?

• Of course, the ultimate plan against sudden lack of DBAs is automation. Make sure all your routine tasks are scripted and automated, and your database may survive the apocalypse even if none of your DBAs do. Some may even argue that automating all routine tasks is a good idea even if the world does not come to an end.

• Now is also an excellent time to review and revise your disaster recovery plans (DRP). Years ago, our department underwent an audit that required us to have a DR plan. At the time, we did not have multiple datacenters or even off-site backups, so we doubted our ability to pass the audit, not to mention surviving an apocalypse. Luckily, our manager was very detail-oriented and noticed that passing the audit just required us to have a DR plan. It did not specify what the plan should actually contain. We drafted a simple DR plan:
  When in trouble or in doubt
  Run in circles, scream and shout.
  We passed the audit.

• I noticed that there is a large number of Friday night parties being promoted as “End of the World Party” or “Party like There’s No Tomorrow”. This is my personal plan for the end of the world – if the music is loud enough and I’m enjoying the dancing, I probably won’t even notice the earthquakes and demons. Luckily for all beer lovers, the US government tested beer safety following a nuclear apocalypse and declared commercially packaged beer safe so we can keep on partying as the world ends.

• Some of us have the  tendency to defer to tomorrow anything that does not absolutely have to be done today. Knowing that the world may end on Friday is the best excuse to procrastinate. If we put our unpleasant tasks off long enough, we may not need to do them at all. Ever.
  For example, the last tip in this list will be published on Monday.

Disclaimer:
The above tips are a joke, yes? We did not validate any of them and performed no dry-runs of the end of the world. Use common sense and keep your databases safe. If you feel a strong need to prepare for an apocalypse, it’s never to early to start worrying about unix-time rollover on January 2038.