Handling the Support Pager

His monitor screen wavers slightly, the slight interference of a incoming mobile/cell call or SMS. And with a sharp beep, the pager goes off. He is on the clock. (There is no clock, but he knows the clock exists in the form of “How soon are we going to be back, Joe?”, estimated downtime, estimated time of arrival of a solution, “Are we there yet?”, and “How long? Users are waiting”.)

Yet he still hasn’t connected to the box to see what fragments of messages in voluminous logs indicate a trail, a scent. The pager beeps again — it is getting serious. The monitoring software, being a software tool, keeps trying and failing and paging. He swiftly silences the tool with well trained muscle memory — choose, click, choose, click. His brain then switches on the experience banks.

Has he seen this before? Is it transient page, the curse of the false positive, the “I’ve got a headache” error? How bad is it really, what destruction of data and indexes, datafiles, and disks awaits?

The prompt is back, the password is in, and bingo! Rote, memory, and experience kick in. He is like a seasoned detective, scanning for the clues. He is quietly confident, he has backups, he has standbys, he has flashback. And he has resources he can bring to bear on the problem — within reach are the minds of dozens of DBAs; and more extensive, a vast case history of problems; and finally a searchable universe called the Net.

The problem–however serious–will be solved, can be solved. Tenacity may be required, leaps of faith, moments of deep thought. Soon, functionality will be restored, and everyone will live happily ever after. But for now, the clock is ticking.