Undoubtedly, the database performance specialist upon first meeting a customer will be told, “but this database functioned great for the first five years and it’s slowed down to a grinding halt at year ten – why??”
As the specialist, you now have to carefully take the time, explaining in layman’s terms, that the natural life of a database is growth and although the pants you bought your child may have fit him wonderfully when he was five years old, he wouldn’t be moving too well in them at the age of ten, either… (We’ll discuss that high-maintenance dry-clean only, misshapen, hand-knitted sweater from Aunt Developer Deena a bit later, but we do want to take this slowly…)
No matter how unlikely it may seem, many companies simply become accustomed to certain lags in performance or even accept outages as normal to their day to day business due to poorly performing database environments. It’s what I refer to as the plague of the database industry and the constant frustration of DBA’s everywhere. Until these lags and outages become huge storms and are impossible to ignore, it is commonly tolerated.
How do environments end up in this situation?
There are many reasons, and it should be noted- rarely are there negative intentions by anyone. People really do want to do what’s best or think they are making the best decision for the database environment. They may simply not have the knowledge to know they are causing harm by not addressing it pro-actively. There isn’t any bold print from Oracle that says to stop the repeat incidents that occur or that “out of sight, out of mind” is not the same as non-existent or why it can be an issue when technical specialists do not have the driving need to know why something happened or the desire to deter it from happening again.
Often, its simply common in emergency outages or performance challenging situations, that most companies perform “triage” and never allocate the time to determine the true cause of the incident. They either don’t have the resources or haven’t been able to follow up to see if it’s a one-time incident or a pattern emerging.
When entering into a new environment, if you speak to the other DBA’s, the user base and management, you will quickly figure out that everyone knew what was wrong at a high level, but often had no idea how to tackle the growing technical storm that was brewing. So the bad news is, yes, it’s almost always daunting to view head on, but the good news is, you’ll wonder what you thought was so difficult in hind sight when it’s been tackled by a performance specialist.
One of the significant benefits that I see to being a client with Pythian that I think all customers should take advantage of is the wealth of performance tuning specialists comprised by one company.
There is a unique opportunity for customers of Pythian to have direct access to these gifted individuals and utilize them to identify issues in an environment before they become, what many call “the perfect storm”. These storms rise up, causing outages, major performance issues and failures. They grow over time, where they may have once only seen small “thunder bursts” in the past, they increase until the database experiences major performance issues, aka “The perfect storm”.
Storm chasing involves making a list of the top 10 issues in an environment and working through those issues with a performance increase goal set for correcting the issues on the list. Ten may seem like a small number, but it is surprising how many storms can be quelled with the “ten to stop the storm” solution.
It is essential to the long-term health of the database environment to grasp the pay-forward benefits of performance tuning. Not all DBA’s are comfortable in these storms, but I truly believe most performance tuning specialists are similar to the seventh president of the United States, Andrew Jackson, who said, “I was born to a storm and the calm does not suit me!” The sheer quantity of “storm chasers” at Pythian is amazing for those customers who have the insight to see the value in allocating time into performance tuning and environment health reporting which results in huge benefits in the way of less outages, simpler code releases and increased customer satisfaction.
No comments