Error Log Head-Scratcher

Posted in: MySQL, Technical Track

As an editor (of https://www.mysqlzine.net), I cringe at the title of this post. However, it is absolutely accurate.  Recently, we had a situation where we had two servers running Sun Solaris 10 on some high-end Sun hardware.  I don’t remember exactly, but it was one of Sun’s upper-end boxes with AMD procs. Nice boxes, really. The two servers are configured in a master-master circular replication setup.

Here is the problem.  On both servers, the error logs were being created incorrectly.  On one of them, it was creating an error log that was 154 megabytes in size.  FLUSH LOGS worked, but the newly-created error log would be the same size. While there was some data in the file that I could use the cat, head or string command to discern, the majority of the file was not text data.

After working on this for a bit, I logged into the secondary server and discovered that the error log on this server didn’t look right either  — the same characteristics of large size with almost no actual text data.  The only difference is that these error logs were around 20 megabytes in size. I googled around a bit and couldn’t discover anyone with a similar problem.

I can’t figure out what was causing this.  We checked everything we could think of, and during some other maintenance, restarted the mysqld daemon. That didn’t work either — when the server came back online it was experiencing exactly the same problem with the error log. Finally, during hardware maintenance to upgrade the memory, the servers were rebooted.  The next morning, I checked them, and found both error logs working exactly as they should. However, it took that server reboot to fix the problem.

I am at a loss to explain what was wrong.  If anyone has any thoughts or a similar experience, I would love to hear from you!

email

Author

Want to talk with an expert? Schedule a call with our team to get the conversation started.

About the Author

I am a MySQL DBA at The Pythian Group (https://www.pythian.com).

2 Comments. Leave new

Carsten Pedersen
June 27, 2008 4:49 am

In some debug versions of MySQL, I’ve seen the error log get filled with binary stuff.

If you’re running a debug version, try using a “clean” one.

Reply
Keith Murphy
June 27, 2008 9:13 am

This isn’t a debug version.

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *