Xaprb

I’ve just committed some changes to diskstats, an I/O analysis tool in Aspersa that’s actually been in the Subversion repository for a long time, but in a barely usable fashion and with no documentation. Now it’s usable and documented.

It is basically a reimplementation of iostat in awk. Why on earth would I reinvent that wheel? Because I spend a lot of time gathering and analyzing raw data from /proc/diskstats, which is vital to really understanding what the storage subsystem is doing. The iostat tool hides important details. Seeing that detail has immediately solved many a disk performance problem and proven SAN vendors wrong, for instance. (I used to do this the old-fashioned way.) Disk performance, of course, is one of the most important things to analyze in a database server that’s struggling.

Also, iostat isn’t interactive, and I wanted an interactive, menu-driven tool to quickly slice and dice the data and drill down into what is happening with I/O. The data it accepts is in the same format as that stored by the stalk and collect tools, which is my default post-mortem toolset. And finally — and I know this might be hard to believe — I’ve been asked to fix problems many times on systems that don’t have iostat and I am not allowed to install it.

And wouldn’t you know it, as I wrote the user’s manual I found a bug, after all my ranting about how other tools show I/O stats wrong. I don’t have time to diagnose or fix the bug right now, so maybe someone else can contribute that. There is a test suite (remind me to explain sometime how I make Bash scripts highly testable) so if we find the problem and fix it, it’ll stay fixed. Contribute your fix to the bug report :-)

A couple of extremely informative recent blog posts have gone to either Planet MySQL or Planet PostgreSQL, but not both, and I think everyone on both aggregators who cares about database internals should be interested in them. Here they are:

• Robert Haas: MySQL vs. PostgreSQL, Part 2: VACUUM vs. Purge
• Ewen Fortune: How InnoDB handles REDO logging

Happy cross-pollination.

I’ll be presenting several sessions at the O’Reilly MySQL Conference & Expo 2011, which is April 11-14 in Santa Clara, California. I recommend this conference to anyone interested in open-source databases including MySQL, PostgreSQL, CouchDB, MongoDB, and others. There is very good coverage of a diverse list of open-source databases.

My sessions are as follows:

• Building on Strengths, Learning from Differences: a keynote address on the open-source database ecosystem, how we got here, and what we can do to make the most of the future.
• Forecasting MySQL Performance and Scalability: mathematical models for forecasting performance and scalability that actually work and are not hard to do. (Example)
• The Aspersa System Administrator’s Toolkit: this is an under-appreciated toolkit at the moment, but it could be the next Maatkit.

In addition, I am listed as presenting Diagnosing and Fixing MySQL Performance Problems, a 3-hour tutorial on how to find and solve performance problems with swift and definite results. However, I actually have a scheduling conflict and a couple of my colleagues will present this instead.