Xaprb

One of my pet peeves is when documentation is wrong. Another pet peeve is keeping documentation right. Crack open a source tarball for many programs and you’ll see a chunk of text that gets printed out when you use the –help option, and elsewhere in the program’s source code you’ll see the definitions of the command-line options. Maintaining a program like this is miserable. Using it is bad, too. I can name a lot of programs that say one thing and do another.

For Maatkit, we solved this problem by making the tool read its own source code and generate command-line options, default values, behaviors, dependencies, data types, and so on directly from its own embedded documentation. This is the same documentation that gets converted into man pages. So when you run the program, view its documentation, ask it for –help, or whatever you do, you get the same information. The documentation is part of the program, and if you change the documentation, you change the program.

For a while I was very unhappy with using Perl to reach outside the boundaries of Perl. It turns out that executing another program, capturing its output, controlling it, capturing its return code, etc is very buggy. So I started to write scripts that need this capability in bash, because it is obviously very good at these tasks. But it’s a bit harder to handle command-line options in bash, and the tools available for it differ or are unavailable on various platforms. So I ended up with usage information in a block of text, and program options defined in program code. Yuck!

I fixed that recently. I wrote a short script that reads the usage text and generates code to implement the options, including default values and options that are constrained to certain valid inputs. Life is good again.

I’ll be presenting several sessions at the O’Reilly MySQL Conference & Expo 2011, which is April 11-14 in Santa Clara, California. I recommend this conference to anyone interested in open-source databases including MySQL, PostgreSQL, CouchDB, MongoDB, and others. There is very good coverage of a diverse list of open-source databases.

My sessions are as follows:

• Building on Strengths, Learning from Differences: a keynote address on the open-source database ecosystem, how we got here, and what we can do to make the most of the future.
• Forecasting MySQL Performance and Scalability: mathematical models for forecasting performance and scalability that actually work and are not hard to do. (Example)
• The Aspersa System Administrator’s Toolkit: this is an under-appreciated toolkit at the moment, but it could be the next Maatkit.

In addition, I am listed as presenting Diagnosing and Fixing MySQL Performance Problems, a 3-hour tutorial on how to find and solve performance problems with swift and definite results. However, I actually have a scheduling conflict and a couple of my colleagues will present this instead.

I’ve written a tool to profile a process’s IO usage. It works by gathering lsof and strace from a process, and then figuring out how the file descriptors, function calls, and filenames are all related to each other. The manual page has examples. I’m curious to see how it works for you. Note that it might be a good idea to run this on your development or staging system before you go running it against your production MySQL server — there are rumors of strace misbehaving on some kernels.