Xaprb

Daniel changed mk-query-digest to use much less memory. It parsed and aggregated a 2GB MySQL slow query log file in a few dozen megabytes of memory for me yesterday. Thanks to Facebook for sponsoring this work.

A lot of times I’m looking at several time-series samples of numbers embedded in free-form text, and I want to know how the numbers change over time. For example, two samples of SHOW INNODB STATUS piped through grep wait might contain the following:

Mutex spin waits 0, rounds 143359179688, OS waits 634106844
RW-shared spins 1224152309, OS waits 38278807; RW-excl spins 2432166425, OS waits 35264871
Mutex spin waits 0, rounds 143386303439, OS waits 634292093
RW-shared spins 1224197048, OS waits 38281423; RW-excl spins 2432347936, OS waits 35271423

How much have the numbers changed in the second sample? My head is too lazy to do that math. So Daniel Nichter and I whipped up Yet Another Snippet to self-discover patterns of text and numbers, and compare each line against the previous line that matches the same pattern. Let’s fetch it:

wget http://maatkit.googlecode.com/svn/trunk/util/rel

Now give it the above input, and it’ll print out something useful (emphasis mine):

Mutex spin waits 0, rounds 143359179688, OS waits 634106844
RW-shared spins 1224152309, OS waits 38278807; RW-excl spins 2432166425, OS waits 35264871
Mutex spin waits 0, rounds 27123751, OS waits 185249
RW-shared spins 44739, OS waits 2616; RW-excl spins 181511, OS waits 6552

My lazy brain likes that much better.

Download Maatkit

There’s a new release with a lot of goodies — speed, efficiency, user-friendliness, and new features. In particular some of Percona’s clients have sponsored features for things they need such as the ability to more frequently verify that slaves are in sync with their masters. If you need features, please ask Percona about it.

Daniel Nichter has written most of the code for this release. Having a dedicated developer is really moving the project forward. Thanks Daniel!

If you check the issues list on the project’s website you’ll see we have some ambitious plans for the project, mostly driven by real-world needs from Percona’s clients, many of whom run very large data centers and such. It makes me really happy that Maatkit is solving serious problems for which no other solution exists.

Here’s the changelog: