Xaprb

Web Operations

Web Operations. By John Allspaw and Jesse Robbins, O’Reilly 2010. (Here’s a link to the publisher’s site).

This book is due out in about a month. It is part of O’Reilly’s Beautiful series, which you might know through Beautiful Code. This one’s about web ops, of course. There are a dozen contributors, including some of my favorites such as Theo Schlossnagle, whose Scalable Internet Architectures is on my (quite short) list of essential books. And then there’s a chapter from me.

My chapter is 32 pages on choosing a relational database architecture for a web application, which of course is slanted towards MySQL, where my expertise lies. I do not discuss fads or cool new things; I write about what the majority of web applications truly need, and how that maps to choices in architecture and technology for the database backend. I focus a lot on how to build a database backend that supports the application’s operational requirements, in keeping with the theme of the book. This is a much simpler topic than it’s often made into. I hope you’ll find it useful.

When I get a dead-tree copy of the rest of the book, I’ll review it. I can’t read books on my computer. But with authors like John, Jesse, Theo, and the others involved, I am pretty confident this one will be worth adding to your library. By the way, the royalties go to a charity, as with all the Beautiful books.

Daniel changed mk-query-digest to use much less memory. It parsed and aggregated a 2GB MySQL slow query log file in a few dozen megabytes of memory for me yesterday. Thanks to Facebook for sponsoring this work.

I spent a little time yesterday doing some things with the “summary” tool from Aspersa. I added support for summarizing status and configuration of Adaptec and LSI MegaRAID controllers. I also figured out how to write a test suite for Bash scripts, so most major parts of the tool are fully tested now. I learned a lot more sed and awk this weekend.

There is really only one way to get status of Adaptec controllers (/usr/StorMan/arcconf), but the LSI controllers can be queried through multiple tools. I added support for MegaCli64, as long as it’s located in the usual place at /opt/MegaRAID/MegaCli/MegaCli64. I am looking for feedback and/or help on supporting other methods of getting status from the LSI controllers, such as megarc and omreport. If you can contribute sample output from these tools, please attach them as a file to a new issue report on the project’s issue tracker. (Don’t paste them as text, please — formatting and whitespace will get mangled. Tabs and spaces need to be preserved.)

I am slowly gaining insight into how best to write a similar summary tool for MySQL servers. The goals of this tool are very specific — including things like diff’able output. I’m figuring out what went wrong with Maatkit’s mk-audit tool and how to go about it differently.