Maatkit in RHEL and CentOS

At least one person (Karanbir Singh) is working to get Maatkit into the CentOS repositories, and I believe there might be movement towards RHEL also. From an email to the Maatkit discussion list a little while ago,

I am in the process of getting maatkit into the CentOS-Extras repositories. The first step for that is that every package needs to go into a CentOS-Testing repo and feedback is required from the project and users on its stability / usability and packaging quality.

maatkit-1887 is now available in the CentOS-Testing[*] repo’s and as soon as we can get some feedback ( needs to be 5 different people, none of whom can be CentOS Developers ) - the packages will move into the main repository so that all users can get access.

I’d appreciate it if people on this were able to give those packages a go and let me know if there are any issues. You can leave feedback :

  • via the maatkit-discuss mailing list (http://sourceforge.net/mailarchive/forum.php?forum_name=maatkit-discuss)
  • on the centos-devel list ( http://lists.centos.org ) or
  • http://bugs.centos.org/ against category ‘maatkit’

[*] : Info about the Testing repo and howto set it up on your machine : http://wiki.centos.org/Repositories

If you’re interested in getting Maatkit into these repositories, please take a moment and give the requested feedback. I can’t do it because it would be a conflict of interest for the main developer to assert that the code is stable and usable.

Technorati Tags:, ,

You might also like:

  1. innotop is available from openSUSE buildservice
  2. New Maatkit release policy
  3. Installing innotop on FreeBSD and Gentoo
  4. How to prelink mozilla-firefox-bin

High Performance MySQL Second Edition Schedule

I just got the rest of the production schedule from the publisher, plus the PDF files for quality control, for our upcoming book. (Now I have to proofreeed the whole book!) This is the first time I’ve seen the entire production schedule. The book is supposed to go to the printer in the first week of June. I don’t know what the on-the-shelf date will be, but I think very shortly after that. The publisher has promised that it’ll physically be on sale at Velocity.

I also took a peek at the PDFs. Without the appendixes, the last page of Chapter 14 (Tools for High Performance) is page 604. The appendixes bring it to 660 pages. That’s real material, not including tables of contents and indexes. So my estimate (620) was not too far off.

660 pages is not bad, considering that the contract was for 384 pages.

Another note: the marketing materials for the book emphasize that it covers MySQL 5.1. While this is true, I want to point out that we took a real-life approach: we write about what we’ve seen in the real world, and 5.1 is not as widely deployed in the real world. However, the book’s real value, as far as version-specific content goes, is its tremendous depth and breadth in MySQL 4.1 and 5.0. These have been “out there” for a long time, and among the four of us we’ve seen about every conceivable scenario with it. So you’ll get a lot of insight about current, production-ready, widely-used versions. Let the other guys speculate — we just report the facts. It’s not like there’s any shortage of things to say about 5.0, right?

Technorati Tags:

You might also like:

  1. High Performance MySQL 2nd Edition is in production
  2. Coming soon: High Performance MySQL, Second Edition
  3. Progress on High Performance MySQL, Second Edition
  4. High Performance MySQL, Second Edition: Backup and Recovery
  5. An alternative to canonical URIs

You have the right to see code samples in an interview

Joel Spolsky writes about 12 steps to better code, and elsewhere about how candidates should write code in interviews.

The reverse conditions are true, too. If you’re a candidate, you should evaluate the employer against the 12 steps, and you should also see code samples. How else will you know what you’re getting into? You really have the right to do this, and you should exercise the right. If you don’t, you’ll get stuck in a crap job maintaining crap code. [dramatic voice] It happened to me.

In many companies, you can see code they’ve released as open-source. (The fact that they’ve done this says a lot about them.) But in others, you’re going to need to surprise someone and say “pick some code that’s not sensitive and show it to me.” Something simple, like the HTML for the search form on their website, or a utility to do some systems administration task. Any company is going to have a lot of code like this that they can show you.

The other approaches I see are to ask about it, assume, or ask the interviewer to write some code for you.

  1. Asking is a valid approach. If you see hesitation, or if someone says “well, it’s not as nice as we’d like, and we’re hoping you will offset that” run don’t walk, is my advice. If you’re reading this as you consider your first job out of college or something, I strongly suggest not getting a job with a company that wants you to improve the way they do things. You should be learning from them, not vice versa.
  2. You can also assume. “Oh, they use Perl? Nevermind.” That’s a stupid approach. Really. Is it acceptable to judge people’s character by the color of their skin? Then why would you judge their code by the language? In all seriousness, I have actually written very elegant, clean VBScript. And I mean, good-quality code by anyone’s standards. It’s hard in VBScript. It’s easy in Perl if you follow the Dog, which is a sign of great intelligence. Think about it this way: people who write beautiful Perl are people you should be eager to work with; they are rocket scientists. You will be the dumbest person in the room, and that should make you happy.
  3. I’ve never asked an interviewer to write code for me. Let me know how it works out for you.
Technorati Tags:, , ,

No related posts.

Summary of beCamp 2008

Yesterday I went to beCamp 2008 along with four roomfuls of other people interested in technology (perhaps close to 100 people total). The conference was a lot of fun. Not everything went as planned, but that was as planned. This was an Open Spaces conference and I thought it worked very well. From an email Eric Pugh sent:

Basically it all boils down to:

Open Space is the Law of Two Feet: if anyone finds themselves in a place where they are neither learning nor contributing they should move to somewhere more productive. And from the law flow four principles:

  • Whoever comes are the right people
  • Whatever happens is the only thing that could have
  • Whenever it starts is the right time
  • When it’s over, it’s over

From Hadoop to Bang-Splat

I used the law of two feet a time or two. In fact, the first session I wanted to go to, which was about Hadoop and MapReduce, had no knowledgeable attendees. Someone overslept. OK, that’s the way it goes: move along.

From there I went to a session about Unix command-line productivity. Most of the sessions I saw were traditional in that they had one person standing up talking and many people sitting and listening, but not all. This one had several clever command-line gurus mentioning their favorite power tips.

I learned about bang-splat and bang-dollar. The bangs have always gotten me in Bash: I avoid them because I’ve never felt like reading the Bash man page section on them. (Am I too lazy, or not lazy enough?) So it was great to hear some people say “bang-splat and bang-dollar are great” and then explain them. That was easy for me, and now I know how they can be useful to me.

This problem-first type of tip is great for me: tell me the problem, then how to solve it, rather than telling me what the solution is and leaving me guessing what kinds of problems I can solve with it. (The Bash man page is solution-first).

In case you’re wondering, bang-splat substitutes the arguments to the last command, and bang-dollar substitutes the last argument of the last command. So, instead of this:

$ touch file1 file2 file3
$ rm file1 file2 file3

I can do this:

$ touch file1 file2 file3
$ rm !*

There were lots of other nice tips too.

MySQL Performance

I ended up doing a talk on MySQL performance basics. I had no idea what the audience was looking for, so I winged it. I did make some slides, but most of the talk isn’t on the slides. You can get the slides from Percona’s slide page. It seemed to be useful to the folks attending, who had a wide variety of experience and knowledge about MySQL.

Cloud Computing

This session began with a demo of how to create an entire application stack in a few minutes with Cohesive Flexible Technologies. Someone else then demoed a similar thing using RightScale. rPath’s Jeff Uphoff was also in the room, but we didn’t get to see a demo of that. During this session the talk turned to various topics including a little bit of the topics I wanted to hear about in the Hadoop session.

Lunch

Lunch was catered Indian food provided by the Rimm-Kaufman Group. Yum.

Large Scale Storage

This session was sort of a round-table. The two people who talked the most were Josh Malone from the National Radio Astronomy Observatory and the Library of Congress, both of whom have a lot of storage needs they are unsure how to meet. Some people from UVA’s library were there as well, but I didn’t ask what they were working on.

This reminded me a lot of a recent keynote Jacek Becla gave at another conference. He’s with the Stanford Linear Accelerator Center, who are going to be generating a lotta data pretty soon.

High Availability Linux

This one started off with more from Josh Malone, who demoed Nagios briefly and then talked about his storage and backup systems. He uses BackupPC, which sounds pretty neat and very smart. We then talked about some of the things he’s looking into doing, with audience suggestions to look into shared storage or DRBD. We also looked at UltraMonkey briefly — it looks like it’s stagnating, though. And the Linux HA project.

Google App Engine

Finally, someone showed us a calculator application they’d built on Google App Engine, including the code and talking about the data model somewhat. It looks like a neat idea, but the lock-in worries me, a sentiment that was voiced by many others in the room.

Technorati Tags:, , , , , , , , , , , , , ,

You might also like:

  1. Come to beCamp 2008
  2. Bash parameter expansion cheatsheet
  3. MySQL Conference and Expo 2008, Day Three

News flash: MySQL 5.1 has zero bugs

Zack Urlocker says MySQL 5.1 has zero bugs. He may have been misquoted, or quoted out of context, but there it is. I’ll quote enough of it that you can’t take it out of context twice:

Mickos also said MySQL 5.1 has upgraded its reliability and ease of use over 2005’s v5.0.

“Now we can admit it, but this version is much improved over 5.0, which we weren’t totally happy with,” Mickos confided.

He reported that more than 1,300 bugs (997 in 2007, 386 so far in 2008) have been fixed in v5.1, and that, according to standard DBT2 benchmarks, the performance of v5.1 is 10 to 15 percent better than the previous version.

“This version now has zero bugs,” Urlocker told eWEEK.

You can check for yourself at the MySQL bug statistics page.

Of course it’s not true. But what did Zack really say, I wonder?

Technorati Tags:, , , ,

No related posts.

Come to beCamp 2008

I’m going to be at beCamp 2008, the followup to the first beCamp, which I sadly missed.

beCamp is a BarCamp un-conference. Tonight was about meeting, greeting, and throwing ideas at the wall to see which ones stick. Literally. We stuck pieces of paper on the wall with our ideas — things we can either talk about or want to hear about — and then scratched our votes on them to see which are popular.

I live and breathe MySQL for a decent part of the day, so I hesitated, but then stuck “MySQL Performance” on the wall. It got quite a few votes, so I assume will be giving a talk on MySQL performance basics at some point during the conference. (The exact schedule is probably being determined right now, in my absence, but I’m so tired right now that I’ll just take my chances on it not being at 8:00 AM tomorrow.) [edit: I just checked the website and there won’t be anything before 9:00, and the schedule is determined tomorrow. I did say I’m tired, right?]

See you there!

PS: if you want to meet some of my colleagues from my former employer, the Rimm-Kaufman Group, they’ll be there too, wearing the “We’re Hiring” t-shirts. They’re hiring, by the way.

Technorati Tags:, , , ,

You might also like:

  1. I have joined Percona
  2. Summary of beCamp 2008
  3. Remember to sign up for MySQL Conference and Expo!
  4. Going to PostgreSQL Conference East
  5. My presentations at the 2008 MySQL Conference and Expo

Pre-Order High Performance MySQL Second Edition

High Performance MySQL

If you’re waiting for High Performance MySQL Second Edition to hit the shelf, you’re not the only one. I am too! I can’t wait to actually hold it in my hands.

But you don’t have to wait idly. No, not at all! You can pre-order it and then you’ll get it as soon as possible. Plus your pre-order will help them figure out how much demand there is, so it doesn’t sell out and make you wait for your own copy.

Technorati Tags:No Tags

You might also like:

  1. L. L. Bean’s privacy policy
  2. High Performance MySQL 2nd Edition is in production

Improved Cacti monitoring templates for MySQL

Download MySQL Cacti templates

As promised, I’ve created some improved software for monitoring MySQL via Cacti. I began using the de facto MySQL Cacti templates a while ago, but found some things I needed to improve about them. As time passed, I rewrote everything from scratch. The resulting templates are much improved.

You can grab the templates by browsing the source repository on the project’s homepage.

In no particular order, here are some things I improved:

  • Standard polling interval and graph size by default.
  • Full captions on every graph; you don’t have to guess at how big the values are. Each graph has current, max, and average values printed at the bottom for every value on it.
  • Much more data is captured. I’ve graphed almost everything I could think of.
  • The graphs are grouped better. Most graphs have only related values. There are some exceptions, but not many.
  • The templates don’t hijack your existing installation. They don’t depend on or alter anything in your default Cacti installation.
  • The script that gathers the data is totally rewritten from scratch, and much improved. For example, the math works on 32-bit systems. It has caching built-in so each poll cycle results in just one request to the server, instead of one request per graph. (This is a weakness of Cacti I’m trying to work around). It also has debugging aids and other good coding stuff.
  • By default, it assumes you have the same username and password across every server you’re monitoring, so you don’t have to fill in a username and password for every single graph you create.
  • One data template == one graph template. This helps work around another Cacti limitation.
  • Lots more. Honestly I can’t really remember everything I’ve done. I’m sure you’ll help me remember by asking me how to get X feature working the way you want, and I’ll go “oh, yeah, that’s another thing I improved…”

Cacti templates are very laborious to create if they’re complex at all; it takes a long time and is very error-prone. Instead of doing it through Cacti’s web interface and exporting a huge XML file, I eliminated the redundancies and created a small, easy-to-maintain file from which I generate the XML template with a Perl script. This gives the added benefit of letting me (or you) generate templates with different parameters such as polling interval or graph size. The README file has the full details. However, I’ve pre-generated a set of templates that matches Cacti’s defaults, so you can probably just use that.

This has taken a lot of time. In particular, I spent a lot of time working on it at my former employer, The Rimm-Kaufman Group (kudos to them for letting me open-source the work) and I just spent most of my weekend writing the scripts to convert from the compact format to XML templates, so it’s possible to maintain these beasts. Plus I had to develop the compact format, too. This took a lot of time because I had to understand the Cacti data model, which is pretty complex.

Please enter issue reports for bugs, feature requests, etc at the Google project homepage, not in the comments of this blog post. I do not look through comments on my blog when I’m trying to remember what I should be working on for a software project.

If these templates help you and you feel like visiting my Amazon.com wishlist and sending something my way, I’d appreciate it!

PS: You may also be interested in Alexey Kovyrin’s list of templates for monitoring servers.

Technorati Tags:, , , , , ,

You might also like:

  1. What’s the best way to choose graph colors?
  2. A new home for innotop in the new year

Baron Schwartz on a podcast at MySQL Conference and Expo 2008

I did an interview with Barton George from Sun while I was at the conference last week. Barton has now posted the interview. If you’re quick, you can listen to it before I do.

Topics: everything and anything, including Maatkit and PostgreSQL.

Technorati Tags:, , , , , ,

You might also like:

  1. Like it or not, it is the MySQL Conference and Expo
  2. My presentations at the 2008 MySQL Conference and Expo
  3. Going to PostgreSQL Conference East
  4. Slides for the innotop workshop at MySQL Conference and Expo 2007
  5. MySQL Conference and Expo 2007 Audio

Like it or not, it is the MySQL Conference and Expo

The conference that many of us just went to is called the MySQL Conference and Expo, but a lot of people don’t call it that. They call it by the name it had in 2006 and earlier: MySQL User’s Conference. In fact, some people say (or blog) that they dislike the new name and they’re going to call it the old name, because [… insert reason here…].

I call it by the new name that some people dislike so much. Why? Because it is a conference and expo, not a user’s conference. There’s no reason to pretend otherwise. The conference is organized and owned by MySQL, not the users. It isn’t a community event. It isn’t about you and me first and foremost. It’s about a company trying to successfully build a business, and other companies paying to be sponsors and show their products in the expo hall. Times have changed.

I’m not saying any of this is bad. Being successful in business is a good thing, and having sponsors and partners is fine too. I’m just pointing out that trying to make it be a user’s conference, just by calling it one, isn’t going to work.

If community members want a community conference, we’ll have to make one. MySQL/Sun cannot do this for us, because then it wouldn’t be a community conference.

There’s a simple test of whether people want this: if it happens, then the community wanted it badly enough to do something about it.

The PostgreSQL East 2008 conference I went to a few weeks ago was a great example of how this works. And the attendance fee was $75, not thousands. A conference doesn’t have to be expensive.

Who wants a conference by, for, and of the community?

Technorati Tags:, , ,

You might also like:

  1. Going to PostgreSQL Conference East
  2. Baron Schwartz on a podcast at MySQL Conference and Expo 2008
  3. MySQL Conference and Expo 2008, Day Two
  4. Remember to sign up for MySQL Conference and Expo!
  5. My presentations at the 2008 MySQL Conference and Expo