I happen to believe this is FUD, and that the reverse might actually be true. (This is one reason why we’re competing head-on with MySQL.) Having a “mother ship” is in the long run, a very complex scenario to fully understand. There are all sorts of causes and effects that play out in subtle ways. I honestly doubt anyone can really know what the true effect is.

So what can we say about the presence or absence of a single controlling organization, then? Simply this: we can look for examples of where power is decentralized, and see how it fares. Let’s see some names:

• The Linux kernel.
• The GNU/Linux operating system in all its variations.
• PostgreSQL.
• Apache.

There are more that I could mention. But these are good enough examples to show that a thriving ecosystem is possible when power is not completely concentrated. In many ways, these projects have been far more successful than MySQL. I invite you to simply take a look at the Postgres hackers mailing list to see the PostgreSQL project’s success first-hand.

I can mention a few effects I’ve seen in the MySQL world that I don’t see elsewhere, and which I attribute to the presence of MySQL the corporation. The single biggest one I see is how the community around MySQL differs from, say, the community around PostgreSQL. As I said in my recent keynote address at the Percona Performance Conference, I think the best way to describe it is “MySQL has a community. PostgreSQL is a community.” (That’s a quote from a PostgreSQL community member.)

PostgreSQL isn’t the only project I can point to — Drizzle, for example, is taking a very good approach in my opinion. But any of these will do as an example. The key point is that there is a fundamentally different outlook on the relationship, both inside and outside MySQL, than there might be. On the outside, I have observed a lot of passivity and negativity. On the inside, I sometimes detect something that might be called paternalism.

Unflattering — yes, I’m being rather unflattering. But let’s be honest. The status quo between MySQL and “the community” is not good, not good at all. It resembles nothing so much as the relationship between users and producers of a proprietary product. That’s not bad for a proprietary product — but really bad for an open-source one. Just the fact that we talk about MySQL and “the community” as though they’re separate reveals a lot.

And I attribute this to the presence of a mother ship, however you might choose to label it. It creates a power relationship that I believe is a roadblock to a lot of good things. I know of examples in the last few weeks where the power relationship caused appalling problems with some of the foremost community members — situations that would never have happened in a relationship of equals. (I’m being discreet for a reason; I don’t want to cause any more heartache for those involved. Let’s just leave it at “the entire community suffered as a result.”) This kind of thing has been happening for years.

I firmly believe those who would truly participate in open source must recognize that there is no true meeting of the minds, unless everyone is an equal. Not just “treated as an equal” — if you hold the power, then you can’t be an equal, no matter how you try to treat others. Anyone who tries to be “more equal” than others is poisoning the well. The only solution I see is to divest oneself of power.

I think there are a lot of opportunities here for everyone involved. And those who are best placed to take those opportunities, but don’t, merely create a vacuum that draws others in to do it in their stead. And that’s why I think that ultimately, the mother ship structure is not only unnecessary and sometimes detrimental, but may even become irrelevant.

I will be very curious to see how things change in the future. MySQL has a lot going for it: an incredible team of talented, dedicated employees; visionary leaders; unprecedented technology landscape; economic conditions — it’s an enviable position. What I’m most interested in is to see how they will nurture the open-source nature of the product, which is its greatest strength — if it is teased out of the closed-source models that have bound it so tightly in recent years. The good news is, I think it’s theirs to lose; they can win if they play it right, but the rest of us win no matter what.

The day we see an announcement like this one for MySQL will be a great day indeed.

Further Reading:

• Does MySQL really have an open-source business model?
• Like it or not, it is the MySQL Conference and Expo
• Why is MySQL more popular than PostgreSQL?
• Oracle is not screwing MySQL
• MySQL: Free Software but not Open Source

• Secure. Use trusted, well-known, encrypted communication. Do not open up new ports.
• Zero install on the monitored system.
• As little installation or modification on the monitoring system as possible.

Over the last several years, I’ve slowly created more and more software to create Cacti graphs via standard POSIX command-line utilities over SSH with key-pair authentication. (I’ve also created similar software for Nagios, but that’s another matter.) The major problem with the work I’ve done is that it’s totally un-publicized.

The system works by passing command-line arguments to a local PHP script like any other Cacti script. This script then executes a remote SSH command, such as ssh somehost uptime and extracts statistics from the result.

The requirements are very simple. On the system to be monitored, a public key must be installed. On the monitoring system, the cacti user account must have a private SSH key that it can read and others can’t. This is standard for any SSH key. The cacti user account must also have the SSH key fingerprint of the monitored system in its known_hosts file.

The work I’ve done thus far is available from Subversion in the mysql-cacti-templates project. This project has the scaffolding for creating Cacti templates easily, so I’m using it.

At this point, the following are available:

• Apache
• Operating system (CPU, memory, load average, etc)
• Memcached
• Nginx

Unfinished work includes network, disk, etc. The techniques to monitor something of which there is a variable number (e.g. there can be many disks, each of which needs its own graph) are a little more complex than simple things like monitoring overall CPU usage. So that’s a work in progress. Once it’s done, it’ll make it really easy to discover and monitor multiples of anything — for example, multiple MySQL servers or memcached servers on a single host — without creating a new host for each resource to monitor.

As with the MySQL templates I created, these templates are comprehensive and have lots of nice properties most templates lack. This is something you get free with my scaffolding. If you’ve ever created templates by hand through the web interface, you should give my work a try. You can turn a three-day project into a few minutes and avoid bugs and other hassles. There are instructions for creating Cacti templates on the project wiki.

Further Reading:

• News on MySQL Cacti Templates
• Improved Cacti monitoring templates for MySQL
• A better way to build Cacti templates
• Version 1.1.2 of improved Cacti templates released
• Version 1.1.7 of Better Cacti Templates released

I intentionally wrote the Maatkit tools so they don’t need to be “installed.” You just run them, that’s all. But I never made them easy to download.

I fixed that. Now, at the command line, you can just run this:

wget http://www.maatkit.org/get/mk-table-sync

Now it’s ready to run. Behind the scenes are some Apache mod_rewrite rules, a Perl script or two, and Subversion. When you do this, you’re getting the latest code from Subversion’s trunk.[1][2] (I like to run on the bleeding edge. Releases are for people who want to install stuff.)

Because there’s some Perl magic behind it, I made it even easier — it does pattern-matching on partial names and Does The Right Thing:

baron@kanga:~$ wget http://www.maatkit.org/get/sync
--21:38:50--  http://www.maatkit.org/get/sync
           => `sync'
Resolving www.maatkit.org... 64.130.10.15
Connecting to www.maatkit.org|64.130.10.15|:80... connected.
HTTP request sent, awaiting response... 302 Moved
Location: http://www.maatkit.org/get/mk-table-sync [following]
--21:38:50--  http://www.maatkit.org/get/mk-table-sync
           => `mk-table-sync'
Connecting to www.maatkit.org|64.130.10.15|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-perl]

    [      <=>                            ] 163,259      136.51K/s             

21:38:51 (136.13 KB/s) - `mk-table-sync' saved [163259]

The redirection is there because otherwise wget will save the file under the name ‘sync’ instead of ‘mk-table-sync’.

And if you’ve forgotten which tools exist, you can just click on over to http://www.maatkit.org/get/ and see.

A quick poll: instead of getting the latest trunk, should this give you the code from the last release? I can do that, if you want.

[1] OK, it’s only refreshed every hour. So you’re getting code that’s up to an hour old.

[2] update: now /get/foo gets the latest release, and /trunk/foo gets the latest trunk code.

Further Reading:

• MySQL command-line tip: compare result sets
• Vim, the ultimate command-line password manager
• A very fast FNV hash function for MySQL
• How fast is MySQL Table Checksum?
• How fast is MySQL replication?

Fixing URLs

As I mentioned, there used to be a blog at /blog/ which must not break. Suppose this blog was about dogs and my wife has recently started blogging about cats. She wants two completely independent blogs: /blog/dogs/ and /blog/cats/. Now the old permalinks structure, e.g. /blog/2006/03/01/dogs-are-great/, must redirect to /blog/dogs/2006/03/01/dogs-are-great/. How to do this?

I’m not a mod_rewrite wizard, but I figured there must be a way. And indeed there is: if an incoming URL doesn’t contain dogs or cats, it can be rewritten and redirected to the new URL. Here’s the code, which goes in /blog/.htaccess:

RewriteBase /blog/
RewriteCond %{REQUEST_URI} !dogs|cats
RewriteRule ^(.*)$ http://www.furryfriends.org/blog/dogs/$1 [R]

(By the way, the furryfriends thing is just an example, not the real site name).

So far, so good. That works just fine: when I access a URL without dogs or cats in it, it redirects me. But I need to do more: I need rewrite rules to match the date-and-title permalinks both blogs will use. I accomplish that like so:

RewriteCond %{REQUEST_URI} dogs|cats
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (dogs|cats) /blog/$1/index.php [L]

This is basically the same thing WordPress usually does, but I’ve made it tolerate either dogs or cats and figure out which installation should get the request. The .htaccess file lives in /blog/, not inside /dogs/ or /cats/ where it would be hard to maintain (it would get wiped out with upgrades). I can see different ways of doing this, but this is the way I chose. So here’s the whole file:

RewriteEngine On

# Anything to the old address (e.g. /blog/foo/bar) goes to the new address
# (e.g. /blog/dogs/foo/bar)
RewriteBase /blog/
RewriteCond %{REQUEST_URI} !dogs|cats
RewriteRule ^(.*)$ http://www.furryfriends.org/blog/dogs/$1 [R]

# If that fired, then we didn't reach this code.  If we did, then this rule
# should do what a normal WP rule does.
RewriteCond %{REQUEST_URI} dogs|cats
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (dogs|cats) /blog/$1/index.php [L]

Are there any better ways of doing this? I’m curious. Leave a comment if you know of one.

Fixing the maintenance headache

Installing two copies of WordPress, then customizing both is a pain. And it makes upgrades harder, too. I’d have to upgrade them both, fiddle with plugins (some of them are customized, too) etc etc. Even backups would be more complicated. It would be all too easy to screw up and delete some data. There are just so many ways this is a bad idea.

It occurred to me that I could use a single copy and turn the dogs/ and cats/ subdirectories in the filesystem into symbolic links. (Windows users, you can stop reading now: this won’t work for you).

To make the blogs, the WordPress installation, and the custom blog theme all independent of each other, I created the following filesystem hierarchy:

blog/
   wordpress/
      2.3.2/
         [The usual WP files are here]
      wp_content/
         plugins/
         uploads/
         themes/
            my_custom_theme/

What I’ve done is separate the custom bits — the parts that don’t ship with WordPress — away from the files I want to upgrade when I upgrade WordPress. How will this work, though?

I’ll make symbolic links from the dogs/ and cats/ directories to the currently installed version of WordPress. So, from the root directory of the website, I type the following at the command line:

$ ln -s wordpress/2.3.2/ dogs
$ ln -s wordpress/2.3.2/ cats
$ cd wordpress/2.3.2/
$ rm -rf wp-content/
$ ln -s ../wp-content wp-content

The directory hierarchy now looks like this:

blog/
   cats/ -> wordpress/
   dogs/ -> wordpress/
   wordpress/
      2.3.2/
         [The usual WP files are here]
         wp-content/ -> ../wp-content
      wp_content/
         plugins/
         uploads/
         themes/
            my_custom_theme/

This is looking pretty good! There’s only one minor detail missing: because both blogs are running literally the same code via the magic of symlinks, each blog is trying to access the same database tables. I need to customize the WordPress configuration file, too. I’ll just give each installation a different table name prefix in wp-config.php:

$table_prefix  = strpos($_SERVER['REQUEST_URI'], 'blog/cats/') ? 'wp_cats_' : 'wp_dogs';

And voila, it works perfectly now. I accessed the two URLs, ran through the installation procedure twice, and have two completely independent blogs running the same code in the same database.

The upgrade procedure

So, this is all a little complicated, right? What if I’ve forgotten how I did it when I upgrade next time, or what if someone else does it instead of me? I wrote myself a little README file to fix this. Here’s what it says:

This is how to upgrade Lynn's blog.

The two blogs are actually using shared files, which are symlinked to make
it so there is only one copy of files.  You can't change the files in one
without changing them in the other.

The wp-content subdirectory is symlinked.

The wp-config file is customized so it will work in either blog:

$table_prefix  = strpos($_SERVER['REQUEST_URI'], 'blog/cats/') ? 'wp_cats_' : 'wp_dogs';

To upgrade, 

 1. Download the latest version and unpack it inside wordpress/ as 2.3.2/
    or whatever version it is.
 2. Then go into that directory.
 3. Remove the wp-content/ directory completely.
 4. Then symlink it like this: ln -s ../wp-content wp-content
 5. Now re-customize wp-config.php
 6. Go back to the blog/ directory.  rm dogs cats
 7. ln -s wordpress/2.3.2/ dogs
 8. ln -s wordpress/2.3.2/ cats

It’s still a manual process, but it should take me all of thirty seconds. I’m okay with that. As long as I remember there’s a README file, that is!

Further Reading:

• How to install beautiful X11 cursors
• Blogs that are good resources for MySQL users
• 3 ways to maintain rollup tables in SQL
• How to install innotop
• How to round to the nearest whole multiple or fraction in SQL