Xaprb » MySQL Cluster

What kind of High Availability do you need?

Xaprb — Mon, 16 May 2011 02:37:26 +0000

Henrik just wrote a good article on different ways of achieving high availability with MySQL. I was going to respond in the comments, but decided it is better not to post such a long comment there.

One of the questions I think is useful to ask is what kind of high availability is desired. It is quite possible for a group of several people to stand in a hallway and talk about high availability, all of them apparently discussing the same thing but really talking about very different things.

Henrik says “At MySQL/Sun we recommended against asynchronous replication as a HA solution so that was the end of it as far as MMM was concerned. Instead we recommended DRBD, shared disk or MySQL Cluster based solutions.” Notice that all of those are synchronous technologies (at least, the way MySQL recommended them to be configured), generally employed to ensure a specific desirable property — no loss of data. But “I must not lose any committed transaction” and “my database must be available” are actually orthogonal requirements. One is about availability, the other is durability.

A lot of people who say they want High Availability actually want High Durability.

There are a great many MySQL users for whom writes are much less valuable than reads. I would point to an advertising-supported website as a canonical example. If the system isn’t available — that is, available to serve read queries — then a lot of money is lost. If someone’s latest comment on a blog post is lost — who cares? Money continues to flow.

This is why a lot of people want a system that keeps the database online, even if some writes are lost. Note that loss of writes is not the same thing as consistency — consistency and durability are also orthogonal for most users’ purposes. So we aren’t talking about eventual consistency or any of the other buzzwords, but simply “the system must respond to read queries.”

Asynchronous replication is well suited to many such users’ availability requirements, as long as replication does not fail (halt) through a write conflict or some other failure mode. (It is often perfectly acceptable for it to fail in other ways, as long as it does not halt.) That’s why a lot of users are interested in the specific type of “high availability” that a system such as MMM is intended to provide (but, as I mentioned, actually doesn’t provide). In other words, MMM would be great for a lot of people, if it worked correctly.

I have also been exposed to applications for which this kind of availability-trumps-durability paradigm is absolutely unacceptable. The advertising system upon which the advertising-supported website relies for its income is a good example. Users know they can build sites that only need to be available for reads, precisely because they are trusting that Google AdSense is highly available for writes! Delegating writes to someone else is the easiest way to build systems.

There is a place for DRBD and MySQL Cluster, and there are also many situations that are served by neither the DRBD nor the MMM type of solution.

Josh Berkus wrote a while back about three types of cluster users, as opposed to three types of clusters. I think it’s helpful to approach the conversation from that angle sometimes too. As a consultant, I almost always do that when I enter a discussion with a customer who wants a “cluster” or “high availability.” Those are basically code phrases that tell me I need to start at the beginning and ensure we are all talking about the same requirements!

I also agree with Henrik about the need to turn off automatic failover. In many, many situations this is by far the best approach. Sometimes people state requirements that, if one steps back and looks at them afresh, quite obviously indicate that an automatic failover is the last thing that’s desirable. For example, if someone tells me that he expects failover to be required less than once a year, this is almost guaranteed not to be a good case for automatic failover. A system that’s tested so infrequently is almost certainly not going to work right when it’s needed. In such cases, it’s far better to leave everything alone until an expert human can resolve the problem, rather than have a stupid machine destroy what would otherwise be a fixable system.

Further Reading:

Josh Berkus helps clarify clustering

Xaprb — Sat, 21 Nov 2009 13:57:51 +0000

If you haven’t seen it, Josh Berkus has a very concise way to look at the confusing mess that is database “clustering” from the point of view of three distinct types of users: transactional, analytic, and online. I think that using this kind of distinction could help keep discussions clear — I’ve seen a lot of conversations around clustering run off the rails due to disagreements about what clustering means. MySQL Cluster, for example, is a huge red herring for a lot of people, but it seems to be a difficult process to learn it well enough to decide. If we called it a clustering solution for transactional users, but not for analytic or online users, it might help a lot.

Further Reading:

MySQL 6.2 is GA, but 5.1 is RC and 6.0 is alpha

Xaprb — Sat, 24 May 2008 02:26:20 +0000

MySQL’s version numbering is getting harder and harder to understand. In fact, it’s getting surreal.

Let me state up front that there’s probably a lot I don’t know here. But if I don’t know, how on earth can the general public figure it out?

Before we begin, let’s define terms: GA is completely done, ready for use. RC is a release candidate: don’t change anything, just fix bugs because we’re charging towards a release here. Beta is possibly unsafe code, use at your own risk. Alpha is known to have significant bugs, but if you’re curious please play with it.

Now for the releases/versions game. Let’s recap:

5.0 has version numbers that leapfrog each other in features and functionality. SHOW PROFILES — now you see it, now you don’t.
5.1 has been “… released to general availability [as] a near-final release candidate,” whatever that means.
5.1 has just had drastic changes in the RC stage. (Remove Federated in 5.1.24, remove RENAME DATABASE, remove Cluster.) And it’s going to have more changes before it’s released, too: Federated will be added back in 5.1.25.
5.2 doesn’t exist. Last year at the MySQL conference, someone made an abrupt decision to skip 5.2 and inflate the version numbers to 6.0, which has big changes in the query optimizer and other areas.
6.0 is alpha, but it includes Falcon, which is beta even though Falcon has extremely bad bugs that its developers claim are not bugs.
6.1 doesn’t exist as far as I know.
6.2 not only exists, but it is GA. Not only that, but it just… appeared as GA, as far as I know. No RC stage, no nothing — at least, nothing on the MySQL website that I see (certainly no manual version). It went from nonexistent to GA instantaneously as far as I know. It was created by extracting the Cluster code from 5.1.
6.2 is GA, but 5.1 is RC.
6.2 is GA, but 6.1 doesn’t exist as far as I know.
6.2 is GA, but 6.0 is alpha. (Hopefully you see the pattern here.)
6.2 is GA, but presumably does not include the changes made in 6.0, since it was derived from 5.1′s code.

What is going on here?

How is this an improved release model? What is improved about this?

How in the world can anyone figure out what versions of the software have what features? Who can make an educated decision about what product to use in this situation? Are people supposed to just rely on the sales people to help them figure out what to use? Boy, is that trusting the fox to guard the henhouse.

Why didn’t they just release 5.1 Cluster as GA separately, if that reflected the reality in the code? They certainly missed an opportunity to show some progress on 5.1. As it is, 5.1 got robbed of its chance to have at least some of its code go GA after more than 2.5 years in development. Now 5.1 looks like even more of an embarrassment — hey 5.1 team, how come you can’t get anything out the door when these 6.2 people are releasing GA products? Not to mention 6.0 — you guys look bad now too! (Just kidding.)

I tried to draw a timeline of MySQL’s release history, in some detail in the 5.0 history and in very basic detail in the 5.1 and 6.0 and 6.2 trees. You can take a look at that. It’s worth studying for 5 minutes or so, even though it’s kind of ugly. There are lots of oddities to notice about it. Enjoy:

The inmates are running the asylum. This gets more and more amusing as time goes on.

Further Reading:

Stock images are too popular

Xaprb — Fri, 28 Mar 2008 03:07:16 +0000

I have an ingrained (possibly even genetic) aversion to stock images. Actually, not all stock: just the vacuous kind. You know what I mean: like the politically-correct, gender-balanced, racially-balanced, age-diverse ones where people are all smiling and pointing at a computer screen you can’t see. Ugh!

(Photo credit: istockphoto.com)

There are many reasons not to use images like this. I guess it’s okay in some situations — for example when you just want a smiling, attractive woman with a customer-service headset to reinforce that you’ve come to the right place for support. However, even these really don’t have to be stock images. One of my former employers used their own employees for such photos, almost exclusively, and it made the site much more real. And there are plenty of examples of companies that use photos of their own employees and get “realness” as a result. If I’m not mistaken, Title Nine does so except for certain things, such as underwear models (for obvious reasons).

However, one great reason to eschew stock: other people will re-use the same image. A famous example from a few years ago: the cover image of Head First Design Patterns was a stock photo that also appeared in a commercial for a feminine hygiene product.

This incident was actually pretty widely linked on the Internet at that time. So no one will ever make that mistake again!

Or will they? Witness: the cover of the MySQL 5.1 Cluster DBA Certification Guide, the xTuple Home Page, and the cover of the MegaRAID Management Suite documentation.

Interestingly, I ran across all three over-usages of this image in one day, completely by accident. Are there other places this image is used? I’d bet there are.

Who cares? Well, the images that go on the cover of your book, your brochure, or your website become part of your image. If someone else then uses the same image, they can (accidentally or otherwise) exert some control over what people think of your product or company.

If this matters — and it almost certainly does — you should just get some of your own employees, hire a good photographer, and go into your own server room (or beg a friend to let you into theirs) for a photo.

On the subject of image, I’ve just gone to a photographer for some new portraits of myself, and I’m also hiring someone to design a logo for Maatkit (for a new website, and for t-shirts to give away at the upcoming conference). I’ll post more about that later.

Further Reading: