Automated, integrated sharding — the new killer database feature
MySQL became wildly successful in part because it had built-in, simple replication. Sure, it had lots of interesting failure scenarios and was not great at first — it is much better these days — but it was nevertheless successful because there was a single, out-of-the-box, not-very-complex way to do replication. I have opined many times before that this was one of the killer features missing from PostgreSQL. I think that can large explain why MySQL became more popular more quickly.
The new killer feature is automatic sharding, in my opinion. If you’re not accustomed to the word, “sharding” means partitioning of a large dataset across many servers.
It is easy to poke fun at MongoDB’s current limitations, but for all that, it has a story to tell about sharding. There is One Right Way To Do It in MongoDB, and it’s a part of the product.
I don’t see sharding being added into the core of MySQL itself, but there are some very interesting efforts headed towards MySQL. There are at least the following companies providing sharding via a proxy or middleware solution, with a lot of other features also available in some products:
In addition, there are community-based efforts, such as Shard-Query and the Spider storage engine. And there’s MySQL (NDB) Cluster, and commercial rip-out-and-plug-in replacements for MySQL such as Clustrix.
Am I missing any? I probably am. You can see and talk to many of these companies at this week’s MySQL conference, by the way.
I missed a big one: Vitess http://code.google.com/p/vitess/
Xaprb
11 Apr 12 at 7:42 pm
I was going to say Vitess.
With all respect to closed source solutions, they have no chance of becoming the “one true” MySQL solution, it seems like they just have a business opportunity to fill the gap until we find the final solution.
In my book Spider and Vitesse will battle it out for becoming the “out of the box” solution. Maybe something for XtraDB Cluster 2.0? You have A good Spider expert in LeFred in house.
Oh no, now I have to guess whether a lion is a cat again. I have less than 50% success rate on that question…
Henrik Ingo
12 Apr 12 at 1:08 pm
Yep. Failed again :-)
Henrik Ingo
12 Apr 12 at 1:08 pm
Hi,
Please could you elaborate on how “easy” it would be to poke fun at MongoDB’s limitations on Sharding.
For those that aren’t accustomed to the term, what might be the weak aspects of it’s sharding implementation?
Many thanks
Rob
Rob Shepherd
15 Apr 12 at 7:47 am
You should re-read what I wrote.
Xaprb
15 Apr 12 at 9:37 am
Ah, got it.
You’re statement is, in other words: Mongo db has limitations ‘in some areas’. But nonetheless it has at least some built in sharding functionality with a prescribed way of using it.
My eyes must have scanned ahead and joined up the sentence incorrectly, apologies.
It’s hard enough to form these inquiries in an objective-enough fashion without it coming across as flame-bait; I should at least read the initial text properly! :)
Cheers, keep up the good blogging – I’m a long time reader.
Rob Shepherd
15 Apr 12 at 1:00 pm
Rob, no worries — my terse reply was testing to see if you were trolling… I didn’t want to engage if so ;-)
There’s another one I didn’t mention: Zimory Scale.
Xaprb
17 Apr 12 at 11:36 am
Baron – in the Web Operations book, sharding was at the top of your list in terms of risky architectures. Just curious if you’ve changed your opinion on that since publication of that book.
Thanks,
-Ed
Ed
25 Apr 12 at 8:21 pm
If you must build a system that is too large to fit onto a single database server, you must partition the data somehow. I just don’t agree with the (once-common, seemingly more balanced recently) advice that you must shard everything from day one.
Xaprb
26 Apr 12 at 8:15 am