Xaprb

A while ago I posted about a comment a Sun performance engineer made about a scalable replacement for InnoDB. At the time, I did not believe it referred to Falcon. In hindsight, it seems even clearer that the Sun performance experts were already working hard on InnoDB itself.

Sun’s engineers have shown that they can produce great results when they really take the problems seriously. And I’m sure that InnoDB’s performance has untapped potential we don’t see right now. However, it does not follow that their work on InnoDB is what was meant by a scalable replacement for InnoDB. Or does it?

General-purpose MVCC transactional storage engines with row-level locking, whatever their performance and scaling characteristics in edge cases, fall into a category together. A person assembling a MySQL server for general-purpose use might choose a different storage engine for various uses — MyISAM here, Memory there… and use “one of those transactional engines” for the bulk of the work. PBXT, InnoDB, Falcon — I don’t see a justification for running more than one of those side by side. The operational costs alone (backups, training the users, etc) would be too high. It is also not at all clear that MySQL itself is ready for multiple transactional storage engines working together (e.g. cross-engine transactions) in the real world.

So what’s left for Falcon? I think they are asking themselves the same question (brilliant gallows humor, by the way). I think Falcon’s ideas and techniques are very interesting, but a storage engine — especially one with such lofty goals — is always a show-me undertaking that will require years to mature and prove itself even after the code is “ready.” With or without the Oracle acquisition, this question has loomed for years: where’s the justification for Falcon politically, functionally, economically? A third party engine such as PBXT, with eyes on replication at the storage engine level and other add-on functionality, has always seemed more likely to really add value than a straight-up InnoDB replacement.

But from my point of view, the biggest win in the short term would still be to drive InnoDB development forward at a consistent and accelerating pace to meet the needs of users and the advances in hardware. Of course, that’s what XtraDB set out to do, and I think the XtraDB project has helped snap InnoDB out of their Percheron-like plod towards improvement. This is nothing but good; when it comes to competition among storage engines, no one should be resting on their laurels. I also see that Sun’s team has more good things in the works, which is great. I’d love for InnoDB to stop being a work horse and start being a quarter horse. We need it to be both scalable and high-performance.

A while back a Sun engineer posted an article claiming that the best way to scale MySQL is to shard your database in many instances on a single server, each of which runs in threads that individually have low performance. The Sun way has always been to get high throughput with high latency. And that’s fine. Others have commented on the real-world applicability of this technique with MySQL, so I won’t.

But what I’m interested in is something the author says in the comments of his own blog post:

btw; we are working on a scalable replacement of InnoDB. Stay tuned….

What is it?

Surely it’s not Falcon. MySQL and Sun have said many times Falcon isn’t meant to be an InnoDB replacement.

The next question: how should we interpret the word “scalable” given the context? Clearly there’s a difference of opinion between MrBenchmark and others on what that word means. So what will the scaling characteristics of this InnoDB replacement really be?

The following is a joke, not meant to be taken seriously: Maybe it will be MyISAM with a fix to the key buffer scalability problems. Start with your database with 50 tables. Make 1 million databases, each with the same 50 tables, and you can scale up to 1 million rows by putting 1 row in each table. Ta-da! Row-level locking with MyISAM!

But seriously, what is it?

I don’t think I have heard anything from the Falcon team for a while. What’s new? Did the project really stall when Jim Starkey left, as Vadim Tkachenko wondered might happen?