Xaprb

Stay curious!

More details about SchoonerSQL performance, please!

with 9 comments

Schooner has a blog post showing that one node of their product beats 9 nodes of Clustrix’s in throughput. But this reduces everything to a single number, and that’s not everything that matters. If you’ve looked at Vadim’s white paper about Clustrix’s (paid-for) performance evaluation with Percona, you see there is a lot of detail about how consistent the throughput and response time are.

I’d love to see that level of details in any product comparison. A single number often isn’t enough to judge how good the performance is — fast is not the only thing that matters.

I have absolutely no doubts that a single node of Schooner’s product can run like a deer. It isn’t doing any cross-node communication, after all, so it had better be faster than something that blends multiple nodes together into a virtual “single database server.” And I think if the full story were told, it would be a great knock-down drag-out fight. Give us more details, Schooner!

Further Reading:

Written by Xaprb

February 2nd, 2012 at 4:37 pm

Posted in SQL

9 Responses to 'More details about SchoonerSQL performance, please!'

Subscribe to comments with RSS

  1. It should include all the necessary information needed to repeat the tests independently.

    My $.02
    G

    Gerry

    2 Feb 12 at 5:20 pm

  2. As I said, I have no doubt that SchoonerSQL is going to beat Clustrix in a raw performance comparison, node for node. I don’t dispute the results, I just want to know more about them.

    Xaprb

    2 Feb 12 at 6:15 pm

  3. Baron,

    At the same time Schooner has published the benchmark
    with the result where async MySQL replication is slower than semi-sync MySQL replication.

    So there is doubt about credibility of their methodology.

    VadimTk

    2 Feb 12 at 7:41 pm

  4. There’s not enough information to doubt :)

    Xaprb

    2 Feb 12 at 9:31 pm

  5. Hello Vadim,
    Long time no chat :)

    We measured MySQL v5.5 and shared results at the April 2011 MySQL User Conference comparing async, semi-sync and Schooner-sync replication between two identical nodes in Master/Slave configuration. For MySQL 5.5 async and semi-sync runs, we ensured slave lag ~< 1 sec by tweaking DBT2 think-time. If this were not done, Slave can get hours behind (slave lag) for every hour of benchmark run.

    I believe you are referring to the results in the slides: http://www.slideshare.net/darpandinker/performance-comparisons-and-trade-offs-for-various-my-sql-replication-schemes-presentation

    Notice the choppy throughput with async Master and Slave vs more stable throughput with semi-sync for MySQL 5.5. I was hoping you have similar runs and can explain it if you see the same.

    If you think MySQL 5.5 async vs semi-sync data doesn't look right, we can run it again for you and share the results. Given the same benchmark conditions, I would still expect to see 4X throughput gains with SchoonerSQL in reruns.

    Darpan Dinker

    3 Feb 12 at 2:50 am

  6. Hello Baron,

    No question Percona did a thorough job writing the white-paper for Clustrix. Having done performance benchmarking, I can surely understand the time it would have taken to execute all those runs, validating data (rerunning benchmarks if necessary) and then plotting all the results.

    I think its easy to get carried away with the title of Pavan’s blog. 9 Clustrix nodes with 48GB DRAM each = 432GB, incidentally the size of the 5000 warehouse tpcc-mysql database. The core question is if the scalability of Clustrix efficient? I picked one data-point to pursue that question. To make that point, I blogged about it here: http://darpanetwork.blogspot.com/2012/01/how-scalable-is-clustrix-scalability.html
    and then put in tpcc-mysql details:
    http://darpanetwork.blogspot.com/2012/01/schoonersql-throughput-with-perconas.html

    BTW I did want to ensure stable environment and results, so I did have a 4 hour long run.

    If you provide a cheat-sheet, in addition to the CPU util chart already posted, I will add throughput and latency charts for the run.

    So let me ask this: lets say you validated my results. Will you ask the question: for what workloads and price/performance will a Clustrix or similar building-block (with a fraction of throughput capability) be better suited for?

    If someone pays me, I will love to do 128 runs for comparison – any takers? ;-)

    Darpan Dinker

    3 Feb 12 at 3:23 am

  7. I remember some years back that SolidDB (now discontinued) published a benchmark ‘proving’ that it handled many times more transactions per second than InnoDB. It could not be verified independently unless a specific testcase was used.

    So I can only agree that all details (configurations, data, queries/scripts) should be available. As long as this is not the case any postulate is as good as another.

    Peter Laursen

    3 Feb 12 at 4:49 am

  8. Peter,
    Good point.

    Let’s see if I can move you along this process:
    Doubt -> Verify -> Trust.

    Darpan

    Darpan Dinker

    3 Feb 12 at 10:29 am

  9. Working with Baron and Vadim (thanks), I was able to decipher the output of tpcc-mysql (it can be definitely improved :) ).

    I have updated my blog with a throughput charts – for folks who want to look at the stability of the same SchoonerSQL run:

    http://darpanetwork.blogspot.com/2012/01/schoonersql-throughput-with-perconas.html

    Yes, I will add more details after understanding the output some more with Vadim’s help.

    Darpan Dinker

    3 Feb 12 at 6:07 pm

Leave a Reply