Xaprb » Andrew Aksyonoff

A review of Introduction to Search with Sphinx by Andrew Aksyonoff

Xaprb — Sun, 28 Aug 2011 21:06:54 +0000

Introduction to Search with Sphinx

Introduction to Search with Sphinx by Andrew Aksyonoff, O’Reilly Media 2011. About 146 pages. (Here’s a link to the publisher’s page.)

This is an engaging short introduction to Sphinx. At 146 pages, you shouldn’t expect it to go into every detail, and it doesn’t. There are major topics that it omits entirely or mentions only tangentially, such as distributed searching across a cluster of machines, real-time updating of indexes and attributes, and so on. But although the book doesn’t boil the ocean, it does a great job at covering an introductory subset of Sphinx. It’s just the book you need to get rid of MyISAM full-text indexes and return MySQL to proper performance again.

This book is about how to get started with Sphinx: getting it up and running, basic configuration, creating and maintaining indexes, and querying with the SphinxAPI and SphinxQL languages. It covers relevance and ranking, so you can understand how Sphinx is different from most relevance engines and why it produces better results.

If you haven’t explored Sphinx, you should start with this book. If you know Sphinx, it might be redundant to you.

Either way, it’s a fun book to read. Andrew is a great writer with a charming sense of geeky humor, and he makes a lot of jokes in his examples. These don’t interfere with the learning, but they do make it a lot less dull. (It isn’t forced or overdone, either, unlike a lot of books that have to make constant Beatles references at every opportunity.)

Further Reading:

Did you know Sphinx can act like a MySQL server?

Xaprb — Wed, 29 Apr 2009 03:15:46 +0000

Peter wrote about this recently, but I don’t know if it was really clear what was going on.

Point One: Sphinx can be contacted by the MySQL protocol. Not “as a MySQL storage engine.” Not “from MySQL.” It understands the MySQL protocol itself. So from the protocol point of view, the Sphinx search daemon can look just like a MySQL server.

Point Two: Sphinx understands a SQL-like query language. Don’t be fooled. You’re not writing SQL. It just looks like you are.

Point Three: Because of point One and point Two, you can use the mysql command-line client program to talk directly to Sphinx, with absolutely no MySQL server anywhere in sight. This also means you can connect to Sphinx from your application and query it, exactly like connecting to a MySQL server and querying it.

Go take a look at Peter’s blog post. He’s not writing MySQL queries. He’s writing queries to Sphinx.

Now think about how cool this is — how easy this is to integrate with your code that already communicates with MySQL. Is there any other external full-text search system that masquerades as a MySQL server? I don’t know of one.

Further Reading:

Sessions of interest at the Percona Performance Conference

Xaprb — Sat, 18 Apr 2009 00:48:02 +0000

Having written about what I think is cool about the upcoming MySQL Conference and the MySQL Camp, now I want to finish up with what I’d like to see at the Percona Performance Conference. Just to recap, this is a conference we created to serve those who want to learn about performance — not “learn about MySQL,” not “learn about database performance,” just learn about performance, period.

I want to see everything. I think this is going to be the single best conference I’ve ever been to. Even the way the conference is organized is exciting. For example, it’s running from early morning till late at night, nonstop. The sessions are also (mostly) only 25 minutes. This means if you decide a session isn’t all that interesting, you didn’t spend much time on it, and you don’t have long to wait for the next one.

So here is a small sample of the sessions:

CouchDB: Behind the Buzz (Jan Lehnardt)
Performance Instrumentation: Beyond What You Do Now (Cary Millsap)
Hive: Distributed Data Warehousing with Hadoop (Ashish Thusoo and Prasad Chakka, Facebook)
High Performance Erlang (Jan Henry Nystrom)

These are not just people who’ve learned about something and want to talk at you. These are the inventors, the originators, the gurus. It is truly the who’s who, and that’s just a few of them. If you aren’t familiar with those names, Google them and see. And after that, why not Google Theo Schlossnagle, Eric Burton, Monty Widenius, Andrew Aksyonoff, and a few others.

I hope to see you there. Bring your business cards and introduce yourself to me!

Further Reading:

Sphinx 0.9.8 is released!

Xaprb — Wed, 16 Jul 2008 00:40:40 +0000

The Sphinx project just released version 0.9.8, with many enhancements since the previous release. There’s never been a better time to try it out. It’s really cool technology.

What is Sphinx? Glad you asked. It’s fast, efficient, scalable, relevant full-text searching and a heck of a lot more. In fact, Sphinx complements MySQL for a lot of non-search queries that MySQL frankly isn’t very good at, including WHERE clauses on low-selectivity columns, ORDER BY with a LIMIT and OFFSET, and GROUP BY. A lot of you are probably running fairly simple queries with these constructs and getting really bad performance in MySQL. I see it a lot when I’m working with clients, and there’s often not much room for optimization. Sphinx can execute a subset of such queries very efficiently, due to its smart I/O algorithms and the way it uses memory. By “subset” I mean you don’t get the full complexity of SQL, but you get enough functionality for lots of the poorly-performing queries I see in the wild. It’s a 95% solution.

Is Sphinx for you? Good question. You can find answers in Appendix C in High Performance MySQL. And yes, that is why I wrote this blog post — to put in a plug for the book. *grin* But before I go, let me put in another plug for Sphinx: go vote for it on Sourceforge! If it’s voted as one of the Community Choice projects of the year, that will be fantastic.

Further Reading:

Progress on High Performance MySQL, Second Edition

Xaprb — Thu, 29 Nov 2007 20:18:05 +0000

It’s been a while since I said anything about the progress on the book. That doesn’t mean we are not still working on it, though.

As Peter wrote a while ago, he is basically wearing the hat of a very advanced technical reviewer at this point. We’ve finished writing all the chapters from his detailed outlines. He has worked through about half the chapters, and I’m continuing to spend my evenings and weekends and holidays (yes, nearly all my free time — just ask my wife!) writing some new material (an appendix on EXPLAIN, for example), finishing unfinished things marked with TODO in the text, and revising chapters after Peter reviews them. Vadim is working on benchmarks. For example, he just finished some benchmarks for something I profiled with SHOW STATUS. I thought that would be good enough to assert something about the performance. Sure enough, SHOW STATUS says it does less work, but Vadim’s benchmarks show it’s slower :-) This is why we check each other’s work!

The core chapters on MySQL performance — beginning with Benchmarking and Profiling, and continuing through Optimizing Server Settings — are the ones Andy Oram, our editor, thinks we should put the most effort into, and I agree. We will probably circle back and go through another review/edit cycle before we release them for technical review. Some of the other chapters, such as Replication, are already out for technical review.

Despite the fact that all of the chapters and appendixes are theoretically a “first draft,” as of several weeks ago, there is still a lot of work to do. Depending on the chapter, it takes me a solid weekend to revise a chapter after Peter reviews it. Each little thing anyone points out (does MySQL version X really do Y by default?) requires some research, testing, benchmarks, or even reading the source code.

Some miscellanea:

The production staff replied to my inquiry to the editor to say that yes, we will be able to have references that point to a specific page number. This was a big relief to me. It requires extra work, but makes the book so much more valuable as a reference work in my opinion. To see why, look at the top of page 151 in the first section, which just refers to chapters and sections by their titles: “See… the “Tools” section…” Now try to find the “Tools” section. If it took you a while… well, the first time I did it, I missed it, and thought it might mean the Tools Chapter. The second edition will say “The X section on page Y” or similar. (Okay, I’ll shut up about this now — everyone has to have a pet peeve, eh?)
We are currently at 425 pages in OpenOffice.org Writer, which by my calculations puts us around 470 pages in print. As I said before, I think we’ll break 500 pages by the time we finish the rest of the missing material.
Andrew Aksyonoff has contributed an appendix on the Sphinx full-text search system. If you don’t know anything about it, check it out. It’s an amazing piece of software that does a lot more than just full-text search.

Well, I’ve run out of my allotted thirty minutes of blogging! Back to the salt mines! Just kidding… I’m actually off to the climbing gym soon to get my mind off it.

Further Reading: