¹ This one’s for Jay.

Related posts:

1. What are your favorite MySQL replication filtering rules?
2. My new favorite comic: The Adventures of Ace, DBA
3. What is your favorite database design book?
4. What are your favorite PostgreSQL performance resources?
5. My favorite wiki is Dokuwiki

Related posts:

1. Get a free sample chapter of High Performance MySQL Second Edition
2. MySQL: Free Software but not Open Source
3. Speaking at NovaRUG on Thursday
4. High Performance MySQL is going to press, again
5. Coming soon: High Performance MySQL, Second Edition

Related posts:

1. MySQL Enterprise/Community split could be renewed under Oracle
2. 50 things to know before migrating Oracle to MySQL
3. Migrating US Government applications from Oracle to MySQL
4. A review of Forecasting Oracle Performance by Craig Shallahamer
5. A review of Optimizing Oracle Performance by Cary Millsap

Related posts:

1. Free webinar on MySQL performance this Thursday
2. Speaking at MySQL Meetup in Northern Virginia
3. Speaking at EdUI Conference 2009
4. Speaking at Surge 2010
5. Speaking at CPOSC 2009

Let’s dive right in and analyze these subtle bugs. Warning: if you don’t understand how SQL handles NULL, you’re not going to understand the following. Many people have a hard time with NULL, which is why these bugs are so hard to understand and avoid. This is one reason why SQL is a hard language to use properly.

Bug 1: a column could be NULL for two reasons, and you can’t distinguish them

If the outer table in your query contains NULL-able columns, and you place a WHERE clause to filter out all but those rows, you’re going to get bugs because a non-matching row in the outer table will be all-NULL. Here’s an example. Let’s start with a plain outer join query:

select * from L left join R on l_id = r_id;
+------+------+---------+
| l_id | r_id | r_other |
+------+------+---------+
|    1 |    1 |       5 | 
|    2 |    2 |    NULL | 
|    3 | NULL |    NULL | 
+------+------+---------+

Here we see that one row in the outer table is missing, and one row (the middle row) has a NULL r_other column. Now, let’s add a WHERE clause:

select * from L left join R on l_id = r_id where r_other is null;
+------+------+---------+
| l_id | r_id | r_other |
+------+------+---------+
|    2 |    2 |    NULL | 
|    3 | NULL |    NULL | 
+------+------+---------+

This query is buggy, because the two rows are returned for completely different reasons, and you can’t be sure which is which. IS NULL clauses can safely be placed on the columns used in the JOIN clause, but not on other columns in the outer table that might be NULL.

Bug 2: an OUTER JOIN is converted to INNER

If you place a non-null-safe comparison operator on any column in the outer table that isn’t part of the JOIN clause, you implicitly disable the outer-ness of the query and convert it to an INNER JOIN. Here’s an example:

select * from L left join R on l_id = r_id where r_other > 1;
+------+------+---------+
| l_id | r_id | r_other |
+------+------+---------+
|    1 |    1 |       5 | 
+------+------+---------+

The left-outer-ness of the above query is what causes the third row to be output in the first query I showed you above. The greater-than operator in this example automatically makes the left-ness impossible, because anytime there’s a row in the inner table that has no match in the outer table, it’ll be filled in with NULLs, and those NULLs will be eliminated by the operator. So the effect is that only matching rows will ever be output.

If you want to ponder variations and subtleties of the above, you can read more discussion on the issue report where we’re hammering out the details of automatically detecting and warning about these sneaky errors.

Related posts:

1. How to simulate FULL OUTER JOIN in MySQL
2. How to write a SQL exclusion join
3. How to write SQL JOIN clauses more compactly
4. The dangerous subtleties of LEFT JOIN and COUNT() in SQL
5. How to write INSERT IF NOT EXISTS queries in standard SQL