Comments on: Why large IN clauses are problematic

By: Venkat

Venkat — Thu, 19 Nov 2009 11:50:58 +0000

yes your article is good enough for optimization but i want to know wheather we make in Clause as Fast enough with creating appropriate indexes

By: Xaprb

Xaprb — Mon, 16 Jun 2008 16:05:36 +0000

I think I wrote this post when I was seeing a particular problem, and before I’d started to see the problems with temp tables. So my dogma switched as I learned :-)

By: Dewey

Dewey — Mon, 16 Jun 2008 14:36:17 +0000

Baron….I thought you were opposed to using temp tables? I love them (especially memory) as a method to get variable data from the app layer with no schema dependency. I’d love to see a post from you regarding when you find them risky, and when this approach is ok.

By: Ben Truitt

Ben Truitt — Wed, 04 Jun 2008 21:57:52 +0000

I just wanted to share additional news on this.

My DBA had a really hard time believing my solution in the first email in this thread should be faster. So after digging some, he ran an ANALYZE on the tables in question.

This helps the PostgreSQL planner out to create more optimal query plans.

It turns out that after analyze the original query that uses an “IN” takes 1ms.

So we went from:
IN Clause: ~4500ms
Inner Join: ~300ms
Analyze with IN Clause: ~1ms

Holy cow.

The lesson is: use explain, use analyze, and consult your DBAs. They are your friends.

By: Ben Truitt

Ben Truitt — Wed, 04 Jun 2008 16:13:42 +0000

In postgreSQL, the suggestion to use inner joins can result in dramatic improvements, because postgreSQL does use ORs as described and proved above.

In our case, we went from ~4500ms for the query to ~300ms

An order of magnitude improvement!

Thanks for the great post!!