SQL Server does have some of the easiest to use tools for things like finding what indexes people should use, something they threw some serious research work into. So their claims of “self-tuning” have at least a bit of technical merit behind the marketing.

PostgreSQL does a bit better without any tuning than some databases simply because it does expect and utilize the operating system buffer cache to be functional. It doesn’t use direct I/O or sync writes in the default setup to try and bypass the OS buffer cache. How well this works depends on the OS. If you’re using Solaris+UFS where that’s going to end up capped at 56MB of buffer cache unless you tune that, using the defaults is going to give you terrible PostgreSQL results. But on Linux or ZFS where disk caching is very aggressive out of the box, PostgreSQL can do just fine at its default settings sometimes.

The simple PostgreSQL tuning wizard program I developed, pgtune, didn’t get anywhere until we decided that it was OK to split the types of workloads it could be expected to work out on and specifically biased toward and make those separate targets (Josh Berkus’s idea, just to credit properly). There’s ones for web apps, OLTP, DW, an “I’m not sure” mixed setting, and one for small desktops that’s still a bit better than the out of the box config.

@Slapo: The only PostgreSQL tuning guide that has enough community members working on it to never fall out of date since its introduction is http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server , which averages an update about every three weeks. It will be updated to cover 9.0 before that version is even released.

http://www.mysqlperformanceblog.com/
http://dimitrik.free.fr/
http://www.westnet.com/~gsmith/

Everyone else is just out to prove something :-D

No single configuration can be optimal for all work loads. So giving the DBA tools (i.e. configuration settings) to say “this is the work load for which I want to be optimized” and allowing them to deliberately sacrifice performance on the occasional odd query to work better on the expected load can be very helpful. The server won’t be able to fall into the reaction trap and have to constantly adjust to the thrashing the settings. OTOH the DBA has to know or easily be able to find out what each setting will actually do and what work loads will benefit from tuning. As Slapo points out the documentation on what each setting does needs to get better and keep more up to date — on ALL of the products. Your recent post on sort_buffer_size demonstrates that. In lieu of better information, people will think “Gee, I do lots of order bys. A bigger sort buffer must mean faster sorting. I think I’ll tune it…” to their detriment (and yes I did check our sort_buffer_size settings, and no we weren’t tuned up). Letting a system self-tune can be better in many cases than ignorant manual tuning.

But to answer your question, if your marketing department came up with the claim, you point out that it was a specialized work load that doesn’t reflect real world usage for which your product is optimized. If the benchmarker or somebody else came up with it, you point out that they are obviously idiots who have no idea how to setup your product and besides, it was a specialized work load that doesn’t reflect real world usage for which your product is optimized with the default settings. Benchmarks are only good if they show your products are the best.