Comments on: Making changes to many tables at once

By: Mark Callaghan

Mark Callaghan — Sun, 21 Feb 2010 17:25:05 +0000

I just noticed queries like this in the slow query log. I use 5.0, so the IS query optimization chapter don’t apply. This server has ~10,000 tables, so I assume this query requires ~10,000 open table handler instances.

# Query_time: 6 Lock_time: 0 Rows_sent: 313 Rows_examined: 313
use foobar;
select TABLE_SCHEMA,TABLE_NAME,AVG_ROW_LENGTH,DATA_LENGTH,
TABLE_ROWS,INDEX_LENGTH
from information_schema.tables
where table_schema=database();

By: Xaprb

Xaprb — Fri, 19 Feb 2010 02:47:43 +0000

In my opinion SHOW TABLES is a problem when the filesystem starts to have a problem, which is fs-dependent but I don’t think 10k is a huge problem. SHOW TABLE STATUS depends both on the number of tables and the size of data, in case InnoDB is used and ::info() is called and statistics are re-sampled (which in my experience they usually seem to be, but I haven’t tried to prove that). Most of what I’ve said on this topic is centered around InnoDB. I noticed that the manual page about I_S optimizations was not overly InnoDB-conscious.

By: Xaprb

Xaprb — Fri, 19 Feb 2010 02:39:43 +0000

I just read the link Sheeri listed. I’m dismayed, too. I wish the data dictionary were stored in InnoDB. I know, Drizzle.

By: Mark Callaghan

Mark Callaghan — Fri, 19 Feb 2010 02:36:39 +0000

At what number of tables does this become a problem? 10,000? 100,000? 1,000,000?

By: Xaprb

Xaprb — Fri, 19 Feb 2010 02:33:09 +0000

SHOW TABLES is not a huge source of trouble in most cases, because it basically lists the directory and trims off .frm and shows the result. That is slow in very few cases. What’s incredibly slow on an increasing number of servers these days is SHOW TABLE STATUS, which lists the directory, iterates the .frm files, creates a handler for each table, and calls ::info() on each handler. And that is also what INFORMATION_SCHEMA.TABLES does — not the same thing as SHOW TABLES.