Comments on: How to calculate table checksums in MySQL http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/ Stay curious! Fri, 16 May 2008 03:49:07 +0000 http://wordpress.org/?v=2.2.2 By: Xaprb http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-14525 Xaprb Thu, 08 May 2008 11:57:44 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-14525 Useful indeed. See Maatkit -- its mk-table-checksum tool does this :) Useful indeed. See Maatkit — its mk-table-checksum tool does this :)

]]> By: Joachim Zobel http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-14524 Joachim Zobel Thu, 08 May 2008 11:41:12 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-14524 I have a useful trick to add. You can INSERT the checksum into a table so that the checksum statement gets replicated. The trick is to then additionally OUTFILE and INFILE the checksum. This way the masters checksum gets replicated to the slaves so it can be easily compared to the local value. I have a useful trick to add. You can INSERT the checksum into a table so that the checksum statement gets replicated. The trick is to then additionally OUTFILE and INFILE the checksum. This way the masters checksum gets replicated to the slaves so it can be easily compared to the local value.

]]> By: Xaprb http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11618 Xaprb Fri, 15 Jun 2007 13:39:46 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11618 The best thing to do is read the embedded Perl documentation, which becomes a man page if you install the tool, so 'man mysql-table-checksum'. If there is anything missing from the documentation please submit a bug report, as I want to make it complete and useful. The best thing to do is read the embedded Perl documentation, which becomes a man page if you install the tool, so ‘man mysql-table-checksum’. If there is anything missing from the documentation please submit a bug report, as I want to make it complete and useful.

]]> By: Jok http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11608 Jok Fri, 15 Jun 2007 08:13:02 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11608 Hi Xaprb, Thank you for your quick answer. Could you be more specific about the tool (mysql-table-checksum) I should use in my case and the way to manipulate it ? Hi Xaprb,

Thank you for your quick answer.
Could you be more specific about the tool (mysql-table-checksum) I should use in my case and the way to manipulate it ?

]]> By: Xaprb http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11589 Xaprb Thu, 14 Jun 2007 20:29:26 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11589 Yes, and I have now developed this technique into a very powerful tool to do so. Look on Sourceforge for <a href="http://sourceforge.net/projects/mysqltoolkit/" rel="nofollow">MySQL Table Checksum</a>.</p> Yes, and I have now developed this technique into a very powerful tool to do so. Look on Sourceforge for MySQL Table Checksum.

]]> By: Jok http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11587 Jok Thu, 14 Jun 2007 16:59:54 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-11587 Hi, I think I have a very similar topic. Have you ever tryed to sign a full table in order to certify it has not been changed ? I work on Mysql 5.0. If you have suggestion to do so... Thx in advance. Jok Hi,

I think I have a very similar topic.
Have you ever tryed to sign a full table in order to certify it has not been changed ?

I work on Mysql 5.0. If you have suggestion to do so…

Thx in advance.

Jok

]]> By: Xaprb http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-6820 Xaprb Tue, 15 May 2007 15:16:34 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-6820 <p>Hi qinyf, I have added this feature to <a href="http://sourceforge.net/projects/mysqltoolkit" rel="nofollow">MySQL Table Checksum</a>. I have not released it, but you can get the code from Subversion.</p> Hi qinyf, I have added this feature to MySQL Table Checksum. I have not released it, but you can get the code from Subversion.

]]> By: qinyf http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-6493 qinyf Fri, 11 May 2007 02:35:19 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-6493 <p>hi Xaprb, Because I'm in vacation these days, very sorry for my late reply. It can be solved by WHERE if the PRIMARY KEY is auto-increase(it works well in my script), dividing the checksum into ranges. However, if the key isn't numeric or sortable, for example, an PRIMARY or UNIQUE KEY (ID1, ID2), a counter maybe needed to get checksum ranges.</p> hi Xaprb,

Because I’m in vacation these days, very sorry for my late reply.

It can be solved by WHERE if the PRIMARY KEY is auto-increase(it works well in my script), dividing the checksum into ranges. However, if the key isn’t numeric or sortable, for example, an PRIMARY or UNIQUE KEY (ID1, ID2), a counter maybe needed to get checksum ranges.

]]> By: Xaprb http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-5956 Xaprb Mon, 23 Apr 2007 14:12:14 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-5956 <p>Hi qinyf,</p> <p>I have limited time right now since I'm at the MySQL conference, but these sound like issues I would like to address on the mysqltoolkit mailing list or bug tracking system, because I think these would be great features for <a href="http://sourceforge.net/projects/mysqltoolkit" rel="nofollow">mysql-table-checksum</a>. Would you please add feature requests there? When I get back from the conference I'll try to address it. I think the issue of which index to use is solved in that tool (but let me know if it doesn't meet your needs), but maybe the tool can figure out a smart way of dividing the checksum into ranges. It can already do a WHERE clause but that doesn't sound like enough for you.</p> Hi qinyf,

I have limited time right now since I’m at the MySQL conference, but these sound like issues I would like to address on the mysqltoolkit mailing list or bug tracking system, because I think these would be great features for mysql-table-checksum. Would you please add feature requests there? When I get back from the conference I’ll try to address it. I think the issue of which index to use is solved in that tool (but let me know if it doesn’t meet your needs), but maybe the tool can figure out a smart way of dividing the checksum into ranges. It can already do a WHERE clause but that doesn’t sound like enough for you.

]]> By: qinyf http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-5947 qinyf Mon, 23 Apr 2007 05:35:28 +0000 http://www.xaprb.com/blog/2007/01/25/how-to-calculate-table-checksums-in-mysql/#comment-5947 <p>So efficient it is! Thanks for your blog, i've checksumed some million-level tables in several minutes with user-variable based tech.</p> <p>For online product DB, the checksum calculation running during several minutes maybe a big pressure to the server. So I add "WHERE ID BETWEEN XX AND YY", in which "XX,YY are pre-calculated PRIMARY ID range, into the SQL and separate those ranges with sleeping several seconds in order to relax the server a little. It's like:</p> <p>calculate checksum, ID from 1 to 50000 sleep(3) calculate checksum, ID from 50001 to 100000 sleep(3) ....</p> <p>Another suggestion is about how to calculating the checksum to a table that do not have an auto-increase PRIMARY(index) ID but an UNIQUE KEY. I'm not sure if it can work okay without "USE INDEX()" on both master and slave(the order maybe different?), maybe we can use "USE INDEX(CERTAIN_UNIQUE_KEY_NAME)" to force the check order.</p> <p>I also want to solve a problem which combine these two issue above, to calculate the checksum for a table has only an UNIQUE-KEY (e.g: ID1-ID2, a two-columns unique key) with separated ranges one by one, can you help me?</p> So efficient it is! Thanks for your blog, i’ve checksumed some million-level tables in several minutes with user-variable based tech.

For online product DB, the checksum calculation running during several minutes maybe a big pressure to the server. So I add “WHERE ID BETWEEN XX AND YY”, in which “XX,YY are pre-calculated PRIMARY ID range, into the SQL and separate those ranges with sleeping several seconds in order to relax the server a little. It’s like:

calculate checksum, ID from 1 to 50000
sleep(3)
calculate checksum, ID from 50001 to 100000
sleep(3)
….

Another suggestion is about how to calculating the checksum to a table that do not have an auto-increase PRIMARY(index) ID but an UNIQUE KEY. I’m not sure if it can work okay without “USE INDEX()” on both master and slave(the order maybe different?), maybe we can use “USE INDEX(CERTAIN_UNIQUE_KEY_NAME)” to force the check order.

I also want to solve a problem which combine these two issue above, to calculate the checksum for a table has only an UNIQUE-KEY (e.g: ID1-ID2, a two-columns unique key) with separated ranges one by one, can you help me?

]]>