Xaprb » monitoring

Version 1.1.2 of improved Cacti templates released

Xaprb — Thu, 07 May 2009 04:31:51 +0000

I’ve packaged up and released version 1.1.2 of the Cacti templates I’ve written for MySQL, Apache, memcached, nginx etc.

Anyone who would like to help write documentation (or do anything else, for that matter) is welcomed to participate. I’ll give commit access at the drop of a hat.

Changelog:

2009-05-07: version 1.1.2

	* The parsing code did not handle InnoDB plugin / XtraDB (issue 52).
	* The servername was hardcoded in ss_get_by_ssh.php (issue 57).
	* Added Handler_ graphs (issue 47).
	* Config files can be used instead of editing the .php file (issue 39).
	* binary log space is now calculated without a MySQL query (issue 48).
	* There was no easy way to force inputs to be filled (issue 45).
	* Some graphs were partially hidden without --lower-limit (issue 43).
	* Flipped some elements across the Y axis (issue 42).
	* Added Apache, Nginx, and GNU/Linux templates.
	* Unknown output is now -1 instead of 0 to prevent spikes in graphs.
	* If you want to use a script server, you must now explicitly configure it.
	* UNIX sockets weren't permitted for MySQL (issue 38).

Further Reading:

Secure, easy Cacti graphing without SNMP

Xaprb — Sat, 25 Apr 2009 19:01:02 +0000

Cacti is a great tool for collecting information about systems and graphing it. However, it likes to use SNMP, and SNMP is often not desirable. Instead, I often see the need for a method that is:

Secure. Use trusted, well-known, encrypted communication. Do not open up new ports.
Zero install on the monitored system.
As little installation or modification on the monitoring system as possible.

Over the last several years, I’ve slowly created more and more software to create Cacti graphs via standard POSIX command-line utilities over SSH with key-pair authentication. (I’ve also created similar software for Nagios, but that’s another matter.) The major problem with the work I’ve done is that it’s totally un-publicized.

The system works by passing command-line arguments to a local PHP script like any other Cacti script. This script then executes a remote SSH command, such as ssh somehost uptime and extracts statistics from the result.

The requirements are very simple. On the system to be monitored, a public key must be installed. On the monitoring system, the cacti user account must have a private SSH key that it can read and others can’t. This is standard for any SSH key. The cacti user account must also have the SSH key fingerprint of the monitored system in its known_hosts file.

The work I’ve done thus far is available from Subversion in the mysql-cacti-templates project. This project has the scaffolding for creating Cacti templates easily, so I’m using it.

At this point, the following are available:

Apache
Operating system (CPU, memory, load average, etc)
Memcached
Nginx

Unfinished work includes network, disk, etc. The techniques to monitor something of which there is a variable number (e.g. there can be many disks, each of which needs its own graph) are a little more complex than simple things like monitoring overall CPU usage. So that’s a work in progress. Once it’s done, it’ll make it really easy to discover and monitor multiples of anything — for example, multiple MySQL servers or memcached servers on a single host — without creating a new host for each resource to monitor.

As with the MySQL templates I created, these templates are comprehensive and have lots of nice properties most templates lack. This is something you get free with my scaffolding. If you’ve ever created templates by hand through the web interface, you should give my work a try. You can turn a three-day project into a few minutes and avoid bugs and other hassles. There are instructions for creating Cacti templates on the project wiki.

Further Reading:

News on MySQL Cacti Templates

Xaprb — Mon, 09 Feb 2009 16:06:48 +0000

It’s been a while since I’ve sent any news about the MySQL Cacti Templates project I started a while ago. Here’s what’s new:

First of all, it’s not just MySQL templates anymore. It’s a generic framework and tools for creating templates, and I’ve written templates for Apache, Memcached, etc. I just haven’t publicized it yet.
Second, there’s a mailing list, http://groups.google.com/group/better-cacti-templates You see it’s called “better cacti templates” which is what I ought to have called the project to begin with :-) I hope you’ll use the mailing list instead of emailing me directly, for all the usual reasons.
There are some open issues I cannot reproduce. I believe they’re problems, but I can’t solve them. Some of them make no sense to me; I don’t know why others are seeing behavior I’m not. However these seem to be problems with Cacti, not problems with the templates. Anyone who wants commit access is welcomed to ask for it.
Some people have been asking about Spine. I haven’t used it and none of my customers has asked me about it, so the short answer is — if you want Spine support, don’t wait for it to happen for free; either contribute it yourself or pay someone (me?) to do it.

Further Reading:

MySQL Cacti templates version 1.1.1 released

Xaprb — Thu, 16 Oct 2008 03:21:14 +0000

I’ve released version 1.1.1 of the MySQL Cacti templates I develop. The new templates work around more Cacti limitations, including the limitation on the length of the data returned from the poller script. There’s also a new graph and many other niceties since the last release.

Note that there are TWO incompatible changes with the previous version of the templates. You don’t have to throw away your RRA files, you just have to delete and re-import over all the existing templates. It’s not my fault, there was no other way to work around the limitations mentioned above. Really. Have fun upgrading!

Version 1.1.1 is actually just a repackaging of 1.1.0.

The changelog follows.

2008-10-15: version 1.1.1

	* The tarball didn't have make-template.pl mysql_definitions.pl (issue 34)

2008-10-14: version 1.1.0

	* Graphs fetched too much data, causing errors (incompatible; issue 28, 23).
	* Output of the poller script is compressed with short value names.
	* Checks can be disabled; no need to fetch INNODB STATUS if unwanted.
	* Queries could cause a MySQL thread stack overflow (issue 19).
	* Older PHP didn't have array_change_key_case function (issue 21).
	* The PROCESS privilege is required for MySQL 5.1.29 with InnoDB (issue 22).
	* Added an aggregated view of SHOW PROCESSLIST; requires PROCESS privilege.
	* The text on the graph could overflow the right-hand edge.
	* Truncated SHOW INNODB STATUS could cause an SQL error (issue 27).
	* The poller script requires proper cmdline options (incompatible change).

Further Reading:

How much memory does MySQL Enterprise Monitor’s agent use?

Xaprb — Tue, 26 Aug 2008 19:14:13 +0000

After last week’s post on agents versus agentless monitoring systems, I got a lot of email. One, from a customer whose name I am not permitted to mention, sent me the following action shot (posted with permission):

Over half a gigabyte; more than twice what MySQL itself is using. So, that raises an interesting question. How much memory would you say your server’s MySQL Enterprise Monitor agent uses? No bashing allowed, tell the truth. And please post if your agent runs in a small amount of memory, too.

Another thing I’m interested in: what factors contribute to the high memory usage? Under what conditions does it use a lot or a little? What can you expect, generally speaking?

On another note, I think it’s almost unfair not to mention the competition, so I will: I was playing with MONyog 2.6 recently and it started to use a lot of CPU. I think it might be architecture-specific (amd64 Ubuntu) but I can’t be sure. It was using 170% CPU on my dual-core system to monitor 2 instances.

Further Reading: