Tag Archive for 'graphing'

Improved Cacti monitoring templates for MySQL

Download MySQL Cacti templates

As promised, I’ve created some improved software for monitoring MySQL via Cacti. I began using the de facto MySQL Cacti templates a while ago, but found some things I needed to improve about them. As time passed, I rewrote everything from scratch. The resulting templates are much improved.

You can grab the templates by browsing the source repository on the project’s homepage.

In no particular order, here are some things I improved:

  • Standard polling interval and graph size by default.
  • Full captions on every graph; you don’t have to guess at how big the values are. Each graph has current, max, and average values printed at the bottom for every value on it.
  • Much more data is captured. I’ve graphed almost everything I could think of.
  • The graphs are grouped better. Most graphs have only related values. There are some exceptions, but not many.
  • The templates don’t hijack your existing installation. They don’t depend on or alter anything in your default Cacti installation.
  • The script that gathers the data is totally rewritten from scratch, and much improved. For example, the math works on 32-bit systems. It has caching built-in so each poll cycle results in just one request to the server, instead of one request per graph. (This is a weakness of Cacti I’m trying to work around). It also has debugging aids and other good coding stuff.
  • By default, it assumes you have the same username and password across every server you’re monitoring, so you don’t have to fill in a username and password for every single graph you create.
  • One data template == one graph template. This helps work around another Cacti limitation.
  • Lots more. Honestly I can’t really remember everything I’ve done. I’m sure you’ll help me remember by asking me how to get X feature working the way you want, and I’ll go “oh, yeah, that’s another thing I improved…”

Cacti templates are very laborious to create if they’re complex at all; it takes a long time and is very error-prone. Instead of doing it through Cacti’s web interface and exporting a huge XML file, I eliminated the redundancies and created a small, easy-to-maintain file from which I generate the XML template with a Perl script. This gives the added benefit of letting me (or you) generate templates with different parameters such as polling interval or graph size. The README file has the full details. However, I’ve pre-generated a set of templates that matches Cacti’s defaults, so you can probably just use that.

This has taken a lot of time. In particular, I spent a lot of time working on it at my former employer, The Rimm-Kaufman Group (kudos to them for letting me open-source the work) and I just spent most of my weekend writing the scripts to convert from the compact format to XML templates, so it’s possible to maintain these beasts. Plus I had to develop the compact format, too. This took a lot of time because I had to understand the Cacti data model, which is pretty complex.

Please enter issue reports for bugs, feature requests, etc at the Google project homepage, not in the comments of this blog post. I do not look through comments on my blog when I’m trying to remember what I should be working on for a software project.

If these templates help you and you feel like visiting my Amazon.com wishlist and sending something my way, I’d appreciate it!

PS: You may also be interested in Alexey Kovyrin’s list of templates for monitoring servers.

Technorati Tags:, , , , , ,

You might also like:

  1. What’s the best way to choose graph colors?
  2. A new home for innotop in the new year

What’s the best way to choose graph colors?

I have an issue I hope someone can help me with. I am generating RRDtool graphs (for Cacti monitoring templates for MySQL, which I’ll release soon) that have up to 11 different metrics on them. With that many lines or areas on a graph, it becomes very hard to pick colors that are easy to see and easy to distinguish from each other. What’s a good way to choose such colors? Is there a way to do it automatically — is there a formal method that will produce good results?

I know some color theory and I have read about how you can distinguish colors from each other (hue, value etc). But I am unsure the best way to choose this many colors. Trying by hand produces garish results or graphs that are just hard to read.

My first attempt to solve this with a program was to simply create a list of every possible completely saturated color in a 32-bit space — essentially, the “pure” colors around the rim of the color wheel — and divide it into the desired number of evenly spaced intervals. This produces pure colors, which is not ideal. They are hard to look at. Did I mention garish?

I can shuffle the order so that they’re not adjacent, but that only helps avoid a “rainbow effect” if I’m stacking areas of color on top of each other, like in the following image:

MySQL Command Counters

Ugh, rainbows (I chose those by hand, not with my program). Lines on a white background might be placed in any order, so shuffling doesn’t help with those graphs.

I modified my little script to let me vary the saturation and value. My thinking was that lines on a white background really shouldn’t be full-value, and when I’m drawing areas instead of lines, I should de-saturate them so they become more pleasing pastels. This doesn’t really help as much as I might have hoped for, either. Colors around 80% saturation and 60% value look pretty good, but they’re still ugly colors. And I can’t get over five colors without them starting to run together again. Here’s an example with only four colors that’s already hard to look at:

InnoDB I/O Activity

Part of the problem, I’m currently thinking, is that I’m varying only one dimension. I could be varying the saturation as well as the hue, for example. But that might be another rabbit hole that will waste more time.

Right now I’m thinking that I should ask for help, instead of continuing to work on this myself. So, any ideas are welcome!

By the way, beautiful colors would be nice… a lot of the colors I choose by hand are very pretty and I’m sure my impartial, evenly-distributing script will never choose them in a million years. Also, it’s actually a good thing when graphs each have their own color scheme (as long as it’s attractive) because it becomes easier to identify graphs without having to read the title. Just some extra food for thought.

Technorati Tags:, , , , , ,

You might also like:

  1. Improved Cacti monitoring templates for MySQL
  2. innotop 1.3.5 released
  3. How to monitor MySQL status and variables with innotop
  4. Advanced HTML table features, Part 2