# The DevOps Identity Crisis

Why DevOps needs a manifesto after all, but may never get one.

DevOps is everywhere! The growth and mindshare of the movement is remarkable. But if you care deeply about DevOps, you might agree with me when I say that although its moment has “arrived,” DevOps is in serious trouble. The movement is fragmented and weakly defined, and is being usurped by those who care more about short-term opportunities than the long-term viability of DevOps.

# Easy on the eyes: the solarized color theme

I recently set up the solarized color theme for my terminal emulator. I’ve been meaning to do this for a while, but procrastinated. However, I finally got really frustrated with the colors I get from “ls” sometimes – I use a dark terminal with light fonts, and the directory listings in particular can become invisible, with dark blue on black. Solarized is much improved. All of the colors work well together and are easy on the eyes.

# Disk latency versus filesystem latency

Brendan Gregg has a very good ongoing series of blog posts about the importance of measuring latency at the layer that’s appropriate for the question you are trying to answer. If you’re wondering whether I/O latency is a problem for MySQL, you need to measure I/O latency at the filesystem layer, not the disk layer. There are a lot of factors to consider. To quote from his latest post: > This isn’t really a problem with iostat(1M) – it’s a great tool for system administrators to understand the usage of their resources.

# How to gather statistics at regular intervals

I gather a lot of statistics such as performance data. Sometimes I have multiple things going on a system and I want to be able to align and compare the resulting data from multiple processes later. That means they need to be aligned on time intervals. Here is a naive way to gather stats at intervals: while sleep 1; do gather-some-stats; done There are two problems: each iteration will take longer than a second, so there will be drift; and the iterations will not be aligned exactly on the clock ticks, so the data isn’t as easy to correlate with other samples.

# Version 1.1.8 of Better Cacti Templates released

I’ve released version 1.1.8 of the Better Cacti Templates project. This release includes a bunch of bug fixes and several important new graphs. There are graphs for the new response-time statistics exposed in Percona Server, and a new set of graphs for MongoDB. VividCortex is the startup I founded in 2012. It’s the easiest way to monitor what your servers are doing in production and I consider it far superior to Cacti.

# Time TCP traffic with tcprstat

I just realized that I didn’t publicize this in the Postgres world, or anywhere but the MySQL blogosphere for that matter. Some folks at my company have released a generic TCP-response-time tool. Very useful for monitoring, benchmarks, historical metrics, and so on. It’s kind of like iostat, but for TCP traffic, and fully focused on time, not traffic size. Performance == time and tasks, and this is a lightweight way to measure that data.

# Beware of svctm in Linux's iostat

I’ve been studying the source of iostat again and trying to understand whether all of its calculations I explained here are valid and correct. Two of the columns did not seem consistent to me. The await and svctm columns are supposed to measure the average time from beginning to end of requests including device queueing, and actual time to service the request on the device, respectively. But there’s really no instrumentation to support that distinction.

# A review of Web Operations by John Allspaw and Jesse Robbins

Web Operations Web Operations. By John Allspaw and Jesse Robbins, O’Reilly 2010, with a chapter by myself. (Here’s a link to the publisher’s site). I wrote a chapter for this book, and it’s now on shelves in bookstores near you. I got my dead-tree copy today and read everyone else’s contributions to it. It’s a good book. A group effort such as this one is necessarily going to have some differences in style and even overlapping content, but overall it works very well.

# How I keep track of notes

This is the follow-up to my post on how I keep track of tasks. It’s important for me to have a good system for keeping notes and other files organized. The problem usually turns out to be that I want them organized several different ways simultaneously: by date, by project, by person, by subject. Alas, if I keep them in files on a hard drive, I can only choose one such organizing strategy, because filesystems are a single hierarchy.

# How I keep track of tasks

I use a super-simple system for keeping track of tasks that are mine personally to manage. I use issue-tracking systems for software projects and consulting work, but there is still a bunch of work-related and personal work that I need to make sure I don’t forget. The main point is not to ensure that I don’t forget, actually. It is to be able to put it out of my mind with confidence that I won’t lose it.

# A better way to build Cacti templates

The traditional way to build Cacti templates is through the Cacti web interface. This is an enormous amount of work, and the result is generally not very consistent or good quality. The process is too error-prone. You can export the templates as XML, but they tend to have problems such as version incompatibilities with other Cacti installations, and it’s hard to adapt them for user preferences such as different graph image sizes and polling intervals.

# How to read Linux's /proc/diskstats easily

These days I spend more time looking at /proc/diskstats than I do at iostat. The problem with iostat is that it lumps reads and writes together, and I want to see them separately. That’s really important on a database server (e.g. MySQL performance analysis). It’s not easy to read /proc/diskstats by looking at them, though. So I usually do the following to get a nice readable table: Grep out the device I want to examine.