Xaprb

I’m going to try to explain how to make software testable in really practical terms. I won’t use words like “dependency injection.” Those things annoy smart programmers and make them stop listening.

Here is a pseudocode snippet. There is some function that parses some IP address out of the output of the “ifconfig” command, and some other code that uses this to get an IP address and do something with it.

parse_ip_address() {
   hostname = system.execute("hostname");
   ifconfig = system.execute("/sbin/ifconfig");
   ip = regex.capture(ifconfig, "/some/regex/" + hostname + "/some/other/regex/");
   return ip;
}

// ... some other code ...

ip = parse_ip_address();
// do something with the ip address.

This code is extremely hard to test. If someone says “it doesn’t work on my computer,” you can only respond “it works on mine and I can’t reproduce it.” The code relies on the server’s hostname and the output of the ifconfig command, so the only way you can reproduce it is if you get access to your reporter’s computer and run the code there. (Imagine if it relied on the time of day or the date!)

Let’s rewrite the code.

parse_ip_address(hostname, ifconfig) {
   ip = regex.capture(ifconfig, "/some/regex/" + hostname + "/some/other/regex/");
   return ip;
}

// ... some other code ...

hostname = system.execute("hostname");
ifconfig = system.execute("/sbin/ifconfig");
ip = parse_ip_address(hostname, ifconfig);
// do something with the ip address.

Now you can write back to the person who reported the issue and say “please send me the output of /sbin/ifconfig, and your hostname.” You can write a test case, verify that it breaks, fix the code, and verify that all of your other tests still pass.

That is the absolutely essential core practice in testing: write code in units (be it functions, modules, programs, or something else) that accept input, cause no side effects, and return output. Then write test suites that begin with known input, execute the code, capture the output, and compare it to what is expected.

Now you’ve learned in ten minutes what took me many years to learn. When they taught me about software engineering in my Computer Science classes, they didn’t teach me how to test. They said “you must test rigorously!” and moved on to the next topic. They left me with the vague understanding that testing was an advanced practice that takes enormous time and effort. It turns out to be simple — if you start out right. And it saves enormous time and effort.

Testing has enabled me to avoid becoming a good programmer. I can’t write good code, but I can write good tests, and with good tests, you can see clearly how broken your code is.

At my previous employer, one of the early decisions that had huge payoffs later was the SQL naming conventions. A good naming convention is more than just a nicety. It lets you write programs that don’t need to be told about the relationships among tables and columns.

There are many ways to do this, I think. But in practice, I think I’ve seen only a few customer systems that have a completely consistent, logical naming convention. And there are levels of convenience; some systems have a couple extra rules that make a big difference.

In this post I’ll explain the components of my ideal naming conventions.

I’m attending the Google Test Automation Conference (GTAC 2007) in Manhattan, New York right now. It’s a two-day event hosted by Google, with mostly non-Google speakers.

The conference is by invitation only and very limited; we all had to either have something Google’s team of judges thought was good enough to present, or our essays had to impress them (I’m not bragging about getting in, I’m telling you why I thought the conference would be great). Unfortunately, I have to say I’m a little underwhelmed. Several of the talks have been very good, especially Allen Hutchison’s GTAC keynote on first principles and Simon Stewart’s informative and very fun talk on Web Driver, but some of the others didn’t zing me very much.

You can view the GTAC YouTube Playlist to see the talks yourself. The first day is up already — amazing! The Google multimedia folks really have it down to a science.

Perhaps I’m a little un-impressed because I think the discipline of test automation, or at least some of those speaking, is too heavily influenced by Extreme/Agile methodologies, which often take a very narrow view of testing and have invented some seriously damaging techniques like Mock Objects. Perhaps because there’s a lot of Java in the room, and Java’s insistence on Everything Shalt Be An Object has twisted natural concepts into very difficult implementations, which other programming languages blindly follow even when they have first-class support for the thing (such as a test) Java represents so awkwardly. And perhaps because there’s so much focus on auto-generated tests, which I think are about as useful as auto-generated documentation. They’re often un-tests, just as documentation generated by inspecting and mentioning class names, method names, and parameter types is un-documentation. Not that auto-generated tests don’t have a place in the world — they do — but it’s limited.

The most interesting talk to me was Adam Porter and Atif Memon’s Skoll project (here’s the Skoll homepage), which is developing a distributed means of building and running test suites in different configurations, very smartly. There’s real computer science going into this. And guess what one of their big test projects is? (Perhaps the only big test project, I’m not sure). Building MySQL source code. Yep, they’re finding real bugs by smartly building different configurations and finding test failures, then iterating to find related configurations that fail. Watch the video for the details of how intelligently they’re doing this.

I decided to skip the last talk and the evening’s socializing, and instead headed over to the MySQL Camp, which is happening just a few miles away in Brooklyn. I spent the evening mooching Japanese food and catching up with friends I met at MySQL Camp 2006. I went to bed late, but it was worth it.

Today I’m also going to skip the Google Test Automation Conference and focus on MySQL Camp. I tried to find out more about today’s GTAC talks, but it’s tough. Google has kind of made it a black box — I didn’t even find out in advance who was going to speak, or get any chance to offer a talk myself. A few days ago they sent an email with the schedule, which listed speaker names and talk titles, but no other information. That was the first I knew of the schedule. There are certain things that are great about how they’re running this, such as having just one track (no tough choices of which talk to attend), but not knowing who was speaking on what made it hard to judge whether and why I wanted to attend. Abstracts would have helped a lot. As far as I can see, today’s talks are going to include more mildly promotional material. I’d be interested in the Lightning Talks and maybe a couple of the others, but time is precious, and given that I know MySQL camp is going to be good, I’m not willing to take the risk that today’s GTAC talks will be uninspiring.