Archive for the 'Coding' Category

You have the right to see code samples in an interview

Published at May 4, 2008 in Perl and Coding. 1 Comment

Joel Spolsky writes about 12 steps to better code, and elsewhere about how candidates should write code in interviews.

The reverse conditions are true, too. If you’re a candidate, you should evaluate the employer against the 12 steps, and you should also see code samples. How else will you know what you’re getting into? You really have the right to do this, and you should exercise the right. If you don’t, you’ll get stuck in a crap job maintaining crap code. [dramatic voice] It happened to me.

In many companies, you can see code they’ve released as open-source. (The fact that they’ve done this says a lot about them.) But in others, you’re going to need to surprise someone and say “pick some code that’s not sensitive and show it to me.” Something simple, like the HTML for the search form on their website, or a utility to do some systems administration task. Any company is going to have a lot of code like this that they can show you.

The other approaches I see are to ask about it, assume, or ask the interviewer to write some code for you.

  1. Asking is a valid approach. If you see hesitation, or if someone says “well, it’s not as nice as we’d like, and we’re hoping you will offset that” run don’t walk, is my advice. If you’re reading this as you consider your first job out of college or something, I strongly suggest not getting a job with a company that wants you to improve the way they do things. You should be learning from them, not vice versa.
  2. You can also assume. “Oh, they use Perl? Nevermind.” That’s a stupid approach. Really. Is it acceptable to judge people’s character by the color of their skin? Then why would you judge their code by the language? In all seriousness, I have actually written very elegant, clean VBScript. And I mean, good-quality code by anyone’s standards. It’s hard in VBScript. It’s easy in Perl if you follow the Dog, which is a sign of great intelligence. Think about it this way: people who write beautiful Perl are people you should be eager to work with; they are rocket scientists. You will be the dumbest person in the room, and that should make you happy.
  3. I’ve never asked an interviewer to write code for me. Let me know how it works out for you.
Technorati Tags:, , ,

No related posts.

Review of Perl Best Practices

Published at April 1, 2008 in Perl, Review and Coding. 0 Comments

In my opinion, every Perl programmer needs at least these two books: the Camel, and the Dog (Perl Best Practices).

The Camel teaches you how to program Perl: the syntax and so on. But the Dog teaches you how to program Perl sanely, by recommending that you use only a subset of Perl’s syntax and abilities.

The Camel tells you that there’s more than one way to do it, which is supposed to be a strength of Perl. The Dog tells you which way is “best.” And it tells you why the other ways are worth avoiding, and in which circumstances, and how badly you can get burned if you’re not careful.

The Dog also shows you a lot of ways to do things that you might not otherwise think about. In that sense, it’s not just stripping away. It’s also adding things that you might not get elsewhere. Example: you might not know about List::Util.

The common complaint about Perl being a write-only language doesn’t have to be true if you follow the Dog’s suggestions. Perl can be a very readable language — and I mean readable for ordinary humans, so don’t think I’m just another Perl programmer insisting that something is readable when it looks like static to you. The Dog encourages readable coding. In fact, in the absence of any other coding standard, if you try to “follow the Dog” you’ll be doing a good job. (There’s really no need to invent your own coding standard, in my opinion. Just follow the Dog.)

It’s not perfect. I don’t always agree with its suggestions. For example, I think the use of “magic comments” to automagically add interactive progress bars to command-line programs is just a bad idea. (Comments should never change the behavior of a program). However, the vast majority of its suggestions have come from long experience. If you don’t agree with one of them, it’s worth following it anyway until you consider yourself expert enough to disagree with authority.

I consider this an essential Perl book. No Perl programmer should be without it.

Technorati Tags:, , ,

You might also like:

  1. You have the right to see code samples in an interview
  2. How to Break Web Software

How to install and maintain multiple WordPress blogs easily

Published at March 11, 2008 in PHP, SQL, World Wide Web and Coding. 13 Comments

My wife has a site that needs two WordPress blog installations. The URLs differ by a subdirectory name. Both blogs need to be (URL-wise) subdirectories of /blog/. They need to be completely independent of each other, yet use the same custom theme. And there used to be just a single blog, which was not in a subdirectory; its permalinks must not break. (It has nice URLs with the date and title in them, not post ID-style URLs). And because I’m the husband, I get to maintain it, so tack “easy to maintain” onto the requirements (it must be easy to upgrade WP in both blogs, for example). In this article I’ll show you how I did it with a single .htaccess file, a single copy of WordPress, two MySQL databases, and a single configuration file.

Fixing URLs

As I mentioned, there used to be a blog at /blog/ which must not break. Suppose this blog was about dogs and my wife has recently started blogging about cats. She wants two completely independent blogs: /blog/dogs/ and /blog/cats/. Now the old permalinks structure, e.g. /blog/2006/03/01/dogs-are-great/, must redirect to /blog/dogs/2006/03/01/dogs-are-great/. How to do this?

I’m not a mod_rewrite wizard, but I figured there must be a way. And indeed there is: if an incoming URL doesn’t contain dogs or cats, it can be rewritten and redirected to the new URL. Here’s the code, which goes in /blog/.htaccess:

RewriteBase /blog/
RewriteCond %{REQUEST_URI} !dogs|cats
RewriteRule ^(.*)$ http://www.furryfriends.org/blog/dogs/$1 [R]

(By the way, the furryfriends thing is just an example, not the real site name).

So far, so good. That works just fine: when I access a URL without dogs or cats in it, it redirects me. But I need to do more: I need rewrite rules to match the date-and-title permalinks both blogs will use. I accomplish that like so:

RewriteCond %{REQUEST_URI} dogs|cats
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (dogs|cats) /blog/$1/index.php [L]

This is basically the same thing WordPress usually does, but I’ve made it tolerate either dogs or cats and figure out which installation should get the request. The .htaccess file lives in /blog/, not inside /dogs/ or /cats/ where it would be hard to maintain (it would get wiped out with upgrades). I can see different ways of doing this, but this is the way I chose. So here’s the whole file:


RewriteEngine On

# Anything to the old address (e.g. /blog/foo/bar) goes to the new address
# (e.g. /blog/dogs/foo/bar)
RewriteBase /blog/
RewriteCond %{REQUEST_URI} !dogs|cats
RewriteRule ^(.*)$ http://www.furryfriends.org/blog/dogs/$1 [R]

# If that fired, then we didn’t reach this code.  If we did, then this rule
# should do what a normal WP rule does.
RewriteCond %{REQUEST_URI} dogs|cats
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (dogs|cats) /blog/$1/index.php [L]

Are there any better ways of doing this? I’m curious. Leave a comment if you know of one.

Fixing the maintenance headache

Installing two copies of Wordpress, then customizing both is a pain. And it makes upgrades harder, too. I’d have to upgrade them both, fiddle with plugins (some of them are customized, too) etc etc. Even backups would be more complicated. It would be all too easy to screw up and delete some data. There are just so many ways this is a bad idea.

It occurred to me that I could use a single copy and turn the dogs/ and cats/ subdirectories in the filesystem into symbolic links. (Windows users, you can stop reading now: this won’t work for you).

To make the blogs, the Wordpress installation, and the custom blog theme all independent of each other, I created the following filesystem hierarchy:

blog/
   wordpress/
      2.3.2/
         [The usual WP files are here]
      wp_content/
         plugins/
         uploads/
         themes/
            my_custom_theme/

What I’ve done is separate the custom bits — the parts that don’t ship with WordPress — away from the files I want to upgrade when I upgrade Wordpress. How will this work, though?

I’ll make symbolic links from the dogs/ and cats/ directories to the currently installed version of Wordpress. So, from the root directory of the website, I type the following at the command line:

$ ln -s wordpress/2.3.2/ dogs
$ ln -s wordpress/2.3.2/ cats
$ cd wordpress/2.3.2/
$ rm -rf wp-content/
$ ln -s ../wp-content wp-content

The directory hierarchy now looks like this: 

blog/
   cats/ -> wordpress/
   dogs/ -> wordpress/
   wordpress/
      2.3.2/
         [The usual WP files are here]
         wp-content/ -> ../wp-content
      wp_content/
         plugins/
         uploads/
         themes/
            my_custom_theme/

This is looking pretty good! There’s only one minor detail missing: because both blogs are running literally the same code via the magic of symlinks, each blog is trying to access the same database tables. I need to customize the Wordpress configuration file, too. I’ll just give each installation a different table name prefix in wp-config.php: 

$table_prefix  = strpos($_SERVER['REQUEST_URI'], 'blog/cats/') ? 'wp_cats_' : 'wp_dogs';

And voila, it works perfectly now. I accessed the two URLs, ran through the installation procedure twice, and have two completely independent blogs running the same code in the same database.

The upgrade procedure

So, this is all a little complicated, right? What if I’ve forgotten how I did it when I upgrade next time, or what if someone else does it instead of me? I wrote myself a little README file to fix this. Here’s what it says:

This is how to upgrade Lynn's blog.

The two blogs are actually using shared files, which are symlinked to make
it so there is only one copy of files.  You can't change the files in one
without changing them in the other.

The wp-content subdirectory is symlinked.

The wp-config file is customized so it will work in either blog:

$table_prefix  = strpos($_SERVER['REQUEST_URI'], 'blog/cats/') ? 'wp_cats_' : 'wp_dogs';

To upgrade, 

 1. Download the latest version and unpack it inside wordpress/ as 2.3.2/
    or whatever version it is.
 2. Then go into that directory.
 3. Remove the wp-content/ directory completely.
 4. Then symlink it like this: ln -s ../wp-content wp-content
 5. Now re-customize wp-config.php
 6. Go back to the blog/ directory.  rm dogs cats
 7. ln -s wordpress/2.3.2/ dogs
 8. ln -s wordpress/2.3.2/ cats

It’s still a manual process, but it should take me all of thirty seconds. I’m okay with that. As long as I remember there’s a README file, that is!

Technorati Tags:, , ,

You might also like:

  1. How to write INSERT IF NOT EXISTS queries in standard SQL
  2. How to exploit an insecure order of access to resources
  3. How to install beautiful X11 cursors
  4. How to make file names cross-platform
  5. Interactive directory merging

A very fast FNV hash function for MySQL

Published at March 9, 2008 in Maatkit, C, SQL and Coding. 6 Comments

I wrote a User-Defined Function that implements the FNV (Fowler-Voll-No) hash function for MySQL. I’m not the first person to do this — in fact, I was inspired by the Google patches for MySQL. But my implementation is a little bit different from most, in a very important way that leads directly to much higher performance, especially suited for the Maatkit tools.

A bit of background: FNV hashing is a very fast hash algorithm that operates in fixed memory. It is widely used in lots of important areas in computer science. My implementation requires absolutely no malloc() calls, which is a darn good thing because I am not to be trusted with malloc(), having spent too many years programming in managed languages. I made it return a 64-bit integer, which matches the size MySQL uses internally for most integer arithmetic.

The most important thing I did was make my UDF accept 1 to infinity arguments. That means you can hash entire rows with a single function call. And that is very useful for the Maatkit table-checksumming tools, which tend to run about 8-10 times faster when they don’t have to make MySQL do a bunch of string concatenation. That translates directly to less impact on the server, and less slave lag (if that is a problem for you).

Here’s how my implementation works:

SELECT FNV_64(col1, col2, col3, .... colN) FROM ...

Compare this to MD5() hashing that accomplishes the same thing:

SELECT MD5(CONCAT_WS('#', col1, col2, col3, .... colN)) FROM ...

The UDF’s code is distributed with Maatkit, and I plan to eventually build it as a binary that can be installed without requiring you to compile it. However, compiling is very easy; there are instructions in the source code comments. Installing is also easy: just a simple SQL statement.

If you’re using Maatkit to make sure your slaves have the same data as their master, you should install the UDF on all your servers for a significant performance boost. You’ll save your servers a lot of work. You don’t need to do anything extra for Maatkit to take advantage of it. Maatkit will auto-detect it if you have it installed.

I’ve been running it in production for a couple of months now with nothing but good results. And the code is drop-dead simple, so I think the chance of bugs is very slim. But if you have questions, problems, bug reports etc, please use the Maatkit project page to report them.

Technorati Tags:, , , ,

You might also like:

  1. Speed up your MySQL replication slaves

Maatkit on Ohloh

Published at December 8, 2007 in Maatkit, Perl, SQL, Coding and Commentary. 3 Comments

Sheeri wrote a post (now a 404 error) referring to Maatkit on Ohloh, which I have never heard of before. I took a look at what Ohloh thinks about Maatkit. It’s kind of neat. Beyond just the obvious “social website” stuff that’s all the rage these days, it actually looks at the project’s SVN history, analyzes the codebase, and so on.

It also estimates 8 person-years of work have gone into the project, and says that at $55,000/year it would cost $450,702 to write the code as it currently exists, which is kind of funny. It took me a whole lot less than 8 years to write. (Perhaps this is why that salary strikes me as unrealistic).

It has a couple of other interesting things, like a visual timeline of source control commits, analysis of licenses it finds in the code, analysis of programming languages, and so on. Really pretty neat overall.

There’s also the ubiquitous popularity rating: how many people have “stacked” the project. I notice it’s been stacked 3 times, coincidentally the same number as MySQL Proxy. It will be interesting to see how that changes over time.

Technorati Tags:, , , , ,

You might also like:

  1. MySQL Community Member of the Year
  2. How good is the new High Performance MySQL going to be?

Things I love about Perl

Published at December 6, 2007 in Perl, Coding and Commentary. 1 Comment

I don’t love everything about Perl, but I love its sense of humor, which I think probably comes from its creators’ senses of humor. From the Perl function documentation for redo:

“last”, “next”, or “redo” may appear within a “continue” block. “last” and “redo” will behave as if they had been executed within the main block. So will “next”, but since it will execute a “continue” block, it may be more entertaining.

“Entertaining,” in this context, means “if we were omniscient and looking over your shoulder while you spend a day debugging your occasional infinite loop or other bizarre behavior, we would be wildly entertained.”

At least that’s how I read it.

Sometimes the sense of humor, especially when imitated by neophytes trying to pretend to be part of The Gang Of Perl Greats, degrades into obnoxious sarcasm that obscures rather than documents. But this is fairly rare in the core documentation or other writings from the language’s authors.

If you’ve never read Programming Perl, you’re missing out on a lot of extremely subtle, very sharp and intelligent wit. I don’t have my copy of the book at hand, but one joke that comes to mind is how to write the Lord of the Rings trilogy with a regular expression substitution:

($lotr = $hobbit) =~ s/bilbo/frodo/g;

Or something like that. There are many fun examples that manage to teach the matter at hand more clearly, and keep me engaged more, than even the clearest straightforward explanation could.

Often imitated, but never equaled in any other book I’ve read. For example, I tried to read Extreme Programming Refactored (I really really tried, honest!) but could not make it through. I found myself getting irritated and wanting them to get to the point.

When Larry Wall et al make a joke about Gandalf, it is the point.

Technorati Tags:, , , , ,

No related posts.

Progress on Maatkit bounty, part 2

Published at December 3, 2007 in Maatkit, Perl, SQL and Coding. 13 Comments

Ironically, the Stream algorithm I wrote as the simplest possible syncing algorithm does what the much more efficient algorithm I wrote some time ago can’t do: sync a table without a primary key, as long as there are no duplicate rows. In fact, it’s so dumb, it will happily sync any table, even if there are no indexes.

The flash of inspiration I had on Friday has turned out to be good insight. It immediately showed me how I can re-use a lot of the hard work I’ve already done for other tools. Chunking and nibbling are the names I’ve given to two algorithms I’ve developed for processing tables a little at a time. Chunking looks for a suitable index and generates an array of WHERE clauses that will divide the table into pieces of approximately a given size (number of rows or number of bytes). I use this on mk-table-checksum and mk-parallel-dump. It requires an indexed column I can treat as a number one way or another. That includes temporal values.

Nibbling is related. It’s an efficient way to do something like LIMIT X, Y without scanning through the first X rows. It also requires a suitable index, but the code I wrote to make it work with mk-archiver is really generous about what “suitable” means. It’ll basically work with any index. It selects some rows with LIMIT Y and uses the last-fetched row to start the next select.

Both algorithms will adapt well to finding and resolving differences in rows, a chunk at a time. The general procedure is to create the WHERE clauses that define boundaries around the chunk of rows, then checksum the whole chunk. The result is a tiny little hash value. If this differs between the source and destination tables, the next step is to checksum the rows individually and fetch their primary or unique key columns. This uses a little more network bandwidth, but it’s still not bad unless the key (or the chunk) is huge. Any rows whose checksums differ can then be fetched by the key and synced.

The more complex algorithms, which were in the original table-sync tool, will come later. They can be potentially much more efficient in terms of network traffic, but they have drawbacks too, such as the granularity of locking. The mk-table-sync tool will soon be able to choose the best algorithm that causes the least locking and just do it without any fuss. For example, if it sees a nice primary key it can use for chunking, and it sees that the table is InnoDB, it’ll just chunk and use SELECT FOR UPDATE. Voila, no table locks, and not much of the table will be locked at a time (it’ll commit between chunks).

Right now I’ve gotten a simple interface for code that finds differences in rows, a plugin-like interface for implementing each of the algorithms uniformly, an interface for resolving differences, and a few other things. I’m about to embark on the Chunk algorithm for syncing.

I don’t think most people will consider this a big deal, but don’t expect the final product to correctly sync tables without a primary key and with duplicate rows. Comparing tables with duplicates is absolutely meaningless. If you can’t write a WHERE clause that uniquely identifies a row, you’re done.

Technorati Tags:, , ,

You might also like:

  1. Progress on Maatkit bounty
  2. Progress on Maatkit bounty, part 3
  3. MySQL Table Checksum 1.1.5 released
  4. A progress report on MySQL Table Sync
  5. Comparison of table sync algorithms

Progress on Maatkit bounty

Published at November 30, 2007 in Maatkit, Perl, SQL, Coding and Tools. 1 Comment

My initial plans got waylaid! I didn’t pull out the checksumming code first, because the code wasn’t at all as I remembered it. Instead, I began writing code to handle the more abstract problem of accepting two sets of rows, finding the differences, and doing something with them. I’m ending up with a little more complicated system than I thought I would. However, it’s also significantly simpler in some ways. Instead of just passing references to subroutines to use as callbacks, I’m object-ifying the entire synchronization concept.

What’s the advantage of doing this? Well, as some of you may know, there are two fairly complex algorithms in the tool at present, which handle synchronization in a hierarchical manner, zooming in on the rows that need to be changed. There are a lot of complexities in them. If I wrap all that up into modules, and make them have a uniform interface (real OO interfaces would be delightful here, but Perl doesn’t support them), I can simplify the project significantly by…

…throwing them out the window! That’s right, I’m tossing out the ‘top-down’ and ‘bottom-up’ algorithms. What I want to develop, first and foremost, is the code that does the synchronization, not the really twisted code that does bitwise XORs on groupwise slices of checksums and has recursion and all that stuff. So I decided on a generic data-syncing interface, and wrote the simplest possible implementation of that, which I’m going to use to help me deal with complexity. This algorithm is called ’stream’ (for lack of a better word). It has no hierarchical drill-down or any other complexities. It amounts to “select * from source, select * from dest, diff and resolve.”

It’s not a very efficient algorithm for comparing and syncing data, at least not by my standards. (It amounts to a FULL OUTER JOIN implemented in Perl). But boy, does it make it easier to start cleaning up the nasty spaghetti code that handles locking, waiting for a slave to catch up, actually changing the data that turns out to be different, and so on.

Of course, I’ll add back the top-down and bottom-up algorithms later, as well as some others. They should turn out to be pretty simple to implement, since they won’t have, for example, locking code intertwined with them. When done, the tool will examine the table and figure out the best algorithm to use. This will go a good way towards another of my goals, which is that you should be able to just point it at two tables and tell it to sync them, and it should do it in the most efficient way possible, without needing lots of command-line options.

Technorati Tags:, , ,

You might also like:

  1. Progress on Maatkit bounty, part 2
  2. Maatkit bounty begins tomorrow
  3. Introducing MySQL Table Sync
  4. A progress report on MySQL Table Sync
  5. Comparison of table sync algorithms

Maatkit bounty begins tomorrow

Published at November 29, 2007 in Maatkit, Perl, SQL and Coding. 1 Comment

Tomorrow is the first of five days I will spend working on mk-table-sync, the data synchronization tool I developed as part of Maatkit. The first thing I’ll do is pull the row-checksumming code out into a module and write a unit test suite for it. I’ll probably add the code to the module that does checksums for mk-table-checksum, since it is not all that different.

My mind is not fresh on the code, but I think the next thing after that will be to pull out the code that finds differences in two sets of rows. It is largely identical for the two algorithms (which I called top-down and bottom-up for lack of better ideas). My plan is to use a callback function to abstract away the functionality that’s not the same. In other words, I’ll write code that accepts two sets of rows and a reference to a subroutine, and when it finds a difference between the rows it will call the subroutine.

This is a bit speculative, but the next step after that is probably to write modules for the top-down and bottom-up code too.

Then the rest of the program becomes “glue” for these tested modules. A lot of the functionality is already in modules I built for other tools, such as the code that parses a table definition, finds an optimal index, etc. I’m not sure how much of the code I’ve already written (and tested) will be able to replace parts of the current non-modular script, but I think it’ll be a lot. And I’ll just have to see what’s left over and how much of that fits into yet more modules. With yet more test suites.

The features I’m planning to implement, as well as the bugs I’m planning to fix, are all in the bug tracker at Sourceforge.

Technorati Tags:, ,

You might also like:

  1. Progress on Maatkit bounty
  2. Maatkit version 1297 released
  3. Progress on Maatkit bounty, part 2
  4. Duplicate index checker improved
  5. Growth limits of open-source vis-a-vis MySQL Toolkit

Growth limits of open-source vis-a-vis MySQL Toolkit

Published at November 5, 2007 in Perl, SQL, Coding, Commentary and Tools. 5 Comments

Si Chen wrote recently about the growth limits of open-source projects. He points out that as a project becomes larger, it gets harder to maintain. I can only agree. As the MySQL Toolkit project has grown, it’s become significantly more work to maintain, document, and enhance. (This is why I’m asking you to sponsor me for a week off my regular job to work on MySQL Table Sync, by the way. Please toss some money in the hat.)

Rewriting code so it’s testable is a major focus for me now. Some of these tools have gotten complicated enough that I can’t keep track of all the code. In other words, they’re collapsing under their own weight.

Back in the project’s humble beginnings, it seemed adequate to just copy and paste a few lines here and there; after all, these are just scripts, right? Right. So I’ll just copy a few lines of code that do command-line option parsing and help screens. Hey, it turns out that several of the tools can connect to more than one server, so simple -u, -h and -p options won’t do; so I invent a DSN-like notation that lets the tools connect to an arbitrary number of servers. Copy and paste that code, too. It’s only ten lines — no big deal. Pretty soon I find out that many of the standard Perl modules aren’t available, for a lot of people. And even when they’re available, people have old versions and can’t upgrade, so I can’t rely on basic things like the quote_identifier() function in DBI modules; time to write my own. Well, that’s only a single line! Surely that’s okay to copy and paste.

As Kurt Vonnegut says, “So it goes.” This is the death not only of quality, but of maintainability and extensibility. The Right Answer ™ is to write everything as modules, with proper test suites, and then make the scripts as minimalistic as possible — essentially gluing the modules together with a few lines of harder-to-test code. That’s how I’m used to working, too, but for some reason I can’t explain, it seemed okay not to work that way with this project. That has turned out to be a big mistake, which I’m slowly correcting out of necessity.

But it turns out it’s not that simple, either. I’ve gotten a lot of emails, phone calls from friends, and bug reports about how hard it is to install or update Perl, or get a CPAN module, on many systems. It turns out that a lot of companies are rightfully suspicious about CPAN (I have a tolerate-hate relationship with it myself), and won’t let my consultant friends install or upgrade any module without a lot of red tape. OK, you say, so bundle and distribute the modules the toolkit needs, and they can be installed locally with the toolkit. That sounds nice, but it’s even worse for a variety of reasons. Just to mention one: did you know that it can be a pain in the butt even to set @INC so a module sitting in the same directory with the script will be found by the script? (Please don’t tell me how easy it is, or I’ll let you respond to the next person trying to get it to work on an obscure platform with a Perl installation from the middle ages). Okay, I’ll mention two reasons: some Perl modules have to be compiled and customized just for the operating system you’re installing them on, or they’ll segfault (of all things)! Don’t get me wrong, I don’t think the grass is greener on the other side; no way do I want to try writing these things in C or Java. Perl is about as portable as it gets.

The net result is that I have to do a lot of little tricks to make these things standalone programs, as much as humanly possible. I’m trying to reduce dependencies on external modules, even those that are part of core Perl. I’m re-inventing functionality because it’s not available in all versions. I’m writing modules that can be tested, but I’m not shipping them as separate modules; I’m basically using sed to copy-and-paste the module’s code into the scripts.

Why am I doing all this work?

Because it’s less work than not doing it.

But it is significantly more work than just whacking together some “scripts” and uploading them to sourceforge. That’s why there is a critical mass beyond which it gets harder to grow a project. The solution to this is to find a way to do things differently, work smarter, not harder. The challenge is to switch the fight against the demons of bad code and maintainability so it’s on my terms. In other words, don’t fight against these characteristics of growth; make them work for me. I won’t say I’ve learned that lesson completely, but I’m starting. That’s why I’m automating basically everything about this project (though for some reason I can’t get WWW::Mechanize to stay logged into Sourceforge, so I’m having a hard time automating part of the release process).

I’m also considering ways to provide this toolkit without taking so much out of my own pocket. What started out as me developing tools for my employer, and them graciously agreeing to let me make them available for Sourceforge, has gone far beyond my employer’s needs now. I can’t ask my employer to carry the weight, so it has fallen to me for a while now. That’s okay for some period while I work out how to do it differently, but not indefinitely. Among other things, it cuts into time I want to spend with my wife. Charging for support has definitely crossed my mind, as has some kind of community/enterprise split (such as the one Zmanda does). I don’t want to go there yet — so I’m just asking for a week of sponsored time off work, to begin with.

By the way, the process of replacing copy/pasted code isn’t without its hitches. I just found and fixed a bug in MySQL Table Checksum that I caused by moving the DSN parsing code to a module. And someone else just reported a different bug in another tool, where it turns out the copy/pasted code wasn’t quite identical and I changed the functionality by moving it to the module. Release early, release often. Rely on users to find bugs and report them. So it goes.

Technorati Tags:, , , , , , , ,

You might also like:

  1. Maatkit bounty begins tomorrow
  2. MySQL Toolkit needs a new name
  3. MySQL Toolkit updated
« Previous Entries