Simple Guidelines For Maintainable Spreadsheets

The spreadsheet is one of the most powerful inventions in the history of computing. But with that power comes responsibility: just as with a programming language, the spreadsheet itself can become difficult to understand and maintain.

Ledger

» Continue Reading (about 1200 words)

Excel Hacks To Ignore Missing Data

I’ve done quite a bit of work with Excel over the last few years, and I’ve found a couple of recurring problems when there’s missing or error data in ranges. I’ve had to work around this enough times that I thought it was worth sharing the solutions I’ve used.

Beautiful green bird eating orange peels

» Continue Reading (about 700 words)

The Age Of Smart Machines

What will life be like in the age of smart machines? According to a Batten Institute briefing on Innovation in the Age of Smart Machines, up to 66% of the U.S. workforce may lose their jobs to computers in the coming decades. The report points out, however, that humans will always be needed to direct the computers. I’m not so sure.

ZT 0001

» Continue Reading (about 900 words)

If Eventual Consistency Seems Hard, Wait Till You Try MVCC

This should sound familiar:

One of the great lies about NoSQL databases is that they’re simple. Simplicity done wrong makes things a lot harder and more complicated to develop and operate. Programmers and operations staff end up reimplementing (badly) things the database should do.

Nobody argued this line of reasoning more vigorously than when trying to defend relational databases, especially during the darkest years (ca. 2009-2010), when NoSQL still meant NO SQL DAMMIT, all sorts of NoSQL databases were sprouting, and most of them were massively overhyped. But as valid as those arguments against NoSQL’s “false economy” simplicity were and are, the arguments against relational databases’ complexity hold true, too.

Puzzle

» Continue Reading (about 3100 words)

A Free Tutorial On Go's Database/SQL Package

Do you use Google’s Go language (golang)? Do you use a relational database such as MySQL or PostgreSQL with it? Do you want to learn how to? Go has a package called database/sql for connecting to relational databases. There’s package documentation, but you’ll need to read the source code if you really want to understand how to use the package. The documentation doesn’t really explain how to use the package, it just explains what it does.

» Continue Reading (about 200 words)

Why Deployment Freezes Don't Prevent Outages

I have $10 that says you’ve experienced this before: there’s a holiday, trade show, or other important event coming up. Management is worried about the risk of an outage during this all-important time, and restricts deployments from the week prior through the end of the event.

What really happens, of course, is that the system in question becomes booby-trapped with extra risk. As a result, problems are more likely, and when there there is even a slight issue, it has the potential to escalate into a major crisis.

Why does this happen? As usual, there’s no single root cause, but a variety of problems combine to create a brittle, risky situation.

freeze

Assumptions

When managers declare a freeze, they’re not being malicious. They’re doing something that seems to make sense. That’s why it’s important to understand the reasoning.

» Continue Reading (about 2100 words)

A Review Of The Docker Book

The Docker Book is a newly published book from James Turnbull, whose name you will recognize if you’re at all familiar with DevOps, Puppet, or Docker itself. It’s a nice introduction to what Docker is and how to get started using it. It’s like Goldilocks — not too detailed, not too superficial, just right. The book starts from the basics, assuming no prior knowledge with Docker, or even most of the core concepts of virtualization, but moves quickly through these topics into installing Docker and getting started with it.

» Continue Reading (about 300 words)

Mac's Time Machine and Symlinks

I use Mac OSX’s built-in Time Machine for backups, and a couple of times I’ve noticed my backups failed and couldn’t be completed successfully. I was unable to fix the problem until I reformatted the backup drive. Today I think I stumbled on the solution.

Time Machine

» Continue Reading (about 200 words)

Unfixable Code

Over the years I’ve come to believe something that I’m not sure others will agree with. I would like to hear your point of view on it.

I posit that some code can become literally unfixable. Programmers can paint themselves into a corner with the code and it becomes impossible to get out again.

humpty

The scenario arises when a specific set of conditions exists:

» Continue Reading (about 300 words)

Can Anomaly Detection Solve Alert Spam?

Anomaly detection is all the buzz these days in the “#monitoringlove” community. The conversation usually goes something like the following: Alerts are spammy and often generate false positives. What you really want to know is when something anomalous is happening. Anomaly detection can replace static thresholds and heuristics. The result will be better accuracy and lower noise. I’m going to give a webinar about the science of statistical anomaly detection on June 17th.

» Continue Reading (about 100 words)

Can MySQL be a 12-factor service?

A while ago I wrote about some of the things that can make MySQL unreliable or hard to operate. Some time after that, in a completely unrelated topic, someone made me aware of a set of principles called 12-factor that I believe originated from experiences building Heroku.

Dodecahedron

That’s been over a year, and I’ve come to increasingly agree with the 12-factor principles. I guess I’m extremely late to the party, but making applications behave in 12-factor-compliant ways has solved a lot of problems for me.

This experience has repeatedly reminded me of one of the applications that continues to cause a lot of the kinds of pain that the 12-factor principles have solved for me: MySQL.

» Continue Reading (about 500 words)

Monitorama 2014: This One Weird Time-Series Math Trick

Monitorama 2014 Portland has been a great show. I’ve enjoyed the technical nature of the talks, the diversity of the speakers, the topics from hilarious to thought-provoking, and the stage in a theater, set up for a Shakespearean tragedy. I have also taken a lot of notes. For example, Toufic from Metafor Software suggested that the audience look into the Kolmogorov-Smirnov test. I am proud of the slide that made its way into my talk as a result:

» Continue Reading (about 200 words)

GopherCon 2014

I spoke at Gophercon last week in Denver, and it was one of the best conferences I’ve attended. I can’t remember learning so much and meeting so many great people in years. I have page after page of notes in my notebook, many of which I’ve yet to follow up on. The conference prompted a burst of learning and a flurry of creativity for me, as well as a huge list of things to study further.

Gophercon

In no particular order, here are some of the many highlights for me:

» Continue Reading (about 600 words)

Go MySQL Drivers

If you’re interested in Google’s Go programming language, perhaps you aren’t sure what drivers to use for MySQL. The good news is there are excellent drivers for MySQL. There are several opensource ones on GitHub and elsewhere, but the driver I recommend is https://github.com/go-sql-driver/mysql/. Why? It is pure Go, not a wrapper around a C library, and is liberally licensed. It is high performance. A lot of work has gone into making it avoid allocations and consume minimal CPU.

» Continue Reading (about 300 words)

Ultima Online and the History of Sharding

Have you heard of sharding a database? Of course you have. Do you know where the term comes from? Someone asked me this at a cocktail party recently. I gave it my best shot.

“The earliest I remember was Google engineers using it to describe the architecture of some things,” I said. “That would have been about 2006.”

“Nope. Much earlier than that,” said my new friend.

ultima

I pondered. “Well, I guess there was the famous LiveJournal architecture article about MySQL. That was, I dunno, 2003?”

The person then told me the following history. I can neither confirm nor deny it; what do you know about it?

» Continue Reading (about 400 words)

Switching from Sublime Text back to Vim

I’ve used Vim for as long as I can remember, but when I started to work with Go at VividCortex, for some reason I started to use Sublime Text instead. It does make a very nice GUI-based editor, but I never felt that it was as powerful as Vim.

Ever notice how the Vim logo looks a little like Superman’s logo? No? Squint a little harder, then.

Superman

» Continue Reading (about 300 words)

Bloom Filters Made Easy

I mentioned Bloom Filters in my talk today at Strata. Afterwards, someone told me it was the first time he’d heard of Bloom Filters, so I thought I’d write a little explanation of what they are, what they do, and how they work. But then I found that Jason Davies already wrote a great article about it. Play with his live demo. I was able to get a false positive through luck in a few keystrokes: add alice, bob, and carol to the filter, then test the filter for candiceaklda.

» Continue Reading (about 200 words)

Generating Realistic Time Series Data

I am interested in compiling a list of techniques to generate fake time-series data that looks and behaves realistically. The goal is to make a mock API for developers to work against, without needing bulky sets of real data, which are annoying to deal with, especially as things change and new types of data are needed. To achieve this, I think several specific things need to be addressed: What common classes or categories of time-series data are there?

» Continue Reading (about 500 words)

Continuous integration and deployment

I’ve been talking to some smart people about deployment. First a little background. One of my colleagues was working on a project that ultimately didn’t bear fruit. It was a system for continuous delivery, and involved reacting to git push by building and shipping to production. But it felt as if the problem shouldn’t be separated from provisioning, and from setting up a development environment, and so these things got folded in, and the effort became a boil-the-ocean project that had to be set aside.

» Continue Reading (about 1200 words)

Speaking at Percona University Sept 12th

I’ll be joining Percona for a free day of MySQL education and insight at their upcoming Percona University Washington DC event on September 12th. My topic is accessing MySQL from Google’s Go programming language. I’ve learned a lot about this over the past year or so, and hopefully I can help you get a quick-start. If you’re not familiar with Go, it’s the darling of the Hacker News crowd these days.

» Continue Reading (about 300 words)