Measuring open-source success by jobs
It’s notoriously hard to measure the usage of open-source software. Software that’s open-source or free can be redistributed far and wide, so the original creators have no idea how many times it’s installed, deployed, or distributed. As a proxy, we often use downloads, but that’s woefully inadequate.
I’ve recently begun trying to figure out how many job openings are mentioning various open-source projects. I think that this might be a better metric because it’s driven by the end result (usage), rather than intermediate processes (downloads, etc). I think that it’s likely that usage and demand for skilled people is somewhat realistically related.
To be more concrete, I’ve been watching RSS feeds from job posting aggregators for several alternative versions of MySQL: Percona Server, MariaDB, and Drizzle. It appears that Percona Server is by far the most in-demand in terms of job skills. (I haven’t seen a job posting for the others at all, so far.)
On the other hand, my sample is skewed; I think Percona Server is better known in America, but MariaDB might be more visible in Europe. And I’m not sure that the sample data set is large enough to be statistically significant. Percona Server jobs are utterly dwarfed by MySQL jobs.
There are other flaws in my method: some software doesn’t really need as much manpower to run as others. I would say that given an equal number of WordPress and Drupal websites, more of the Drupal websites are going to be trying to hire experts to manage their sites. So nothing is apples to apples.
What do you think about this metric and its merits or drawbacks? Is there a better way to figure out how much adoption a project really has?
Yeah very hard to quantify MariaDB/Percona usage as it isn’t like web servers where there’s a signature tag to identify usage which allows for metrics such as netcraft’s web server survey http://news.netcraft.com/archives/category/web-server-survey/ for apache, IIS, nginx usage tally. Of course the signature version can be turned off so not entirely accurate.
Job demand as a metric probably too skewed depending on usage. i.e. if a large web hosting company adopts Percona or MariaDB, they could deploy it on 10,000s of servers but still have a fixed number of system admins handling those instances. It would hide real usage figures as such.
Maybe measure the number of software updates as versions are released ??
George Liu
4 Jul 11 at 11:17 pm
Had a quick thought what about having such a signature tag on install of percona/mariadb you could setup a apache alias with empty file /percona51, /percona55, /mariadb52, /mysql50, /mysql51, /mysql55
You could then do what netcraft does to find out what sites use which version ?
For non-apache sites using nginx, litespeed etc, would have to figure out a way to setup such an alias though.
George Liu
4 Jul 11 at 11:28 pm
I’m interested in these job posting aggregators that you talk about. I’m surprised that they’re mentioning Percona Server by name as opposed to just MySQL (like I see normally). Please do furnish us with the URLs. Thanks!
Colin Charles
4 Jul 11 at 11:59 pm
It is not certain that there ever was a vacancy announcements before a job is positioned. I think most people in Monty Programs/MariaDB and SkySQL just became employed because they were ‘known faces’ and some unofficial contact initially went either way.
A little polemical I’d say that if Percona needs to announce vacancies to attract people they have a problem! :-)
I have also no method for answering the question on how to measure open-source success.
Peter Laursen
5 Jul 11 at 5:36 am
Debian has a popularity contest page for this!
http://qa.debian.org/popcon.php?package=mysql-5.1
I think you should use a formula like this
debian_popcon*0.1+jobs*0.3+downloads*0.1+google_results*0.05
Daniël van Eeden
5 Jul 11 at 5:55 am
Colin, I’ve just done some Yahoo Pipes feeds from JuJu, Simply Hired, etc for searches for “Percona” and “MariaDB” and the like.
Xaprb
5 Jul 11 at 8:40 am
Peter, I’m not talking about people who work for Percona or SkySQL or Monty Program, but rather your average company who wants to hire a DBA with skills for MySQL/MariaDB/Percona Server/Drizzle.
Xaprb
5 Jul 11 at 8:40 am
I think my feeds are broken. I’m manually searching for MariaDB and Drizzle now, and I see some listings. Please disregard my statement about that in the body of the post.
Xaprb
5 Jul 11 at 9:35 am