Desktop archive at Xaprb

Archive for the 'Desktop' Category

Favorite USB wireless card for Ubuntu?

Published at November 28, 2007 in Desktop and GNU/Linux. 4 Comments

Dear LazyWeb,

Do you have a favorite USB wifi network card for a laptop running Ubuntu?

Sincerely, Xaprb

Technorati Tags:ubuntu, usb, wireless networking

How to set up dual monitors in Ubuntu on Dell Inspiron 1501

Published at August 10, 2007 in Desktop and GNU/Linux. 4 Comments

It took me about five minutes to get dual monitors working on my Dell Inspiron 1501 under Ubuntu 7.04. Here’s how I did it.

I tried the xorg driver, which had been working fine on just the laptop display, but it wouldn’t work; I could get either the external display or the internal display to show, but not both. If you get it working, post a comment and let me know.
I enabled the proprietary driver via Ubuntu’s Restricted Drivers Manager.
I read the Gentoo Wiki page on dual monitors.
As root, I ran
aticonfig --initial=dual-head --screen-layout=above -v
I rebooted.

At the moment, I’m typing into Firefox on the laptop monitor. My Dell 1800FP monitor is perched right above it; it’s the same width in pixels, and almost the same physical width. I have no windows open on that monitor, but I do have a nice background image! XFCE configures the backgrounds for each display separately.

I’ve been using this setup for about a week now. It’s not flawless, but the flaws don’t get in my way much. Here are the problems I’ve noticed:

The driver is proprietary. Okay, this is a major flaw.
The driver doesn’t work perfectly. Occasionally a weird defect in the screen image appears and won’t go away until X is restarted. It looks like a bar code and usually shows up near the bottom right of one of the monitors, often right over the clock in my system tray.
If I log out of XFCE, my system hangs and I have to use the Alt-PrntScrn keys to shut it down and restart it.
When I start my laptop, only the display on my laptop shows anything (perhaps the login screen isn’t dual-monitor capable). As soon as I log in, both monitors become active. While this is happening, my mouse randomly jumps between the top-left corner of the two monitors.
The two monitors are running two different X displays. I’m not crystal-clear on how this should normally work, but I get the idea Xinerama isn’t the same as this and should work better (I don’t know if I can set up Xinerama with the proprietary driver, but I don’t think so). This has a variety of side effects:
1. Windows I open on one display can’t be moved to the other (oddly, I can drag and drop between displays, which I didn’t expect).
2. I can’t alt-tab to the other display.
3. When I click on a link in Thunderbird, if Firefox is running on the other display, it says Firefox is already running but not responding, and refuses to open the link. I can’t get Firefox running on both displays at once.
4. My XFCE panel on the laptop display doesn’t show windows on the other display. I tried creating a panel for that display too (XFCE recognizes that there are two displays and lets me place the panel on either one), but I couldn’t place it at the bottom of the display. When I chose to place it at the bottom, it seemed to place itself 768 pixels from the top (the external monitor is 1280×1024, but the laptop display is 1280×768). So I placed it at the top of the display, added a taskbar applet to it, and voila I had what I wanted — but when I rebooted, the panel showed up on the laptop display again.

Otherwise I haven’t noticed any troubles. Anyone who has suggestions on these issues, feel free to post a comment!

Technorati Tags:dell inspiron 1501, dual monitors, ubuntu, xfce, xorg

How to create stepping slides in OpenOffice.org Impress

Published at August 6, 2007 in Desktop. 6 Comments

If you’ve used Microsoft Powerpoint to create “stepping” slides — slides that appear one bullet point at a time — and can’t figure out how to do it with OpenOffice.org Impress, this article is for you.

What are stepping slides?

“Stepping” slides are the only animation or transition effect I allow myself in slideshows. I don’t generally like animations or other distractions; when I give talks, I am always painfully conscious of how much the audience tends to focus on the slides. I’ve seen some research that suggests people’s brains turn off when they look at slides, so I try to minimize that by making as few slides as possible and engaging the audience.

But enough about me, what do you think about my shirt?

Seriously: stepping slides one bullet point at a time is helpful. It lets me pack a lot more onto a slide, so I can build the story around a concept gradually, increasing the mental density of whatever’s on screen. If the whole slide pops up at once, it’s a distraction. If I split it into a bunch of slides, it’s a distraction.

How do you do it with OpenOffice.org Impress?

Don’t search the Help files for “stepping” — I went that route and crashed OpenOffice.org. Strangely, at the moment the search function crashed, I was writing a presentation about search and indexing algorithms!

You have to do it as an animation. One step at a time (no pun intended):

First, create a slideshow and add a bulleted list:
Next, select the list text and choose the “Custom Animation” sub-pane in the right-hand side:
Click the Add… button in the Custom Animation pane and select Appear, then OK to dismiss the dialog box:
If your Custom Animation pane is large enough, you’ll see a small preview of the bullet points at the bottom. Notice there’s a mouse-click icon next to the first one, and the “Start” pull-down menu is blank (no selection). At this point, all the bullet points are going to be animated as a unit:
The last step is to make the animation start upon clicking, and make that apply to each bullet point. Pull down the “Start” drop-down and select “On click”. You should now see a little mouse-click icon next to each bullet point:

You’re done! Test your slideshow just to be sure.

Technorati Tags:animation, impress, microsoft powerpoint, openoffice, slides, stepping

Ubuntu on Dell Inspiron 1501

Published at July 24, 2007 in Desktop, Review and GNU/Linux. 11 Comments

I recently bought a Dell Inspiron 1501, which I got a great deal on thanks to the fine people at DealNews. The base system was $449 shipped, and I chose to upgrade the processor to dual AMD64s. But I didn’t buy the system that came with Ubuntu pre-installed; for whatever reason, the one that came with Windows offered a special discount (normally the Windows tax for otherwise identical machines appears to be around $150, and I’m certainly not going to run Windows).

Therefore, I was not sure Ubuntu would support all the hardware. It’s the same story it’s been for as long as I’ve been using computers: hardware manufacturers withhold specifications from the Free Software world, so there is always a chance something will be a trouble. The good news is, I’ve only noticed two very minor incompatibilities out of the box.

One is that the Fn+arrow keys won’t change my screen brightness, at least under XFCE. Strangely, my ancient Dell laptop had no trouble with that. I assume the old one was a hardware-controlled feature and this one needs some software support, but I could be wrong.

The other thing is the built-in wireless card, which isn’t supported with Ubuntu 7.04’s drivers out of the box. However, I quickly found a set of drivers for the Broadcom Corporation Dell Wireless 1390 card, and was up and running shortly thereafter. The only thing I had to do after installing the drivers was press the Fn+F2 key, which turns the card on.

Otherwise everything works brilliantly.

And now for a rant: click through to that page about the drivers, and you’ll see an example of what I consider the Ubuntu sudo disease. There’s even a screenshot of someone typing sudo uname -a and using sudo to remove a file he didn’t create with sudo. I think unfortunately, Ubuntu’s policy of allowing one to run any command with sudo has created a crop of people who don’t understand what should and shouldn’t be privileged; some of them seem to believe that ’sudo’ is what you type at the beginning of every command. It completely defeats the purpose and circumvents the security gained by not running as root. For my part, when I want to administer my system, I become root, do what I need to do, and then quit again. I rarely sudo any command other than sudo su -.

But that’s just me.

Technorati Tags:dealnews, dell, hardware, networking, ubuntu

More GnuCash to MySQL tools and queries

Published at November 29, 2006 in Desktop, SQL, GNU/Linux and Tools. 4 Comments

Download gnucash2mysql and queries

I wrote a while ago about a program I wrote to export GnuCash data into a MySQL database, including a couple of queries against the resulting schema. I’ve made some improvements since then to allow a simple overlay of my wife’s categories onto the GnuCash hierarchy. This article explains the improved schema, and includes some more useful tools and queries.

The improved schema

The heart of the schema that holds the GnuCash data remains the same, but I’ve added a table to overlay our categorization system onto it. The fundamental issue is the old multiple-hierarchy problem: some transactions should live in different places in the GnuCash account hierarchy than they belong in our expense summary. To remedy this, I just created a table to define our own categories. The resulting system is very simple, and defines a single level of hierarchy, where everything is at the top level; there are no nested categories (though you can simulate nested categories with a naming convention, such as “Auto:Repair”).

This system is about as simple as possible. Here’s the query to create the new table:

create table account_category (
   account char(31) not null primary key,
   category char(20)
) ENGINE=InnoDB;

It would be more normalized to place this data in the account table itself, but that makes it too easy to wipe out your category data when you recreate the schema. For that reason I decided to separate the account and its category.

Category setup

There’s a required initial setup before this is usable, of course: you have to define the categories and associate accounts with them. To ease this step, you should go through your GnuCash file and check the “placeholder” properties checkbox for any accounts you know only exist to hold other accounts. This way you won’t have to worry about assigning them to categories. If you do this, you should run gnucash2mysql to re-create the schema and re-import the data.

Next, run the included setup-categories.pl script. It will ask you to enter a category name for each unassigned account. It prints the account hierarchy so you can easily tell what the account is. If you want to leave an account un-categorized, just press Enter and it’ll be skipped.

The categorization requires that you understand the double-entry accounting principles on which GnuCash is built. The updated queries I’ve included in the download will include transaction splits that transfer money both into and out of a GnuCash account, so think about that as you assign the categories. For example, our GnuCash account “Bills:Groceries” is associated with “Groceries.” When there is a transaction involving “Bills:Groceries,” the money has to come from somewhere — our checking account, for example. We don’t want to include the checking account in any categories, or that money will get counted twice — once as it leaves the checking account, once as it enters the “Bills::Groceries” expense account. This is the fundamental simplification I made to create a non-double-entry report of what we spend. On the other hand, maybe you want to do it differently. Suit yourself :-)

When you’re done

When you’re done associating categories with accounts, you will find new queries in the package for your enjoyment.

Miscellaneous improvements

In the time since I first wrote this program I’ve realized the benefits of using Perl libraries that are already installed on most systems. I’ve gotten rid of the dependencies on non-standard libraries in the new versions. That change should make the scripts easier to install and use. The scripts also read your .my.cnf file if it exists, so you don’t have to specify MySQL options.

At the end of the day, these little scripts and queries are about having more than one way to do things. I hope you find them useful.

Technorati Tags:No Tags

To Gentoo or not to Gentoo?

Published at November 21, 2006 in Desktop, SQL, GNU/Linux and Commentary. 21 Comments

Some people who know I’ve used Gentoo asked me my thoughts on using it for MySQL servers. Here are my opinions and experiences while using Gentoo, both for desktop systems and for servers.

This is long, but hopefully the brain dump is useful to you. I have so many thoughts on this subject that I’m having a hard time really organizing them well, and I’m censoring myself too because I know I represent only a very small, focused viewpoint on Gentoo.

This article is highly subjective, so please take what I say with a spoonful of salt. Your mileage may vary.

Finally, before I begin: I have a lot of reasons for moving away from Gentoo myself, and this article is mostly about why I don’t think Gentoo is a good idea for what I use computers for, but I don’t mean to criticize anyone or say it’s no good in general (in fact I know of some uses where it works great, and lots of places use it). Nothing here is personal. This is simply the train of thought I went through when deciding not to use a source-based, continuous upgrade distribution for my own systems.

The 50,000 foot view

I recently switched away from Gentoo, after using it since 2001 or 2002. I used Red Hat, Slackware and other distributions before that, and my main gripe with them was the unavailability of reasonably recent software; I felt they became stale and forced you to upgrade your entire system to get something other than an ancient version of [insert package name here]. There were also endless dependency conflicts. By contrast, Gentoo let me build a recent version of whatever I wanted, and I felt a lot happier with it.

In recent years though, I’ve become more aware of the pain of a constant, incremental upgrade process, which is what Gentoo gave me. Even though in theory it’s not necessary to constantly keep updated with the most recent version of all installed packages, in practice I find it is. I found if I didn’t keep abreast of changes, I couldn’t catch back up again without a lot of pain.

I also started to question some of the things people assert about Gentoo, such “it is small, light and focused on speed.” I find packages compiled under Gentoo, with the precisely right CFLAGS customized just for my processor and hardware, and with just the right USE flags customized to only install what’s needed, are often much less performant than software compiled for generic i386 architecture with no optimizations. Finally, some of the software I’ve installed under Gentoo seems to be broken.

The following are more details of my experiences.

Upgrading is painful

Upgrading can be an enormous pain:

First you sync,
Then you run emerge -avuD --newuse world and look at what’s going to be upgraded,
Then you cancel and tweak some USE flags,
Then you pull your hair out as you still can’t understand what package is trying to install 301MB of dependencies you don’t want,
Then you use equery and friends to try to find out,
Then you pull your hair out because these tools say there is no such package, or can’t find the files for it, or find the package but say nothing at all,
Finally you get on the forums and figure it out, and get the flags/whatever right. It turns out there’s a new default USE flag for some package that doesn’t show up in the upgrade list, and it wants to install a bunch of “dependencies” it doesn’t need. For example, there are lots of things that want to install multiple versions of QT, including QT itself. If you let it install version 3 for just one program, it will want to simultaneously upgrade itself to 4 and keep the version 3, because that program needs it,
Then you let it upgrade, which still has to download 194MB (hey! you saved 105MB, be happy!) over your DSL connection and takes 4 hours to compile on a fast 64-bit machine loaded with RAM — why? Why, because of course GCC has to be upgraded from 4.1.1-pre5_r16a to 4.1.1-pre5_r16b, which is probably three lines of source that changed. But as I say later, you don’t dare not upgrade GCC.

Now your system is upgraded, right? Wrong, everything is broken because it isn’t linked against the new glibc/QT/whatever. Time to revdep-rebuild — oh goody, it wants to rebuild your entire system. Another 7 hours of compiling. Cozy up to the back of your computer’s case, it’s cold in the basement and that thing is warm. Who needs a fireplace? Aaahhhhh. Watch a movie.

You won’t have to do this again for another three days, don’t worry.

I’m exaggerating for effect. It isn’t like this every three days. There is something to upgrade every three days, or even every day, but it’s usually small and incremental. However, if you let it slip, it becomes a nightmare, and even when you keep up with it, occasionally you still get this kind of mess. I am absolutely not exaggerating about the size of these. Recent memory contains painful un-repressed experiences with xorg and GCC, which took weeks to solve on certain machines (how much pain you experience totally varies from system to system, depending on what you’ve installed and how it’s configured). And even when this doesn’t happen, it is a royal pain to do this every few days. A constant upgrade process is just way too much work.

Upgrading breaks things

Upgrading can badly break things too frequently for my taste. For example, these are things that used to break on my desktop machine all the time:

My atheros network card. Every time I upgraded anything to do with networking (such as my kernel, or the drivers), networking broke. I know it has to be this way, but when upgrading a kernel, it’s tough to remember everything else that has to be rebuilt against it. I usually figured it out by rebooting and noticing that the network wouldn’t come up, at which point I didn’t have a working network to fetch the latest madwifi drivers for building (oops, I should have been upgrading constantly; I’d have the .tar.gz files cached if I did).
Samba. Any idea how many things in a typical Samba-enabled desktop system are linked against Samba? Me neither, I’m trying to forget. Upgrading a kernel means re-compiling all that stuff (if there’s a way to avoid it, I never found it, and that’s what revdep-rebuild told me to do — and the only thing that worked for me).

There is no way around this, as far as I know.

Even in the best case, it’s too slow

Downloading and building everything from source may promise “infinite flexibility” (more on this later), but it’s too slow. It doesn’t seem that way at first, but after a while, it gets tedious. Compiling everything is just a waste of time, for most people and most purposes. Why should everyone have to start from source code with every package? If we all compiled all our own software, we’d waste billions of hours of processor time every year. There’s something to be said for doing things once and then enjoying the benefits, and compilation is a good example, in my opinion.

Downloading the source is also usually a larger download than getting pre-compiled packages. It’s a waste of bandwidth.

Gentoo is not optimized for my hardware!

As I said above, the whole “Gentoo is optimized” train of thought doesn’t ring true for me. Here are some experiences I’ve had where the Gentoo “optimized” build ran very badly on my machine:

Firefox. When I used an official binary package, Firefox ran like a dream. But if I did this, I couldn’t link against it for building the desktop environment, so then that desktop environment wanted to build the Mozilla suite. So one way or another, I ended up either compiling Firefox from source, or Mozilla from source. And Firefox from source ran like a dog. Even scrolling in a text field was slow. I know, I know — I read all the forum entries about locales and blah blah, and it helped, but it still ran badly. One note about desktop environments — I like to use Fluxbox or XFCE, but there are certain things that have at least some dependency on gnome-base. I don’t mind having that installed, except that it in turn depends on things… and that makes for a lot more things to upgrade.
Any DVD playing software.
OpenOffice.org
MySQL server (I think — preliminary results discussed below).

These are the places where I most notice the speed difference. It beats me why packages I compile from source are slower. I bet I’ve spent a hundred hours trying to learn enough to fix these and other speed problems, but I never could.

The only thing I think might have been more responsive on Gentoo, before I switched to a binary distribution on one machine, is GTK+. On this machine now I notice a small lag in bringing up dialog boxes and so forth. (QT has always been lightning quick for me, no matter what).

Gentoo doesn’t have the latest and greatest

While Gentoo usually has more up-to-date versions of packages in its source tree than Red Hat sometimes used to in 5.1 (ah, the good old days), it doesn’t always have updated versions. I can’t think of a good example right now, except for how long it took to get XFCE 4.2 into the tree, but I know I’ve seen it — common desktop apps that are a year and a half out of date or something like that. Sorry I’m getting a bit hand-wavy here, but I’m sure if you browse Portage you’ll find something really stale, which binary distributions provide much more up-to-date. In general, I don’t think Gentoo is any better or worse than a binary distro.

Gentoo builds are sometimes really broken

Other things I’ve written about are painful, but this is worse. Slow is bad, incorrect is worse. I can’t tell you all the programs I’ve seen behave bizarrely or crash after being built from source on Gentoo (and I stress I don’t do anything exotic with USE flags or compiler optimizations). Most programs work fine, it’s true, but some don’t. Here’s a partial list:

Kexi was completely and bizarrely broken on my Gentoo box. Even though it was supposedly version 1.6.something, dialog boxes popped up saying version 0.8, large sections of functionality simply didn’t exist or didn’t work, and everything crashed constantly. I installed a lower version number on a binary distribution and everything worked like a dream. This was really strange; it was as though the Gentoo build was mixing together source from different trees!
MySQL Administrator and MySQL Query Browser were similarly busted beyond all recognition. Yet when I downloaded the official builds from MySQL and ran them, everything worked flawlessly. I tried this multiple times on many different machines. Why can’t the Gentoo version — built from the same source, I assume — be made to work? Granted, it was masked in portage, but why should it be if the official builds work great?
Perl’s MySQL support broke, then unbroke, then broke… I don’t know whether it was MySQL, Perl, or the DBI or DBD libraries (or something else). I couldn’t connect with AutoCommit disabled; it would throw an error saying “transactions are not supported.” I found a number of places where other people complained of the same problem, but never found a solution, and recompiling endlessly never fixed it.
- Update: this is not Gentoo’s fault, it’s MySQL’s fault (bug 21829). The reason I’ve never seen it elsewhere is because other distributions choose a non-broken MySQL version. In this case I shot myself in the foot by choosing a broken MySQL.
GRUB mysteriously broke one day after an upgrade. I could never get it to work again. It would start and then have an error, which I forget now, and drop into a command-line. I had to manually type the boot commands at the GRUB command-line every time I booted. I probably spent five or six hours trying to solve this too. Again I saw many other people had the same problem, and none of the tutorials or FAQs fixed it, even when they described the problem exactly.
The madwifi libraries were flakey at best. Half the time when I booted, the Atheros module wouldn’t find the card because the device didn’t exist. I suspected something was wrong with my hardware, but then I switched distributions. Nary a problem since then.

Experienced developers will likely see this list and see I’m confused about the source of some problem, or “oh, you just need to do X” or something. Why does it take an experienced Gentoo developer to properly configure a Gentoo machine? Have you ever noticed that’s who gives the answers to the sticky problems on the forums? Users shouldn’t have to be C programmers to know how to configure their system. Unfortunately, when you run into these issues you do, as far as I can tell — and that’s a risk.

Infinite flexibility and minimality are myths

Gentoo isn’t infinitely flexible, no matter how many USE flags there are. Here are two examples where you can’t get just some part of the required functionality.

The first is MySQL. You can’t just get the mysql client, or C libraries, which means you have to download and build the entire server if anything on your system needs MySQL support. For example, suppose you’re setting up a web server, which will run Perl web apps that talk to other servers running MySQL. You can’t just build Perl with MySQL support. You build dev-perl/DBD-mysql, and it depends on dev-db/mysql. Look at the ebuild:

DEPEND="dev-lang/perl
	dev-perl/DBI
	dev-db/mysql"

If you want anything to do with MySQL, you just installed a MySQL server, even if you wanted your web servers to be “minimal.” Other distributions let you get just the client programs or C libraries.

The next example is Samba. It’s much the same — you can’t just get Samba client libraries and programs, you install a Samba server too.

Even USE flags don’t give you full flexibility. For example, mytop will use ANSI color codes and hi-res time if there’s support for them, but it doesn’t really need them. However, they are listed as dependencies in Portage, so even though they’re optional for mytop, they’re not optional in Gentoo. In theory you could add USE flags for that, but in practice, you don’t get the choice. In theory, Gentoo is infinitely flexible, but in practice it’s not. This should not be a surprise.

I’m sure there are other examples, but I don’t want to get into it too much. I don’t mean to put down the developers’ hard work, but in these cases this just doesn’t do what I want. I want a system that can talk to MySQL or Samba servers, without being one itself, and without having to compile all that extra code.

It’s reinventing the wheel

As a distribution, Gentoo has been and continues to be great. I can’t thank the Gentoo people enough. But I personally don’t want to run any system that I can’t just install and run, with a modicum of knowledge and reasonable installation/configuration time. I’m not a “protect me from complexity because it scares me” person. I’m a “don’t make me do all this work, my time is scarce” person. I do not want to solve the same problems other people are solving, over and over. I want a distribution that solves the problems and gives me the result. If you don’t think Gentoo users reinvent the wheel thousands of times a day, go read the forums!

Package staleness vs. flexibility vs. stable release cycles

Gentoo is all about choice and flexibility. While that is true, as I said infinite flexibility isn’t a reality. What you get instead is the ability to choose exactly which versions of packages you want (theoretically).

For example, if I use a binary distribution with regular “snapshot” releases and only security upgrades, I might be limited to versions 1, 2 and 3 of certain packages. Maybe I want those packages to be versions 2, 2 and 3, but when I upgrade to a newer release of my binary distribution, I can only get versions 2, 3 and 3. Gentoo will likely let me install exactly the versions I want.

That’s the upside of being able to mix and match versions of software. The downside is “infinite flexibility” really means much greater system complexity, when you consider the number of combinations of packages, versions, and interdependencies between them.

Binary or “snapshot” releases are much simpler and the packages can be much better tested with each other. Simple math shows this: if you have N packages in a release snapshot, you have a graph with potentially (N-1)^2 edges. If you have a distribution that lets you install many versions of a given package, say 5 on average (I think this is about right for Gentoo), you have (5(N-1))^2 edges in the graph — 25 times as many. In real numbers, Gentoo currently has 11453 packages in the Portage tree. A hypothetical snapshot release would have about 131 million package inter-relationships, while Gentoo with an average of 5 versions per package would have about 3.27 billion inter-relationships.

In practice, most of these packages aren’t installed and don’t actually interact with each other, so the number of relationships to test and integrate is much smaller, but you can see how the combinatorial explosion makes it much less likely that any given set of packages has really been installed widely and tested together widely.

Combine this with the fact that most binary distro users are using basically the same system, starting from the same installation CD, and you get an idea how much less variability there is in, say, people running the stable version of Ubuntu. A person who installs Gentoo, doesn’t tweak USE flags or other settings at all, syncs her Portage tree, and runs emerge -uD world probably ends up with an entirely unique system, different from any other system anywhere in the world. Just like a snowflake. That’s bad, not good.

Gentoo is like a continuously variable function, whereas a snapshot-based distribution is like a step function.

There’s something else I haven’t added into the mix: USE flags and compiler and CPU settings. I doubt your web browser has enough zeroes to express the combinatorial explosion that causes.

For this reason, I feel far less comfortable with building everything from source and continuously upgrading, than accepting something that’s a few months old but better tested.

This isn’t just theory. Many packages won’t even build unless you get everything just right. For example, a lot of “package won’t compile” forum entries are solved with answers like “you can’t build that with GCC 4.1, you have to upgrade to 4.1.1.” Who knew?

While we’re on the subject…

Certain upgrades can be either difficult or impossible, depending on how much software you have installed. When the most recent “large” GCC upgrade appeared, I tried to put off upgrading my home desktop machine to it because I saw forum threads from people having huge problems. I thought maybe these would get ironed out over time. But then I needed to upgrade or install something myself, and it wouldn’t build with my current GCC version. The only option was to upgrade GCC, I think to 4.1.1.

“Nightmare” doesn’t really capture the experience. Everything broke, and I mean everything. It would have been far easier to just re-install the system from scratch — except that when you install your system, you get whatever GCC is on the CD, so you still have to complete the upgrade. When you go through one of these upgrades, you have to re-compile your entire system, all at once in the proper order. If it breaks partway through — which it will unless you’ve got a really minimal system (e.g. not a desktop machine) — you have to restart, because the dependencies and order of compilation really matters. And there’s no facility for only re-compiling the things that aren’t already re-compiled. After about three or four cycles of emerge --emptytree world, watch it break after eight or ten hours, read the forums and try something, restart only to have the same package break again, it was just too much for me. I switched to Ubuntu.

I’ve had a good bit of experience with other people’s installations too. I know of several servers at various sites that have bumped up against that GCC upgrade cliff. I have only seen it solved by rebuilding the machine from scratch, and upgrading the system before adding non-base-system packages in. I have seen servers remain in the queue a long time for that, as they can’t always be taken down and rebuilt at will. If you try to put it off, every time you try to change anything you’ll likely find its Portage tree is out of sync and refers to packages that don’t exist anymore, so you have to re-sync Portage, and then tons of stuff wants to be upgraded as a dependency. Sometimes I’ve seen machines get so stale they want to upgrade hundreds of packages, but that can’t happen or everything would bust. This is an ugly reality of life with Gentoo, as far as I’m concerned.

If you’re running a desktop machine on Gentoo, there are other painful upgrades too. My main home desktop machine somehow had no problems upgrading xorg to the modular distribution, but on another machine, I basically had to nuke xorg and install it from scratch — and then the modular builds failed, because they were really buggy and didn’t properly list all dependencies, etc (and when there are over 300 modular xorg packages alone to build, this is pretty ugly too). So everything failed and I didn’t have a graphical desktop. I’m okay working in that environment — I spend most of my day at a terminal anyway — but it’s nice to have a web browser. I could not resolve this, and spending days reading forums was simply not an option. I may have had that luxury in the past when I wasn’t really busy on other things (ha, ha), but not anymore. I ended up upgrading that machine to Ubuntu as well.

Configuration management is tough

It’s really tough to get all your machines uniform. There are ways around it, but you still have to expend that effort, and it can be significant. I know of people who set up a build machine that will host binaries for other machines, such as white-box workhorses. This machine is a proxy for the others, and they sync up with it. You can also use g-cpan to manage Perl code in a separate Portage overlay, so you’ll have uniform Perl module versions too. But this is a fair amount of work no matter what, at least with the tools I know about.

That might only be feasible with cheap white-box workhorses. What if you need this same process for really expensive machines? Can you spare a really expensive machine big-iron server just to act as a build host? Even if you do, do all your machines have the same hardware? That’s kinda doubtful. Virtual machines are only a partial solution — a virtual machine isn’t the same as the server it runs on, and installing a virtual machine by definition makes the server different from the others in its “class.”

You might try documenting your install process. I guess you could get machines reasonably close to identical, as long as you have a single Portage tree you’re syncing against. (Whatever you do, don’t sync against the main Gentoo mirrors, or you’re finished before you start. It changes constantly.) I’ve personally had mixed success with this type of process. I don’t think it works as well as it might. There are still too many places to mess up, too many things that introduce variability between machines.

There are ways to deal with all this, but they require extra work, and are basically imposing a “snapshot release cycle” on top of Portage’s stream-of-consciousness continuous change, which is a wheel that’s been invented in almost every other distro out there. I think Gentoo is enough work already.

When you have a problem, where is the problem?

When you’ve compiled everything from source, with your own USE flags, linked against your own libraries etc, you have a version of the software that’s different from anyone else’s anywhere else in the world. And when you have problems with that, you don’t know what the problem is. Is it a bug, or is it just your system?

You can’t always know. Based on my other experiences with things being broken, I am not prepared to say any Gentoo build isn’t subtly buggered.

For example, I’ve seen a business have trouble with MySQL replication. Is it MySQL, is it the way they’ve been doing replication, is it the setup and configuration, or is it that they have a broken MySQL build? I don’t know. They are not executing the same bits as anyone else. I don’t even feel comfortable submitting certain of their bugs as bug reports to MySQL because of this.

I suggest using the official MySQL builds

One site using Gentoo recently decided to download MySQL’s official build and untar it instead of building it from Gentoo’s source tree. Preliminary results indicate this was a wise decision. For one thing, the official build seems to be getting dramatically higher throughput. There were queries that ran in 90+ minutes on Gentoo’s build, and after switching to the official build, ran in 20 minutes (granted, they also went from 5.0.21-log to 5.0.26-standard, but that isn’t the cause of the increased performance!). This is not the only indicator of better performance, as the disk activity increased significantly (by about 2x) on the official build too. That might indicate the CPU is being used much more efficiently. In fact, the official build seems to saturate the I/O, so the disk is now the bottleneck, which is as it should be in that site’s workload.

Regardless of whether you run Gentoo or a binary distribution, I generally advocate running the same bits as other MySQL users. I’ve chatted with dozens of people on IRC who are having trouble with broken builds from third parties, not just Gentoo users. The official builds are tested by thousands of people already, so the bits are much more likely to be good. And if there’s a problem, it makes it dramatically easier to troubleshoot and/or give good bug reports.

This has its downsides too. Getting other things in Portage to link against MySQL libraries, etc etc, without installing the MySQL package from Portage, can be non-trivial. You probably don’t want to install both, or you’ll never know what your programs are linked against.

Conclusion

That’s a heck of a brain dump, but hopefully you can sort through it and glean something from it. I would summarize by saying any source-based, continuous-upgrade distribution is less likely to be well-tested and widely verified, the complexity is much higher, and it is probably much slower and more difficult to install, maintain and upgrade, and keep everything at the same configuration.

If you’re using Gentoo for a MySQL server, I would at a minimum install the official builds, not build from source. I don’t want to say what distribution to use for a MySQL server, but my personal experience has made me want to stay away from Gentoo.

But that’s just my opinion :-)

Technorati Tags:No Tags

Permit Cookies: a Firefox extension that makes cookie whitelisting easy

Published at October 23, 2006 in Desktop, World Wide Web and Review. 6 Comments

I’ve been writing a lot lately about Perl, MySQL and so forth, and neglecting another topic that interests me: the Web. I’m going to finish and publish some of the drafts I have on the Web before I continue with another massive database marathon. This one is about someone who read my mind and built exactly the Firefox extension I was about to build myself: an easy way to block all cookies and allow them on a case-by-case basis without nagging.

By default, cookie whitelisting in Firefox isn’t easy

As you may know, I take a “say no by default” approach to blocking cookies on the Web. I disable all cookies by default, and only enable them on sites I want to have them (a “whitelist”).

This cookie whitelist is a bit of a pain, though. Every time I browse to a site and want to allow cookies, I have to open the cookie preferences and add the site to the whitelist. That ends up being eight clicks, and I have to type the domain too.

I could tell Firefox to ask me about every cookie, but that’s worse than no privacy at all. I don’t even know why there is such an option. It makes browsing so unpleasant, I’d rather just let everyone set cookies and be done with it. So that’s not a solution.

What I need is a non-disruptive way to 1) see when cookies aren’t allowed, and 2) allow them with just a click or two. I looked around the Mozilla extension site and didn’t find what I wanted. I wanted a bare-bones extension, and everything I saw was a complicated mess. That was quite a while ago.

I thought I’d have to write my own extension, but never got around to it. In the meantime, either someone wrote exactly what I wanted, or I got better at finding things. The Permit Cookies extension by Daniel Lindkvist is about as perfect as it gets.

How Permit Cookies works

It puts a small icon in the status bar:

When I want to allow cookies for a site, a single click on the icon pops up a dialog with the top-level domain of the site I’m browsing already filled in:

Permit Cookies Dialog

The little “C” in the status bar turns green to indicate cookies are now allowed. Another click on that same icon later will let me change my setting for that site, to delete the setting and disallow cookies once more.

Thanks, Daniel! You made Firefox a lot easier for me to use!

Technorati Tags:No Tags

A review of the Glom graphical database front-end

Published at September 4, 2006 in Desktop, SQL, Review, GNU/Linux and Tools. 27 Comments

Glom is an interesting graphical database front-end I’ve been meaning to try out for some time. Someone asked about graphical database front-ends on the #mysql IRC channel recently, and that prompted me to install Glom and learn how to use it. My overall impressions? It lands squarely in the middle of its target audience’s needs, but still has a quirk here and there. With a bit of polish it will be a fine product, and it’s already a winner over Microsoft Access and Filemaker, two similar programs with which you might be familiar. In this article I’ll walk through installing and configuring Glom, a simple database design, a quick peek under the hood, an archaeologist’s experiences using it, and give my opinions about Glom in detail.

Introduction to Glom

Glom is a GTK+ front-end to PostgreSQL, advanced open-source relational database software that is used widely for mission-critical business applications. Having a powerful relational back-end means it is built on serious technology, with all the advantages that gives. The choice of front-end technology gives it a very nice cross-platform graphical interface that runs natively on the Gnome desktop, and can run on Microsoft Windows and other platforms (though I don’t think Glom itself has been ported to anything besides GNU/Linux).

Glom lets you create simple database interfaces in a manner similar to Filemaker Pro or Microsoft Access, though it is far easier to use and has a true relational database behind it. With Glom you create the database and the interface together, not separately. Glom requires no programming — it just requires understanding your data model, which of course is a prerequisite to doing quality work in any similar product. If you know Python and want to add more functionality to your interface, you can add Python code to buttons and for generating calculated fields.

Glom has just one developer at this time, the talented Murray Cumming, who is also a developer or lead developer for several other important Free/Open-Source projects.

Installation

As I said, a message on an IRC channel spurred me to set aside time to install and use Glom. I’ve been reading articles about it for quite a while, with mounting interest each time, but never made time to try it out. I decided I would write an article about my experience, so I approached the project with the goal of doing a “typical” installation and creating some “typical” database just for fun. I searched the web for information about Glom and found very little written about it ahead of time, so I decided to try to fill in some gaps in what I did find.

Glom is packaged for installation on Ubuntu GNU/Linux, and is said to be dead simple to install. I decided not to install it on my Ubuntu laptop, and take the road slightly less travelled — I installed it on my Gentoo desktop machine instead. I did this because it seems people who install Glom on Ubuntu generally have a smooth experience, but I couldn’t find much written about installing it elsewhere.

Glom’s website has information about installing it on Gentoo by using the popular Break My Gentoo unofficial e-build repository. Though it sounds like an invitation to disaster, in fact it is just a repository of e-builds that have not yet gotten into the main Gentoo Portage tree, and you should not fear it. That said, this was my first experience using a Portage overlay. Assuming your Gentoo system has Subversion installed, and you don’t yet have any Portage overlays either, the following commands (executed as root) should do the trick:

cd /usr/local/
svn co https://svn.breakmygentoo.org/bmg-main/
echo 'PORTDIR_OVERLAY="/usr/local/bmg-main"' >> /etc/make.conf
echo dev-db/glom >> /etc/portage/package.keywords
echo dev-cpp/bakery >> /etc/portage/package.keywords
echo dev-cpp/libgdamm >> /etc/portage/package.keywords
emerge -av dev-db/glom

In words: I checked out the Break My Gentoo tree to /usr/local/bmg-main, added that directory as a Portage overlay, unmasked Glom and two of its dependencies, and then started the emerge process. I ran this on my AMD64 system on 2006-09-01 and it worked fine for me. I ended up with version 1.0.4 of Glom, 2.4.0 of bakery, and 1.3.7 of libgdamm from the Break My Gentoo tree. Everything else was installed from the standard Gentoo Portage tree.

After I added the Portage overlay directory, newer packages for much of my system became available, and I don’t want to stray outside Portage for most of my system software, so I removed the overlay from /etc/make.conf.

I already had the latest stable PostgreSQL installed from Portage, which is version 8.0.8. I decided to stay with this version instead of upgrading to version 8.1, which is considered stable on many other distributions, because I wanted to do a fairly “conservative” installation, mimicking the type of system a risk-averse business might want to build.

Configuring PostgreSQL

You need to create a database user that can create and edit databases. Glom also requires PostgreSQL to accept connections over TCP/IP, which it doesn’t do by default. If you already have PostgreSQL configured to allow remote connections, you can skip this step. Otherwise, you should follow the instructions on Glom’s website, which you can find by following a link from the Download page. Here’s what I did to set everything up correctly:

I started PostgreSQL and added it to the default runlevel, so it will start automatically:
```
/etc/init.d/postgresql start
rc-update add postgresql default
```

I added a PostgreSQL user called glom. For reasons I’ll explain in a bit, I also created a glom database:

xaprb@tigger ~ $ su -
root@tigger ~ # su - postgres
postgres@tigger ~ $ createuser -P
Enter name of user to add: glom
Enter password for new user: 
Enter it again: 
Shall the new user be allowed to create databases? (y/n) y
Shall the new user be allowed to create more new users? (y/n) y
CREATE USER
postgres@tigger ~ $ createdb glom
CREATE DATABASE
postgres@tigger ~ $ exit

You don’t have to create the glom database, but it makes it easier to verify your user is set up correctly, as you’ll see later.

As root, I edited the PostgreSQL configuration files to allow TCP/IP connections:

root@tigger ~ # vim /var/lib/postgresql/data/postgresql.conf
# I added the following line:
listen_addresses = '*'
root@tigger ~ # vim /var/lib/postgresql/data/pg_hba.conf
# I added the following line:
host    all         all         0.0.0.0 0.0.0.0               md5

These steps took a careful eye; I made some mistakes at first, because I’m not that familiar with configuring PostgreSQL.

I restarted PostgreSQL. It is not necessary to restart the computer:
```
/etc/init.d/postgresql restart
```
If you made mistakes, as I did, you may think it restarted, but it actually didn’t. I took a look at the log to see what was the matter:
```
root@tigger ~ # tail /var/lib/postgresql/data/postgresql.log
FATAL:  unrecognized configuration parameter "listen_address"
FATAL:  unrecognized configuration parameter "listen_address"
```
I needed to say listen_addresses in postgresql.conf. Unfortunately Gentoo thought PostgreSQL was already started, so trying to start or stop it failed; I had to “zap” it to proceed:
```
/etc/init.d/postgresql zap
```

Eventually I got the user and database created, and PostgreSQL listening for TCP/IP connections.

At this point, I recommend you make sure the user you created can connect to PostgreSQL. If it can’t, you may have misconfigured PostgreSQL, created the user wrong, or had some other trouble. To verify all is well, connect via the command-line client:

xaprb@tigger ~ $ psql -h localhost -W -U glom
Password: 
Welcome to psql 8.0.8, the PostgreSQL interactive terminal.

If you see the welcome message, everything is fine. Explicitly specifying -h localhost causes psql to connect via TCP/IP, so this is a good way to verify your configuration is correct. This is why I created a glom database before, by the way. If I didn’t do that, my connection attempt would be rejected because it automatically tries to connect to a database named the same as the user. Maybe a PostgreSQL expert can write in a better way to do this, but I’m still new at this, so I just took the simple route.

Starting Glom

Once the configuration is all done, you’re ready to start Glom. I recommend you do this by opening a terminal and typing glom, so you have a place to see any debugging output, instead of starting it from a system menu. This helped me troubleshoot configuration issues.

In Xfce, my preferred graphical environment, Glom appears under the “Office” entry in the desktop menu. I don’t know where it is in Gnome or KDE.

If all is well, Glom will start and ask you if you want to create a new database, create a new database from a sample, or open an existing database.

Configuration troubles

At this point, I ran into some troubles. The first issue was Glom couldn’t connect to PostgreSQL, even though I could connect on the command-line. Fortunately a quick web search turned up the issue: libgda, which Glom uses as a database abstraction library, wasn’t compiled with support for PostgreSQL. This is my fault. I have the postgresql USE flag turned off globally in /etc/make.conf, to avoid building extra dependencies for many packages. I added the USE flag and re-compiled the library:

root@tigger ~ # echo gnome-extra/libgda postgres >> /etc/portage/package.use
root@tigger ~ # emerge libgda

This solved the connection issues. Now I could log in as the glom user I created before, and I was able to create a new database, but I couldn’t switch Glom from Operator mode to Developer mode. I could create an empty database, but pretty much nothing else. I couldn’t create tables, and many menus and menu entries were grayed out. When I tried to use the User Level menu to switch from Operator to Developer mode, Glom displayed an error dialog saying “Developer mode not available. Developer mode is not available. Check that you have sufficient database access rights and that the glom file is not read-only.” Here’s a screenshot:

This also turned out to be a minor configuration issue, but Glom’s documentation is a bit thin at the time of writing, so it took me a while to understand what was wrong. Fortunately, because I’d started Glom from a terminal, I was also able to see some debugging output, which gave me a hint:

DEBUG: User=glom is _not_ in the developer group on the server.

My first attempt to solve this was just to add a developer group in PostgreSQL, and add the glom user to the group:

xaprb@tigger ~ $ psql -h localhost -W -U glom
   [ snipped  some output ]
glom=# create group developer;
CREATE GROUP
glom=# alter group developer add user glom;
ALTER GROUP
glom=# \q

This didn’t solve the problem. I dropped the developer group to clean up the clutter I’d just created, and tried web searches. I saw some messages on the Glom mailing list about the same problem, starting with this thread about problems with Glom not letting the user switch to Developer mode. According to the message and its follow-ups, Glom wanted my user to belong to a group called glom-developer, not just developer. This still didn’t solve the issue, though!

I was getting a bit frustrated with the lack of documentation, and was just about to go read the source code and find the problem, but I stumbled upon another message on the mailing list. This message said the issues could be solved by creating a new database from the provided sample file, which would set up any needed groups in PostgreSQL. I opened the example, which was in /usr/share/glom/doc/examples/, and it added a group called glom_developer (underscore, not hyphen). This fixed the problem at last.

Though this was a minor issue, it really was annoying. I’ve since edited the installation documentation to indicate the required group memberships. If you want to configure PostgreSQL correctly without using the example database, run the following from within psql:

glom=# create group glom_developer;
CREATE GROUP
glom=# alter group glom_developer add user glom;
ALTER GROUP

This is for PostgreSQL prior to version 8.1, which has a significantly different permission system; I imagine in 8.1 you would say CREATE ROLE... ALTER ROLE, though I’ve not tried it.

Creating a simple database

Finally, I was really ready to create a simple database project. I decided to create a database that would keep track of servers and software in a corporate network. I wanted to store information about servers, software, and which software is installed on which servers. When Glom showed me the initial screen, I chose “New,” typed the name of the XML file defining the database (”servertest”), and accepted the default database title (”Servertest”). Glom next presented me with a login dialog, asking me to connect to PostgreSQL:

It filled the username field with my Unix username, not “glom,” so I changed that to “glom” and typed the PostgreSQL password. The next screen I saw was the “Tables in database” dialog, which showed no tables. I created three tables: Server, Program, and Installation. This screen doesn’t allow you to define columns for tables; all you can do is create named tables:

After I closed this dialog, Glom displayed the Installation table in list view, I guess because it’s the first table in alphabetical order:

Glom creates three columns for every table by default: description, comments, and an auto-incrementing integer ID field. I decided not to change the Server and Program tables, and just accepted the defaults, but I needed to record which machine and program an installation represents. I chose the Developer->Fields menu to edit the Installation table, and added columns called server and program:

Then I switched to the Details view of the Installation table, which will eventually allow me to enter information about where a program is installed:

This isn’t very useful for entering data as it is. I want a pull-down menu of servers and programs, and I want those fields first in the display, before the Description and Comments fields. To create the pull-down menus, I first defined relationships between the tables. I selected the “Developer->Relationships for this table” menu entry, which showed the Relationships dialog. I added relationships from my newly added columns to the ID columns of the Program and Server tables:

Next I chose the Developer->Layout menu, which shows the fields and some controls for formatting and editing them:

I selected the server field and clicked the Formatting button to control how it’s presented. I chose “Use custom formatting” and selected “Choices from Related Records,” then chose the Installation_To_Server relationship. I also chose to restrict the data to these choices, so a user can’t enter a server that doesn’t exist:

I repeated the process for the program field. Next, I brought those fields up to the front of the display, by selecting them in the Layout dialog and using the Up buttons. After I accepted these changes, the Details view looks much more usable:

If I had not restricted the data to the choices in the related records, the pull-down menus would have been combo boxes, and if I had selected the checkbox in the Relationships dialog to allow editing related records, changes made by typing in the combo boxes could be propagated through to the related records. I chose not to use these more advanced features for this simple demo, though. I also chose not to use a wealth of other advanced features, such as date fields, which automatically generate a GTK+ date chooser, etc etc.

In order to create entries in the Installation table, I need to create some entries in Server and Program. I did this easily from the list views of those tables, but first, I edited the tables so their ID columns are auto-incrementing. If I’m going to have a surrogate key, it might as well auto-increment! Once I created those entries, my pull-down menus were nicely populated:

And the resulting data, in list mode:

That’s as far as I’ll go with this basic intro to designing and using databases with Glom. If you are interested in the more advanced features, you should take a look at the sample applications that come with it.

How the backend is created

I poked around to see what Glom really does behind the scenes. This is something I’m always curious about when I use any abstraction or front-end, especially a front-end to a database, which I’m used to tuning and tweaking to my exact specifications.

First of all, when creating a new database, Glom names it with a glom_ prefix, and creates two tables as soon as you submit your password. It never shows you these tables, but I looked at the newly created database through psql to see what was there before I created any user-defined tables. In my case the database is called glom_servertest. Here are the two default tables:

glom_servertest=# \d
                  List of relations
 Schema |            Name            | Type  | Owner 
--------+----------------------------+-------+-------
 public | glom_system_autoincrements | table | glom
 public | glom_system_preferences    | table | glom
(2 rows)

The user-defined tables aren’t named with a prefix, so after I defined tables for my sample application, I had the following tables:

glom_servertest=# \d
                  List of relations
 Schema |            Name            | Type  | Owner 
--------+----------------------------+-------+-------
 public | Installation               | table | glom
 public | Program                    | table | glom
 public | Server                     | table | glom
 public | glom_system_autoincrements | table | glom
 public | glom_system_preferences    | table | glom
(5 rows)

I looked at the columns in the tables. In every case a hidden glom_lock column was added to the table. Otherwise the tables look pretty straightforward, and the columns use generic data types.

I don’t understand why Glom creates these extra tables. It stores so much data in the .glom XML file, I’d expect it to put this data there, too. Either that, or put it all into the database. I find the mixture a little strange.

Quirks, bugs, and annoyances

Glom is not perfect. I found many areas where something could be improved, but I don’t want to give the impression that I didn’t like it. Nevertheless, the list is pretty long:

Documentation is skimpy. The best documentation on what Glom can do, and how to do it, is hidden in the “screenshots” section of the Glom website. Fortunately, it’s a wiki, so anyone can edit it (hint, hint).
The interface sometimes does things you don’t expect. It’s easy to create spurious records, for example. Likewise, it’s easy to think you’ve edited something in the designer interface, and navigate away from a screen, then discover later your work didn’t get saved.
The designer interface isn’t quite consistent everywhere. For instance, it’s possible to open the Formatting dialog from a couple different places, but I’ve only found one way to have the changes really saved.
The editing interface seems to assume every table will have an auto-incrementing numeric primary key, but the primary key field the table editor creates isn’t auto-increment; you have to drill down into an editor dialog to assign that property, even though the field appears in a list of auto-increment fields in the Developer->Database Preferences dialog.
The database user always defaults to my Unix login, even after I’ve logged in as a different user. This can be changed by hand-editing the .glom XML file, but apparently nothing in Glom sets this value through a preference setting or anything like that.
Some features can crash the program completely, for instance certain functions while creating reports.
Reports are related to tables, so it can be confusing to not find the reports you expect just because you have a different table selected. In fact, at first it seemed the reports were appearing and disappearing randomly.
When I tried to create a new database called Glom-test-1, there was a database error, which leads me to believe Glom may not be quoting things correctly when it sends SQL to the database server.

Glom doesn’t like multi-column primary keys. When I tried changing a PK to be multi-column, Glom completely crashed:

Glom  Base_DB::query_execute(): Error while executing SQL
  ALTER TABLE "Installation" ADD PRIMARY KEY ("server")
Internal error (Database): ERROR:  multiple primary keys for table "Installation" are not allowed

Apparently Glom wants everything to have a single auto-incrementing numeric primary key. I tried to remove the auto-increment columns, and just use natural data for the keys, but then it had issues entering records in Details mode, giving me errors about not being able to save the data because no primary key was defined. In fact it did save the entry, but not the non-primary-key data, which is very strange. I also noticed that sometimes when I’m in list view and switch to details view, it creates a new entry I don’t intend it to make. And it won’t let me delete it from there because it says the entry has no primary key. I think it just has issues with non-numeric primary keys.
The relationship editor has some issues showing the proper fields in the parent table; I chose the Server table and it showed me available columns from Program. Clicking around to other items in the dialog, then returning to that relationship editor, solved the problem; apparently it just needed to freshen its view of the data or something.

At this point, Glom has many features and capabilities, and some need improvement, but how does this really affect its usability? To answer this question, I conducted rigorous usability testing.

Usability from a novice’s point of view

Okay, so I didn’t really conduct rigorous tests. Okay, I didn’t even “conduct tests.” What actually happened was my wife looked over my shoulder and asked what I was doing. She’s an archaeologist and longtime Mac user, and when I said I was testing out a program that is similar to Filemaker, she got very excited and stole the mouse and keyboard from me. To keep up appearances and pretend it was really my idea to do this anyway, I got a piece of paper and a clipboard, stood behind her and wrote down what happened as she tried to use the program.

You may not think an archaeologist is a good person to do this kind of test, but archaeologists are ideal users for Glom. Archaeologists record tons of data about what they find, and in my experience, many of them use Filemaker to do it. Glom’s developer likens Glom to Filemaker Pro, so getting someone familiar with Filemaker Pro to test-drive it is a great test. Not only did my wife know what she wanted to do, she also had preconceptions about how she ought to be able to do it, which was revealing.

One thing I did right in the test: I let her do it on her own. Okay, I mostly did. This actually happened just at the instant I got everything configured correctly, so I hadn’t had a chance to form any of my own preconceived notions. I really couldn’t give her good guidance even if I wanted to.

She alternately designed, fought, and went in circles for about half an hour, trying to create a simple database that would keep track of our travels together, when they happened, and how much we spent on them. She bumped into lots of Glom’s bugs, caused it to crash a lot, and got very confused by her notions about “files” (I’m not sure whether her idea of “file” maps to a table, a database, or something else). It was a bit painful to watch her struggle with the interface quirks, her limited knowledge of relational design, and the program’s bugs. However, at the end of the half-hour, she had a database that essentially did what she wanted, and she knew the program well enough that she’d probably be fine if she had to use it for her work.

And that concludes my, ahem, rigorous testing.

The three bears: is Glom too simple, too complex, or just right?

What is Glom’s target audience? It seems to be small businesses, archaeologists and similar professionals or academics who need a graphical interface, and individuals who want to design a database and data-entry interface that is simplified, but powerful enough to serve basic needs. Another way to look at it is that Glom should be a replacement for Microsoft Access and Filemaker Pro, but in the Free Software world. How well does it achieve these goals?

First of all, consider how Glom hides the database back-end. Glom is an abstraction over PostgreSQL, and as Joel Spolsky famously points out, every abstraction leaks (worth reading if you’re not familiar with that article). Glom tries to protect the developer or operator from really knowing what a database is all about. Data types are genericized; relationships are too. And for the target audience, this really helps. But when things go wrong — when the underlying database leaks through the abstraction — it gets frustrating and confusing. I’m no PostgreSQL expert, but I know enough that if it takes me hours to troubleshoot the issues I ran into, an archaeologist really doesn’t stand a chance. This would be OK if more of the quirks and bugs were fixed. It’s OK to have the abstraction leak very occasionally. In this sense, Glom is too complex. There are still too many things that can go wrong installing and using it. Once more of the bugs are fixed and there’s better documentation, I think it will be “just right.”

Next, consider the interface itself. How much power does it give the user? Not enough, too much, or just right? I’d have to say “just right.” Consider this: if a user needs more power, or has more specific needs for the data, the user either a) is a data expert, and can either make Glom sing and dance as desired, b) is an expert, and can design and use databases directly, or b) needs to hire an expert. One similar system that gives the user significantly more power and complexity is Microsoft Access. I would strongly argue this is a problem. I’ve used Access for really complex things, such as to design a system to track patient information at a health clinic (I know, I know… I didn’t want to use Access either). Access is a real mess. It’s a hodgepodge of Visual Basic and spreadsheets, with a healthy dose of unreliability and stupidity, mixed together with highly complex features. This has led to it being used where it never should be used, such as health clinics for example. If you need that much power, you need to use a real relational database. At some point it’s time to stop over-extending simple tools. Likewise, if you need more power than Glom gives you, just use PostgreSQL.

On the flip side, one of Glom’s design decisions is not to let the user arrange the interface with infinite precision. This is both a blessing and a curse. It’s a blessing because it simplifies, and it’s a curse because it simplifies. If you need to make a field a little bit narrower, and move it next to another field, and turn its background bright orange, and… you can’t do it, as far as I can see. Those of you who’ve used “databases” that look like Halloween decorations will agree this is a blessing, especially since these features are like clip-art — if you provide them, people will use them lavishly and tastelessly (witness the ubiquitous garishness of most applications developed in-house at my last employer). Besides, Glom actually does a pretty good job of laying out fields nicely — much better than most people will do by hand, in my experience. And it does give you limited control over grouping fields together, ordering them, and so on. All in all, I think again it’s more than good enough for most people’s needs.

I have one gripe with Glom’s auto-layout, and I expect Filemaker Pro users will agree with this one: to make data entry easy and efficient, one must be able not only to define field order, but tab order as well. Often the tab order and field order must be different. And multi-line text fields in Glom, at least on my system, can be tabbed to, but pressing the tab key doesn’t leave the field; it enters a tab character in it. This makes Glom cumbersome for rapid data entry.

On the whole, I think Glom is a well-balanced compromise between simplicity and power, and with a bit more polish and documentation, it’ll be even better.

Other similar programs

I’m actually surprised about this, but there don’t seem to be many other similar products for the Free Software world. The only other one I found is Bond:

Bond is a rapid application development framework for building applications for the linux desktop and the web. Using bond you can quickly build database forms for Gnome Linux, or the web via standard HTML and AJAX or for windows using our win32 version.

I haven’t used Bond, so I really can’t comment on it other than to say it looks like it’s more complex and has more features than Glom.

Another surprise is that neither Glom nor Bond uses MySQL as a back-end, though both say it would be possible.

Conclusion

As I’ve shown you, Glom is a simple yet powerful front-end to PostgreSQL, which I think is a good match for many database users’ needs in the Free Software world. It has some quirks and lacks good documentation, but is a very promising product. Even though there are still bugs, my wife’s experience proved the 1.0.4 version number isn’t inflated or boastful; it really is ready for competent users. And since it’s built on PostgreSQL instead of just being a front-end to a file, it’s potentially much more powerful and useful than proprietary, expensive, non-Free alternatives built on inferior technology.

I hope this article has given you a good overview of Glom. If you found it useful, you should subscribe to stay current with my upcoming articles.

Technorati Tags:No Tags

How to convert text to columns in OpenOffice.org Calc

Published at June 26, 2006 in Desktop. 39 Comments

Unlike Microsoft Excel, OpenOffice.org 2.0 Calc doesn’t have a built-in “text to columns” feature, which is hard to live without once you’re used to it. OpenOffice.org has an extensible add-on architecture, and someone has written a “text to columns” add-on, but installation may be confusing. In this article I’ll explain how to install the add-on.

I hope this feature will be added into the office suite at some point. Oddly, it seems to already be implemented, but not in the way it’s needed. Writer has a text-to-columns feature already, and Calc’s Open process has what looks to me like the needed functionality too — when opening a delimited text file, it brings up a dialog that does exactly what I’d do with the text-to-columns feature in Excel.

Update

Here is something I’ve noticed under GNU/Linux since writing this article: if I’ve copied text to the primary selection, for example by highlighting it in a terminal window, I can get OO.org to “text to columns” the text just by middle-click-pasting it into the spreadsheet. Instead of actually pasting it, this opens up the “Text Import (Pasted Data)” dialog, which lets me choose delimiters, etc — exactly what I need. (Of course, it’d be nice if it were even smarter and auto-detected that for me). So far I have not found any other way to cause this dialog to appear, which is puzzling.

Getting the add-on

The add-on is available through SourceForge and the OOoMacros site, which is both a source of add-ons and help and examples for those wishing to make their own add-ons. You can download text to columns for Calc here. There are actually two add-ons, but apparently the “Text2Columns fixed width” one is redundant, since the “Text to Columns” one does that too.

How to install an add-on in OpenOffice.org

This is the part I found confusing. The difference between macros and add-ons in OpenOffice.org isn’t very clear to me, and the tools to manage them aren’t either. I went down the wrong path with macros until I realized this isn’t a macro, it’s an add-on. Then I tried to learn how to install an add-on. I found lots of references to something called unopkg, but nothing about where it’s installed. It wasn’t in my PATH. I searched my filesystem and found it in /usr/lib/openoffice/program/unopkg. Then I ran it:

xaprb $ /usr/lib/openoffice/program/unopkg gui

OpenOffice.org Package Manager

Lo and behold, it brought up the same dialog I can access through the Tools > Package Manager menu entry. All that searching for nothing. I recommend not running it from the command-line; just run it through Tools > Package Manager!

Installing and using the package

Once I found it, installing the package was as easy as selecting the My Packages entry and pressing Add.. to browse for the file. It installs itself and shows up under the Tools > Add-Ons menu. This is really easy to do, but it took me a while to abandon my misdirected efforts to install it as a macro.

Here’s a screenshot of what it looks like (click the screenshot for a full-size look).

So far it has worked fine for myself and my coworker. I hope it’s useful to you too.

Update For those of you using GNU/Linux, Gnumeric has a built-in text-to-columns converter that’s very nice. Gnumeric also loads much faster and runs with much less memory than OpenOffice.org.

Technorati Tags:No Tags

How to auto-mount removable devices in GNU/Linux

Published at May 20, 2006 in Desktop and GNU/Linux. 6 Comments

Most operating systems have an auto-mount feature to detect and mount devices, such as USB flash drives, when they’re plugged in. GNU/Linux is no exception. It’s easy to install and just as easy to use. In this article I’ll show you how to get auto-mounting without changing /etc/fstab, writing udev rules, or installing a huge desktop environment such as Gnome or KDE.

There’s more than one way to do it

There are lots of ways to get things auto-mounted. I’ve browsed around the Gentoo forums, and I see dozens of ways people do it. Some people write udev rules to recognize devices. Others write custom shell scripts, tinker with /etc/fstab, or do a combination of lots of things. Probably the most common methods, though, are to use the built-in functionality that comes with the KDE or Gnome desktop environments — or to use the software they use, but not use the desktop environment itself.

Gnome uses gnome-volume-manager and gnome-volume-properties to auto-mount devices, and when appropriate runs a program, such as a gphoto2 in the case of a digital camera. I’m not sure exactly how KDE does automounting, since I haven’t used it in many years. What I am fairly sure of is that they both use the same programs to get their work done under the hood. These programs are hal (the Hardware Abstraction Layer), dbus (a system message bus), and udev (userspace device filesystems).

In my opinion, some methods are definitely easier than others. I suggest you either install Gnome or KDE, or if you’re a minimalist like myself, just install a system that doesn’t require a huge set of dependencies.

Using Gnome

If you’re running Gnome, or if you have it installed but run another desktop environment, such as XFCE, you need gnome-volume-manager to be running. Gnome usually starts it automatically. If you use another desktop environment, you need to start it so it runs in the background. For example, you could put this in your .xinitrcfile:

gnome-volume-manager &
startxfce4

That starts it running in the background, then starts XFCE.

Once you’ve got it running, you can run gnome-volume-properties to configure what it should do when it detects and mounts a volume. It can run programs, open a file browser, and so forth.

My favorite method

I prefer not to use gnome-volume-manager. For one thing, it requires an X server to be running, so it won’t work if I don’t want to start a windowing environment. Sometimes I just want to log in and work at the command line. It also requires me to install tons of other “stuff” I just don’t really need. Don’t get me wrong, I think Gnome is nice. I just think less is more.

So, I prefer to run software that does nothing but sit in the background waiting to be notified that something has been plugged in. The ivman program does that very nicely, and it’s very small.

For this to work, it needs the same things Gnome relies on: hal, dbus, and udev. All three need to be running as daemons. If you’re using Gentoo, you almost certainly have udev already, since it’s been part of the standard installation instructions for years. Other distributions should have udev as well. Installing hal through your distribution’s package management system should install dbus as a dependency. Likewise, starting the hald daemon should start dbus as a dependency. In Gentoo, installing ivman will install these as dependencies, so it’s as easy as

tigger ~ # emerge ivman
tigger ~ # rc-update add hald default

Now all the software I need is installed, and hald and dbus will start when the computer boots. All that remains is to give myself the proper permissions, and start ivman. As root, I can run vigr or use usermod to make myself a member of the plugdev group. I need to log out and back in for this to take effect. To start ivman automatically, I can add it to the end of my .bashrc file:

ivman --nofork > /dev/null 2>&1 &

Now I’m done. When I plug a device in, ivman finds it and mounts it (actually, it can do a lot more than just mount it; read the man page). I have the permissions I need to change files on the device and unmount it again. It appears under /media. If I’m running a file manager such as Nautilus, it shows up on the desktop and in the left-hand pane of the browser view, and I can unmount it with a right-click.

I’m running ivman as myself, not system-wide. There are security and convenience reasons for doing so. The man page explains more about it. It can also automatically run itself as a daemon in the background, but I’m choosing not to do that so a) it quits when I log out for security, and b) I don’t get multiple instances running when I log in and out repeatedly.

Summary

I recommend one of two methods:

Use Gnome or KDE
Install ivman, make yourself a member of the plugdev group, and start ivman from your .bashrc

So that’s it, it just works. I hope this helps you avoid a bunch of hacking, udev rules, and shell scripts.

Technorati Tags:No Tags

Archive for the 'Desktop' Category

You might also like:

You might also like:

What are stepping slides?

How do you do it with OpenOffice.org Impress?

You might also like:

You might also like:

The improved schema

Category setup

When you’re done

Miscellaneous improvements

You might also like:

The 50,000 foot view

Upgrading is painful

Upgrading breaks things

Even in the best case, it’s too slow

Gentoo is not optimized for my hardware!

Gentoo doesn’t have the latest and greatest

Gentoo builds are sometimes really broken

Infinite flexibility and minimality are myths

It’s reinventing the wheel

Package staleness vs. flexibility vs. stable release cycles

While we’re on the subject…

Configuration management is tough

When you have a problem, where is the problem?

I suggest using the official MySQL builds

Conclusion

You might also like:

By default, cookie whitelisting in Firefox isn’t easy

How Permit Cookies works

You might also like:

Introduction to Glom

Installation

Configuring PostgreSQL

Starting Glom

Configuration troubles

Creating a simple database

How the backend is created

Quirks, bugs, and annoyances

Usability from a novice’s point of view

The three bears: is Glom too simple, too complex, or just right?

Other similar programs

Conclusion

You might also like:

Update

Getting the add-on

How to install an add-on in OpenOffice.org

Installing and using the package

You might also like:

There’s more than one way to do it

Using Gnome

My favorite method

Summary

You might also like:

You Ought To Know

Hot Hot Headlines