Comments on: Brian Aker: 20GB doesn’t fit on a single server

By: Robert Treat

Robert Treat — Fri, 16 Apr 2010 16:40:09 +0000

Just a counter point, we have servers running postgres on boxes with 32 cores and 512GB of ram. From our experience, it scales pretty well on that size hardware; by far our largest problems are (still) disk related. Now, if your argument is that you could scale so much better on Oracle on that kind of hardware, I’m not going to argue that point, but Postgres on that kind of hardware would do the job for 99% of the people out there.

By: William

William — Mon, 12 Apr 2010 17:12:55 +0000

I agree with Barron. I had to re-read that article a few times. Its pretty clear to me that it implies Brian doesn’t think it can handle 20 GB. I was left to conclude that the author misquoted/misrepresented what he was saying, or there is an interesting use case ( maybe heavy write /update use case) where even 20 GB doesn’t work on a single box.

By: Bill

Bill — Mon, 12 Apr 2010 14:12:04 +0000

A few years ago, I came up with an idea here at work, and led a team of 3 (myself included) to develop a distributed processing system in SQL Server. From design to implementation was 9 months. It’s currently one of our most important systems. We get about 6 million new records per day, and run about 300 SQL statements on each of them on average. I looked into existing solutions like Map Reduce, and found that for what most database systems need, they aren’t well suited. I’ll qualify that by stating that they weren’t well suited based on my understanding of them.

Our processing requirements required doing lookups on many other tables, both small and large. We weren’t lucky enough to do simple aggregates (this word is on that webpage 9 times). A lot of the lookups are slightly complex (get the newest lookup record that was in place on this date at this time, etc.).

I would agree that 20G of data isn’t nearly enough to require distributing it, unless you are expecting explosive growth in the near future. The worst time to switch do a larger/better solution is when you “need” do. I’d always prefer doing that ahead of the data flood when possible.

By: Xaprb

Xaprb — Sun, 11 Apr 2010 18:36:55 +0000

PS, note: I know Brian is smart, and knows that 20gb fits onto a computer, which is why I said the interview “quoted him as saying” instead of directly attributing it to him :-)

By: Jay Pipes

Jay Pipes — Sun, 11 Apr 2010 18:35:23 +0000

see you soon :)

/me goes off to hit the weights for the cage match.