This weekend I wen’t on a frenzy. I turned beeets.com from a single VPS enterprise to 4 VPSs: 2 web (haproxy, nginx, php-fpm, sphinx, memcached, ndb_mgmd) and 2 database servers (ndmtd). There’s still some work to do, but the entire setup seems to be functioning well.

I had a few problems though. In PHP (just PHP, and nothing else) hosts were not resolving. The linux OS was resolving hosts just fine, but PHP couldn’t. It was frustrating. Also, I was unable to sudo. I kept checking permissions on all my files in /etc, rebooting, checking again, etc.

The fix

Then I looked again. /etc itself was owned by andrew:users. Huh? I changed permissions back root:root, chmod 755. Everything works. Now some background.

A while back, I wrote some software (bash + php) that makes it insanely easy to install software to several servers at once, and sync configurations for different sets of servers. It’s called “ssync.” It’s not ready for release yet, but I can say without it, I’d have about 10% of the work done that I’d finished already. Ssync is a command-line utility that lets you set up servers (host, internal ip, external ip) and create groups. Each group has a set of install scripts and configuration files that can be synced to /etc. The configuration files are PHP scriptable, so instead of, say, adding all my hosts by hand to the /etc/hosts file, I can just loop over all servers in the group and add them automatically. Same with my www group, I can add a server to the “www” group in ssync, and all of a sudden the HAproxy config knows about the server.

Here’s the problem. When ssync was sending configuration files to /etc on remote servers, it was also setting permissions on those files (and folders) by default. This was because I was using -vaz, which attempts to preserve ownership, groupship, and permissions from the source (not good). I added some new params (so now it’s “-vaz –no-p –no-g –no-o”). Completely fixed it.

Compared are Linux 2.4, 2.6, FreeBSD, NetBSD, and OpenBSD. Really well-performed benchmarks, with graphs.

http://bulk.fefe.de/scalability

Linux 2.6 was hands down the winner, which makes me feel good about Slackware (2.6 linux but actually stable) as a server. I’m sure Windows would have won if only it was benchmarked. One thing to keep in mind – from what I gathered, the machine tested was a single-processor, single-core machine…this means that SMP scalability was not tested, a HUGE consideration for more modern servers (what server now doesn’t have multiple cores?) and may skew the modern-day results, especially between the two leads, FreeBSD and Linux 2.6.

As all of my hundreds of thousands of readers know, I installed Ubuntu 9.04 on my girlfriend’s laptop. Aside from the confusingly random names they give Ubuntu releases, I have to say I’m extremely impressed. The basic idea is that if you want a server linux, go with Debian or Slackware…both are extremely stable, highly configurable, and FAST. Slackware is a bit more for advanced users than Debian (not a big deal if you know how to edit config files) and perhaps a tad faster, but both are well-oiled machines ready to handle anything you can throw at them.

If you want a desktop machine, check out Ubuntu. All the things I try expecting it not to work, it just works. For example, I was testing flash on it and the volume was too loud. Just for shits and giggles, I tried the volume buttons on the laptop. Holy shit, it works. Next, I’m installing printer drivers on the VirtualBox XP instance. Well guess what? I plugged in the printer thinking, well thank god I have windows installed because it will autodetect it. Instantly, I see a window popup in Ubuntu telling me that my printer has been installed and is ready for use. Now that’s service.

Now, all of this plug and chug computing surely comes at some sort of performance hit, but who the hell cares if it makes your life easier and you don’t have a production website running on that box that gets 1000 requests/s.

There is the issue of auto-updates. Sure you need to update buggy or hackable software…but ahem, why is it on there in the first place? The reason distros like slack and deb don’t need constant auto-updating is because they choose packages that have been known for a long time to be rock solid. I can’t say I agree with the “constant contact with the update server” methodology that windows instilled in all of us. That’s something you’d never use on a server. If something sucks or has security problems, you hand-update that package and test it a billion times with your software. Once again though, this is a desktop machine, not a server…and the auto update worked pretty damn well.

So while this review is short and sweet, so has my experience been with Ubuntu. Perhaps someday I’ll install it on one of my own laptops and take it for a whirl.

Ubuntu: two thumbs up for the new or casual desktop linux user. Not for server usage. I’m sure there are some servers that use it successfully, but none of mine ever will =).

I gotta say, even though Linode is the first provider I’ve had a VPS through, they kick ass.

Their control panel is boss, and adding units is apparently very easy (haven’t messed with it quite yet). I’m excited to have beeets on there.

Having full control of the environment (and the fact that it’s Slackware!!) gives me a boner. I’m on there tinkering too much. I almost hope the site never gets big, because once it does I can’t fuck with it anymore (or I’ll have to have a test machine, I guess).

Anyway, there’s not enough positive information about these guys on the net. It was between them and Slicehost, but I ended up going with Linode because they were a) a bit cheaper, and b) not as “hip.” I tend to shy away from trendy companies.

Good work, Linode. Keep it up. Oh yeah, and thanks for offering Slack ;)