Off you go… into the purple yonder!

I moved a bunch of servers from one colocation facility to another last night. I had been preparing this move for a few weeks. While everything went well in the end, things took much longer than anticipated. Here are a few lessons learned.
1. preparation is *everything*

I spent a lot of time preparing for the move, [...]

I have a bunch of WRT54GL units that still run OpenWRT Kamikaze 7.07 (latest version is 7.09).
I needed to swap out a unit that is one half of a WDS pair, but preferred not to touch the other half. Cloning the wan mac address on these things is easy, but cloning the wifi mac address [...]

Getting access to the serial port in a Xen 3.2 dom0 is somewhat complicated. This is the magic incantation for your grub menu.lst file to get console at 115200 bps on the first physical serial port, as well as on the screen.

serial –unit=0 –speed=115200
terminal –timeout=5 serial console

title Xen 3
root (hd0,0)
kernel [...]

I don’t know about you, but I like to have my system logs split day by day, particularly on busy machines. I also like to have a full timestamp (including year and timezone thank you) in system log files.
The venerable syslog package can not do these things - but syslog-ng can.
On Debian/Ubuntu, the default [...]

Testing smtp auth can be a bit of a pain sometimes, particularly if SSL or TLS is involved. Mail clients often don’t really show what they are doing and/or don’t allow you to set all variables, like which authentication mechanism to use.
John M. Simpson has a nice page that describes how to do AUTH PLAIN [...]

This morning, a user on one of our machines (inadvertently) created a mail loop with a bad procmail script:
09:42:05 up 120 days, 9:23, 20 users, load average: 3367.40, 3265.08, 2751.75
I had seen machines go up to about 200 before, but never this high. If you ever wonder about the stability [...]

Explicit Congestion Notification (ECN, RFC 3168) has been around for quite a while now, but there are still lots of devices and hosts out there that don’t support it properly. For instance, http://www.npr.org, and apparently all HP’s ILO and ILO2 baseboard management systems.
If you’re on a GNU/Linux system, just try it:
echo 0×1 > /proc/sys/net/ipv4/tcp_ecn
Now you [...]

Interesting interview with Jeff Schiller, MIT’s main network guy, in Network World. MIT is going to activate a direct link to New York city later this month - 72 10G waves. Yes. That’s 720Gbit/s. From Boston. To New York.

If you’re trying to do a remote Debian install on a BL20 blade from HP via the ILO interface, you should use the sarge install cd (I used the business card edition) via the ILO’s virtual CD feature, and use ‘expert26′ to make the installer boot with the 2.6 kernel. The default 2.4 kernel does [...]

Debian released a security advisory on Horde yesterday, so I had to upgrade a bunch of machines.
I run Horde/IMP from source. Upgrading to a newer version is kind of annoying; I lost a lot of time doing it. In case you need to upgrade Horde and/or some of its modules, here’s a (relatively) easy way.
1. [...]