Off you go… into the purple yonder!

• Make: Nikon
• Model: E995
• ExposureTime: 10/603 sec
• FNumber: f/3
• ISOSpeedRatings: 100

Google has announced it is going to build a real broadband network in the US, to test ultra-high speed applications and networks. They intend to provide service to at least 50,000 and possibly up to 500,000 people. It will be a fiber to the home network with speeds over 1 gigabit/second.

That’s way, way, way faster than anything commonly considered ‘broadband’ in the US. It’s on par with speeds residential users can get in parts of the most advanced broadband nation in the world – Japan. If you dig statistics and want to see how pathetic the broadband situation is in the US, the OECD has a ton of numbers on this topic.

Google is going to build this as an open access network. That means they will own the fiber but they will share access to that fiber with many ISPs. Users will be able to sign up for service with an ISP of their choice, which will then presumably handle all billling and pay Google a share of proceeds for the use of the fiber.

DSL used to be operated in a similar way in the US. That changed when our regulators and legislators rolled over and allowed incumbent telephone companies (Verizon and co) to kill off most of the companies they had to share phone lines with. The incumbents did that largely by pricing the alternative ISPs (CLECs) out of business: they charge them higher wholesale prices than what they charge their own DSL end users.

The difference with other countries is stark. The countries where open access is mandated by law and heavily regulated so that the company that ‘owns’ the cable can not abuse its position tend to have far higher availability of high-speed connections, at a fraction of the cost per megabit that is common in the US.

So, assuming that Google does the right thing with this new fiber (as in, does not undercut or sabotage competitor ISPs that share its fiber), and/or regulators and legislators get the guts and sense to actually enforce open access on all access networks, this announcement is really good news for broadband competition.

Google’s looking for state, county and city officials who want their communities to participate in this project. Google’s also asking non-officials to nominate their communties.

Now, if they could be convinced to put that fiber in the ground in Somerville, MA…

In order to install Rockbox on an iPod, it needs to be formatted in FAT32, not HFS+. The relevant wiki page over at the Rockbox site suggest either connecting the iPod to an iTunes install on Windows, or using one of the bootsectors they have available for download from that page.

Those boot sectors assume your iPod has one of the factory disks installed. I’ve got an old 4th gen iPod that I converted to compact flash after its disk died. It happens to have an 8G CF card in there.

Since I don’t do Windows, I downloaded the 20G 4th gen bootsector, put that on the iPod, and used fdisk to change the size of the FAT32 partition. And that worked fine.

My biggest problem with exercising is how boring it is. Enter the Google bike hack which allows you to ‘cycle’ through Google maps. Awesome idea.

Link via Hack a day.

I’ve never understood why it is common in the US to run power, telephone and cable on poles, rather than underground. Particularly so in areas with much more extreme weather than the part of Europe that I’m from – where most utilities run underground.

Sometimes, though, the pole-and-wire people get a little bit too creative. An old pole was removed yesterday near my house. Both old and new pole had been in place next to each other for quite a while – giving the utilities the chance to move the wires from the old pole to the new pole. I’m guessing one of the utilities did not get the memo. This is how they ’secured’ some of the wiring that was still on the old pole to the new pole.

Dell’s DRAC5 is based on free software. The source has been rumored to be available but hard to find – until today, when this helpful e-mail popped up on the linux-poweredge list.

Here’s the link to an ISO with the DRAC5 source:

http://linux.dell.com/files/drac5/source/

I found this script while looking for a simple script to monitor mdadm arrays. The script is fine, but it has a subtle bug – it will never report an error because the –detail parameter is missing in the call to mdadm. I modified the script a bit, like so:

#!/bin/sh
# (c) 2008 Jasper Spaans 

worst=0
msg=""

for dev in /dev/md?* ; do \
  mdadm --misc -t --detail $dev >/dev/null
  status=$?
  if [ $status == 0 ]; then
    msg="${msg} ${dev}: ok"
  elif [ $status == 1 ] ; then
    if [ worst != 2 ] ; then
      worst=1
    fi
    msg="${msg} ${dev}: degraded"
  elif [ $status == 2 ] ; then
    worst=2
    msg="${msg} ${dev}: degraded - unusable"
  fi
done

echo "mdadm:$msg"
exit $worst

which I saved as /usr/local/bin/check-mdadm.sh.

Add in a bit of snmpd.conf config (and set up sudo accordingly, of course):

...
exec   mdadm /usr/bin/sudo /usr/local/bin/check-mdadm.sh

and a small script on the nagios side (/usr/local/bin/nagios-check-mdadm):

#!/bin/sh

SNMP=`snmpwalk -v1 -c YOUR-PUBLIC $1 extOutput |grep mdadm`
TMP1=`echo $SNMP |grep degraded`
TMP2=`echo $SNMP |sed -e 's/^.*mdadm: //'`

if [ "$TMP1" = "" ]; then
  echo "OK: $TMP2"
  return 0
else
  echo "ERROR: $TMP2"
  return 2
fi

add a bit of nagios config:

define command {
       command_name check_mdadm
       command_line /usr/local/bin/nagios-check-mdadm $HOSTADDRESS$
}

define service {
       use      defaults
       name     check_mdadm
       description   MDADM
       check_command check_mdadm
}

And voila, nagios notifications when disks fall out of the array.

I’ve been running an old Shuttle with a 2.4GHz celeron CPU, 512MB of ram and two 500GB disks in raid-1 as home server for the past 5 years or so. Well, I upgraded the disks in February 2008, before that it had 2x 200GB in raid-1. The thing has no UPS and runs in the closet here at home. And yet:

13:40:34 up 569 days, 17:04,  2 users,  load average: 1.26, 0.94, 0.45

Yeah, home power is pretty reliable around here.

This machine serves as the central network storage for our home, and I also use it to back up a bunch of servers that live at a nearby colo facility, with the rather fantastic BackupPC. The Shuttle has served well over the years but it is getting a bit old – I was starting to expect it to fail. Its power draw is rather high: 78W while idle (that’s after applying all of powertop’s suggestions), and a whopping 100W while doing heavy disk activity.

I was running out of disk space again, so I bought two 1TB ‘green’ WD drives (WD10EADS-00L) that are rated at 5.4W active, 2.8W idle, and 0.4W standby/sleep.

Next – a replacement for the Shuttle. First I looked at a QNAP TS-219p which is a rather awesome little NAS device. It’s based on Marvell’s Kirkwood ARM core, which is the same as the one used in the Sheevaplug, clocked at 1.2GHz. This thing is pretty fast. Its power specs are also impressive:

Sleep mode: 5W
In operation: 21W (with 2 x 500GB HDD installed)

I was of course looking to run Debian on it, which is perfectly possible. People like the firmware that the thing comes with, but it’s proprietary so I’d rather not use that. Plus, I need to be able to run BackupPC.

The major downside is price – the TS-219P costs about $400, without disks. Since the Sheevaplug costs about $100, I would have thought a price in the $200-250 range for the TS-219P would have been reasonable.

Meanwhile I came across some really good NAS reviews over at SmallNetBuilder, and in particular their price/performance NAS chart.

Looking at that chart, the MSI Wind PC performance is pretty much on par with the TS-219P, for a fraction of the price. Extra bonus: it does not come with proprietary software preinstalled, because the Wind is really a bare-bones PC. The Wind has one 3.5″ bay, and one 5.15″ bay. It also has an on-board CF adapter. It has a dual-core Intel Atom 230 (1.6GHz).

I purchased

$134.99    MSI Wind PC
 $26.99    G.SKILL 2GB 200-Pin DDR2 SO-DIMM DDR2 533
 $43.99    Transcend 16GB Compact Flash (CF) Flash Card Model TS16GCF133
  $9.99    StarTech BRACKET Metal 3.5" to 5.25" Drive Adapter Bracket

Total: $215.96 + shipping

The drive bay adaptor turned out to be not only severely overpriced, but also not practical for the Wind – I had to drill a few holes in the damn thing to make the second hard drive fit in the Wind. Don’t buy this kind, or don’t pay $10 for it!

I installed Debian on the CF card (leaving it read-only during normal operation) and use the two disks purely for data – in raid-1 of course. If I did this again I’d buy a smaller CF card – 8GB would be plenty, even 4GB would be enough for the non-volatile bits of /.

Power use, as tested: idle 27W, with heavy disk activity 33W. In other words, this will take 50-70W off our household power budget, which should work out to a savings of $7 to $10/month.

I bought an Intel X-25M SSD drive for my laptop in early June. I got the 80GB version, and this was to replace a Hitachi 7K200-160 – a 160GB 7200rpm drive. Note that the X25-M is generation 1; Intel has since released an updated, cheaper version of the drive that is slightly faster and uses a little less power. The Intel SSD drives generally blow the competition out of the water in real world applications because they have extraordinary random write performance. SSDs are typically marketed with their theoretical maximum sequential write performance, which – barring a few very specific use cases – is far less important than random write performance for your average desktop or server.

The difference in performance between the old mechanical drive and this SSD is *spectacular*. The machine feels 100 times faster. In other words, you want one of these for your desktop/laptop

As for servers – I can’t wait until these things become big enough/cheap enough to replace mechanical drives there, too…

Here are some bonnie++ results:

Hitachi 7K200-160:

$ bonnie++
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
countzero        4G 14755  55 15887  20 13568  13 18775  58 31430  11  72.4   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 21341  83 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
countzero,4G,14755,55,15887,20,13568,13,18775,58,31430,11,72.4,0,16,21341,83,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

Intel X25-M gen 1:

$ bonnie++
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
countzero        4G 51217  98 76395  31 42853  25 43666  80 150391  45 14762  61
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
countzero,4G,51217,98,76395,31,42853,25,43666,80,150391,45,14762.2,61,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

A customer reported problems with capistrany deploys that would just die like this:

 ** [XXX.XXX.XXX :: err] svn: REPORT request failed on '/!svn/vcc/default'
 ** svn: REPORT of '/!svn/vcc/default': Chunk delimiter was invalid (http://XXX.XXX.XXX)
    command finished

After disabling gzip compression on the server for text/xml documents, the error became

 ** [XXX.XXX.XXX :: err] svn: REPORT request failed on '/!svn/vcc/default'
 ** svn: REPORT of '/!svn/vcc/default': Could not read response body: connection was closed by server (http://XXX.XXX.XXX)

The server side logs said:

[Fri Jul 03 10:53:57 2009] [error] [client XX.XX.XX.XX] Provider encountered an error while streaming a REPORT response.  [500, #0]
[Fri Jul 03 10:53:57 2009] [error] [client XX.XX.XX.XX] A failure occurred while driving the update report editor  [500, #190004]

Googling was not very helpful – there are many reports of these errors going back years, and many different solutions, none of which applied to my setup. In general, these errors seem to mean that there was some sort of network problem.

I tried to reproduce the problem by running the offending svn command manually. Out of hundreds of tries, I only managed to make it fail like that just once. And yet running cap deploy, which in turn calls the svn command, it would happen much more often.

I finally tracked this down to an agressive send/receive timeout in Apache’s config. It was set to 3 seconds to prevent too many inactive connections from taking up server resources. Apparently the subversion client sometimes takes a while to get back to the http server its talking to – in this particular situation when run via capistrano, more than 3 seconds. So the server would disconnect the svn client, which would then just fall over with that obscure error message.

In other words, check your server timeouts if you see this kind of intermittent error…