Dell’s GNU/Linux support leaves to be desired

So I have a poweredge 2800 with some hardware issues (voltage sensors on the riser card seeing things they should not…). The Dell support folks wanted me to run a DSET report with this tool


http://support.euro.dell.com/support/downloads/download.aspx?c=uk&l=en&s=gen&releaseid=R155882&formatcnt=2&libid=0&fileid=208066

Man – what a pile of crap. This thing is a self-extracting archive for rpm-based systems (redhat/suse). Fine – except that I run Debian on that machine. Anyway, you can run it with –extract to extract the contents:

./delldset_v1.4.0.8.bin --help


Command-line options for Dell System E-Support Tool (DSET)

Usage:  [options...]

Options:

-h,--help        : Display command-line usage help
-i,--install     : Install/upgrade the DSET application
-v,--version     : Display version information
--list           : Display contents of package (+)
--extract  : Extract files to specified path (+)

Using no options will display the interactive user
interface which will guide you through using DSET.

These contents are basically an rpm file and some support files:

-r-xr-xr-x  1 ward ward     4133 Apr  2  2007 Dell_License*
-r-xr-xr-x  1 ward ward     9909 May  8  2007 README*
-r-xr-xr-x  1 ward ward       58 May  8  2007 Version.txt*
-r-xr-xr-x  1 ward ward 15200341 May  9  2007 delldset-1.4.0-8.i386.rpm*
-rw-r--r--  1 ward ward 15187240 Oct 27 15:57 delldset_1.4.0-9_i386.deb
-r-xr-xr-x  1 ward ward    10271 Apr  2  2007 install.sh*
-r-xr-xr-x  1 ward ward      472 Apr  2  2007 sphelp.txt*
-r-xr-xr-x  1 ward ward       24 May  4  2005 test.sh*
-r-xr-xr-x  1 ward ward     9877 Apr  2  2007 utility.sh*

The rpm can be transformed into a .deb with alien:

$ alien delldset-1.4.0-8.i386.rpm 
Warning: Skipping conversion of scripts in package delldset: postinst postrm preinst prerm
Warning: Use the --scripts parameter to include the scripts.
delldset_1.4.0-9_i386.deb generated
$ alien delldset-1.4.0-8.i386.rpm --scripts
delldset_1.4.0-9_i386.deb generated

Which installs fine – but you really want to be using a kernel that has ipmi functionality compiled as modules; otherwise the package will not install properly until you add

hapi.allow.user.mode=yes

to /etc/omreg.cfg. In any case, you need to have ipmi as modules for the rest of this sequence of commands, so you might as well boot into a stock Debian kernel.

You’ll probably want to have installed Dell’s open manage before you run the DSET report. To do that, add

  deb ftp://ftp.sara.nl/pub/sara-omsa dell sara

to your /etc/apt/sources.list file and

  apt-get update
  apt-get install dellomsa

So, then you would be ready to run the dellsysteminfo command but WAIT – don’t do it yet. That script is dangerous. It does stuff like this, probing for the running services (from /opt/dell/dset/bin/dell-sysreport.sh):

#Dell: Extra OS stuff we want

echo "     Getting status of services ..." ${TEE2LOG}

#Dell: SuSE doesn't have a "service status" method so do it the manual way 
#catifexec "/sbin/service" "--status-all" 
 
rm -f $ROOT/service > /dev/null 2>&1 
 
SERVICEDIR=/etc/init.d 
 
#pushd ${SERVICEDIR} > /dev/null 
          for SERVICE in `ls -1 ${SERVICEDIR}` ; do 
 
            case "${SERVICE}" in 
#Skip over these "services" since they tend to hang some systems when requested for status 
              functions | halt | killall | single| linuxconf| kudzu | bgpd | boot | reboot | single | nscd | \ 
                  halt.* | rc | boot.* | *rpmorig | *rpmnew | *rpmsave | *~ | *.orig) 
      ;; 
              *) 
                if [ -x "${SERVICEDIR}/${SERVICE}" -a -f "${SERVICEDIR}/${SERVICE}" ]; then 
        echo "Service Name: ${SERVICE}" >> $ROOT/service 
                 /usr/bin/env -i LANG=$LANG PATH=$PATH TERM=$TERM "${SERVICEDIR}/${SERVICE}" status >> $ROOT/service 2>&1 
    echo >> $ROOT/service 
    echo "---------------------------------------------------------------------------" >> $ROOT/service 
               fi 
                ;; 
            esac 
          done 
#popd > /dev/null

That didn’t go over too well on my system – a bunch of things stopped working (like the ‘w’ command), samba started segfaulting, etc. It’s clearly calling some init scripts that don’t deal well with being asked for a status.

So I commented out that part of the script, which made things a lot happier – but of course services are now not mentioned in the report.

The other problem with the dset package is that it is riddled with calls to awk, sort, etc with absolute paths. Seriously, this is not 1995. You’ll need to fix tons of calls – just grep for ‘/bin/awk’, ‘/bin/grep’, ‘/bin/sort’ etc in /opt/dell/dset.

When all that’s done, you can generate the report

  dellsysteminfo

which will drop a zip file with a horribly long name (including the use of round brackets, sigh) in /root/ with the password ‘dell’. Awesome security guys. Why is there even a password if it’s so trivial?

The Dell tech I’m e-mailing with (who is very helpful btw, this post is not a criticism about Dell’s support people – only about their GNU/Linux tools) said I’d need to upgrade the system bios and the ‘ESM’, which is the ‘system firmware’. The former was easy with the help of this blog post. In a nutshell:

aptitude install libsmbios-bin libsmbios1 libsmbiosxml1
getSystemId

That will print out your ‘System ID’. Look up the latest system bios for that System ID at


http://linux.dell.com/repo/software/bios-hdrs/

and download it. Then

modprobe dell_rbu
dellBiosUpdate -u -f bios.hdr

and reboot, at which point your bios will be upgraded if all goes well. Note that if your cmos battery is dead, you can’t upgrade your bios anymore because all the command line tool does is load it into ram and flip a switch to tell the bios to upgrade itself on the next boot; without cmos battery either that switch or the bios in ram may not survive a reboot.

So that worked fine. The ESM update was a little trickier. I downloaded another .BIN file from


http://support.euro.dell.com/support/downloads/download.aspx?c=fr&l=fr&s=gen&releaseid=R147948&SystemID=PWE_PNT_2800&servicetag=8YGYQ1J&os=WNET&osl=fr&deviceid=5814&devlib=0&typecnt=0&vercnt=7&catid=-1&impid=-1&formatcnt=4&libid=29&fileid=196749

At least that thing didn’t try to do all sorts of rpm manipulations, it just runs a few scripts (again, you can verify with –extract).

The tricky thing here is that running the BIN file conflicts with the ipmi modules. Here’s what I did:

Boot into single user mode, and stop openmanage and ipmievd:

/etc/init.d/instsvcdrv stop
/etc/init.d/ipmievd stop

Then unload all ipmi modules:

rmmod ipmi_devintf
rmmod ipmi_si 
rmmod ipmi_msghandler

Then start screen, and start the .BIN file. It will look around, cause the ipmi modules to be loaded again and if all is well ask you if you really want to update your ESM, [Y/N].

At this point, break out of screen with ctrl-a d, and unload the ipmi modules again (I kid you not!):

rmmod ipmi_devintf
rmmod ipmi_si 
rmmod ipmi_msghandler

Now return to screen

screen -r

and press ‘Y’. Don’t worry, it’s going to load the ipmi modules yet again, and then starts printing dots on the screen as the upgrade progresses. It takes a while – at least 5 minutes on my system. So just wait until it is done.

Eventually the script said ‘update complete’. I read this trick with screen and the unloading of the ipmi modules somewhere on a dell wiki, but I can’t find the link now (sorry!) so I can’t give credit.

Anyway, and I was able to verify the upgrade with ipmitool (I upgraded to firmware revision 1.72):

# ipmitool -I open mc info
Device ID                 : 32
Device Revision           : 0
Firmware Revision         : 1.72
IPMI Version              : 1.5
Manufacturer ID           : 674
Manufacturer Name         : Unknown (0x2a2)
Product ID                : 0 (0x0000)
Device Available          : yes
Provides Device SDRs      : yes
Additional Device Support :
    Sensor Device
    SDR Repository Device
    SEL Device
    FRU Inventory Device
    IPMB Event Receiver
    Chassis Device
Aux Firmware Rev Info     :
    0x00
    0x00
    0x00
    0x00

So in the end I got almost everything to work (except for the dset output which is still partially missing). But seriously, this was way harder than it should be.

That DSET package is a total mess and needs some serious reworking.

The system bios upgrade procedure is elegant and simple – there Dell deserves credit.

The ESM upgrade… not so much. Seriously – what’s up with those ipmi modules getting in the way twice?

In a nutshell, I’d like Dell to hire some people to clean up the GNU/Linux tools they provide. But they have to be people who know about more than just rpm-based distros, and who are interested in truly cross-distribution solutions and want to work with all (popular) distributions. Basically, if Dell can get their tools into Debian, Fedora and OpenSuse, things will trickle down to almost every other popular flavor of GNU/Linux. There is a lot of work to do…

This entry was posted in Free Software/Open Source, Hardware. Bookmark the permalink.

One Response to Dell’s GNU/Linux support leaves to be desired

  1. oclua says:

    Thanks a lot for the comments. Otherwise I could have not used
    the DSET.

    Just for the sake of complete documentation:
    The delldsetXXX.bin has a header thata claims to be run by
    /bin/sh in the #| line. Actually it needs some derivative of
    the Korn shell. You can run it using

    $ ksh ./dellsetXXXX.bin –extract /path/where/to/extract

Leave a Reply