So I have a poweredge 2800 with some hardware issues (voltage sensors on the riser card seeing things they should not…). The Dell support folks wanted me to run a DSET report with this tool
http://support.euro.dell.com/support/downloads/download.aspx?c=uk&l=en&s=gen&releaseid=R155882&formatcnt=2&libid=0&fileid=208066
Man – what a pile of crap. This thing is a self-extracting archive for rpm-based systems (redhat/suse). Fine – except that I run Debian on that machine. Anyway, you can run it with –extract to extract the contents:
./delldset_v1.4.0.8.bin --help
Command-line options for Dell System E-Support Tool (DSET)
Usage: [options...]
Options:
-h,--help : Display command-line usage help
-i,--install : Install/upgrade the DSET application
-v,--version : Display version information
--list : Display contents of package (+)
--extract : Extract files to specified path (+)
Using no options will display the interactive user
interface which will guide you through using DSET.
These contents are basically an rpm file and some support files:
-r-xr-xr-x 1 ward ward 4133 Apr 2 2007 Dell_License*
-r-xr-xr-x 1 ward ward 9909 May 8 2007 README*
-r-xr-xr-x 1 ward ward 58 May 8 2007 Version.txt*
-r-xr-xr-x 1 ward ward 15200341 May 9 2007 delldset-1.4.0-8.i386.rpm*
-rw-r--r-- 1 ward ward 15187240 Oct 27 15:57 delldset_1.4.0-9_i386.deb
-r-xr-xr-x 1 ward ward 10271 Apr 2 2007 install.sh*
-r-xr-xr-x 1 ward ward 472 Apr 2 2007 sphelp.txt*
-r-xr-xr-x 1 ward ward 24 May 4 2005 test.sh*
-r-xr-xr-x 1 ward ward 9877 Apr 2 2007 utility.sh*
The rpm can be transformed into a .deb with alien:
$ alien delldset-1.4.0-8.i386.rpm
Warning: Skipping conversion of scripts in package delldset: postinst postrm preinst prerm
Warning: Use the --scripts parameter to include the scripts.
delldset_1.4.0-9_i386.deb generated
$ alien delldset-1.4.0-8.i386.rpm --scripts
delldset_1.4.0-9_i386.deb generated
Which installs fine – but you really want to be using a kernel that has ipmi functionality compiled as modules; otherwise the package will not install properly until you add
hapi.allow.user.mode=yes
to /etc/omreg.cfg. In any case, you need to have ipmi as modules for the rest of this sequence of commands, so you might as well boot into a stock Debian kernel.
You’ll probably want to have installed Dell’s open manage before you run the DSET report. To do that, add
deb ftp://ftp.sara.nl/pub/sara-omsa dell sara
to your /etc/apt/sources.list file and
apt-get update
apt-get install dellomsa
So, then you would be ready to run the dellsysteminfo command but WAIT – don’t do it yet. That script is dangerous. It does stuff like this, probing for the running services (from /opt/dell/dset/bin/dell-sysreport.sh):
#Dell: Extra OS stuff we want
echo " Getting status of services ..." ${TEE2LOG}
#Dell: SuSE doesn't have a "service status" method so do it the manual way
#catifexec "/sbin/service" "--status-all"
rm -f $ROOT/service > /dev/null 2>&1
SERVICEDIR=/etc/init.d
#pushd ${SERVICEDIR} > /dev/null
for SERVICE in `ls -1 ${SERVICEDIR}` ; do
case "${SERVICE}" in
#Skip over these "services" since they tend to hang some systems when requested for status
functions | halt | killall | single| linuxconf| kudzu | bgpd | boot | reboot | single | nscd | \
halt.* | rc | boot.* | *rpmorig | *rpmnew | *rpmsave | *~ | *.orig)
;;
*)
if [ -x "${SERVICEDIR}/${SERVICE}" -a -f "${SERVICEDIR}/${SERVICE}" ]; then
echo "Service Name: ${SERVICE}" >> $ROOT/service
/usr/bin/env -i LANG=$LANG PATH=$PATH TERM=$TERM "${SERVICEDIR}/${SERVICE}" status >> $ROOT/service 2>&1
echo >> $ROOT/service
echo "---------------------------------------------------------------------------" >> $ROOT/service
fi
;;
esac
done
#popd > /dev/null
That didn’t go over too well on my system – a bunch of things stopped working (like the ‘w’ command), samba started segfaulting, etc. It’s clearly calling some init scripts that don’t deal well with being asked for a status.
So I commented out that part of the script, which made things a lot happier – but of course services are now not mentioned in the report.
The other problem with the dset package is that it is riddled with calls to awk, sort, etc with absolute paths. Seriously, this is not 1995. You’ll need to fix tons of calls – just grep for ‘/bin/awk’, ‘/bin/grep’, ‘/bin/sort’ etc in /opt/dell/dset.
When all that’s done, you can generate the report
dellsysteminfo
which will drop a zip file with a horribly long name (including the use of round brackets, sigh) in /root/ with the password ‘dell’. Awesome security guys. Why is there even a password if it’s so trivial?
The Dell tech I’m e-mailing with (who is very helpful btw, this post is not a criticism about Dell’s support people – only about their GNU/Linux tools) said I’d need to upgrade the system bios and the ‘ESM’, which is the ‘system firmware’. The former was easy with the help of this blog post. In a nutshell:
aptitude install libsmbios-bin libsmbios1 libsmbiosxml1
getSystemId
That will print out your ‘System ID’. Look up the latest system bios for that System ID at
http://linux.dell.com/repo/software/bios-hdrs/
and download it. Then
modprobe dell_rbu
dellBiosUpdate -u -f bios.hdr
and reboot, at which point your bios will be upgraded if all goes well. Note that if your cmos battery is dead, you can’t upgrade your bios anymore because all the command line tool does is load it into ram and flip a switch to tell the bios to upgrade itself on the next boot; without cmos battery either that switch or the bios in ram may not survive a reboot.
So that worked fine. The ESM update was a little trickier. I downloaded another .BIN file from
http://support.euro.dell.com/support/downloads/download.aspx?c=fr&l=fr&s=gen&releaseid=R147948&SystemID=PWE_PNT_2800&servicetag=8YGYQ1J&os=WNET&osl=fr&deviceid=5814&devlib=0&typecnt=0&vercnt=7&catid=-1&impid=-1&formatcnt=4&libid=29&fileid=196749
At least that thing didn’t try to do all sorts of rpm manipulations, it just runs a few scripts (again, you can verify with –extract).
The tricky thing here is that running the BIN file conflicts with the ipmi modules. Here’s what I did:
Boot into single user mode, and stop openmanage and ipmievd:
/etc/init.d/instsvcdrv stop
/etc/init.d/ipmievd stop
Then unload all ipmi modules:
rmmod ipmi_devintf
rmmod ipmi_si
rmmod ipmi_msghandler
Then start screen, and start the .BIN file. It will look around, cause the ipmi modules to be loaded again and if all is well ask you if you really want to update your ESM, [Y/N].
At this point, break out of screen with ctrl-a d, and unload the ipmi modules again (I kid you not!):
rmmod ipmi_devintf
rmmod ipmi_si
rmmod ipmi_msghandler
Now return to screen
screen -r
and press ‘Y’. Don’t worry, it’s going to load the ipmi modules yet again, and then starts printing dots on the screen as the upgrade progresses. It takes a while – at least 5 minutes on my system. So just wait until it is done.
Eventually the script said ‘update complete’. I read this trick with screen and the unloading of the ipmi modules somewhere on a dell wiki, but I can’t find the link now (sorry!) so I can’t give credit.
Anyway, and I was able to verify the upgrade with ipmitool (I upgraded to firmware revision 1.72):
# ipmitool -I open mc info
Device ID : 32
Device Revision : 0
Firmware Revision : 1.72
IPMI Version : 1.5
Manufacturer ID : 674
Manufacturer Name : Unknown (0x2a2)
Product ID : 0 (0x0000)
Device Available : yes
Provides Device SDRs : yes
Additional Device Support :
Sensor Device
SDR Repository Device
SEL Device
FRU Inventory Device
IPMB Event Receiver
Chassis Device
Aux Firmware Rev Info :
0x00
0x00
0x00
0x00
So in the end I got almost everything to work (except for the dset output which is still partially missing). But seriously, this was way harder than it should be.
That DSET package is a total mess and needs some serious reworking.
The system bios upgrade procedure is elegant and simple – there Dell deserves credit.
The ESM upgrade… not so much. Seriously – what’s up with those ipmi modules getting in the way twice?
In a nutshell, I’d like Dell to hire some people to clean up the GNU/Linux tools they provide. But they have to be people who know about more than just rpm-based distros, and who are interested in truly cross-distribution solutions and want to work with all (popular) distributions. Basically, if Dell can get their tools into Debian, Fedora and OpenSuse, things will trickle down to almost every other popular flavor of GNU/Linux. There is a lot of work to do…