SuperMicro TwinBlades, Software Setup

So, I've covered most of the hardware for the SuperBlades.  But what about a software configuration?  These machines are acting as front-end web servers, which pull code and data from a NAS. However, there's no reason to have enterprise disks in each of these blades just for the OS, so let's get some diskless booting happening!


The first TwinBlade got the same 40W CPUs as the others, but each got a pair of 600GB Velociraptors in RAID 1.  There we go, single disk failures are taken care of.  But what about an OS crash, a fat-fingered remote shutdown, or other issue?  Well, there is a nifty piece of software called DRBD, which essentially acts as RAID 1 across a network.  So, complete data sets exist on 4 disks, across two systems.  By tying in a couple of other packages, active or passive takeover can be achieved if one system fails.

NOTE:  These will NOT be exhaustive steps, just the important bits to get it all to work.
Take your primary and backup machines, and install DRBD on them.  Take your disk or partition, and set them up as a DRBD device.  If you're going to use DRBD, you need to understand the mechanics, so read their documentation for setup info.


Then, mount your DRBD device.  We'll use "/export" as a mount point for this example.

Now, first you need to make sure that each machine gets the right info, so let's do some DHCP work:

host blade-1 {
hardware ethernet 01:02:03:04:05:06;
fixed-address 10.11.12.13;
next-server 10.11.12.14;
if ((exists user-class) and (option user-class = "gPXE")) {
        option root-path "iscsi:10.0.1.28::::iqn.2012-04.com.foobar.iscsi1:blade-1";
        filename "";
  } else {
        filename "undionly.kpxe";
  }
}

This does a few things.  First, it gives the machine a fixed IP address.  It then tells it where to look for the next bit of the boot sequence, via the "next-server" statement.  Then, the if-else bracket tells it to grab the "undionly.kpxe" bootloader (courtesy of gPXE) if is acquiring a DHCP address for the first time.  But, once it has that, and is executing the gPXE environment and asks for the DHCP info again, it instead gives it the necessary info for gPXE to get the iSCSI disk going.  Crazy stuff.

So, then you need a TFTP server on 10.11,12.14, handing out the "undionly.kpxe" file - which can be compiled from the gPXE source.  Read-only, don't be a security fool.

Then, create a disk file on your iSCSI target:

dd if=/dev/zero of=./blade-1-disk.img bs=1 count=1 seek=17179869183

Why 17179869183?  Well, that's 16 gigs minus the one byte which we will write, so the file ends up at exactly 16 gigs.  Since we only wrote a single byte to the end of the file, not only is the creation fast, it also takes advantage of Linux's ability to use sparse files, which will save you some disk space - at least until you actually fill that up.

Now, let's set up IET to serve that out.  First, in your ietd.conf:

Target iqn.2012-04.com.foobar.iscsi1:blade-1
        Lun 0 Path=/export/blade-1-disk.img,Type=fileio,ScsiSN=Blade1
        InitialR2T no
        ImmediateData yes

Then, in your initiators.allow so that only the relevant machine sees the LUN:
iqn.2012-04.com.foobar.iscsi1:blade-1-disk.img 10.11.12.13

Now, you should be able to start up your CentOS installer, choose "advanced storage configuration", install to the iSCSI target, and rock and roll.