Stateless Desktop Computing...

[MultimediaMan]

Target Configuration...

I am using Windows Storage Server 2008 R2, with the iSCSI Target v3.4 software... Unfortunately this is not available unless you are a Software Assurance MSDN user. If you want a reasonable replacement and don't mind Linux, by all means look into the EIT (Enterprise iSCSI Target) for Linux or FreeNAS for the BSD Crowd. FreeNAS supports all you would probably ever need to even a mid-sized business if you built the underlying hardware robust enough.

In Storage Server2008 R2 you create a new iSCSI Target in the Storage Manager. The Wizard will populate most of the data...



You need to set access rules for the Target... I use simple IQN addresses.



Finally you either create or mount an existing *.VHD file...



You can test this by mounting the iSCSI Target with a software initiator, or go all in and test with iPXE...I tested with a "blank" iPXE bootloader from command line.

[Jammrock]

Yes, you can... Bitlocker doesn't necessarily Encrypt everything. But administering it can be difficult without a TPM...and a CA, and AD/LDAP.

There is a very small unencrypted partition that contains a few boot files needed to load the decryption stuff. You'd have to load both the boot and system (encrypted) partitions from iSCSI/iPXE boot, but yes it's certainly possible.

Without a TPM you either need a USB drive of floppy drive to store the decryption key. In Hyper-V you can get around the TPM issue by keeping a virtual floppy attached to the VM.

A CA doesn't do anything with BitLocker, but having AD to store your decryption keys is handy. The keys can be stored in a flat file though. For a business this is impractical but for home use it's fine.

Nice work on this project MMM!

[Jammrock]

I am using Windows Storage Server 2008 R2, with the iSCSI Target v3.4 software... Unfortunately this is not available unless you are a Software Assurance MSDN user.

It was actually released to the web for free back in April.

http://blogs.technet.com/b/virtualization/archive/2011/04/04/free-microsoft-iscsi-target.aspx

Version 3.3 at least. I haven't seen 3.4 before.

[MultimediaMan]

Putting it all together...

Everything now explained (more or less)... here's how it works:

The PC Boots from PXE/ TFTP...



and loads the iPXE Bootstrap program...



with the following parameters:

#!ipxe
set keep-san 1
set initiator-iqn iqn.2003-12.org.centos:proteus-iscsi.olympus.bv.ar.us.local
dhcp net0
sanhook --drive 0x80 iscsi:atlas.olympus.bv.ar.us.local::::iqn.1991-05.microsoft:atlas-proteus-0-target
sanboot --drive 0x81 --no-describe http://epimetheus.olympus.bv.ar.us.local/centos/5.7/isos/i386/CentOS-5.7-i386-netinstall.iso

1) keep-san tells the bootstrap program to write data to the iBFT.
2) Initiator-iqn sets the iqn of the client to pass to the iBFT.
3) The DHCP command tells the bootstrap program to send a DHCP Request. If you have compiled the UNDI driver, "net0" represents whatever NIC has received a PXE response... in subquent kernels during the boot sequence...net0 becomes eth1.
4) Pay particular attention to the SANHOOK command... it logs into the iSCSI Target and assigns the Local LUN as Disk 0, but does NOT attempt to boot from it. (This information is written to the iBFT).
5) The SANBOOT after the SANHOOK command tells the the initiator to assign the Local LUN as Disk 1, but NOT to write this information to the iBFT (Using the --no-describe argument), and to Boot from a standard HTTP URL path is designated for the boot image.

*The IP Discrepencies are due to the fact it took several boots to grab the Registered SAN Device shot... a screengrab of a video would have been better.

Some interesting things to note... during the PXE Boot, the Bootstrap program is the only thing downloaded using TFTP, and hence, using the UDP Protocol. There is much less chance of corruption with a small binary. After the iPXE Bootstrap program is downloaded, all connections are TCP/IP... that reason alone is justification for using iPXE...the ISO image mount is about 10x faster than a UDP Download.

[MultimediaMan]

Update to iSCSI Boot scripts...

The iPXE Stack supports both embedding and calling scripts... it also has a shell which can interrogate certain systems properties. Like the UUID of a system, for instance... this allows the placement of large numbers of differentiated boot scripts to be largely automated.

The real solution here is to use PHP to dynamically create boot scripts which tell the machine what network, path and LUN to boot from and how. Server-side control of the boot process is what is achieved when you go with a database-driven set of definitions.

But before I delve into setting up a Database for this and a PHP front end, let me show a simpler way to manage your maintenance of boot scripts...

Create a Generic iPXE boot script to embed into the Bootstrapper...

Embedded code in iPXE:

#!ipxe
#Generic Boot Script
:retry
dhcp
chain http://%WEBSERVER_DNS_NAME%/ipxe/boot/${uuid}.ipxe || goto retry

The string "${uuid}" is actually a global variable which iPXE populates at the time the bootstrapper is executed... I use that as part of a filename to execute a boot script with that name. Note, there are no dependencies beyond having a correct Webserver DNS name embedded into this script and a BIOS which supports a UUID (everything from 2006 on does... most before 2006 do as well).

Now, in the webserver create a directory structure to keep your boot scripts in;

In WebServer Directory:

%WEB_ROOT_DIRECTORY%/ipxe/boot/%UUID%.ipxe

#!ipxe
goto startme
:retry
sleep 5
:startme
set keep-san 1
set initiator-iqn %IQN_QUALIFIER%:%CLIENT_DNS_NAME%
sanboot --drive 0x80 iscsi:%TARGET_DNS_NAME%::::%IQN_QUALIFIER%:%TARGET_NAME% ||
sanboot --drive 0x81 --no-describe http://%TARGET_DNS_NAME%/scientific_linux/6.1/x86_64/iso/SL-61-x86_64-2011-07-27-boot.iso || goto retry

This has been working very well whilst I get my PHP environment working...

[dZeus]

Today I finally got round to trying this with Ubuntu... and it works! Basically a chainloading of gPXE and then booting Ubuntu off the iSCSI target. It's not as hard as I imagined beforehand.

In the setup I've tried:
- dnsmasq as DHCP and TFTP server (OpenWRT router)
- no http server involved; dnsmasq allows to give different dhcp options depending on the MAC and for iPXE clients; the following lines were added to /etc/dnsmasq.conf

# enable tftp and chainloading for iPXE
enable-tftp
tftp-root=/tmp/tftp
dhcp-match=ipxe,175
dhcp-boot=net:#ipxe,undionly.kpxe
dhcp-option=net:ipxe,17,"iscsi:192.168.1.5::::iqn.2012-01.lan.storage-server:storage.disk1"

- I first installed Ubuntu onto the iSCSI target using VirtualBox. I used VirtualBox' iSCSI initiator to transparently mount the iscsi target before the installation. After I had Ubuntu 11.10 set up (which unfortunately took many hours to install), I set up the PXE/iPXE chainloading boot. This requires iscsi-initiator support to be added to Ubuntu and the iscsi target passed to the bootloader. The etherboot wiki explains how to do this.
- It should be possible to boot the same Ubuntu image from both real machines and virtual machines, if the iPXE firmware is added as a disk image to VirtualBox (haven't verified yet; just did the first chainloaded PXE boot)

Ubuntu is nice without access to a version of Windows that doesn't require activation when hardware changes are detected; it will work fine regardless of the hardware used (as long as its x86).

This is basically a very nice way to test OSes without needing to install them to a local drive. Or to run machines without any drives.

[MultimediaMan]

Correct: you can migrate between physical hardware and virtual pretty much at will with a newer OS (including Windows 7/2008R2 and Windows 8); Windows doesn't much like the fact that the UUID of the system changing, but in a KMS environment, you'd never notice it.

You can also migrate between hypervisors seemlessly... The storage is completely separated from the local hardware and the Hypervisors.

[dZeus]

Just successfully tested booting the Ubuntu from the iscsi-target in Virtualbox, without using the build-in VirtualBox iscsi-initiator. It's not necessary to mount any iPXE image in the virtual machine, PXE took care of it just like for the non virtual machines.

Note that VirtualBox requires installation of an add-on pack to get PXE boot on some of the virtualised network controllers (the Intel ones). Another thing is that the iPXE rom doesn't reliably get the DHCP responses. Rebooting once or twice usually makes it get the response. This might be due to promiscuous mode not being set on the host machine's network interface with the guest OS network interface bridged to the host OS one, but I haven't verified.

Unfortunately, VirtualBox is horrendously slow with Ubuntu (even with the VirtualBox drivers installed onto the guest OS). I haven't tried Xen or Microsoft's solutions yet for desktop OS (my webserver though runs in a Xen VPS of a cloud provider).

Might be interesting to try...

[MultimediaMan]

The DHCP Problems are VirtualBox-specific. Even a regular PXE ROM has a problem from time to time.

DHCP Option 175 is a "nice-to-have", but it is NOT required.

I have deployed a fully virtualized Microsoft SQL Cluster in a VMware Environment with the "A" and the "B" nodes residing in different logical clusters (vCenter Version 4.1 Update 2/ ESXi 4.1 U2). The Nodes boot iPXE using iSCSI Targets, and the Shared Cluster Disk and Tiebreaker disks are also iSCSI Connections: It works like a charm.