August 2009 Archives

ISO Files

Most Linux distributions and a number of CD utilities like Clonezilla or GPARTED-Live come as iso-images, ready to be burned to a CD medium, to start an installation or running a live session for some useful purpose. But all these iso files are supposed to be booted inside an ordinary CD drive. What if you just don't have a CD burner at hand or if your CD drive is damaged or missing all together, because you are testing the shiny new motherboard without a CD drive?

Don't think you are stuck, there's help around the corner that will lead you into a habit of booting your iso files in a much more creative fashion.

"Stop for a moment, could I not simply put the iso file on a USB key using the dd command? Most modern BIOSes are perfectly capable of booting USB keys and surely there is a USB slot on your new motherboard you can use."

Of course you could do that, the iso file contains a valid iso9660 filesystem together with the bootloader for CDs. And the dd command will transfer the file to your USB key without adding any other structure. But, don't expect the new USB key to boot like a CD, because after you have changed your BIOS settings to allow booting from a USB hard disk, the BISO will expect the structure of a hard disk not that of an iso-filesystem. It will look for a MasterBootRecord (MBR) in sector 0 in the same way as if it would boot your ordinary hard disk. And an iso-filesystem doesn't have a MBR and a partition table where the BIOS expects it to be. So forget booting iso files directly.

"Ok, seems as if there is no way around putting a bootloader like grub on the USB key, and ... , wait, there is a file called iso9660_stage1_5 in the grub directory. It looks like grub has iso9660 support already. Can we use this to boot the iso file?"

Well, unfortunately no. The iso9660 support built into grub can be used to create a bootable CD with grub as the bootloader. Your file is being used when a new iso file containing your grub bootloader is created. What we need instead is the ability to read the kernel and the initramdisk inside the iso-filesystem at boot time when no filesystem is mounted.

"Sounds as if we need a loop device to get at the files inside the iso filesystem. Does grub support mounting a iso-filesystem using a loop device?"

The New Grub

Yes and no. The traditional grub (version 1) that everyone uses does not have this loopback support, but the grub development team has been working on grub2 for a number of years now, which has loopback support already. The new grub2 is still under development, so it's not yet ready to replace the legacy grub version 1, but grub2 is exactly what we need to boot our iso files directly.

A Word Of Caution

So we are about to prepare a USB key to boot iso files with the grub2 bootloader written into the key's master boot record. Writing MBRs on a Linux system usually raises blood pressure a bit, as we have to be extra cautious not to ruin our running Linux system. If you want to follow me from here, please make sure that you do not perform the grub2 installation on your company's production server and double-check everything before issuing commands, to be on the safe side. And for those of you, who hesitate to write MBRs at all, I have an offer for you at the end of this posting, read on.

Preparing The USB Boot Key

Like any other software grub2 can be installed using the distros repositories, at least with Fedora 11 that I use now. The installation will not replace our legacy grub software we are using to boot our computer, as the new files go into a directory "/boot/grub2" without changing anything of the current grub setup unless grub2 is deliberately used to overwrite the MBR. And this is the only thing we have to avoid. But for now, our objective is to get the grub2 software onto our USB key, writing the key's MBR will then be the crowning event before we start testing the key.

For preparing the boot key it is absolutely essential that you know how the Linux kernel recognizes your USB key. To find out, you can observe the log file "/var/log/messages" while you plug your key into the computer. For now I assume that you have one hard disk in your computer (/dev/sda) and your USB key will probably show up as /dev/sdb. First we re-partition the key and create two Linux partitions, a small one of 10 MBytes for the grub2 software and a larger one where all our iso files can be stored:

You can now mount the second partition and copy all the iso files you wish to boot into this location. Writing the MBR now requires that the first partition is mounted on a well know mount point like /mnt, because we will use this directory to write the grub2 software to the key with the following command:

Please double-check that your key is mounted and is called /dev/sdb before you use this command, which populates the directory /mnt/boot/grub2 with grub2 modules and writes the MBR into /dev/sdb.

As you may know there is one task still left before we can start testing our boot key, we need to create a menu list, which is called "/boot/grub2/grub.cfg" now. A number of changes have been made to the config file compared to the old grub, so let's have a look at the new configuration:

The most remarkable change is the numbering of partitions which has become more natural as the first partition now is (hd0,1), while hard disks still start with number 0. Then the old title entry is replaced by menuentry and loading the kernel is now done with the linux command. Apart from all these cosmetics we can now create a new device called (loop) that will replace a real partition like (hd0,2) to access files inside the iso-filesystem, once we have connected the iso file with this new device using the loopback command.

It is even possible to list the files that grub2 can see on the (loop) device using the grub2 command line:

Now it's time to check if the ISOLINUX menuentry works. Reboot and see your new boot key fire up the CentOS netinstall process. Voila.

Download Your Bootable USB Key

I have prepared an image of my boot key for you that you can download, if you want to create a boot key without the hassle of following the steps above. The small 32 MByte image file contains all the magic and can be copied to your USB key with the dd command. Please send me an email and I will give you a download link with the necessary instructions. Once you have your key working, you can delete and re-create the second partition to make space for all your live CD iso files on the key. All you need to edit is the config file.

That way you can carry all your distros on one key and boot them all from the grub2 menu without having to change the Linux system on your hard disk.

A few weeks ago I happened to install CentOS-5.3 on a HP blade server, and while scrolling through the software selection dialog, I ticked everything that had to do with Virtualisation and Clustering. Consequently, the new system came up with a shiny Xen kernel and displayed a new icon in the system tools menu, the "Virtual Machine Manager".

Everything else was still pretty normal, but the XEN kernel was constantly reminding me to use it for what it's for, starting a virtual machine. As you may know Xen is not the easiest way to get such a virtual system in place even thought there is a graphical frontend for Xen as well. But there is one advantage of Xen that makes up for all the trouble you are having to get the thing going and that is speed.

Unlike other virtualisation software like VirtualBox that are designed to run unmodified operating systems within an application environment that emulates everything, Xen aims at running on bare hardware as directly as possible. Xen promises to achieve near native performance by putting a kernel software called hypervisor between the hardware and the guest operating system. The hypervisor occupies the privileged part of the CPU driving the guests away into the non-privileged areas of the CPU, but offering to execute privileged code on behalf of the guests whenever they need to use privileged code. It is this design, that speeds up performance but on the other hand makes it necessary that the guests find their way to the hypervisor by using a special Xen-enabled kernel themselves. So operating systems that cannot be modified to have a Xen kernel (like Microsoft Windows) cannot run in the speedy paravirtualized mode.
 

Beware Of Using Defaults

In my experience, what makes Xen very tricky to use at first, is that the default values after a fresh installation can easily break your network connection, leaving you with plenty new network interfaces but without a working local network, let alone the internet. On boot there are two services that are being started for the Xen system, the libvirtd and the xend daemons. These services read a single, very short  configuration file "/etc/xen/xend-config.sxp" which consists of only nine lines:


And line no. 4 is the culprit that renames a working ethernet interface, recreates it and introduces two bridge interfaces leaving a mess in which nothing does work any more. I strongly suspect that this is not what defaults are for. Fortunately a small modification gets Xen on track again. Simply don't use the script "network-bridge" and replace it by /bin/true, problem solved. It seems to me that the Xen defaults are striving to isolate the new virtual machines from the ethernet as much as possible, while someone starting to use Xen might want the virtual machine to use the already working internet adapter with a new fixed IP address in the LAN.

Use Your Own Bridge

If you are after a new virtual machine on the same LAN you can create a config file for a bridge (br0) that replaces your ethernet interface and connect your ethernet interface to that bridge. All firewall settings for your host can remain the same as your new bridge takes over the old fixed IP address and nothing seems to have changed for the LAN. Once your virtual machine is ready to run, just use the new bridge br0 as network interface in the config file and you can assign an unused IP for the guest on the LAN with the default gateway for internet access, and everything works well.

/etc/xen/xend-config.sxp

/etc/sysconfig/network-scripts/ifcfg-br0

/etc/sysconfig/network-scripts/ifcfg-eth0

Try it, and you are beginning to love Xen!