Solutions from the road: Red Hat Enterprise Linux Rescue Mode over PXE (part I)

One of the many uses of having a Red Hat Consultant on site is to provide “best practices” in regard to using or deploying Red Hat Linux or any of Red Hat’s applications. These best practices come from the way a product was written as well as how the product was originally intended to be used. Although we lean towards sticking to these best practices and procedures, sometimes they need to be modified and adjusted to meet the special needs of a client. Knowing how to make a product work within a customer’s specialized environment is also a great use of a Red Hat Consultant.

Recently, a client of mine needed a means by which a server could be booted in a safe environment for network restore. My first thought was, “boot to rescue mode using disk 1 of the installation media”. Here is where the specialized environment of the client came in to play.

The sensitive nature of the business, and it’s data, meant that all desktop machines were ordered without CD-ROM drives and although the servers had remote console access, the ability to use virtual media was effectively disabled. The fact that the support team was in one city and half of the servers were in another city meant that physical access was going to be an issue approximately half of the time.

What is a consultant to do? The sum of these small hurdles begs the question, “how were the servers installed in the first place?” If the servers were installed in such an environment, then chances are we can come up with a rescue solution for that same environment. The answers to the install question are PXE and Kickstart. The short descriptions of these are “a means of booting a host from a network accessible kernel” and “an automated way of deploying many like servers in a repeatable manner” respectively. We are only concerned with the PXE portion of the equation here.

This article assumes that PXE is already set up in the environment, and that the reader is familiar with setting up PXE/tftp as well as bringing up a host in rescue mode (at least in a more conventional manner). A tutorial on setting up a very basic PXE server can be found here. I’m simply going to illustrate adding a rescue configuration to an existing PXE environment.

The default PXE environment exists under /tftpboot/linux-install, so any relative paths below will refer to this. The first thing to edit is the pxelinux.cfg/default file. Assuming there is already at least one entry, like the following,

label 1
        kernel RH-AS-64U4/vmlinuz
        append ks=http://kickstart.server.com/pub/ks.cfg initrd=RH-AS4-64U4/initrd.img

add the following lines underneath.

label r1
       kernel RH-AS-64U4/vmlinuz
       append rescue initrd=RH-AS4-64U4/initrd.img

Under the original “label 1”, the append line signifies that an install over HTTP is about to be initiated and the relative paths for the initrd.img and vmlinuz files are provided. Under the “label r1” that was just added, the append line has been altered to reflect a rescue boot. Keep in mind that the initrd and kernel need to match the server being rescued. The labels are named arbitrarily, they just have to match between the msgs/boot.msg file and the pxelinux.cfg/default file.

Next, edit the msgs/boot.msg file to provide a human readable entry listing the new PXE profile.

[snip]
1 Default Install (RH-AS-4-64-U4)
r1 Rescue Env (RH-AS-4-64-U4)

Now we need to make the media available. Copy the disk 1 ISO image for the exact version of RHEL that you are rescuing to an NFS exportable directory. This step can be repeated for each of the versions of RHEL that your environment supports. Mount the ISO image on the loopback device and then export the mounted ISO. (In all fairness, this could have been done with HTTP or FTP as well.)

Lastly, boot the server in question, choosing PXE (or network, depending on the make/model), and wait for the PXE menu to come up. Choose your newly created “r1” label, and follow the prompts for network device, language, and keyboard. When prompted for rescue method, simply type the NFS server name and directory. The rest of the process follows standard rescue protocol.

The customer just needed a painless method of getting to rescue mode that fit the confines of their environment. I’m sure if we spent enough time and money we could have come up with something big and expensive, but many times its just better to keep it simple. Hopefully this is worthy of your “bag of tricks”.

By the way, if your “bag of tricks” is full of Linux goodies, you might want to check out the Consultant job postings.

This entry was posted by Jon Benedict on Thursday, July 5th, 2007 at 1:21 pm and is filed under Red Hat Enterprise Linux, tips and tricks, technical, documentation. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.