Gentoo PXE Server

Have you ever had to boot up a rescue LiveCD/LiveDVD/LiveUSB and found that you either had:

1. Lost the Live* boot medium
2. Had the wrong Live* boot medium
3. The Live* boot medium was corrupt
4. The Live* boot medium just simply didn't work for any other reason

Well I have, and I was sick of having to dig around to create a new bootable medium for my computers at home, so I came up with a solution!

Enter Preboot Execution Environment (also known as PXE).

PXE in a nutshell is a system by which a computer connected to a wired network can query that network for boot images when starting up, allowing the device to boot from information found on the network instead of from some device directly connected to it.

This little project is my endeavors into getting PXE working on my own home network.

I have a few prerequisites that I am going to assume are met, since setting them up individually is an effort in and of themselves and information can be found online (Google it!) for these things.

I expect the following:

This might seem silly, but PXE is only supported (AFAIK) in wired devices, as the device has to be able to query the network without any prior setup (such as wireless configurations and passwords).

I use a Linksys router with DD-WRT installed on it, and expect your network to have a router with it installed as well. Not only is it awesome firmware to begin with, it takes care of our DHCP configuration for us through the use of DNSMasq.

PXE requires a server to be connected to the network that will host the boot images. When a device network-boots it is told by the DHCP server (in our case the router) to look for boot images on this server.

Finally, you need to be able to actually PXE boot your devices. Most modern network cards support this. Be sure to check your BIOS and make sure PXE is enabled on your network cards.

For the purposes of this document, I will assume the following (you should alter this information for your own setup:

• ${NETRANGE} = 10.168.100.0/24 - the IP range of your local network
• ${GENTOO_IP} = 10.168.100.3 - the IP address of the Gentoo server
• ${GENTOO_HN} = konata - the hostname of the Gentoo server
• ${PXE_DIR} = /pxe - a directory on the Gentoo server to store all of the PXE stuff

The first step in getting PXE setup is to set up a TFTP server (Trivial FTP). These are extremely simple FTP servers that the network boot protocol will use to download a boot loader of some kind to your network device.

First up we need to install aTFTP:

emerge atftp

Next you will need to edit its configuration file found it /etc/conf.d/atftp. Be sure to set TFTPD_ROOT to whatever ${PXE_DIR} is appropriate for your setup.

It should look like this when you are done:

/etc/conf.d/atftp

# Config file for tftp server
 
TFTPD_ROOT="/pxe"
TFTPD_OPTS="--daemon --user nobody --group nobody"

Finally, start up the server and set it to run at boot.

/etc/init.d/atftp start
rc-update add atftp default

Although an NFS server (Network File System) isn't needed for PXE to function, it is extremely useful to have for your PXE boot images, as Linux can boot off of an NFS root.

First you will need to install the nfs-utils package.

emerge nfs-utils

Next you will need to set up your exports. Exports are essentially a list of directories that can be shared via your NFS server, as well as who is allowed to access them and what properties they can be accessed with. This information is stored in /etc/exports.

You should edit this file to look something like this:

/etc/exports

# /etc/exports: NFS file systems being exported.  See exports(5).
/pxe 10.168.100.0/24(ro,sync,no_root_squash,no_subtree_check)

Here is a breakdown of that file:

• /pxe - this is the exported directory (should be set to your ${PXE_DIR})
• 10.168.100.0/24 - this is the IP range to share to (should be set to your ${NETRANGE})
• ro - allow read-only access (this should be rw if your boot images will need read-write access to NFS, mine don't)
• sync - NFS will only reply to requests after all current requests are committed to disk
• no_root_squash - allow requests from a root user to be treated as such
• no_subtree_check - increase performance and reliability by disabling subtree checking

Finally, start up NFS and make sure it starts by default.

/etc/init.d/nfs start
rc-update add nfs default

Now that we have our servers set up, we need to create a bootloader to send to our network devices that wish to PXE boot. Since I have always been a fan of GRUB (the GRand Unified Bootloader), I have opted to use their latest incarnation: GRUB2.

Since GRUB2 is technically still unstable, we will not be installing it directly on our server, but instead will be downloading and compiling it ourselves in order to generate our PXE-bootable image.

First step is to download a copy of GRUB2's source code. You should be able to find a copy of it on their ftp server: ftp://ftp.gnu.org/gnu/grub/.

I personally downloaded version 1.99 (to a random directory to work in):

wget ftp://ftp.gnu.org/gnu/grub/grub-1.99.tar.gz
tar -zxvf grub-1.99.tar.gz

Compiling GRUB2 should be relatively easy. Simply run the following commands:

cd grub-1.99
./configure
make

We will now create a boot directory for GRUB2 in our ${PXE_DIR}

mkdir -p /pxe/boot/grub

We now need to copy over all of the *.lst and *.mod files for GRUB2 to use. The *.mod files are modules that GRUB2 loads in order to perform various tasks, while the *.lst files contain metadata for those modules (dependencies and such).

cp ./grub-core/*.{mod,lst} /pxe/boot/grub/

==== Generate the GRUB2 PXE boot image

Finally, we can now generate our PXE boot image from our compiled GRUB2:

./grub-mkimage \
    -d /pxe/boot/grub \
    -p '(pxe)/boot/grub' \
    -O i386-pc-pxe \
    -o /pxe/boot/grub/grub.pxe \
    pxe pxecmd

This creates a GRUB2 boot image using the following information:

• -d ./grub-core - tells the program where it can find required files to create the image
• -p '(pxe)/boot/grub' - create an image where the prefix for all of the grub files is under /boot/grub under the TFTP root.
• -O i386-pc-pxe - create a i386 PC PXE boot image (standard PC PXE)
• -o /pxe/boot/grub/grub.pxe - output file
• pxe pxecmd - include various PXE modules by default

With that you should now have a ./boot/grub folder within your ${PXE_DIR} that contains many *.mod and *.lst files, as well as your grub.pxe boot image.

Now you have everything setup to allow network booting, the last thing you need is to actually set up your router to point new devices to the Gentoo server for their information.

To do this you must first go into your DD-WRT panel and be sure to enable DNSMasq as your DHCP server. This is done under the Setup→Basic Setup tab, under Network Setup: Network Address Server Settings (DHCP). You must make sure you have DHCP enabled (the radio button) and that you use DNSMasq as your server (the checkbox).

Below I have highlighted the options you need to set.

The last step is to go into the Services→Services tab, under DNSMasq, be sure to enable DNSMasq, and set the options to look like the following:

dhcp-vendorclass=pxe,PXEClient
dhcp-boot=net:pxe,/boot/grub/grub.pxe,konata,10.168.100.3

These options perform the following actions:

• dhcp-vendorclass=pxe,PXEClient - create a pxe namespace that we can query against based on whether or not the client connecting is a PXE client
• dhcp-boot=net:pxe,/boot/grub/grub.pxe,konata,10.168.100.3 - if the client connecting is a PXE client (net:pxe), then tell them that they can find our GRUB2 PXE bootloader on the TFTP server at IP 10.168.100.3, with hostname konata, and at TFTP location /boot/grub/grub.pxe.

Below is a screenshot of my settings.

With this a device should be able to boot up into GRUB2 from PXE, although there isn't yet a configuration file or an actual kernel to boot yet.

The final step in getting your PXE booting network set up is to actually prepare a bootable image for your Gentoo server to send out to devices on boot.

For the purposes of this project, I chose the x86 Gentoo Minimal Install CD. I chose this because it has some nice features in its kernel that make it easy to set this all up.

The first step is to go to Gentoo.org and grab the x86 Minimal Installation ISO. You should save this to some temporary location. We will be extracting the needed files from it.

Once downloaded it, you should extract its contents to the TFTP root like so:

# make a directory for our install cd contents in PXE root
mkdir -p /pxe/x86_gentoo_install
 
# make a directory to mount the ISO
mkdir -p /tmp/isomnt
 
# mount the ISO
mount -o loop /path/to/x86_gentoo_minimal_installation.iso /tmp/isomnt
 
# copy the contents
cp -a /tmp/isomnt/* /pxe/x86_gentoo_install
 
# clean up
umount /tmp/isomnt
rmdir /tmp/isomnt

Now we need to grab the kernel and initrd images from the extracted install ISO.

cp /pxe/x86_gentoo_install/isolinux/gentoo /pxe/boot/x86_gentoo_install.kernel
cp /pxe/x86_gentoo_install/isolinux/gentoo.igz /pxe/boot/x86_gentoo_install.initrd

Finally, we need to create a configuration file for GRUB2 and add a menu entry for our network boot image. This file should be created at /pxe/boot/grub/grub.cfg.

/pxe/boot/grub/grub.cfg

set timeout=10
set default=0
menuentry 'Gentoo Minimal Install (x86)' {
  set gfxmode=1024x768,auto
  set root=(pxe)
  linux /boot/x86_gentoo_install.kernel root=/dev/ram0 init=/linuxrc looptype=squashfs loop=/image.squashfs cdroot real_root=/dev/nfs nfsroot=${pxe_default_server}:/pxe/x86_gentoo_install docache
  initrd /boot/x86_gentoo_install.initrd
}

A lot of this information is pulled directly from the install ISO's boot configuration, found in /pxe/x86_gentoo_install/isolinux/isolinux.cfg, but here is an explanation of the various parameters:

• set timeout=10 - upon booting grub, give the user 10 seconds to pick an image to boot before booting the default
• set default=0 - zero-based offset of the default image to boot if the user doesn't interact with grub
• menuentry 'Gentoo Minimal Install (x86)' { … } - create a menu entry with the given name
  • set gfxmode=1024×768,auto - attempt to run in 1024×768 graphics mode. if the video driver doesn't support it then fallback to auto
  • set root=(pxe) - use our PXE server as our root for GRUB2 to download files from
  • linux /boot/x86_gentoo_install.kernel … - defines the kernel to load as well as options to pass to it
    • root=/dev/ram0 - use a ramdisk as our rootfs
    • init=/linuxrc - run /linuxrc as our initialization process
    • looptype=squashfs - our real root is stored on a squashfs loop image
    • loop=/image.squashfs - location of our squashed loop in the livecd root
    • cdroot - treat our initial root as a cdroot, so it will remount the loop image
    • real_root=/dev/nfs - tell the kernel to find our real livecd root on NFS
    • nfsroot=${pxe_default_server}:/pxe/x86_gentoo_install - tell the kernel where to mount our NFS root from (${pxe_default_server} will be auto-populated by GRUB2 with the IP of our TFTP server, which is also our NFS server. convenient, no?)
    • docache - cache the entire livecd root into tmpfs (RAM), so that after boot we can unmount our NFS share

Once this is all set up, you should be able to take an x86 computer, plug it into your router, and fire up a PXE boot session to boot into the Gentoo Linux Installation CD.

This opens up some awesome possibilities since GRUB2 is PXE-aware, and can pull configuration files and images from the (pxe) root. I am currently playing with different possibilities.

One awesome aspect of this is for XEN Hypervisor. With this setup you can run a Domain0 XEN system that then boots all subdomains via the PXE server.

Also, remember that I set my NFS share to be read-only, but you don't have to. It is entirely possible to boot a diskless system by specifying its root partition as an NFS share.

• The Gentoo Linux alternative installation method HOWTO - Useful for some of the Gentoo-specific setup components
• Gentoo Linux based Netboot HOWTO - Information about TFTP setup
• PXE - DD-WRT Wiki - Information on using DD-WRT to handle your DHCP boot parameters
• Diskless booting - OSDev Wiki - Using GRUB2 to boot from PXE
• GNU GRUB Manual 1.99 - Setting up network boot for GRUB2.