Describe the steps you need to build and test a 1.8 Lustre system (MGS, MDT, MDS, OSS, OST, client) on a CentOS 5.5 or 5.6 machine. The only difference between building on 5.5 and 5.6 is that a different kernel SRPM is required. Minor differences may appear in the output of some commands below.
This walk-thou has reportedly been used successfully on CentOS 6.0. One significant difference is that CentOS 6.0 replaces
/etc/modprobe.conf with a directory:
/etc/modprobe.d/. For CentOS 6.0 clients, the advice is to create a file
/etc/modprobe.d/lustre.conf to contain Lustre specific module configurations.
- A newly installed CentOS 5.5 x86_64 machine with the hostname: client-10.
- EPEL Repository: this is a convenient source for git.
Lustre 1.8 servers require a patched and compiled kernel. Patches are readily available in the Whamcloud git source repository. A test suite is included with the Lustre 1.8 source. This document walks through the steps of patching the kernel, building Lustre and running a basic test of the complete system.
The procedure requires that a OS is setup for development - this includes Lustre sources, kernel source and build tools. Once setup, a new kernel can be patched, compiled, run and tested. Further reading on building a CentOS RPM based kernel is available on the CentOS site.
Once CentOS 5.5 is newly installed on client-10 login as root.
- Install required kernel development tools.
- Create a user
buildwith the home directory
- Switch to the user
buildand change to the build
- Get the 1.8 branch from the Whamcloud git account.
- Resolve any outstanding dependencies until
autogen.shcompletes successfully. Success will look like:
Prepare the kernel source
In this walk-thru, the kernel is built using rpmbuild - a tool specific to RPM based distributions.
- Get the kernel source. First create the directory structure, then get the source from the RPM. Create a
.rpmmacrosfile to install the kernel source in our user dir.
- Install the kernel source:
RHEL periodically releases updates to the kernel. The Lustre Master branch tracks the most recent kernel from Red Hat. In the event that the link above is not completely up-to-date, you should visit the Red Hat source RPM download site or CentOS source RPM download site and manually ensure you are downloading the most recent kernel.
NOTE If you are performing this walk-thru on CentOS 5.6, the kernel source can be found here:
- Expand the source. Using rpmbuild will also apply CentOS patches.
This will end with:
At this point, we now have kernel souce, with all the CentOS patches applied, residing in the directory
Patch the kernel source with the Lustre code.
- Add a unique build id so we can be certain our kernel is booted. Edit
~build/kernel/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.x86_64/Makefileand modify line 4, the
- enter the directory
- overwrite the
- link the Lustre series and patches
- Apply the patches to the kernel source using quilt
Build the new kernel as an RPM.
- Go into the kernel source directory and issue the following commands to build a kernel rpm.
- A successful build will return:
NOTE If you receive a request to generate more entropy, you need to trigger some disk I/O or keyboard I/O. I would recommend (in another terminal):
At this point, you should have a fresh kernel RPM
Configure and build Lustre
- Configure Lustre source
- make rpms:
- You should now have build the following, similarly named, rpms:
Installing the Lustre kernel and rebooting.
- As root, Install the kernel
- Create a initrd
- Check that
/boot/grub/menu.lstis configured to boot the new kernel. Add the following lines to
/boot/grub/menu.lsthas the Lustre kernel is selected: This is 1:
- view the login prompt with satisfaction:
- Change to
rootand Change directory into
- Install modules
lustre-modulesand user space tools
e2fsprogs is needed to run the test suite.
- Download e2fsprogs from http://downloads.whamcloud.com/
- Install with
rpm -ivh e2fsprogs
A quick test
- you will now have a Lustre filesystem available at
- NOTE: if you receive an error:
mkfs.lustre: Can't parse NID 'client-10@tcp'you'll need to associate the ip address of a non-loopback interface with name of your machine into the /etc/hosts file.