Describe the steps you need to build and test a 1.8 Lustre system (MGS, MDT, MDS, OSS, OST, client) on a CentOS 5.5 or 5.6 machine. The only difference between building on 5.5 and 5.6 is that a different kernel SRPM is required. Minor differences may appear in the output of some commands below.
This walk-thou has reportedly been used successfully on CentOS 6.0. One significant difference is that CentOS 6.0 replaces
/etc/modprobe.conf with a directory:
/etc/modprobe.d/. For CentOS 6.0 clients, the advice is to create a file
/etc/modprobe.d/lustre.conf to contain Lustre specific module configurations.
- A newly installed CentOS 5.5 x86_64 machine with the hostname: client-10.
- EPEL Repository: this is a convenient source for git.
Lustre 1.8 servers require a patched and compiled kernel. Patches are readily available in the Whamcloud git source repository. A test suite is included with the Lustre 1.8 source. This document walks through the steps of patching the kernel, building Lustre and running a basic test of the complete system.
The procedure requires that a OS is setup for development - this includes Lustre sources, kernel source and build tools. Once setup, a new kernel can be patched, compiled, run and tested. Further reading on building a CentOS RPM based kernel is available on the CentOS site.
Once CentOS 5.5 is newly installed on client-10 login as root.
- Install required kernel development tools.
- Create a user
buildwith the home directory
- Switch to the user
buildand change to the build
- Get the 1.8 branch from the Whamcloud git account.
- Resolve any outstanding dependencies until
autogen.shcompletes successfully. Success will look like:
Prepare the kernel source
In this walk-thru, the kernel is built using rpmbuild - a tool specific to RPM based distributions.
- Get the kernel source. First create the directory structure, then get the source from the RPM. Create a
.rpmmacrosfile to install the kernel source in our user dir.
- Install the kernel source:
NOTE If you are performing this walk-thru on CentOS 5.6, the kernel source can be found here:
- Expand the source. Using rpmbuild will also apply CentOS patches.
This will end with:
At this point, we now have kernel souce, with all the CentOS patches applied, residing in the directory
Patch the kernel source with the Lustre code.
- Add a unique build id so we can be certain our kernel is booted. Edit
~build/kernel/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.x86_64/Makefileand modify line 4, the
- enter the directory
- overwrite the
- link the Lustre series and patches
- Apply the patches to the kernel source using quilt
Build the new kernel as an RPM.
- Go into the kernel source directory and issue the following commands to build a kernel rpm.
- A successful build will return:
NOTE If you receive a request to generate more entropy, you need to trigger some disk I/O or keyboard I/O. I would recommend (in another terminal):
At this point, you should have a fresh kernel RPM
Configure and build Lustre
- Configure Lustre source
- make rpms:
- You should now have build the following, similarly named, rpms:
Installing the Lustre kernel and rebooting.
- As root, Install the kernel
- Create a initrd
- Check that
/boot/grub/menu.lstis configured to boot the new kernel. Add the following lines to
/boot/grub/menu.lsthas the Lustre kernel is selected: This is 1:
- view the login prompt with satisfaction:
- Change to
rootand Change directory into
- Install modules
lustre-modulesand user space tools
e2fsprogs is needed to run the test suite.
- Download e2fsprogs from http://downloads.whamcloud.com/
- Install with
rpm -ivh e2fsprogs
A quick test
- you will now have a Lustre filesystem available at
- NOTE: if you receive an error:
mkfs.lustre: Can't parse NID 'client-10@tcp'you'll need to associate the ip address of a non-loopback interface with name of your machine into the /etc/hosts file.