Build it and you will learn: Building an RPi2 cluster


Probably the most fundamental tool used in cluster administration is a parallel shell. With pdsh [6], you can perform a wide range of administrative tasks on a cluster. A parallel shell tool lets you enter a command once (in this case, on the master node) and the command is then executed on all (or some) of the compute nodes. When you think of the size of many HPC systems, with thousands of compute nodes, you'll understand why entering a command once to execute on all the systems is much easier than undertaking the configuration of each node separately. For our tiny HPC system, the time savings is less pronounced; however, this system will hardly seem like a real HPC cluster without the capacity for parallel administration.

The options used to build and install pdsh on the master node from /work/pi/src/pdsh-<version> are:

$ ./configure --prefix=/work \
  -with-ssh -without-rsh
$ make
$ sudo make install

These options are discussed in detail online [7]. By default, pdsh uses the obsolete rsh shell. RSH has some pretty serious security issues, so it's better to use ssh. The command option -without-rsh disables RSH and the option -with-ssh enables SSH instead.

Pdsh was installed into /work, which means the binaries will be in /work/bin or /work/sbin. These directories need to be in the standard path regardless of what modules are loaded. Therefore, you need to add these directories to your $PATH variable. In either your .bashrc file (user pi) or the global file, /etc/bash.bashrc, you just add the following line to the end of the file:

export PATH="$PATH:/work/bin:/work/sbin:/work/include"

Pdsh is pretty easy to test from the home directory:

$ pdsh -w uname -r 3.18.11-v7+
$ pdsh -w raspberrypi uname -r
raspberrypi: 3.18.11-v7+

Notice that using the IP address or the hostname should produce the same output. Once you get pdsh set up on your system, you'll need to tweak it for your own environment to use it effectively as a cluster management tool. See the box titled "Configuring Pdsh."

ConFiguring Pdsh

To make pdsh easier to use, you have to do a little configuration work. For example, if you create an environment variable named WCOLL and point it to a file with the list of hosts you want to use with a pdsh command, they will be used by default (i.e., you don't have to specify the host names or IP addresses every time you enter a command). To accomplish this, first create a list of hosts you want to use by default. It is a good idea to put this list on a shared storage device for all nodes.

The WCOLL environment variable can be placed in the user's ./bashrc file or in the global bash configuration file. I elected to put the variable in the global bash configuration file. In addition to the WCOLL environment variable, you need to add one other:

$ more /etc/bash.bashrc
# pdsh:
export PATH=??
export PDSH_RCMD_TYPE=ssh
export WCOLL=/work/PDSH/hosts

PDSH_RCMD points to the type of protocol used for remote access with pdsh. The environment variable WCOLL points to the file containing the list of default hosts. Logging out and logging back in causes the system to read the global bashrc file.

If you just run pdsh by itself, it will use SSH because of the environment variable PDSH_RCMD. It will also use the list of default hosts in the file pointed to by WCOLL,

$ pdsh uname -r 3.18.11-v7+

as in the above example.


MPICH [8] is one of the key MPI libraries for parallel programming. Use the configure options and commands below to build MPICH (in /work/pi/src/mpich-<version>):

$ ./configure --prefix=/work/apps/gnu_4.6 \
  /mpich/3.1.4 --enable-fortran=all
$ make
$ sudo make install

MPICH is installed in /work/apps/gnu_4.6/mpich/3.1.4 (for the versions I used), which is not in the standard path. An Lmod module file will be written that allows MPICH library/tools to be "loaded" into the user environment. This method basically puts it into the environment variable $PATH so that these binaries are used first (i.e., mpif90 will be used before any other mpif90 binaries on the system). I will discuss this in a little more detail in the section on Lmod.

The installation path requires a bit more explanation. From McLay, the recommended best practice for Lmod is to install the compilers in /work/apps/gnu/4.6.3; then, the MPI libraries should be installed in /work/apps/gnu_4.6. The underscore indicates the everything installed "under" this directory is dependent on the gnu-4.6 compilers. Therefore, MPICH is installed in /work/apps/gnu_4.6/mpich/3.1.4. If version 3.2.0 of MPICH comes out, then it would be installed in /work/apps/gnu_4.6/mpich/3.2.0.

If you want to use a different compiler with mpich-3.1.4 (e.g., gcc-5.2.0), the compilers would be installed in /work/apps/gnu/5.2.0, and mpich-3.1.4 would be built and installed in /work/apps/gnu_5.2/mpich/3.1.4.

In the case of Raspbian on the RPi2, the GNU 4.6.3 compilers are installed by default in /usr/bin; therefore, they can't be installed in /work/apps/gnu/4.6.3. To maintain the naming structure, tools that depend on the GNU 4.6.3 compilers will be installed in /work/apps/gnu_4.6/.

Buy this article as PDF

Express-Checkout as PDF

Pages: 2

Price $2.95
(incl. VAT)

Buy Raspberry Pi Geek

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content