Build it and you will learn: Building an RPi2 cluster

The biggest and fastest computers in the world are the HPC systems used by science, government, and industry to perform complex calculations for data mining, 3D modeling, and other scientific problems. Large HPC systems sometimes use thousands of processors.

Behind the scenes, every HPC system is actually a network of computers working together in what is called a cluster. A typical high-performance cluster consists of a master node and a collection of compute nodes (see Figure 1). The master node subdivides the task (typically a complex mathematical equation) into a collection of smaller tasks that are assigned to the compute nodes. The compute nodes crank through the calculations, sometimes displaying the output in realtime and sometimes storing the results to assemble later into a final solution.

Figure 1: Hardware configuration.

An HPC system thus consists of several smaller pieces. How small? I decided to try my luck with building an HPC system using the Raspberry Pi 2. Of course, my HPC system won't set any speed records, but I can still use it for testing and demonstrations.

A small, energy-efficient cluster fits well with my travel plans – I can take it in my carry-on baggage. Many commercial aircraft also have power outlets, so it would be very cool indeed to fire up a small cluster from my seat and start running HPC applications at 36,000 feet.

Even if you don't plan on testing parallel applications and flying to hi-tech conferences with an HPC cluster on your tray table, building your own Raspberry Pi 2 cluster will teach you a little about the tools and techniques used for high performance computing.

Why the Pi 2?

The Raspberry Pi 2 (RPi2) is inexpensive (~$35), has four cores (representative of current modern hardware), has reasonable performance for the price, and uses very little power. The classic Raspberry Pi B Rev. 2 is a little light on resources, even for a test HPC system, but the Pi 2, with its quad-core processor and 1GB of memory, has just enough to make our project worth the effort. The Raspberry Foundation claims the Pi 2 is six times faster than the old model.

Large-scale HPC projects use tools such as Warewulf [1] to deploy an operating system on each of the compute nodes. For this mini-project, I decided to install the OS directly on the microSD card for each system and then install the necessary cluster tools on an NFS-exported volume from the master node to the other nodes.

Hardware Requirements/Configuration

The cluster configuration is pretty classic: a master node, some compute nodes, and a private cluster network. Figure 1 shows the configuration I chose for this project.

This particular project comprises four compute nodes, but the ellipsis in the diagram indicates that you can make the system much larger if you want. Attached to the master node is a USB drive, which will be NFS-exported to the compute nodes so all nodes will have access to the storage. A separate attached device is not strictly necessary; you can also just NFS-export the /home directory from the master node to the compute nodes.

The compute nodes are put on a private network to keep them separate from the outside world, which is how almost all clusters are constructed. A Gigabit Ethernet switch is used for the computational network. Although the RPi2 isn't capable of GigE speeds, the price of a GigE switch is very low. You could also use a Fast Ethernet switch.

You'll also need to attach a keyboard, mouse, and monitor to the master node.