An experiment in creating clusters and distributing loads with Raspberry Pis

If your project has exhausted the capabilities of a Raspberry Pi, it can be worth your while to connect several of these devices together to make a cluster. Doing so will create new resources and give you an opportunity to learn about techniques for building a computer cluster with a load balancer.

Typically, a cluster is a group of computers that act to guard against an outage or improve the performance of a service. The cluster in this example is made from four Rasp Pis. Three of the diminutive machines run locally with the GlusterFS [1] distributed filesystem (Figure 1), thus combining the read bandwidth of the three devices. The next step involves adding geo-replication with the fourth Raspberry Pi, which the other Rasp Pis can only reach over the Internet.

Figure 1: A handful of Rasp Pis is all it takes to learn how a cluster and load balancer work.

Each of the individual nodes receives a web server. Finally, a separate web server assumes the task of balancing the load to make the cluster resources available on the network.

Preparation

A new setup for a Rasp Pi should not include expansion of the filesystem. In the case at hand, you need the free space to install a partition for GlusterFS. Using raspi-config as root, choose Advanced Options | Hostname. You should give the devices names like cluster01 to cluster04. Using this setup, cluster01 to cluster03 will function as local servers and cluster04 as the remote computer.

Each node in the cluster must be running in the same time zone, so go to Internationalisation Options to set the time zone, remembering also to change passwords when you do in the Change User Password section. After specifying these settings, you should restart the computers so that the changes will take effect on each host.

Setting Up a Network

Now it is time to create a stable network environment. One possible approach is to assign permanent IP addresses and then resolve the names via /etc/hosts. A number of different router models let the user assign a static IP address to a particular MAC address. Additionally, some models let the user resolve names by means of DNS.

The test model did not have a DNS function. As a result, the only way to proceed was to assign a static IP address to each MAC address and then resolve the name using the local file /etc/hosts. This is the only workable method to use with computers that can only be accessed via the Internet. Listing 1 shows an example for a corresponding file that you will need to distribute to all of the computers that participate in the cluster.

127.0.0.1     localhost
127.0.1.1     lin01
192.168.2.5   cluster01
192.168.2.7   cluster02
192.168.2.6   cluster03
<External IP Address>    cluster04

The client computers that will later use the clustered filesystem need to have the appropriate entries. The cluster04 computer is an exception; it cannot reach the cluster01 to cluster03 machines via the Internet.

Now it should be possible to create a connection to all of the Rasp Pis via SSH. If this doesn't work, the first thing you should do is double-check the network configuration.