An experiment in creating clusters and distributing loads with Raspberry Pis

Setting Up Apache

Before installing the web server, it is a good idea to make some room on the root filesystem. The following commands remove software you don't need for this project, such as dependencies that are no longer needed and temporary files:

$ sudo apt-get remove x11-common midori lxde python3 python3-minimal
$ sudo apt-get autoremove
$ sudo apt-get clean

You should repeat these three steps on all four computers. Following this approach in this test, I was able to create 1.2GB of free space on each system. Figure 2 shows how the setup looks when complete.

Figure 2: The cluster configuration.

Next, you can mount the volpi volume on each node of the cluster and enter the command

sudo mkdir /mnt/www

to specify a mountpoint for the volume. You can then use the following command to mount the exported filesystem on each local filesystem, repeating this step in an identical manner for each of cluster02, cluster03, and cluster04:

$ sudo mount -t glusterfs cluster01:/volpi /mnt/www/

Listing 11 shows the /etc/fstab entries corresponding to those in the /etc/hosts file (Listing 1) that mounts the remote filesystem permanently.

Listing 11

/etc/fstab

cluster01:/volpi /mnt/www glusterfs defaults,_netdev 0 0
cluster02:/volpi /mnt/www glusterfs defaults,_netdev 0 0
cluster03:/volpi /mnt/www glusterfs defaults,_netdev 0 0
cluster04:/volpi /mnt/www glusterfs defaults,_netdev 0 0

Now, you can install the web server with the following command:

apt-get install apache2

in a terminal window on all four computers. Next, take a look at the DocumentRoot variable on all of the nodes. Listing 12 shows what you need to change (e.g., line 3 and 8-13).

Listing 12

DocumentRoot Changes

01 <VirtualHost *:80>
02   ServerAdmin webmaster@localhost
03   DocumentRoot /mnt/www
04   <Directory />
05     Options FollowSymLinks
06     AllowOverride None
07   </Directory>
08   <Directory /mnt/www/>
09     Options Indexes FollowSymLinks MultiViews
10     AllowOverride None
11     Order allow,deny
12     allow from all
13   </Directory>
14 [...]
15 </VirtualHost>

Now you should re-start the web server with

/etc/init.d/apache2 restart

so that it accepts the changes. To bring the distributed web servers to the Internet via one URL, you need a little trick. First, set up access to the individual clusters through the firewall. Figure 3 shows how the corresponding configuration looks on the router used in this test.

Figure 3: Forwarding the corresponding ports.

A low-cost root server, which can cost less than $5 a month, makes an Apache proxy available on the web that can connect to the four nodes in the cluster. You will need to adapt the /etc/hosts file in such a way that you are able to use the names that have already been assigned. Then, install Apache and load the corresponding modules:

$ sudo apt-get install apache2
$ sudo a2enmod proxy_balancer
$ sudo a2enmod proxy
$ sudo a2enmod proxy_http

Next, you should create a virtual host that contains the proxy configuration from Listing 13. To distribute the load, you won't need an entry for DocumentRoot.

Listing 13

Proxy Configuration

<VirtualHost *:80>
  ServerName <host.de>
  ServerAlias <www.host.de>
  <Proxy balancer://mycluster>
    BalancerMember http://cluster01:8081
    BalancerMember http://cluster02:8082
    BalancerMember http://cluster03:8083
    BalancerMember http://cluster04:8480
  </Proxy>
  ProxyPass /test balancer://mycluster
  ErrorLog /var/log/apache2/<host.de>.error.log
  LogLevel warn
  CustomLog /var/log/apache2/<host.de>.access.log combined
</VirtualHost>

In the BalancerMember variable, you should be able to enter DynDNS entries. In the configuration that now exists, the server will distribute the load in a round-robin fashion.

If one of the nodes fails, then the web server will simply retrieve the page from the next computer. The proxy module of the Apache server can accomplish a lot more. On the homepage, you will find all the information you need to build more complex environments [5].

Conclusion

Putting a handful of Raspberry Pis together lets you build a functioning cluster that can be distributed over the Internet. Naturally, you shouldn't expect fantastic performance, especially if you are considering some serious applications with lots of users, because Raspberry Pis are not blessed with a lot of resources. However, using the small computers to create a functioning cluster can be a very rewarding learning experience.

Buy this article as PDF

Express-Checkout as PDF

Pages: 6

Price $2.95
(incl. VAT)

Buy Raspberry Pi Geek

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content