Using a script to automate a Raspbian installation

roverto, 123RF.com

AutoPi

It does not take long to install and configure Raspbian on an SD card. However, for users who need to repeat the same steps time and again on multiple Pis, there is an easier, less boring way to get the job done. This comes in the form of simple shell scripts which can automate installation.

The installation process for Raspbian is simple. It consists of downloading an image, copying the image to an SD card, booting, and executing the initial configuration. Next you update all of packages via the sudo apt-get update and sudo apt-get upgrade commands. Additional maintenance of the system, including a complete switch over to a newer operating system, can be accomplished by means of package administration.

In spite of the ease with which you can complete installation, the enthusiastic Pi hacker will be able to identify a significant drawback: The initial configuration which includes selecting a language, expanding storage to include all of the SD card, and possibly also changing settings to accomodate over clocking all need to be performed manually. Those who maintain a single system will not have any issues with this. However, going through the steps laid out in the Pi config menu over and over again can get annoying. Since the system creates new computer certificates on the initial boot, remote access from other devices can be problematic. Each user has to first delete all traces of the old device from the ~/.ssh/known_hosts file before access is granted via the same host name.

Unfortunately, there are a number of concrete reasons for having to start over each time. Various hardware devices have different software requirements. You may also wish to start a new project with a fresh install for the sake of simplicity. Although system maintenance is made simple through package administration, new applications are not automatically added to the Pi.

Figure 1: An automated Pi installation using Apiinst.

Your own image

The solution to the problem of repeated installation is to use your own Raspbian image. This approach involves a quasi-reversal of the standard installation process. Once Raspbian has been installed and configured how you wish, remove the memory card and insert it into a card reader on your computer. Windows users can use Win32 Disk Imager to write the Raspbian image . The program can create an image from astorage medium. Linux and Mac users cansimply reverse the dd command (Listing 1). This is done by replacing /dev/sdX with the corresponding device name of the card reader. One way to find the correct device ID is via lsblk.

Listing 1

sdcard.img

$ sudo dd if=/dev/sdX of=sdcard.img bs=1M

This approach makes it easy for you to quickly create an image and write it to the SD card. However,if the image is not immediately copied to the card after the first configuration, it will not be a blank slate. Additional work is required to prevent issues such as the assignment of one certificate to all of the clones. An image from one card also may not fit on another card of the same nominal size. This is because actual card capacity can vary from one manufacturer to another.

The write process will be interrupted if the image is too large for the storage medium. If the file system in the image is not too full, the write program will often still be able to manage to copy over the image. However, you should correct the size of the partition and the file system afterwards. This is likely to require some expertise and persistence.

It makes sense to automate the manual steps for a standard installation. Linux offers a powerful script language for this purpose via the shell. As a rule however, any script language should work. You will not have to write your own program.. The complete apiinst script can be found in the author's Git repository [2]. See the Download and Installation box for more information.

Download and Installation

The scripts described in this article can be cloned from the author's Git repository [2]. Alternatively, you can retrieve the scripts in ZIP format both from the repository. The package includes scripts for the offline phase (bin/apiinst/), a script for collecting modified files (bin/colfiles), and the example file for the template directory (files/usr/local/sbin/apiinst2) which is found under files/. The latter will need to be adapted according to the user's needs.

Preparing an image

The underlying purpose of automatic installation is to provide maximum flexibility. The installation is divided into two phases (Figure 1). The script copies the image to the SD card and also modifies the hardware settings. This is the offline phase. Other configurations are carried out on the target Pi only during the online phase.

The script takes care of the partitioning and copies the image to the SD card during the offline phase. Next, the script moves the files from the template directory into the Pi system. Afterwards, you should take the card out of the reader, insert it into the Pi and start the computer. Immediately following the first boot, a second script starts work on the remainder of the tasks, such as installation of any packages that are still missing.

You will need to do some prepatory work to make sure the install occurs automatically.. For the most part, this will involve a standard installation and configuration of the Pi, as well as collating any files that have been modified in the initial configuration for the template directory. Instead of using dd, standard installation is done with the command from Listing 2.

Listing 2

wheezy.img

$ sudo apiinst -i 2015-05-05-wheezy.img -t /dev/sdb -P

Modify the installation path , /dev/sdb, to the drive of your choice. In contrast to dd, the script will check to see whether the partition of the destination device has already been mounted. Even so, caution is advised, as mistakes can lead to data loss. Although Apiinst uses dd internally, it can also handle zip archives of Raspbian images. In addition, the application updates the system time stored on the image. This simplifies the process of searching for modified files. See Pi Clock info box.

Pi Clock

The Pi has no internal real time clock (RTC). Instead Raspbian retrieves current time via the network time protocol (NTP) over the Internet. In the absence of an Internet connection, the he system time starts from zero. This creates practical problems since zero time in Unix is January 1, 1970. Raspbian gets around the zero time issue by writing the current shutdown time to the file /etc/fake-hwclock.data. Time then stands still for the Pi until the next boot, at which point the system time will start from that of the most recent shut down.

Before booting the Pi with the images you created, you should copy the program colfiles from the bin directory onto a USB stick. The bin directory is found on the Github project listed above. The program stores only files modified during the initial configuration so requires very little space.

Boot the Raspberry Pi with the SD card you prepared, but without connecting it to the Internet. This prevents the Pi from retrieving the current time via a server, which would make searching for the files modified in the initial configuration much more difficult. Once booted, configure the system as usual with Raspberry Pi Configuration. Do not expand the file system at this stage.

Now the USB stick that you prepared earlier comes into play. Insert it into the Pi, open a terminal and enter the commands from Listing 3. Occasionally, the sticks will contain partitions. If so, then replace /dev/sda with something like /dev/sda1. Once finished, there will be a tarball named apiinst_files.tgz on the stick.

Listing 3

mount

$ sudo mount /dev/sda /mnt
$ /mnt/colfiles
$ sudo umount /mnt

The colfiles script consists of two commands. The find command looks for all files that have been changed over the last hour and gives the list to a tar command which copies them to an archive. This would of course work without a script. However, a number of files and directories need to be excluded from the search. Therefore, the corresponding find command is not exactly trivial and should be encapsulated in an auxiliary program, as shown in Listing 4.

Listing 4

find command

find / /boot -xdev \
  \( -path "/var" -o -path "/home" -o -path "/tmp" \) \
  -prune -o \( -type f -o -type l \) \
  ! -path "/etc/mtab" \
  ! -path "/etc/.fstab" \
  ! -path "/etc/dhcpcd.duid" \
  -mtime -1 -print | \
  tar -cvzpf "/mnt/apiinst_files.tgz" -T -

Decompress the archive on a computer using either graphical compression tools or via a command in the template directory (Listing 5). This makes the minimum number of files available for subsequent automatic installation. You can add other files as needed such as those from /etc/network/interfaces) or your own configuration files from etc/ or the home directory. You should remember to copy the files with administrative rights, such as root or via the sudo command. This will allow you to retain the correct file permissions.

Listing 5

apiinst_files.tgz

$ sudo tar -xvzpf /mnt/apiinst_files.tgz -C /path/to/folder

Your reward comes in the form of speedy installation for additional Pis. The command from Listing 6 will copy the image from a card inserted into a card reader (/dev/sdb in the example) to the memory card of the Pi. The script will expand the root file system and copy the contents of the template directory to the card.

Listing 6

Copying the Image

$ sudo apiinst -i 2015-05-05-wheezy.zip -t /dev/sdb /path/to/template_folder

If you still need a home partition, then you can use the -H value option. Wnter a fixed size in MB or GB or simply enter rest, in which case the script will not expand the root partition. Instead, the remaining free space will be added to the partition. It is a good idea to select an oversized value as apiinst does not currently support shrinking the root partition.

Buy this article as PDF

Express-Checkout as PDF

Pages: 4

Price $2.95
(incl. VAT)

Buy Raspberry Pi Geek

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content