Using Packer & Vagrant to bootstrap a testing infrastructure (Part 2)

In the previous blog post of this series we've learned a few reasons to use Packer & Vagrant. In this post we'll see how to create Packer images/ boxes that can be used by Vagrant to setup new VMs.

The first step is to install Packer itself. Download the package and make it available to your shell (add to PATH variable). Read the docs for details.

To get Packer running you need to prepare a Packer template. The components of the template are described in the Packer documentation.

Let's take a look at a real-world example from my Packer repository on Github and go through the lines:

  • L2: variables: A section containing a list of user-defined variables that are used in the builders, provisioners and post-processors. These variables can be overridden on the command-line.
  • L3: iso_url: HTTP URI to Installer image that will be dowloaded on demand.
  • L8: boxes_directory: directory where the final box image will be saved.
  • L12: headless: Show the VM GUI console if available (e.g. on VirtualBox).
  • L14: http_directory: path to the directory that will be made available with an internal webserver (might be available on VirtualBox only).
  • L15: custom_provisioners: A user-defined list of scripts (shell provisioners) that will be executed after OS installation.
  • L16: custom_provisioners_context: Context variables for the shell provisioners.
  • L19: builders: A section containing a list of settings that are used to build the VM and install the OS.
  • L23: http_directory: A setting using the user-defined variable called http_directory as value: {{ user `<name-of-variable>` }}.
  • L24: boot_command: A sequence of commands (<esc>, sleep, tab) and command-line commands that are entered after booting the ISO installer image.
  • L64: provisioners: A section containing a list of provisioners that will be executed after OS installation to provision the naked OS.
  • L66-68: Provisioner config that doesn't more than simply executing the command mkdir -p /tmp/packer-provision.
  • L72-74: Provisioner config of type file which lets Packer upload a local file/ directory into the VM.
  • L82: Again a provisioner of type shell but executing a shell script instead of an arbitrary command (like mkdir).
  • L86: A list of shell environment variables that will be made available to the shell that executes the script listed in L84.
  • L95: post-processors: A section containing a list of actions that should be taken after provisioning the VM.
  • L97-100: A post-processor of type vagrant converts the generated image into a Vagrant compatible format called Vagrant box. Read the Vagrant documentation to learn more about the box format.

We've learned what the template consists of and are now going to clone a Git repository containing the template listed above and create our first Packer image:

$ cd ~/Downloads/
$ git clone --recursive
$ tree -I '.git|.vagrant|.*\.swp|packer_cache' --matchdirs -L 2 vagrant-devenv
|-- .gitignore
|-- .gitmodules
|-- README.rst
|-- Vagrantfile.template
|-- assets/
|   |-- .gitignore
|   |-- AUTHORS
|   |-- CHANGELOG.rst
|   |-- LICENSE
|   |-- README.rst
|   |-- scripts/
|   |-- snippets/
|   |-- tools/
|   `-- update-docs*
|-- build-boxes*
|-- configs/
|   |-- centos-6/
|   |-- centos-7/
|   |-- debian-7/
|   `-- debian-8/
|-- shared/
|   `-- boxes/
|-- templates/
|   |-- centos-6/
|   |-- centos-7/
|   |-- debian-7/
|   `-- debian-8/
`-- update-docs*

Take a look into templates/debian-7/, you will find our template.json file that is shown above. The assets/scripts/ directory contains scripts (assets/scripts/bootstrap) to prepare the OS with some basic packages, package updates and executes our custom provision (shell) scripts (assets/scripts/provision).

In configs/debian-7/ you will find a file called minimal.json:

This file contains nothing more than values for our user-defined variables (template.json: L11/ L99, L13/ L87, L12/ L31, L3/ L24, L4/ L25). To go one inception level further we can override the values within minimal.json by setting command-line arguments when executing Packer, which is what we are going to do now:

$ cd vagrant-devenv
$ packer build \
  -var release=v1.0 \
  -var headless=false \
  -only=virtualbox \
  -var-file=configs/debian-7/minimal.json \

We override the user-defined variables release and headless, specify that we use VirtualBox to create our Vagrant box, specify a file path containing user variables (minimal.json) and specify the packer template (template.json) as the last argument.

If everything works well Packer downloads the ISO file, creates the VM, boots the VM, enters the boot_command, waits for completion of OS installation, logs into the VM, executes the provisioners and uses the post-processor to create the Vagrant box. See the screencast linked in the README file for demonstration.

$ tree shared/boxes
|-- .gitignore
$ file shared/boxes/
shared/boxes/ gzip compressed data

Great! We now have a Vagrant box that can be used by Vagrant to create new VMs. See the next blog post that will handle the common usage of Vagrant.