Red Green Repeat Adventures of a Spec Driven Junkie

Customizing Virtualboxes

Developing on virtualboxes instead of on the (host) machine directly is my preference because it is easy to:

  • make an environment similar to production servers
  • take or restore a snapshot of the virtualbox
  • sandbox my work area from other apps
  • ramp up system resources as needed

On the other hand, setting up virtualboxes for development, is really frustrating. There are small things to build and configure such as:

  • downloading system installation images
  • setting up host and guests: IP addresses, ports, & ssh keys
  • installing system packages to install
  • installing programming tools
  • installing & configuring support applications (i.e. webservers, databases, etc.)

Just getting one box setup is a lot of work. Most of the time, once I customize a box, I never update until it’s too late, until now.

Enter: Vagrant

This is a tool that works with virtual boxes, like VirtualBox. Vagrant manages the downloading of system and setting up of the guest machine on the host programmically.

After installing the program vagrant on your computer, which is the host computer, to download Ubuntu trusty 64bit, install it, and setup IP addresses, ports, SSH keys, etc. a simple Vagrantfile containing:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty64"
end

and running commands on the command line:

(host)$ vagrant up
...
    default: Guest Additions Version: 4.3.36
    default: VirtualBox Version: 5.1
==> default: Mounting shared folders...
    default: /vagrant => /Users/aleung/Developer/temp_vagrant

Voila, a new Linux system to work on.

Additional Configurations?

What about customizing and configuring the system further? The current system is a plain linux install. Emacs is not even installed!

(host)$ vagrant ssh
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-125-generic x86_64)
...
(guest):~$ emacs
The program 'emacs' can be found in the following packages:
 * emacs24
 * emacs24-nox
 * e3
 * emacs23
 * emacs23-lucid
 * emacs23-nox
 * emacs24-lucid
 * jove
Ask your administrator to install one of them

Blasphemous!

Enter: Provisioning

This is part of vagrant that allows one to customize a system after the initial installation step.

Vagrant has different ways to configure the system after installation called: provisioning. One way is to specify a shell script, which is a script vagrant will run as the root user post-installation.

Example: emacs

Let’s go through an example of installing the newest version of emacs using vagrant’s shell provisioning system.

In order to install the latest version of emacs on a fresh system, these are the steps I have found to be the shortest way:

Action Command
install python software properties sudo apt-get install -y python-software-properties
add emacs repository sudo apt-add-repository -y ppa:ubuntu-elisp
update repository listings sudo apt-get update
install emacs sudo apt-get install -y emacs-snapshot

Instead of searching for or looking up these instructions each time, I can have vagrant’s shell provisioning do this after creating a box:

$script = <<SCRIPT
apt-get install -y python-software-properties
apt-add-repository -y ppa:ubuntu-elisp
apt-get update
apt-get install -y emacs-snapshot
echo I am provisioning the newest version of emacs...
date > /etc/vagrant_provisioned_at
SCRIPT

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.provision "shell", inline: $script
end

note: the -y option is important as this option will tell apt-get to respond yes to any questions (i.e. Do you want to install?)

Recreate with Provisioning

Let’s destroy the previous virtualbox vagrant created and create a new one using the above Vagrantfile:

(host)$ vagrant destroy
(host)$ vagrant up

Vagrant will use the previously downloaded Linux install image and once done installing, execute $script.


==> default: Setting up emacs-snapshot (3:20161025-1ea669d~ubuntu14.04.1) ...
...
==> default: update-alternatives:
==> default: using /usr/bin/etags-snapshot to provide /usr/bin/etags (etags) in auto mode
==> default: Processing triggers for libc-bin (2.19-0ubuntu6.13) ...
==> default: Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.30.7-0ubuntu1.6) ...
==> default: I am provisioning the newest version of emacs...

Verify Configuration

Let’s double check if the provisioning worked by logging in and check the version of emacs installed:

(host)$ vagrant ssh
...
(guest):~$ emacs --version
GNU Emacs 26.0.50.2
Copyright (C) 2016 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

Perfect! The provisioning installed the newest version of emacs and is ready to go.

Provisioning File

Now that the shell script in Vagrantfile worked, let’s put it in a separate file that accessible by the Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.provision "shell", path: "emacs.sh"
end

and the emacs.sh file, with some comments, now in the same folder as the above Vagrantfile:

#!/usr/bin/env bash

# apt-add-repository depends on this
apt-get install -y python-software-properties

# setup emacs
# https://emacs.stackexchange.com/a/12774
apt-add-repository -y ppa:ubuntu-elisp
apt-get update
apt-get install -y emacs-snapshot

The path option in the Vagrantfile, allows vagrant to refer to the file.

Customizing More!

Of course, once provisioning is setup, it is easy to provision a more extensive list of tools, like python, pip, node, npm, tmux, etc.

Here’s a sample of my current provisioning file:

#!/usr/bin/env bash

# apt-add-repository depends on this
apt-get install -y python-software-properties

# setup emacs
# https://emacs.stackexchange.com/a/12774
apt-add-repository -y ppa:ubuntu-elisp
apt-get update
apt-get install -y emacs-snapshot

# install git
apt-get install -y git

# get common python build tools:
# https://github.com/pyenv/pyenv/wiki/Common-build-problems
apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev

# install 6.x version of node for the system (instead of v0.6.x)
# https://nodesource.com/blog/installing-node-js-tutorial-ubuntu/
curl -sL https://deb.nodesource.com/setup_6.x | bash -
apt-get update
apt-get install -y nodejs

# update npm with newer version of node
npm install npm --global

# install tmux
apt-get install -y tmux

# install vagrant user to sudoers list
echo 'vagrant ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers

# install python-pip and virtualenv
apt-get install -y python-pip
apt-get install -y python-virtualenv

# install silversearcher (blazing grep even on a single core CPU!)
apt-get install -y silversearcher-ag

Conclusion

What started out as a way to consistently install the newest emacs on a fresh linux install turned into a understanding vagrant’s shell provisioning system.

Thanks to shell provisioning, I now have a way to:

  • consistently build and fully configure virtualboxes
  • easily make virtualboxes I like
  • sandbox different environments from each other

It is so easy to create new boxes all the time. Want to make sure the install instructions on the README.md file is up to date? Try it out on a fresh install. Regretted installing a package globally? Roll all the way back and do it right. Why fix an install when one can rebuild?