AWS Automation based on Vagrant — Part 3: Creating a Docker Host on AWS in 10 Minutes using Vagrant


Okay, I am cheating a little bit with respect of the time of 10 minutes mentioned in the title: I assume that this step by step guide has been accomplished already. This might take you an hour or so.

After that you are ready to run a Docker host on AWS within 10 minutes with only 2 lines of additional code. With a few more clicks in the Amazon web portal (the AWS EC2 console) you are ready to access the newly created Docker host. After downloading a Docker Python image you will print a Python-created “Hello World!” to the console of the Docker host.

The series is divided into three parts:

  • In Part 1, we will introduce Amazon Web Services (AWS) and will show how to sign into a free trial of Amazon, create, start, shut down and terminate a virtual machine on the AWS EC2 console.
  • Part 2 will lead you through the process how to use Vagrant to perform the same tasks you have performed in part 1, but now we will use local Vagrantfiles in order to automate the process.
  • Part 3 (this blog post) is the shortest part and will show, how Vagrant helps you to go beyond simple creation, startup, shutdown and termination of a virtual machine. In less than 10 minutes, you will be able to install a Docker host on AWS. With a few additional clicks on the AWS EC2 console, you are ready to start your first Docker container in the AWS cloud.

Document versions

v1 (2016-04-06): intial release of this document
v2 (2016-04-12): documented a provisioning error I have hit in the Caveats section at the end.

Prerequisites

  • Your Firewall allows you to access systems via Internet using SSH with no proxy in between. In most cases, this is possible from a home network or a hot spot, but in most cases, this is not permitted from within a corporate network using HTTP proxies.
  • You have followed this step by step guide in order to set up Vagrant as a AWS provider. After this, you will have…
    • … signed into AWS
    • … created an AWS user with the appropriate priviledges
    • … installed Vagrant and the Vagrant AWS Provider
    • … created a Vagrantfile with the appropriate information to connect to AWS
    • … tested the creation and termination of an Ubuntu image on AWS by using the local Vagrant command line interface

Step by Step Guide

Step 1: Adapt the Vagrant File

Add the two config.vm.provision lines to the the existing Vagrantfile created in the other step by step guide

# Vagrantfile
...
Vagrant.configure(2) do |config|
  ...
  config.vm.provision :shell, :inline => "sudo wget https://raw.githubusercontent.com/oveits/docker-enabled-vagrant/master/ubuntu-trusty/vagrant-provision.sh -O /tmp/vagrant-provision.sh", :privileged => true
  config.vm.provision :shell, :inline => "sudo bash /tmp/vagrant-provision.sh", :privileged => true
end

Step 2: Launch and Provision Instance

Back on the local command line, issue the command:

vagrant up --provision

to create and launch the new instance on AWS and install docker with many useful docker tools.
Or, if the image is already up and running, we do not want to create the instance, but only install Docker on the existing image by issuing the command:

vagrant provision

If you happen to hit a curl error here, please see the Caveats section at the end.

After that, you will be able to observe in the local console, that lots of software is downloaded (this is quite quick, when run in the cloud, since AWS has a good Internet connection. The log file will end with some error commands that can be savely ignored:

==> default: e67def44f1a2: Download complete
==> default: e67def44f1a2: Pull complete
==> default: e67def44f1a2: Pull complete
==> default: a3ed95caeb02: Pull complete
==> default: a3ed95caeb02: Pull complete
==> default: Digest: sha256:c46c830e33c04cadebcd09d4c89faf5a0f1ccb46b4d8cfc4d72900e401869c7a
==> default: Status: Downloaded newer image for weaveworks/plugin:1.4.6
==> default: docker: "rm" requires a minimum of 1 argument.
==> default: See 'docker rm --help'.
==> default:
==> default: Usage: docker rm [OPTIONS] CONTAINER [CONTAINER...]
==> default:
==> default:
==> default: Remove one or more containers
==> default: Failed to remove image (busybox): Error response from daemon: No such image: busybox:latest
[/f/veits/Vagrant/ubuntu-trusty64-docker-aws-test]

Step 3: Update the Security Policy

In the EC2 console, under Network&Security -> Security Groups (in my case in EU Central 1: https://eu-central-1.console.aws.amazon.com/ec2/v2/home?region=eu-central-1#SecurityGroups:sort=groupId), we can find the default security group. We need to edit the inbound rule to allow the current source IP address. For that, select the policy group, click on the “Inbound” tab on the bottom, specify “My IP” as source and save the policy:

2016.04.01-13_05_18-hc_001

Now we should be able to access the system.

Step 4: Access the System

Note: This step and the following steps will work only, if your firewall allows you to access systems in the Internet using SSH.

When you log in, you can issue your first docker commands. Note that you might need to update your security setting in order to allow for access from your IP address, like described in the other step by step guide, or see below the Appendix A.

$vagrant ssh
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-74-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Sat Apr  2 20:24:18 UTC 2016

  System load:  0.01              Processes:              111
  Usage of /:   18.9% of 7.74GB   Users logged in:        1
  Memory usage: 14%               IP address for eth0:    172.31.30.67
  Swap usage:   0%                IP address for docker0: 172.17.0.1

  Graph this data and manage this system at:
    https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud


*** System restart required ***
ubuntu@ip-172-31-30-67:~$ sudo docker search python
NAME                     DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
python                   Python is an interpreted, interactive, obj...   738       [OK]

Step 5: Test a docker image with a Python hello world

Now let us perform a Python hello world, using the corresponding python docker image:

$echo 'print("hello world!")' > helloworld.py
$docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp python python helloworld.py

Or, we can set a new alias, which allows us for a simpler syntax in future (not that the alias will not survive a reboot, if not written to .bashrc. Moreoever, it will not survive a termination/creation cycle, if the alias is not provisioned via Vagrantfile):

ubuntu@localhost:~$ alias python='docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp python python'
ubuntu@localhost:~$ python helloworld.py
hello world!

Caveats

After trying again to perform vagrant provision in order to verify the 10 minutes installation time, I hit the following problem on line 125 of /tmp/vagrant-provision.sh (a file that is uploaded automatically as specified by the Vagrantfile):

default: curl: (56) SSL read: error:00000000:lib(0):func(0):reason(0), errno 104

The problem seems to be caused in line

curl -o docker-machine -L https://github.com/docker/machine/releases/download/$MACHINE_VERSION/docker-machine-`uname -s`-`uname -m`

I have not found a reason for the error yet. My workaround was to issue vagrant provision a second time. Docker seems to work thereafter.

Summary

In this blog post, we have shown how Vagrant can be used to perform more sophisticated provisioning tasks than creation and termination of virtual machines. From our local Vagrant console, we have installed lots of useful Docker Software in less than 10 minutes and we have verified the results by downloading and testing the Python Docker image.


<< Part 1 | Part 2 |Part 3 >>

 


4 thoughts on “AWS Automation based on Vagrant — Part 3: Creating a Docker Host on AWS in 10 Minutes using Vagrant

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s