1

Jenkins Part 4.2: Code Quality Tests via Checkstyle


2017-01-19-23_30_11

Today, we will show how to use Checkstyle for improving the style of Java code. First, we will add Checkstyle to Gradle in order to create XML reports for a single build. Jenkins allows us to visualize the results of more than one test/build run into historic reports. After that, we will show, how a developer can use the Eclipse Checkstyle plugin in order to create better code:

2017-02-01-04_43_16-github-triggered-build-jenkins

This blog post series is divided into following parts:

    • Part 1: Installation and Configuration of Jenkins, loading Plugins
    • Part 2: Creating our first Jenkins job: GitHub download and Software build
    • Part 3: Periodic and automatically triggered Builds
    • Part 4.1: running automated tests: Functional Tests via Java JUnit
    • Part 4.2: running automated tests: Code Quality Test via Checkstyle (this post)
    • Part 4.3: running automated tests: Performance Tests with JMeter (work in progress)

What is Jenkins?

Jenkins is the leading open source automation server mostly used in continuous integration and continuous deployment pipelines. Jenkins provides hundreds of plugins to support building, deploying and automating any project.

 

Jenkins build, test and deployment pipeline
Jenkins build, test and deployment pipeline

A typical workflow is visualized above: a developer checks in the code changes into the repository. Jenkins will detect the change, build (compile) the software, test it and prepare to deploy it on a system. Depending on the configuration, the deployment is triggered by a human person, or automatically performed by Jenkins.

For more information, see the introduction found in part 1 of this blog series.

Checking Code with Checkstyle

In this post, we will show how to configure Jenkins for automated code checking as part of the Post-Build Tests:

2017-01-19-04_15_46

After this tutorial has been followed, we will have learned how to apply standard or custom checks on the code quality using Checkstyle in Eclipse and Jenkins.

Tools & Versions used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.32.1
        • Checkstyle Plug-in 3.47
      • Eclipse Kepler Service Release 2 (Build id: 20140224-0627)
        • Checkstyle Plug-in 7.2.0.201611082205

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB
      • Docker Host is available, Jenkins is installed and a build process is configured. For that, perform all steps in part 1 to part 3 of this blog series (new: you now can skip part 1, if you wish)
      • Tested with 2 vCPU (1 vCPU might work as well)

Step 1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

We have created this entry in part 1 already, but I have seen that the entries were gone again, which seems to be a VirtualBox bug. I have added it again now.

Log in with the admin account we have created in the last session:

2016-12-09-10_24_00-jenkins

Step 3: Code Analysis: Checkstyle

With Gradle, we can invoke the Checkstyle plugin as follows:

Step 3.1: Prepare Gradle for performing Checkstyle

Add to build.gradle:

apply plugin: 'checkstyle'

tasks.withType(Checkstyle) {
 ignoreFailures = true
 reports {
 html.enabled = true
 }
}

We have set ignoreFailures to true, since we do not want the Gradle build to fail for now. We are just interested in the Checkstyle reports for now.

We can download an example Checkstyle configuration file from the Apache Camel repository, for example:

git clone <yourprojectURL>
mkdir -p <yourprojectDir>/config/checkstyle/
curl https://raw.githubusercontent.com/apache/camel/master/buildingtools/src/main/resources/camel-checkstyle.xml > <yourprojectDir>/config/checkstyle/checkstyle.xml

Step 3.2 (optional): Test Checkstyle locally

If you have no Git and/or no Gradle installed, you may want to skip this step and directly proceed to the next step, so Jenkins is performing this task for you.

We can locally invoke CheckStyle as follows:

gradle check

Step 3.3: Configure Jenkins to invoke Checkstyle

Adding Gradle Checkstyle tests to be performed before each build is as simple as performing Step 4.1 and then adding “check” as a goal to the list of Jenkins Build Gradle Tasks:

On Dashboard -> Click on Project name -> Configure -> Build, add “check” before the jar task:

2016-12-28-15_33_58-github-triggered-build-config-jenkins

Click Save.

Now we verify the setting by either checking changed code into the SW repository (now is a good time to commit and push the changes performed in Step 4.1) or by clicking “Build now” -> Click on Build Link in Build History -> Console Output in the Project home:

2016-12-28-15_39_37-github-triggered-build-725-console-jenkins2016-12-28-15_40_08-github-triggered-build-725-console-jenkins

We have received a very long list of CheckStyle Errors, but, as configured, the build does not fail nevertheless.

At the same time, CheckStyle Reports should be available on Jenkins now:

The Links specified in the output are only available on Jenkins, but since Jenkins is running as a Docker container on Vagrant VM residing in

D:\veits\Vagrant\ubuntu-trusty64-docker_openshift-installer\jenkins_home

I need to access the files on

file:///D:/veits/Vagrant/ubuntu-trusty64-docker_openshift-installer/jenkins_home/workspace/GitHub%20Triggered%20Build/build/reports/checkstyle/

2016-12-28-15_48_11-index-von-d__veits_vagrant_ubuntu-trusty64-docker_openshift-installer_jenkins_ho

And on main.html we find:

2016-12-28-15_49_04-main-html

Wow, it seems like I really need to clean the code…

Step 4: Visualize the CheckStyle Warnings and Errors to the Developer

Usually Jenkins is not running as a Docker container on the Developer’s PC or Notebook, so he has no access to the above report files. We need to publish the statistics via the Jenkins portal. For that, we need to install the CheckStyle Jenkins plugin:

Step 4.1 (optional): Install the “Static Analysis Utilities”

Note: I have not tried it out, but I believe that this step is not necessary, since the next step will automatically install all plugins the Checksytle plug-in depends on.

On Jenkins -> Manage Jenkins -> Manage Plugins -> Available

In the Filter field, type “Static Analysis U”

2016-12-28-22_44_53-update-center-jenkins

Check the checkbox of “Static Analysis Utilities” and Install without restart.

2016-12-28-22_47_06-update-center-jenkins

Step 4.2: Install Checkstyle Plugin

On Jenkins -> Manage Jenkins -> Manage Plugins -> Available

In the Filter field, type “Checkstyle ” (with white space at the end; this will limit the number of hits):

2016-12-28-22_56_26-update-center-jenkins

Check the checkbox of “Checkstyle Plug-in” and Install without restart.

2016-12-28-22_58_22-update-center-jenkins

Step 4.3: Configure Checkstyle Reporting in Jenkins

On Dashboard -> <your Project> -> Configure -> Post-build Actions -> Add post-build action, choose

Publish Checkstyle analysis results

Now add the path, where Gradle is placing its result xml files:

**/build/reports/checkstyle/*.xml

2016-12-28-23_10_57-github-triggered-build-config-jenkins

And click Save.

Step 4.4: Manually trigger a new Build

On the Project page, click “Build now”, then click on the build and then “Console output”:

2016-12-28-23_17_16-github-triggered-build-726-console-jenkins

We now can see [CHECKSTYLE] messages after the build, telling us, that the reports were collected. Now, where can we see them?

Step 4.5: Review Checkstyle Statistics

On the Project page, choose Status:

2016-12-28-23_20_31-github-triggered-build-726-jenkins-v2

and click on Checkstyle Warnings on the left, or the warnings link in the center of the page, and we get a graphical representation of the Checkstyle statistics:

2016-12-29-12_27_34-jenkins

When clicking on one of the File Links (MyRouteBuilder.java in this case), we can get an overview of the Warning types for this file:

2016-12-29-12_28_37-jenkins

We choose the category Indentation and get details on the warnings:

2016-12-29-09_03_58-jenkins

and after clicking on one of the links in the Warnings field, we see the java code causing the warning:

2016-12-29-09_05_56-jenkins

Okay, Camel’s Checkstyle configuration does not like my style of grouping each route’s first line with a smaller indent than the rest of the route:

2016-12-29-09_10_26-jenkins

And it does not seem to accept my style of putting the ; in a new line at the end of a route as seen by choosing the Whitespace category and then choosing an occurence:

2016-12-29-12_34_10-jenkins

I either need to change this style, or I need to adapte the checkstyle.xml configuration file to ignore those warnings.

Step 5: Improve Code Style

For the developer, it is very inconvenient to use the Jenkins Checkstyle messages from the console and match them with the code. We need something better than that: the Eclipse Checkstyle plugin.

Step 5.1: Install Eclipse Checkstyle Plugin via local Installation

Since the recommended installation via Marketplace did not work in my case (see Appendix A), I have followed some hints about a local installation found on StackOverflow:

Download Checkstyle from Sourceforge.

2016-12-30-13_54_36-add-repository

2016-12-30-13_55_14-install

In the next window, you are asked to specify some credentials we do not have. However, you can just ignore the window and click Cancel:

2016-12-30-14_01_54-login-required

->Cancel

Then the installation proceeds:

2016-12-30-14_04_17-install

2016-12-30-14_04_26-install

2016-12-30-14_04_33-installing-software

Now I had to klick OK on security warnings twice:

2016-12-29-19_55_50-security-warning

At the end, I had to restart Eclipse:

2016-12-30-19_09_15-software-updates

Now, the Checkstyle plugin is installed on Eclipse.

Step 5.2: Configure Project for Checkstyle Usage

The project in question must be enabled for Checkstyle usage by editing the Project Properties:

2017-01-07-23_14_44

Choosing the Checkstyle style. For now, let us choose the Google Checks in the drop-down list:

2017-01-07-23_18_41-properties-for-simple-restful-file-storage

Then confirm that the project is being re-built:

2017-01-07-23_18_50-rebuild-suggested

Now the code is more yellow than white, with many hints how to improve the code:

2017-01-07-23_28_00-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

However, the hints do not go away, if you correct the code. Do we need to rebuild again? Let us test:

2017-01-07-23_30_36-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Google style does not like that there is no empty line before the package line (sorry, in German):

2017-01-07-23_29_57-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

So, let us add an empty line and save the file. However, the style warning does not change:

2017-01-07-23_31_53-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Let us rebuild the project:

2017-01-07-23_33_05

Yes, after the re-build: the warning has disappeared:

2017-01-07-23_43_01-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Step 5.3: Download and Create Custom Checkstyle Profile in Eclipse

In the Jenkins Checkstyle tests above, we have used following custom Checkstyle configuration file:

$ curl https://raw.githubusercontent.com/apache/camel/master/buildingtools/src/main/resources/camel-checkstyle.xml > <yourprojectDir>/config/checkstyle/checkstyle.xml

I.e. the Checkstyle file is found on <yourprojectDir>/config/checkstyle/checkstyle.xml

Correct:

2017-01-07-23_49_13-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

2017-01-07-23_52_04-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

2017-01-07-23_52_57-preferences

2017-01-07-23_55_39-check-configuration-properties

Step 5.4: Assign Custom Checkstyle Profile to the Project

To assign the new Checkstyle profile to the project, we change the project’s Checkstyle properties by

Project->Properties -> Checkstyle

2017-01-07-23_14_44

-> Choose new Checkstyle profile -> OK

2017-01-08-00_01_13-properties-for-simple-restful-file-storage

On the Rebuild suggested window -> Yes

2017-01-08-00_01_18-rebuild-suggested

This works fine:

2017-01-18-02_29_51-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles-v2

In the code, we can see the Checkstyle warnings. To get more information on the specific Checkstyle warning, the warning text can be retrieved via the mouse over function on the left of the code line, or on the markers tab on the lower pane of Eclipse.

Step 5.5: Improve Code Style

Step 5.5.1: Change Code

In order to test, how the developer can improve the code style, let us replace some of the tabs by four spaces here:

2017-01-18-02_48_39-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Save the file now.

Step 5.5.2: Update Maven

Unfortunately, the Checkstyle warnings update process is a little cumbersome for custom Checkstyle profiles, it seems: we need to

  1. save the changed file,
  2. update Maven and
  3. rebuild the project.

Let us update Maven first:

right-click the project folder in the left pane -> Maven -> Update Project -> OK

2017-01-18-02_54_032017-01-18-02_58_21-update-maven-project

Then all Checkstyle markers are gone (although I have not changed all occurrences of a tab):

2017-01-18-02_59_32-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Step 5.5.3 Rebuild the Project

To get the Checkstyle warnings back, we need to rebuild the project:

Project -> Build Project

2017-01-18-03_02_56

Now we can see that some of the Checkstyle warnings are gone:

2017-01-18-03_04_05-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Next time, you check in the code to the Gir repository, you will see that the number of Checkstyle warnings we get from Jenkins via Gradle will decrease…

Step 6: Verify Jenkins Results

Since we have improved the source code, we expect the Jenkins Checkstyle warnings to decrease. We can verify this by doing the following:

-> save, commit and push the improved code -> log into Jenkins -> check out the build process that is triggered by the code push (or we can manually trigger the build process by clicking project -> Build now)

On the dashboard, we will see, that the Checkstyle statistics have (very) slightly improved:

2017-01-18-04_37_06-github-triggered-build-jenkins-v2

On the upper right edge of the figure, the number of warnings is slightly lower. The code quality is far from being perfect, but we now have all tools and plugins needed to improve the situation.

After changing all tabs by 4 spaces each, the number of Checkstyle violations goes down by ~50%. That is a good start.

2017-01-19-22_51_58-github-triggered-build-jenkins-v2

Perfect, we have learned how to use the Checkstyle plugin for Eclipse in order to produce better code. And the Jenkins Checkstyle plugin allows us to admire the progress we make.

😉

thumps_up_3

Appendix A: Problems with installing Checkstyle Eclipse Plugin via Marketplace

Note: this way of installation is recommended officially, but has failed in my case. If you hit the same problem, try the local installation as described in step 5.1 above.

To improve the style, it would be much too cumbersome to click through all 360 style warnings, edit the Java code, build the Code and check again. It is much better to give the programmer immediate feedback of the warning within the IDE. I am using Eclipse, so I need to install the Checkstyle Eclipse plugin as follows:

Choose Eclipse -> Help -> Eclipse Marketplace

2016-12-29-09_16_30-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Search for “Checkstyle” and click install:

2016-12-29-09_19_00-eclipse-marketplace

And then “confirm”:

2016-12-29-09_20_01-eclipse-marketplace

What is that?

2016-12-29-09_21_18-proceed-with-installation_

I install it anyway. At this point, it is hanging quite a while:

2016-12-29-09_24_24-eclipse-marketplace

so, let me get a morning coffee…

After approximately two minutes, I can see it to proceed to 4 / 15. Good sign.

After the coffee, I still see 4 / 15. Not a good sign:

2016-12-29-09_44_41-eclipse-marketplace

Meanwhile I am researching the steps needed for performance testing…

After 2 hours or so: 6/15

This will take all day!

2016-12-29-11_31_35-eclipse-marketplace

Some hours later, I checked again, and I have seen the following:

2016-12-29-19_49_10-eclipse-marketplace

I have confirmed, confirmed the license:

2016-12-29-19_50_51-eclipse-marketplace

And have pressed Finish.

Then software gets installed:

2016-12-29-19_52_04-installing-software

I hope I will not break my good old Eclipse installation (it is installed locally, not in a virtual machine or container and it has ever worked better than any new version I have tested…).

After a two or three minutes:

2016-12-29-19_55_57-security-warning

I have confirmed with “OK”…

Then I had been asked to restart Eclipse and I have confirmed.

Problem: however, Checkstyle is still not installed:

Help -> Eclipse Marketplace

2016-12-30-13_20_09-eclipse-marketplace

Let us try again by clicking “Install”:

2016-12-30-13_24_45-eclipse-marketplace

2016-12-30-13_24_59-proceed-with-installation_

This does not work

Workaround

Instead of installing Checkstyle via the Eclipse marketplace, better install the Eclipse Checkstyle Plugin via download (see Step 5.1)

Summary

In this blog post we have performed following tasks:

  1. Started Jenkins in a Docker container
  2. Installed the Checkstyle Gradle Plugin to create Checkstyle plugins as XML files
  3. Installed the Checkstyle Jenkins Plugin to summarize the XML files into graphical historic reports
  4. Installed the Checkstyle Eclipse Plugin to be able to improve the code
  5. Installed custom Checkstyle policies
  6. Visualized the Code Improvement
  7. were happy

All steps apart from the installation of the Eclipse Checkstyle plugin were quite straightforward. For the Eclipse Checkstyle installation, we had to revert back to a local download and installation method described in step 5.1: the installation via Eclipse marketplace had failed. At the end, we could reduce the number of Checkstyle warnings by 50% without much effort.

Further Reading

7

Jenkins Part 3.1: periodic vs triggered Builds


2016-11-30-18_19_38

Today, we will make sure that Jenkins will detect a code change in the software repository without manual intervention. We will show two methods to do so:

  1. Periodic Builds via Schedulers: Jenkins periodically asks the software repository for any code changes
  2. Triggered Builds via Webhooks: Jenkins is triggered by the software repository to perform the build task

We will see that the triggering build processes is more challenging to set up, but has quite some advantages in terms of economics and handling, once it is set up properly. See also the Summary at the end of this post.

This blog post series is divided into following parts:

    • Part 1: Installation and Configuration of Jenkins, loading Plugins
    • Part 2: Creating our first Jenkins job: GitHub download and Software build
    • Part 3 (this blog): Periodic and automatically triggered Builds
    • Part 4 (planned): running automated tests

What is Jenkins?

Jenkins is the leading open source automation server mostly used in continuous integration and continuous deployment pipelines. Jenkins provides hundreds of plugins to support building, deploying and automating any project.

 

Jenkins build, test and deployment pipeline
Jenkins build, test and deployment pipeline

A typical workflow is visualized above: a developer checks in the code changes into the repository. Jenkins will detect the change, build (compile) the software, test it and prepare to deploy it on a system. Depending on the configuration, the deployment is triggered by a human person, or automatically performed by Jenkins.

For more information, see the introduction found in part 1 of this blog series.

Automatic Jenkins Workflow: Periodic Polling

In this chapter, we will show how we need to configure Jenkins for automatic polling of the Software repository and start the build process, if code changes are detected.

2016-12-09-10_12_31

Tools used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.19.3

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB
      • Docker Host is available, Jenkins is installed and a build process is configured. For that, perform all steps in part 1 and part 2 of this blog series
      • Tested with 2 vCPU (1 vCPU might work as well)

Step 1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

We have created this entry in part 1 already, but I have seen that the entries were gone again, which seems to be a VirtualBox bug. I have added it again now.

Log in with the admin account we have created in the last session:

2016-12-09-10_24_00-jenkins

Step 3: Configure Project for periodic Polling of SW Repository

Step 3.1: Goto Build Trigger Configuration

On the Jenkins Dashboard, find the hidden triangle right of the project name,

2016-12-09-18_17_35-dashboard-jenkins

In the drop-down list, choose “Configure”

2016-12-09-18_18_06-dashboard-jenkins

(also possible: on the Dashboard, click on the project name and then “Configure”).

Step 3.2: Configure a Schedule

We scroll down to “Build Triggers” and check “Build periodically” and specify that it will be done every 10 minutes (H/10 * * * *). I do not recommend to use lower values than that since I have seen that even my monster notebook with i7-6700HQ and 64GB RAM is quite a bit stressed by the build those many build processes.

2016-12-22-23_54_06-github-triggered-build-config-jenkins

Note that this is a very short polling period for our test purposes only; we do not want to wait very long after a code change is detected.

Note also: you can click the Blue Question Markright of the Schedule text box to get help with the scheduler syntax.

Step 3.2: Save

Click Save

Step 4: Change the content of the Software Repository

Now we expect that a change of the SW repository is detected latest 2 minutes after new code is checked in. Let us do so now: In this case, I have changed the content of README.md and commited the change:

(local repository)$ git add README.md
(local repository)$ git commit -m "changed README"
(local repository)$ git push

Within 2 minutes, I see a new job #24 running on the lower left:

2016-12-09-18_35_13-dashboard-jenkins

It seems that the page needs to be reloaded by refreshing the browser, so the dashboard displays the #24 build process as “Last Success”:

The build process was very quick, since we have not changed any relevant source code. The console log can be reached via the Jenkins -> Project Link -> Build History -> click on build number -> Console:

2016-12-11-21_55_22-github-triggered-build-687-console-jenkins

As you can see, after some hours, the git repository is downloaded even if there was no code change at all. However, Gradle will detect that the JAR file is up-to-date and it will not re-build the JAR file, unless there is a code change.

The disadvantage of a scheduled build process with high frequency is that the number of builds in the build history is increasing quickly:

2016-12-11-22_02_24-github-triggered-build-jenkins

Note: The build history is spammed by many successful builds with no code change, and it is not easy to find the interesting build among all those many unnecessary builds. Let us try to improve the situation by replacing periodic, scheduled builds by triggered builds:

Step 5: Triggered Builds

In Step 4, we have seen that periodic builds should not be performed in a very short timeframe, because:

  1. the Jenkins server is stressed quite a bit by configuring a too low build frequency
  2. the build history is polluted by information of many irrelevant build processes with no changed code.

Therefore, it is much better to create a triggered build. The target is to trigger a build process every time the developer is checking in new code to the software repository:

2016-12-21-15_12_25

In this way, a periodic build is not necessary, or can be done much less frequently.

What do we need to do?

  1. Make sure that the Jenkins server is reachable from the SW repository
  2. Configure the SW repository with a web hook for informing Jenkins upon each code change
  3. Configure Jenkins for triggered build

Let us start:

Step 5.1 Configure Jenkins for triggered Build

On the Jenkins Dashboard, click on the project:

2016-12-22-18_56_44-dashboard-jenkins

and then “Configure” on the left pane:

2016-12-22-18_58_28-github-triggered-build-jenkins

Scroll down to Build Triggers and check the “Trigger build remotels (e..g. , from scripts)” checkbox and choose an individual secret token (do not use the one you see here):

2016-12-22-19_03_16-github-triggered-build-config-jenkins

You will be provided with the build trigger URL, which is in my case:

JENKINS_URL/job/GitHub%20Triggered%20Build/build?token=TOKEN_NAME

And the JENKINS_URL is the URL needed to be contacted by the Git Repository. Save the URL above for later use.

Now click Save.

Step 5.2 Test Trigger URL locally

Now we can test the trigger URL locally on the Docker Host as follows (as found on this StackOverflow Q&A):

We need to retrieve a so-called Jenkins-Crumb:

(dockerhost)$ CRUMB=$(curl -s 'http://admin:your_admin_password@localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
(dockerhost)$ echo $CRUMB
Jenkins-Crumb:CCCCCCCCCCCCCCCCCCCCCCCCCC

Please make a note of the returned Jenkins-Crumb, since we will need this value in the next step.

Then we can use the Jenkins-Crumb as header in the build trigger request:

(dockerhost)$ curl -H $CRUMB 'http://admin:your_admin_password@localhost:8080/job/GitHub%20Triggered%20Build/build?token=hdsghewriohwziowrhwsn'

This should trigger a new build on Jenkins:

2016-12-22-21_56_44-dashboard-jenkins-v2

By clicking on the build and then the “Console Output”, we see a successful build with no changed data:

2016-12-22-22_01_36-github-triggered-build-712-console-jenkins

Step 5.3 Make sure that the Jenkins Server is reachable from the SW repository

We are running the Jenkins server as a Docker container within a Vagrant VM as host. In step 2 we have made sure that the Docker container is reachable from the local network by exposing the Docker ports and by configuring port forwarding in VirtualBox. However, the Docker container is not yet reachable from the Git Repository, since the Router will block all requests, as long as no pot forwarding is configured on the router:

2016-12-21-16_02_27

Let us fix that now:

In my case, the (sorry, German) input mask of the router looks like follows:

2016-12-22-19_30_22-fritzbox-7490

I am mapping outside port 8080 to the internal machine running the Docker Host VM.

Now, the routing should work. We will test this in the next step.

2016-12-22-23_59_49

Step 5.4: Add Webhook to Git SW Repository

Now we need to add a Webhook to the Git repository. In my case, the repository is located at https://github.com/oveits/simple-restful-file-storage. On that page, goto

Settings -> Webhooks-> Add webhook -> Confirm password

Then copy&paste the URL of Step 5.1 into the Payload URL with following changes:

  • Change JENKINS_URL by the IP address or DNS name your router is reachable from the Internet.
  • Choose a port that you intend to open for this service (e.g. 8080) in the next step.
  • Add admin:your_admin_password@ before the JENKINS_URL; use your own username and password here
  • append &Jenkins-Crumb=CCCCCCCCCCCCCCCC to the URL with the value of the Jenkins-Crumb we have retrieved in the previous step

Example with the items to change in red:

http://admin:your_admin_password@your_public_ip_or_name:8080/job/GitHub%20Triggered%20Build/build?token=TTTTTTTTTTTTTTTTT&Jenkins-Crumb=CCCCCCCCCCCCCCCCCCCC

2016-12-22-22_08_39-webhook-http___veits-no-ip-biz_8080_job_github%20triggered%20build_build

 

For the other fields, keep the defaults and klick Add Webhook.

If everything works fine, we already should see a successful delivery of the trigger on the lower end of the Github page:

2016-12-22-22_17_57-webhook-http___veits-no-ip-biz_8080_job_github%20triggered%20build_build-v2

If it was not successful, you can see more details by clicking on the request:

2016-12-22-22_20_52-webhook-http___veits-no-ip-biz_8080_job_github%20triggered%20build_build

Step 6: Test triggered Build upon Code Push

This is the final step of this tutorial: we now will test that a build is triggered each time a user pushes new code to the repository.

Step 6.1: Install Git locally

If Git is not installed locally, so do it now.

Step 6.2: Download the Project Repository

We now clone the project by issuing the command

$ git clone https://github.com/oveits/simple-restful-file-storage

Step 6.3: Change Code

You can perform a minor change the content of the README.md in order to test the triggered build.

Step 6.4: Push Code to the Repository

With the commands

$ git commit -am "Minor change of README.md to trigger a Jenkins build"
$ git push

We push the changed code to the SW repository.

If everything works correctly, we will immediately see, that Git has triggered Jenkins to perform a build by reloading the Jenkins Dashboard (32 sec ago, in this screenshot):

2016-12-22-22_50_14-dashboard-jenkins-v2

We can check the build by clicking on the Last Success build and then “Console Output”:

2016-12-22-22_53_08-github-triggered-build-713-console-jenkins

Gradle was clever enough to detect that no relevant code had been changed, so everything is still up to date.

With this procedure we have made sure that the Software repository will trigger a new build process on each and every code change. Moreover, the Jenkins server is not polluted with unnecessary builds anymore, since we have switched off periodic builds.
thumps_up_3

Summary

In this blog post we have performed following tasks:

  1. Started Jenkins in a Docker container
  2. Configured and tested periodic builds
  3. Configured and tested triggered builds
  4. Made sure that the Git Software repository is triggering such a build at every code change

As in the other parts of this series, we have run Jenkins interactively in a Docker container. See below a discussion of the advantages of periodic and triggered builds:

Periodic Builds vs Triggered Builds

When we compare periodic builds with triggered builds, we see following advantages/disadvantages:

Complexity of Setup: periodic builds are much easier to set up. They only need to be configured on Jenkins. Triggered builds requires setup steps on the Jenkins Server, the Software Repository and intermediate Firewalls, if the Jenkins Server is located in a private network.

Economics: Triggered builds are more economic in terms of Jenkins Server load. The build processes run only, when needed.

Handling: Triggered builds have important handling advantages compared to triggered builds: firstly, each and every code change can be tested helping the programmer to get near immediate feedback for every code change. Secondly, the build log is not polluted by hundreds of irrelevant builds.

In my opinion, a clear winner is: triggered builds. Those may be combined with periodic clean builds at certain milestones.

2016-12-22-23_39_41

 

References

 

3

Java Build Automation Part 2: Create executable jar using Gradle


Original title: How to build a lean JAR File with Gradle

2016-11-14-19_15_52

In this step by step guide, we will show that Gradle is a good alternative to Maven for packaging java code into executable jar files. In order to keep the executable jar files “lean”, we will keep the dependent jar files outside of the jar in a separate folder.

Tools Used

  1. Maven 3.3.9
  2. JDK 1.8.0_101
  3. log4j 1.2.17 (downloaded automatically)
  4. Joda-time 2.5 (downloaded automatically)
  5. Git-2.8.4 with GNU bash 4.3.42(5)

Why using Gradle for a Maven Project?

In this blog post, we will show how Gradle can be used to create a executable/runnable jar. The task has been accomplished on this popular Mkyong blog post by using Maven. Why would we want to do the same task using Gradle?

By working with both, Maven and Gradle, I have found that:

  • Gradle allows me to move any resource file to outside of the jar without the need of any additional Linux script or alike;
  • Gradle allows me to easily create an executable/runnable jar for the JUnit tests, even if those are not separated into a separate project.

Moreover, while Maven is descriptive, Gradle is procedural in nature. With Maven, you describe the goal and you rely on Maven and its plugins to perform the steps you had in mind. Whereas with Gradle, you have explicit control on each step of the build process. Gradle is easy to understand for programmers and it gives them fine-grained control over the build process.

The Goal: a lean, executable JAR File

In the following step by step guide, we will create a lean executable jar file with all dependent libraries and resources.

Step 1 Download Hello World Maven Project of Mkyong

Download this hello world Maven project you can find on this popular HowTo page from Mkyong:

curl -OJ http://www.mkyong.com/wp-content/uploads/2012/11/maven-create-a-jar.zip
unzip maven-create-a-jar.zip
cd dateUtils

Logs:

$ curl -OJ http://www.mkyong.com/wp-content/uploads/2012/11/maven-create-a-jar.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7439  100  7439    0     0  23722      0 --:--:-- --:--:-- --:--:-- 24963

olive@LAPTOP-P5GHOHB7  /d/veits/eclipseWorkspaceRecent/MkYong/ttt
$ unzip maven-create-a-jar.zip
Archive:  maven-create-a-jar.zip
   creating: dateUtils/
  inflating: dateUtils/.classpath
  inflating: dateUtils/.DS_Store
   creating: __MACOSX/
   creating: __MACOSX/dateUtils/
  inflating: __MACOSX/dateUtils/._.DS_Store
  inflating: dateUtils/.project
   creating: dateUtils/.settings/
  inflating: dateUtils/.settings/org.eclipse.jdt.core.prefs
  inflating: dateUtils/log4j.properties
  inflating: dateUtils/pom.xml
   creating: dateUtils/src/
   creating: dateUtils/src/main/
   creating: dateUtils/src/main/java/
   creating: dateUtils/src/main/java/com/
   creating: dateUtils/src/main/java/com/mkyong/
   creating: dateUtils/src/main/java/com/mkyong/core/
   creating: dateUtils/src/main/java/com/mkyong/core/utils/
  inflating: dateUtils/src/main/java/com/mkyong/core/utils/App.java
   creating: dateUtils/src/main/resources/
  inflating: dateUtils/src/main/resources/log4j.properties
   creating: dateUtils/src/test/
   creating: dateUtils/src/test/java/
   creating: dateUtils/src/test/java/com/
   creating: dateUtils/src/test/java/com/mkyong/
   creating: dateUtils/src/test/java/com/mkyong/core/
   creating: dateUtils/src/test/java/com/mkyong/core/utils/
  inflating: dateUtils/src/test/java/com/mkyong/core/utils/AppTest.java
olive@LAPTOP-P5GHOHB7  /d/veits/eclipseWorkspaceRecent/MkYong/ttt
$ cd dateUtils/

olive@LAPTOP-P5GHOHB7  /d/veits/eclipseWorkspaceRecent/MkYong/ttt/dateUtils
$ 

Step 2 (optional): Create GIT Repository

In order to see, which files have been changed by which step, we can create a local GIT repository like follows

git init
# echo "Converting Maven to Gradle" > Readme.txt
git add .
git commit -m "first commit"

After each step, you then can perform the last two commands with a different message, so you can always go back to a previous step, if you need to do so. If you have made changes in a step that you have not committed yet, you can go back easily to the last clean commit state by issuing the command

# go back to status of last commit:
git stash -u

Warning: this will delete any new files you have created since the last commit.

Step 3 (required): Initialize Gradle

gradle init

This will automatically create a file build.gradle file from the Maven POM file with following content:

apply plugin: 'java'
apply plugin: 'maven'

group = 'com.mkyong.core.utils'
version = '1.0-SNAPSHOT'

description = """dateUtils"""

sourceCompatibility = 1.7
targetCompatibility = 1.7

repositories {

     maven { url "http://repo.maven.apache.org/maven2" }
}
dependencies {
    compile group: 'joda-time', name: 'joda-time', version:'2.5'
    compile group: 'log4j', name: 'log4j', version:'1.2.17'
    testCompile group: 'junit', name: 'junit', version:'4.11'
}

Step 4 (required): Gather Data

Since we are starting from a Maven project, which is prepared to create a runnable JAR via Maven already, we can extract the needed data from the POM.xml file:

MAINCLASS=`grep '<mainClass' pom.xml | cut -f2 -d">" | cut -f1 -d"<"`

Note: In cases with non-existing maven plugin, you need to set the MAINCLASS manually, e.g.

MAINCLASS=com.mkyong.core.utils.App

We also can define, where the dependency jars will be copied to later:

DEPENDENCY_JARS=dependency-jars

Logs:

$ MAINCLASS=`grep '<mainClass' pom.xml | cut -f2 -d">" | cut -f1 -d"<"`
$ echo $MAINCLASS
com.mkyong.core.utils.App
$ DEPENDENCY_JARS=dependency-jars
echo $DEPENDENCY_JARS
dependency-jars

Step 5 (required): Prepare to copy dependent Jars

Here, we will add instructions to the build.gradle file, which dependency JAR files are to be copied into a directory accessible by the executable jar.

We will need to copy the jars, we depend on, to a folder the runnable jar will access later on. See e.g. this StackOverflow question on this topic.

cat << END >> build.gradle

// copy dependency jars to build/libs/$DEPENDENCY_JARS 
task copyJarsToLib (type: Copy) {
    def toDir = "build/libs/$DEPENDENCY_JARS"

    // create directories, if not already done:
    file(toDir).mkdirs()

    // copy jars to lib folder:
    from configurations.compile
    into toDir
}
END

Step 6 (required): Prepare the Creation of an executable JAR File

In this step, we define in the build.gradle file, how to create an executable jar file.

cat << END >> build.gradle
jar {
    // exclude log properties (recommended)
    exclude ("log4j.properties")

    // make jar executable: see http://stackoverflow.com/questions/21721119/creating-runnable-jar-with-gradle
    manifest {
        attributes (
            'Main-Class': '$MAINCLASS',
            // add classpath to Manifest; see http://stackoverflow.com/questions/30087427/add-classpath-in-manifest-file-of-jar-in-gradle
            "Class-Path": '. dependency-jars/' + configurations.compile.collect { it.getName() }.join(' dependency-jars/')
            )
    }
}
END

Step 7 (required): Define build Dependencies

Up to now, a task copyJarsToLib was defined, but this task will not be executed, unless we tell Gradle to do so. In this step, we will specify that each time, a Jar is created, the copyJarsToLib task is to be performed beforehand. This can be done by telling Gradle that the jar goal depends on the copyJarsToLib task like follows:

cat << END >> build.gradle

// always call copyJarsToLib when building jars:
jar.dependsOn copyJarsToLib
END

Step 8 (required): Build Project

Meanwhile, the build.gradle file should have following content:

apply plugin: 'java'
apply plugin: 'maven'

group = 'com.mkyong.core.utils'
version = '1.0-SNAPSHOT'

description = """dateUtils"""

sourceCompatibility = 1.7
targetCompatibility = 1.7

repositories {

     maven { url "http://repo.maven.apache.org/maven2" }
}
dependencies {
    compile group: 'joda-time', name: 'joda-time', version:'2.5'
    compile group: 'log4j', name: 'log4j', version:'1.2.17'
    testCompile group: 'junit', name: 'junit', version:'4.11'
}

// copy dependency jars to build/libs/dependency-jars
task copyJarsToLib (type: Copy) {
    def toDir = "build/libs/dependency-jars"

    // create directories, if not already done:
    file(toDir).mkdirs()

    // copy jars to lib folder:
    from configurations.compile
    into toDir
}

jar {
    // exclude log properties (recommended)
    exclude ("log4j.properties")

    // make jar executable: see http://stackoverflow.com/questions/21721119/creating-runnable-jar-with-gradle
    manifest {
        attributes (
            'Main-Class': 'com.mkyong.core.utils.App',
            // add classpath to Manifest; see http://stackoverflow.com/questions/30087427/add-classpath-in-manifest-file-of-jar-in-gradle
            "Class-Path": '. dependency-jars/' + configurations.compile.collect { it.getName() }.join(' dependency-jars/')
            )
    }
}

// always call copyJarsToLib when building jars:
jar.dependsOn copyJarsToLib

Now is the time to create the runnable jar file:

gradle build

Note: Be patient at this step: it can appear to be hanging for several minutes, if it is run the first time, while it is working in the background.

This will create the runnable jar on build/libs/dateUtils-1.0-SNAPSHOT.jar and will copy the dependency jars to build/libs/dependency-jars/

Logs:

$ gradle build
:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.7
1 warning
:processResources
:classes
:copyJarsToLib
:jar
:assemble
:compileTestJava
warning: [options] bootstrap class path not set in conjunction with -source 1.7
1 warning
:processTestResources UP-TO-DATE
:testClasses
:test
:check
:build

BUILD SUCCESSFUL

Total time: 3.183 secs

$ ls build/libs/
dateUtils-1.0-SNAPSHOT.jar dependency-jars

$ ls build/libs/dependency-jars/
joda-time-2.5.jar log4j-1.2.17.jar

Step 9: Execute the JAR file

It is best practice to exclude the log4j.properties file from the runnable jar file, and place it outside of the jar file, since we want to be able to change logging levels at runtime. This is, why we had excluded the properties file in step 6. In order to avoid an error “No appenders could be found for logger”, we need not specify the location of the log4j.properties properly on the command-line.

Step 9.1 Execute JAR file on Linux

On a Linux system, we run the command like follows:

java -jar -Dlog4j.configuration=file:full_path_to_log4j.properties build/libs/dateUtils-1.0-SNAPSHOT.jar

Example:

$ java -jar -Dlog4j.configuration=file:/usr/home/me/dateUtils/log4j.properties build/libs/dateUtils-1.0-SNAPSHOT.jar
11:47:33,018 DEBUG App:18 - getLocalCurrentDate() is executed!
2016-11-14

Note: if the log4j.properties file is on the current directory on a Linux machine, we also can create a batch file run.sh with the content

#!/usr/bin/env bash
java -jar -Dlog4j.configuration=file:`pwd`/log4j.properties build/libs/dateUtils-1.0-SNAPSHOT.jar

and run it via bash run.sh

Step 9.1 Execute JAR file on Windows

In case of Windows in a CMD shell all paths need to be in Windows style:

java -jar -Dlog4j.configuration=file:D:\veits\eclipseWorkspaceRecent\MkYong\dateUtils\log4j.properties build\libs\dateUtils-1.0-SNAPSHOT.jar
11:45:30,007 DEBUG App:18 - getLocalCurrentDate() is executed!
2016-11-14

If we run the command on a Windows GNU bash shell, the syntax is kind of mixed: the path to the jar file is in Linux style while the path to the log properties file needs to be in Windows style (this is, how the Windows java.exe file expects the input of this option):

$ java -jar -Dlog4j.configuration=file:'D:\veits\eclipseWorkspaceRecent\MkYong\dateUtils\log4j.properties' build/libs/dateUtils-1.0-SNAPSHOT.jar
11:45:30,007 DEBUG App:18 - getLocalCurrentDate() is executed!
2016-11-14

Inverted commas have been used in order to avoid the necessity of escaped backslashes like D:\\veits\\eclipseWorkspaceRecent\\… needed on a Windows system.

Note: if the log4j.properties file is on the current directory on a Windows machine, we also can create a batch file run.bat with the content

java -jar -Dlog4j.configuration=file:%cd%\log4j.properties build\libs\dateUtils-1.0-SNAPSHOT.jar

To run the bat file on GNU bash on Windows, just type ./run.bat

Yepp, that is it: the hello world executable file is printing the date to the console, just as it did in Mkyong’s blog post, where the executable file was created using Maven.

simpleicons_interface_folder-download-symbol-svg

Download the source code from GIT.

Note: in the source code, you also will find a file named prepare_build.gradle.sh, which can be run on a bash shell and will replace the manual steps 4 to 7.

References

Next Steps

  • create an even leaner jar with resource files kept outside of the executable jar. This opens the opportunity to changing resource files at runtime.
  • create an executable jar file that will run the JUnit tests.

 

1

LXD vs. Docker — or: getting started with LXD Containers


Container technology is not new: it had existed long before the Docker hype around container technology has started after 2013. Now, with Docker containers having reached mainstream usage, there is a high potential of getting confused about available container types like Docker, LXC, LXD and CoreOS rocket. In this blog post we will explain, why LXD is not competing with Docker.

We will show in a few steps how to install and run LXC containers using LXD container management functions. For that, we will make use of an automated installation process based on Vagrant and VirtualBox.

What is LXD and why not using it as a Docker Replacement?

After working with Docker for quite some time, I have stumbled over another container technology: Ubuntu’s LXD (say “lex-dee”). What is the difference to Docker, and do they really compete with each other, as an article in the German “Linux Magazin” (May 2015) states?

As the developers of LXD point out, a main difference between Docker and LXD is that Docker focuses on application delivery from development to production, while LXD’s focus is system containers. This is, why LXD is more likely to compete with classical hypervisors like XEN and KVM and it is less likely to compete with Docker.

Ubuntu’s web page points out that LXD’s main goal is to provide a user experience that’s similar to that of virtual machines but using Linux containers rather than hardware virtualization.

For providing a user experience that is similar to that of virtual machines, Ubuntu integrates LXD with OpenStack through its REST API.  Although there are attempts to integrate Docker with OpenStack (project Magnum), Ubuntu comes much closer to feature parity with real hypervisors like XEN and KVM by offering features like snapshots and live migration. As any container technology, LXD offers a much lower resource footprint than virtual machines: this is, why LXD is sometimes called lightervisor.

One of the main remaining concerns of IT operations teams against usage of the container technology is that containers leave a “larger surface” to attackers than virtual machines. Canonical, the creators of Ubuntu and LXD is tackling security concerns by making LXD-based containers secure by default. Still, any low level security feature developed for LXC potentially is available for both, Docker and LXD, since they are based on LXC technology.

What does this mean for us?

We have learned that Docker offers a great way to deliver applications, while LXD offers a great way to reduce the footprint of virtual-machine-like containers. What, if you want to leverage the best of both worlds? One way is to run Docker containers within LXD containers. This and its current restrictions are described in this blog post of Stéphane Graber.

Okay; one step after the other: let us postpone the Docker in LXD discussion and let us get started with LXD now.

LXD Getting Started: a Step by Step Guide

This chapter largely follows this getting started web page. However, instead of trying to be complete, we will go through a simple end to end example. Moreover, we will add some important commands found on this nice LXD cheat sheet. In addition, we will explicitly record the example output of the commands.

Prerequisites:

  • Administration rights on you computer.
  • I have performed my tests with direct access to the Internet: via a Firewall, but without HTTP proxy. However, if you cannot get rid of your HTTP proxy, read this blog post.

Step 1: Install VirtualBox

If not already done, you need to install VirtualBox found here. See appendix A, if you encounter installation problems on Windows with the error message “Setup Wizard ended prematurely”. For my tests, I am using the already installed VirtualBox 5.0.20 r106931 on Windows 10.

Step 2: Install Vagrant

If not already done, you need to install Vagrant found here. For my tests, I am using an already installed Vagrant version 1.8.1 on my Windows 10 machine.

Step 3: Initialize and download an Ubuntu 16.0.4 Vagrant Box

In a future blog post, we want to test Docker in LXD containers. This is supported in Ubuntu 16.0.4 and higher. Therefore, we download the latest daily build of the corresponding Vagrant box. As a preparation, we create a Vagrantfile in a separate directory by issuing the following command:

vagrant init ubuntu/xenial64

You can skip the next command and directly run the vagrant up command, if you wish, since the box will downloaded automatically, if nocurrent version of the Vagrant box is found locally. However, I prefer to download the box first, and run the box later, since it is easier to observe, what happens during the boot.

vagrant box add ubuntu/xenial64

Depending on the speed of your Internet connection, you can take a break here.

Step 4: Boot the Vagrant Box as VirtualBox Image and connect to it

Then, we will boot the box with:

vagrant up

Note: if you encounter an error message like “VT-x is not available”, this may be caused by booting Windows 10 with Hyper-V enabled or by nested virtualization. According to this stackoverflow Q&A, running Virtualbox without VT-x is possible, if you make sure that the number of CPUs is one. For that, try to set vb.cpus = 1 in the Vagrantfile. Remove any statement like vb.customize ["modifyvm", :id, "--cpus", "2"] in the Vagrantfile. If you prefer to use VT-x on your Windows 10 machine, you need to disable Hyper-V. The Appendix: “Error message: “VT-x is not available” describes how to add a boot menu item that allows to boot without Hyper-V enabled.

Now let us connect to the machine:

vagrant ssh

Step 5: Install and initialize LXD

Now we need to install LXD on the Vagrant image by issuing the commands

sudo apt-get update
sudo apt-get install -y lxd
newgrp lxd

Now we need to initialize LXD with the lxd init interactive command:

ubuntu@ubuntu-xenial:~$ sudo lxd init
sudo: unable to resolve host ubuntu-xenial
Name of the storage backend to use (dir or zfs) [default=zfs]: dir
Would you like LXD to be available over the network (yes/no) [default=no]? yes
Address to bind LXD to (not including port) [default=0.0.0.0]:
Port to bind LXD to [default=8443]:
Trust password for new clients:
Again:
Do you want to configure the LXD bridge (yes/no) [default=yes]? no
LXD has been successfully configured.

I have decided to use dir as storage (since zfs was not enabled), have configured the LXD server to be available via the default network port 8443, and I have chosen to start without LXD bridge, since this article was pointing out that the LXD bridge does not allow SSH connections per default.

The configuration is written to a key value store and, which can be read with the lxc config get commands, e.g.

ubuntu@ubuntu-xenial:~$ lxc config get core.https_address
0.0.0.0:8443

The list of available system config keys can be found on this Git-hosted document. However, I have not found the storage backend type “dir”, I have configured. I guess, the system assumes that “dir” is used as long as the zfs and lvm variables are not set. Also, it is a little bit confusing that we configure LXD, but the config is read out via LXC commands.

Step 6: Download and start an LXC Image

Step 6.1 (optional): List remote LXC Repository Servers:

The images are stored on image repositories. Apart from the local repository, the default repositories have aliases images, ubuntu and ubuntu-daily:

ubuntu@ubuntu-xenial:~$ sudo lxc remote list
sudo: unable to resolve host ubuntu-xenial
+-----------------+------------------------------------------+---------------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    | PUBLIC | STATIC |
+-----------------+------------------------------------------+---------------+--------+--------+
| images          | https://images.linuxcontainers.org       | simplestreams | YES    | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| local (default) | unix://                                  | lxd           | NO     | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
Step 6.2 (optional): List remote LXC Images:

List all available ubuntu images for amd64 systems on the images repository:

ubuntu@ubuntu-xenial:~$ sudo lxc image list images: amd64 ubuntu
sudo: unable to resolve host ubuntu-xenial
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+
|          ALIAS          | FINGERPRINT  | PUBLIC |              DESCRIPTION              |  ARCH  |  SIZE   |         UPLOAD DATE          |
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+
| ubuntu/precise (3 more) | adb92b46d8fc | yes    | Ubuntu precise amd64 (20160906_03:49) | x86_64 | 77.47MB | Sep 6, 2016 at 12:00am (UTC) |
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+
| ubuntu/trusty (3 more)  | 844bbb45f440 | yes    | Ubuntu trusty amd64 (20160906_03:49)  | x86_64 | 77.29MB | Sep 6, 2016 at 12:00am (UTC) |
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+
| ubuntu/wily (3 more)    | 478624089403 | yes    | Ubuntu wily amd64 (20160906_03:49)    | x86_64 | 85.37MB | Sep 6, 2016 at 12:00am (UTC) |
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+
| ubuntu/xenial (3 more)  | c4804e00842e | yes    | Ubuntu xenial amd64 (20160906_03:49)  | x86_64 | 80.93MB | Sep 6, 2016 at 12:00am (UTC) |
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+
| ubuntu/yakkety (3 more) | c8155713ecdf | yes    | Ubuntu yakkety amd64 (20160906_03:49) | x86_64 | 79.16MB | Sep 6, 2016 at 12:00am (UTC) |
+-------------------------+--------------+--------+---------------------------------------+--------+---------+------------------------------+

Instead of the “ubuntu” filter keyword in the image list command above, you can use any filter expression. E.g. sudo lxc image list images: amd64 suse will find OpenSuse images available for x86_64.

Step 6.3 (optional): Copy remote LXC Image to local Repository:

This command is optional, since the download will be done automatically with the lxc launch command below, if the image is not found on the local repository already.

lxc image copy images:ubuntu/trusty local:
Step 6.4 (optional): List local LXC Images:

We can list the locally stored images with the following image list command. If you have not skipped the last step, you will find following output:

ubuntu@ubuntu-xenial:~$ sudo lxc image list
sudo: unable to resolve host ubuntu-xenial
+-------+--------------+--------+-------------------------------------------+--------+----------+------------------------------+
| ALIAS | FINGERPRINT  | PUBLIC |                DESCRIPTION                |  ARCH  |   SIZE   |         UPLOAD DATE          |
+-------+--------------+--------+-------------------------------------------+--------+----------+------------------------------+
|       | 844bbb45f440 | no     | Ubuntu trusty amd64 (20160906_03:49)      | x86_64 | 77.29MB  | Sep 6, 2016 at 5:04pm (UTC)  |
+-------+--------------+--------+-------------------------------------------+--------+----------+------------------------------+
Step 6.5 (mandatory): Launch LXC Container from Image

With the lxc launch command, a container is created from the image. Moreover, if the image is not available in the local repository, it will automatically download the image.

ubuntu@ubuntu-xenial:~$ lxc launch images:ubuntu/trusty myTrustyContainer
Creating myTrustyContainer
Retrieving image: 100%
Starting myTrustyContainer

If the image is already in the local repository, the Retrievien image line is missing and the container can start within seconds (~6-7 sec in my case).

Step 7 (optional): List running Containers

We can list the running containers with the lxc list command, similar to a docker ps -a for those, who know Docker:

ubuntu@ubuntu-xenial:~$ lxc list
+-------------------+---------+------+------+------------+-----------+
|       NAME        |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+-------------------+---------+------+------+------------+-----------+
| myTrustyContainer | RUNNING |      |      | PERSISTENT | 0         |
+-------------------+---------+------+------+------------+-----------+

Step 8: Run a Command on the LXC Container

Now we are ready to run our first command on the container:

ubuntu@ubuntu-xenial:~$ sudo lxc exec myTrustyContainer ls /
sudo: unable to resolve host ubuntu-xenial
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Step 9: Log into and exit the LXC Container

We can log into the container just by running the shell with the lxc exec command:

ubuntu@ubuntu-xenial:~$ sudo lxc exec myTrustyContainer bash
sudo: unable to resolve host ubuntu-xenial
root@myTrustyContainer:~# exit
exit
ubuntu@ubuntu-xenial:~$

The container can be exited just by issuing the “exit” command. Different from Docker containers, this will not stop the container.

Step 10: Stop the LXC Container

The container can be stopped with the sudo lxc stop command:

ubuntu@ubuntu-xenial:~$ sudo lxc stop myTrustyContainer
sudo: unable to resolve host ubuntu-xenial
ubuntu@ubuntu-xenial:~$ lxc list
+-------------------+---------+------+------+------------+-----------+
|       NAME        |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+-------------------+---------+------+------+------------+-----------+
| myTrustyContainer | STOPPED |      |      | PERSISTENT | 0         |
+-------------------+---------+------+------+------------+-----------+

Summary

We have discussed the differences of Docker and LXD. We have found that Docker focuses on application delivery, while LXD seeks to offer Linux virtual environments as systems.

We have provided the steps needed to get started with LXD by showing how to

  • install the software,
  • download images,
  • start containers from the images and
  • running simple Linux commands on the images.

Next steps:

Here is a list of possible next steps on the path to Docker in LXC:

  • Networking
  • Docker in LXC container
  • LXD: Integration into OpenStack
  • Put it all together

Appendix A: VirtualBox Installation Problems: “Setup Wizard ended prematurely”

  • Download the VirtualBox installer
  • When I start the installer, everything seems to be on track until I see “rolling back action” and I finally get this:
    “Oracle VM Virtualbox x.x.x Setup Wizard ended prematurely”

Resolution of the “Setup Wizard ended prematurely” Problem

Let us try to resolve the problem: the installer of Virtualbox downloaded from Oracle shows the exact same error: “…ended prematurely”. This is not a docker bug. Playing with conversion tools from Virtualbox to VMware did not lead to the desired results.

The Solution: Google is your friend: the winner is: https://forums.virtualbox.org/viewtopic.php?f=6&t=61785. After backing up the registry and changing the registry entry

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Network -> MaxFilters from 8 to 20 (decimal)

and a reboot of the Laptop, the installation of Virtualbox is successful.

Note: while this workaround has worked on my Windows 7 notebook, it has not worked on my new Windows 10 machine. However, I have managed to install VirtualBox on Windows 10 by de-selecting the USB support module during the VirtualBox installation process. I remember having seen a forum post pointing to that workaround, with the additional information that the USB drivers were installed automatically at the first time a USB device was added to a host (not yet tested on my side).

Appendix B: Vagrant VirtualBox Error message: “VT-x is not available”

Error:

If you get an error message during vagrant up telling you that VT-x is not available, a reason may be that you have enabled Hyper-V on your Windows 10 machine: VirtualBox and Hyper-V cannot share the VT-x CPU:

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'thesteve0/openshift-origin' is up to date...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
 default: Adapter 1: nat
 default: Adapter 2: hostonly
==> default: Forwarding ports...
 default: 8443 (guest) => 8443 (host) (adapter 1)
 default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "8ec20c4c-d017-4dcf-8224-6cf530ee530e", "--type", "headless"]

Stderr: VBoxManage.exe: error: VT-x is not available (VERR_VMX_NO_VMX)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole

Resolution:

Step 1: prepare your Windows machine for dual boot with and without Hyper-V

As Administrator, open a CMD and issue the commands

bcdedit /copy "{current}" /d "Hyper-V" 
bcdedit /set "{current}" hypervisorlaunchtype off
bcdedit /set "{current}" description "non Hyper-V"

Step 2: Reboot the machine and choose the “non Hyper-V” option.

Now, the vagrant up command should not show the “VT-x is not available” error message anymore.

1

Choosing the right IaaS Provider for a custom Appliance or: how hard is it to install from ISO in the cloud?


Which Cloud Infrastructure provider allows to install custom appliances via ISO? The short answer is: none of the IaaS market leaders Amazon Web Serviese (AWS), Microsoft Azure offer the requested functionality, but they offer the workaround to locally install the virtual machine (VM) and upload the VM to the cloud. The cheaper alternative DigitalOcean does not offer any of those possibilities.

At the end, I thought I have found  the perfect solution to my problem: Ravello Systems a.k.a. Oracle ravello is a meta cloud infrastructure provider (they call it “nested virtualization provider”), which is re-selling the infrastructure from other IaaS providers like Amazon AWS and Google Engine. They offer a portal that supports the installation of a VM from an ISO in the cloud. Details see below. They write:

2016.03.30-13_57_04-hc_001

However, Ravello was ignoring my request for a trial for more than two months.

Ravello’s trial seems to be open for companies only? I even told them that I am about to found my own company, this did not help.

2016.03.30-13_08_51-hc_001

If you are representing a large company and if you are offering them a prospect to earn a lot of money, they might be reacting differently in your case, though. Good luck.

I am back at installing the image locally and uploading it to Amazon AWS. Maybe this is the cheaper alternative, anyway. I am back at the bright side of life…

2016.03.30-13_06_53-hc_001

At the end, after more than 2 months, I have got the activation link. The ISO Upload tool has some challenges with HTTP proxies, but is seems to work now.

Document Versions

v1.0 (2016-03-14): initially published version
v1.1 (2016-03-21): added a note on ravello’s nested virtualization solution, which makes the solution suitable for VMware testing on public non-VMware clouds
v1.2 (2016-03-23): added a note of my problems of getting a trial account; I have created a service ticket.
v1.3 (2016-03-30): Ravello has chosen to close my ticket without helping me. I am looking for alternatives.
v1.4 (2016-04-09): After I have complained about the closed ticket, they wrote a note that they are sorry on 2016-03-30. However, I have still not got an account. I have sent a new email asking for the status today.
v1.5 (2016-05-25): I have received an activation link on May, 11th. It has taken more than 2 months to get it. I am not sure, if I am still interested…

The Use Case

Integration of high tech systems with legacy systems is fun. At least, it is fun, if you have easy access from you development machine to the legacy systems. In my case, I was lucky enough: the legacy systems I am dealing with are modern communication systems that can be run on VMware. Using a two year old software version of the system, I have run the legacy system on my development machine. With that I could run my integration software against the real target system.

But why have I used a two year old software version of the legacy system? That is, why: the most recent versions of that system have such a high demand on the virtual resources (vCPU, DRAM) that it has outgrown my development machine: it was quite a bit…

2016.03.14-19_48_01-hc_001

…overloaded.

Possible Solutions

How to deal with this? Some thoughts of mine are:

  • I could buy a new notebook with, say 64 GB RAM.2016.03.14-20_25_24-hc_001
    • this is an expensive option. Moreover, I am a road warrior type of developer and do a lot of coding in the train. Most notebooks with 64GB RAM are bulky and heavy and you need to take a power plant with you if you do not want to run out of energy during your trip.
  • I could develop a lightweight simulator that is mocking the behavior of the legacy system.
    • In the long run, I need to do something along those lines anyway: I want to come closer to Continuous Integration+Deployment process and for the automated tests in the CI/CD system, it is much simpler to run a simulator as part of the software than to run the tests against bulky legacy systems.
  • I could develop and test (including integration tests) in the IaaS cloud.

2016.03.14-18_57_14-hc_001

The Cloud Way of Providing a Test Environment

Yes, the IaaS cloud option is a particularly interesting one; especially, if development is done as a side job because:

  • I need to pay only for resources I use.
  • For most functional tests, I do not need full performance. I can go with cheaper, shared resources.
  • I can pimp up the legacy system and reserve resources for performance tests, while freeing up the resources again after finish of the test.
  • Last but not least, I am a cloud evangelist and therefore I should eat my own dog food (or drink my own champagne, I hope).

However: which are the potential challenges?

2016.03.14-19_01_14-hc_001

  1. Installation challenges of the legacy system in the cloud.
  2. How much do you pay for the VM, if it is shut down? Open Topic, but will not (yet) investigated in this blog post.
  3. How long does it take from opening the lid of the development notebook until I can access the legacy system? Open Topic, but will not (yet) investigated in this blog post.

First things first: in this post, I will concentrate on challenge 1.

The Cloud Way of installing (a custom appliance from ISO)

2016.03.14-19_04_03-hc_001

In my case, the legacy system must be installed from ISO. From my first investigation, it seems that this is a challenge with many IaaS providers. Let us have a closer look:

Comparison of IaaS Providers

2016.03.14-19_05_07-hc_001

  • DigitalOcean: they do not support the installation from ISO. See this forum post.
    • there is no workaround like local installation and upload of the image, see here. Shoot. 😦

2016.03.14-19_06_58-hc_001

  • AWS: same thing: no ISO installation support.
    1. For AWS, the workaround is to install the system locally and to upload and convert the VM. See this stackoverflow post.
      One moment: didn’t I say the legacy system is too large for my notebook? Not a good option. 😦
    2. Another workaround for AWS is to use a nested virtualization provider like ravello systems: they claim here that the installation of an AWS image from ISO is no problem.
      Note: ravello’s nested virtualization solution places an additional Hypervisor on top of AWS’ XEN hypervisor, in order to run VMware VMs on public clouds that do not support VMware VMs natively. This will not increase the performance, though and is intended for test environments only. However, this is exactly, what I am aiming at (for now).

Ravello claims: “With Ravello, uploading an ISO file is as simple as uploading your files to dropbox. Once the file is in your library in Ravello simply add the CD-ROM device to your VM and select your customer ISO file from your library.”2016.03.14-19_38_13-hc_001

2016.03.14-19_08_58-hc_001

  • Microsoft Azure: not fully clear…
    • I have found here the information that an ISO can be attached to an existing VM. I do not know, though, whether or not the VM can be installed from the ISO by booting from ISO.
    • you can create a local image in VHD format and upload it to the cloud. However, the only (convenient) way to create this image is to install the VM on Hyper-V. I do not have access to Hyper-V and I do not want to spend any time on this for now. 😦

Among those options, it seems like only AWS and ravello are possible feasible for me.

Even so, I need to take the risk caused by the fact that my legacy systems are supported on VMware only. However, this is a risk I need to accept, if I want to go with a low cost mainstream IaaS provider. A private cloud on dedicated VMware infrastructure is prohibitive with respect to effort and price.

Decision:

I have a more powerful notebook at home and I could install the image locally. However, I will give the meta IaaS provider Ravello Systems a try and I will install the legacy system via their overlay cloud. Within Ravello systems, I will choose AWS as the backend IaaS provider, because AWS is the number one IaaS provider (see this article pointing to the Gartner report) and therefore I want to gain some experience with AWS.

Note about the pricing comparison between AWS and ravello: I believe that ravello comes at higher rates (estimated 30-50%). But please do not take this for granted and calculate yourself, using the AWS monthly calculator and the ravello pricing page.

HowTo

More than 2 months after my application, I finally got an activation link. Looking for how to import the ISO, I have found this ravello link. However, the documentation is not good. They write:

To download the tool, click Download VM Import Tool on the Library page.

However, there is no Download VM Import Tool on the Library page. Instead, you can choose Library->Disk Images ->Import Disk Image in order to reach the import tool download page (or click this direct link).

After installing the GUI tool on Windows using the exe file, I am redirected to the browser login page of the tool:

2016.05.24-14_53_30-hc_001

If you are behind a proxy, you will receive the following connectivity problem error:

2016.05.24-14_59_19-hc_001

The link will lead here. The process is to create a config.properties file on a folder named .ravello in the user’s home directory (%HOME%\.ravello).

Note: be sure to use %HOME%\.ravello and not %USERPROFILE%\.ravello, if those two pathes differ in your case (in my case they do: %HOME% is my local Git directory on F:\veits\git).

The file config.properties needs to have following content:

[upload]
proxy_address = <ip address of proxy server>
proxy_port = <port on which the proxy server accepts connections>

The nasty thing is, that you need to kill the RavelloImageImportServer.exe task in case of Windows or the ravello-vm-upload process in case of Linux.

The problem is, that

  1. they do not tell you how to restart the process. In my case, I have found RavelloImageImportServer.exe on C:\Program Files (x86)\Ravello Systems\Ravello import utility. I have restarted it.
  2. Even though I have created the properties file, the import tool does not find the proxy configuration on %USERPROFILE%\.ravello. Crap! I have found out that the import tool is looking for %HOME%\.ravello instead, which has been set by my local git installation to be on F:\veits\git. I was close to giving up…

Finally, I have managed to upload the ISO:

2016.05.24-16_01_49-hc_001

From there, it should be possible to create an empty VM, attach the ISO on it and boot the VM from ISO…

No luck: after some time, the upload is stopped due to no apparent reason:

2016.05.24-20_48_42-hc_001

The pause button as well as the resume button are greyed out. No way to resume the upload. Well thought, but not so good implemented. Okay, the service is quite new. Let us see, how ravello works, if we give them a few additional months…

After connection to the Internet without HTTP proxy (my notebook was in standby for a while), I have seen, that I could not log into the local GUI upload tool anymore. The process was consuming a constant 25% of my dual core CPU. Workaround: renaming the config.properties file (or maybe remove/comment out its content), killing and restarting of the process brought back the GUI upload process to normal.

Summary

I have shortly investigated, which options I have to run a legacy system on an IaaS provider cloud network.

Before I found out that ravello’s service times are sub-optimal, I initially thought that the meta IaaS provider called Ravello Systems is the winner of this investigation:

2016.03.14-19_38_13-hc_001

However, I see following problems:

  • it has taken ravello more than two (!) months to provide me with an activation link.
  • An ISO or VM upload requires the installation of a local tool
  • the GUI tool has problems to handle HTTP proxies. I have followed their instructions, but I could not get it to work, initially. At the end, I have found out, that the tool is not looking in Maybe the tool is looking in %USERPROFILE%\.ravello, but in %HOME%\.ravello, which is a GIT home directory and does not match C:\Users\myusername in my case.
  • another problem might be that Ravello is running the VMware VMs on top of a Hypervisor layer, which in turn translates the VM to the underlying infrastructure. There is a high risk that this will work only for test labs with low CPU consumptions. This is to be tested.

In the short time I have invested into the investigation, I have found that

  1. ravello had seemed to be the best alternative, since the system can be installed in the cloud with
  2. A reader of my blog suggests to check out Vultr. Since ravello has its own drawbacks (service: long time to respond, longer time to help, GUI import tools seems to have weaknesses: I could not get it to work from behind a HTTP proxy, even if I follow the instructions), Vultr might be a real good alternative with low pricing.
  3. Amazon AWS is an alternative, if it is O.K. for you not to install from ISO, but to install locally and upload the created custom VM.

The following alternatives have major drawbacks:

  • Microsoft Azure requires the local installation of the VM using Hyper-V and I do not have such a system. I have not found a statement, whether it is possible to boot a Microsoft Azure VM from ISO (do you know that?).
  • DigitalOcean neither supports an installation from ISO, nor does it support the upload of custom VMs.

See also:

Next steps:

  • once the ISO is uploaded, create a VM and try to boot the VM from ISO.
  • Try out Vultr.

Update 2016-03-21: I have applied for a trial with ravello on March 17th, but no reaction so far, apart from the automatic email reply. I have opened a ticket yesterday and I got an email that they will come back to me…

Danger_No Entry

Update 2016-03-23: still waiting…

Update 2016-03-30: instead of helping me, Ravello’s support has sent an email that they did not get any response from me (response about what?) and they have closed the ticket, along with a link with the possibility to give feedback. My feedback was “not satisfied”. Let us see, how they react.

Update 2016-05-11: I have received the activation link, more than 2 months after my application. I have signed in although I do not know, if I am still interested. I have added the HowTo chapter, but I have failed to upload the ISO via a HTTP proxy, even though I have followed the instructions closely.

Meanwhile, I have signed up for a native AWS account. The intent of this blog was to find a provider that makes it more easy to install an image from ISO: I did not want to install locally, and then upload and convert the image, because my SSD disk is notoriously full. Ravello was the only alternative I had found in a quick Internet research. However, Ravello had failed to provide me with a valid registration within 2 months.

14

IT Automation Part I: Ansible “Hello World” Example using Ansible on Docker


This is part I of a little “Hello World” example using Ansible, an IT automation tool.

The post has following content:

  • Popularity of Ansible, Salt, Chef and Puppet
  • Installation based on Docker
  • Playbooks (i.e. tasks) and Inventories (i.e. groups of targets)
  • Remote shell script execution

As a convenient and quick way of Ansible installation on Windows (or Mac), we are choosing a Docker Ansible image on an Vagrant Ubuntu Docker Host that has won a java performance competition against CoreOS and boot2docker (see this blog post).

NEW: Try it out!

Get a feeling for Ansible in a real console without the need to install anything!

  1. Quick sign-in to Katacoda via Github, LinkedIn, Twitter Google or email
  2. click the console below or on https://www.katacoda.com/oliverveits/scenarios/ansible-bootstrap

Posts of this series:

  • Part I: Ansible Hello World (this post) with a comparison of Ansible vs. Salt vs. Chef vs. Puppet and a hello world example with focus on Playbooks (i.e. tasks), Inventories (i.e. groups of targets) and remote shell script execution.
  • Part II: Ansible Hello World reloaded with focus on templating: create and upload files based on jinja2 templates.
  • Part III: Salt Hello World example: same content as part I, but with Salt instead of Ansible
  • Part IV: Ansible Tower Hello World: investigates Ansible Tower, a professional Web Portal for Ansible

 

2015.11.18-13_40_14-hc_001

Versions

2015-11-09: initial release
2016-06-08: I have added command line prompts basesystem#, dockerhost# and container#, so the reader can see more easily, on which layer the command is issued.
2017-01-06: Added linked Table of Contents

Table of Contents

Why Ansible?

For the “hello world” tests, I have chosen Ansible. Ansible is a relatively new member of a larger family of IT automation toolsThis InfoWorld article from 2013 compares four popular members, namely Puppet, Chef, Ansible and Salt. Here you and here find more recent comparisons, if not as comprehensive as the InfoWorld article.

In order to explore the popularity of the software, let us look at a google trend analysis of those four tools (puppet/ansible/chef/salt + “automation”) (status November 2015; for a discussion of the somewhat more confusing recent results, please consult Appendix F below):

2015.11.09-13_52_20-hc_001

Okay, in the google trends analysis we can see that Ansible is relatively new and that is does not seem to replace Puppet, Chef or Salt. However, Ansible offers a fully maintained, RESTful API on the Ansible Web application called Ansible Tower (which comes at a cost, though).  Moreover, I have seen another article stating that Ansible is the very popular among docker developers. Since I have learned to love docker (it was not love at first sight), let us dig into Ansible, even though Puppet and Chef seems to be more popular in google searches.

For a discussion of REST and Web UI capabilities of the four tools, see Appendix D.

Ansible “Hello World” Example – the Docker Way

We plan to install Ansible, prepare Linux and Windows targets and perform simple tests like follows:

  1. Install a Docker Host
  2. Create an Ansible Docker Image
    • Download an Ansible Onbuild Image from Docker Hub
    • Start and configure the Ansible Container
    • Locally test the Installation
  3. Remote Access to a Linux System via SSH
    • –> Create a key pair, prepare the Linux target, access the Linux target
    • Note: we will use the Docker Host as Linux Target
  4. Working with Playbooks
    • Ansible “ping” to single system specified on command line
    • Run a shell script on single system specified on command line
  5.  Working with Inventory Files
    • Ansible “ping” to inventory items
    • Run a shell script on inventory items

1. Install a Docker Host

Are you new to Docker? Then you might want to read this blog post.

Installing Docker on Windows and Mac can be a real challenge, but no worries: we will show an easy way here, that is much quicker than the one described in Docker’s official documentation:

Prerequisites:
  • I recommend to have direct access to the Internet: via Firewall, but without HTTP proxy. However, if you cannot get rid of your HTTP proxy, read Appendix B.
  • Administration rights on you computer.
Steps to install a Docker Host VirtualBox VM:

Download and install Virtualbox (if the installation fails with error message “Setup Wizard ended prematurely” see Appendix A: Virtualbox Installation Workaround below)

1. Download and Install Vagrant (requires a reboot)

2. Download Vagrant Box containing an Ubuntu-based Docker Host and create a VirtualBox VM like follows (assumed a Linux-like system or bash on Windows):

(basesystem)$ mkdir ubuntu-trusty64-docker ; cd ubuntu-trusty64-docker
(basesystem)$ vagrant init williamyeh/ubuntu-trusty64-docker
(basesystem)$ vagrant up
(basesystem)$ vagrant ssh

Now you are logged into the Docker host and we are ready for the next step: to create the Ansible Docker image.

Note: I have experienced problems with the vi editor when running vagrant ssh in a Windows terminal. In case of Windows, consider to follow Appendix C and to use putty instead.

2. Create an Ansible Docker Image

1. Download an Ansible Onbuild Image from Docker Hub

In order to check that the docker host has Internet access, we issue following command:

dockerhost# docker search ansible

This command will lead to an error, if the you work behind a HTTP proxy, since we have not (yet) configured the docker host for usage behind a HTTP proxy. I recommend to get direct Internet access without HTTP proxy for now. However, if you cannot get rid of your HTTP proxy, read Appendix B.

Now we download the ansible image:

dockerhost# docker pull williamyeh/ansible:ubuntu14.04-onbuild

2. Start and configure the Ansible Container

dockerhost# docker run -it williamyeh/ansible:ubuntu14.04-onbuild /bin/bash

The -it (interactive terminal) flags starts an interactive session in the container. Now you are logged into the Docker Container and you can prepare the Ansible configuration files, namely the inventory file (hosts file) and the playbook.yml:

3. Locally test the Installation

container# ansible all -i 'localhost,' -u vagrant -c local -m ping

Note that the -i is a CSV list of target hosts and needs to end with a comma. With -u, we define the remote user, and the private key file needs to be specified. The response should look like follows:

localhost | success >> {
 "changed": false,
 "ping": "pong"
}

3. Remote Access to a Linux System via SSH

Now let us test the remote access to a Linux system.

We could perform our tests with any target system with a running SSH service and installed Python >v2.0 on /usr/bin/python (see the FAQs). However, the Ubuntu Docker host is up and running already, so why not use it as target system? In this case, the tested architecture looks like follows:

2015.11.18-15_59_31-hc_001

Ansible is agent-less, but we still need to prepare the target system: Ansible’s default remote access method is to use SSH with public key authentication. The best way is to create an RSA key pair on the Ansible machine (if not already available) and to add the corresponding public key as “authorized key” on the target system.

1. Create an RSA key pair on the Ansible container:
container# ssh-keygen -t rsa

and go through the list of question. For a proof of concept, and if you are not concerned about security, you can just hit <enter> several times. Here is a log from my case:

root@930360e7db68:/etc/ssh# ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
fe:eb:bf:de:30:d6:5a:8c:1b:4a:c0:dd:cb:5f:3b:80 root@930360e7db68
...

On the target machine (i.e. the ubuntu-trusty64-docker system), create a file name /tmp/ansible_id_rsa.pub and copy the content of ansible’s ~/.ssh/id_rsa.pub file into that file. Then:

dockerhost# cat /tmp/ansible_id_rsa.pub >> ~/.ssh/authorized_keys
2. test remote access via SSH:

Now, we should be able to access the target system from the ansible container using this key. To test, try:

container# ssh vagrant@192.168.33.10 -C "echo hello world"

This should echo a “hello world” to your screen. Here 192.168.33.10 is a reachable IP address of the docker host (issue ifconfig on the docker host to check the IP address in your case).  For troubleshooting, you can call ssh with the -vvv option.

3. Remote access via Ansible:

Check that Python version > 2.0 (better: > 2.4) is installed on your target machine. In our case of the ubuntu-trusty64-docker image, this is a pre-installed package and we get:

dockerhost# python --version
Python 2.7.6

Now also a remote Ansible connection should be possible from the container:

container# ansible all -i '192.168.33.10,' -u vagrant -m ping

which results in following output:

192.168.33.10 | success >> {
 "changed": false,
 "ping": "pong"
}

This was your first successful Ansible connection via SSH. Now let us also perform a change on the remote target. For that, we perform a remote shell command:

container# ansible all -i '192.168.33.10,' -u vagrant -m shell -a "echo hello world\! > hello_world"

This time the module is a “shell” and the module’s argument is a echo hello world command. We should get the feedback

192.168.33.10 | success | rc=0 >>

On the target, we can check the result with:

dockerhost# cat hello_world
hello world!

4. Working with Playbooks

This was your first remote shell action via Ansible. Now we want to have a look to playbooks, which are the Ansible way to document and automate the tasks.

1. Create a playbook

On the ansible container terminal, we create a playbook.yml file:

container# vi playbook.yml

and we add and save the following content:

---
# This playbook uses the ping module to test connectivity to Linux hosts
- name: Ping
  hosts: 192.168.33.10
  remote_user: vagrant 

  tasks: 
  - name: ping 
    ping: 
  - name: echo 
    shell: echo Hello World! > hello_world

Note: If you have problems with formatting of the characters in the terminal (I have experienced problems in a Windows terminal), then I recommend to use a putty terminal instead of using vagrant ssh. For that, see Appendix C.

Note also that the number of white spaces are relevant in a yml file. However, note that the ‘!’ does not need to be escaped in the playbook (it was necessare on the command line, though). Now, we perform following command on the Ansible container:

container# ansible-playbook -i '192.168.33.10,' playbook.yml

This time, the -u flag is not needed (it is ignored, if specified), since we have specified the user in the playbook. We get the following feedback:

PLAY [Ping] *******************************************************************

GATHERING FACTS ***************************************************************
ok: [192.168.33.10]

TASK: [ping] ******************************************************************
ok: [192.168.33.10]

TASK: [echo] ******************************************************************
changed: [192.168.33.10]

PLAY RECAP ********************************************************************
192.168.33.10 : ok=3 changed=1 unreachable=0 failed=0

We check the result on the target machine again:

dockerhost# cat hello_world
Hello World!

We see that the file hello_world was overwritten (“Hello World!” with capital letters instead of “hello world!”).

5. Working with Inventory Files

Instead of specifying singular hosts in the playbook.yml, Ansible offers the more elegant way to work with groups of machines. Those are defined in the inventory file.

More information about inventory files can be found the official documentation. However, note that this page describes new 2.0 features that do not work on the Docker image (currently ansible 1.9.4). See Appendix E for more details on the problem and for information on how to upgrade to the latest development build. The features tested in this blog post work on ansible 1.9.4, though; so you do not need to upgrade now.

1. Now we do the same as before, but we use an inventory file to define the target IP addresses:

container# vi /etc/ansible/hosts

and add the following lines:

[vagranthosts]
192.168.33.10

In the playbook.yml we replace 192.168.33.10 by a group name, e.g. vagranthosts

---
# This playbook uses the win_ping module to test connectivity to Windows hosts
- name: Ping
  hosts: vagranthosts
  remote_user: vagranthost

  tasks:
  - name: ping
    ping:
  - name: echo
    shell: echo HELLO WORLD! > hello_world

In order to see the difference, we also have changed the hello world to all capital letters.

Now we perform:

container# ansible-playbook -i /etc/ansible/hosts playbook.yml

Here, we have replaced the list of hosts by a reference to the inventory file.

The output looks like follows:

PLAY [Ping] *******************************************************************

GATHERING FACTS ***************************************************************
ok: [192.168.33.10]

TASK: [ping] ******************************************************************
ok: [192.168.33.10]

TASK: [echo] ******************************************************************
changed: [192.168.33.10]

PLAY RECAP ********************************************************************
192.168.33.10 : ok=3 changed=1 unreachable=0 failed=0

We check the result on the target machine again:

dockerhost# cat hello_world
HELLO WORLD!

We see that the file hello_world was overwritten again (all capital letters). Success!

Summary

We have shown, how you can download and run Ansible as a docker container on any machine (Windows in my case). We have prepared the Ansible container and the target for SSH connections and have shown, how to perform connectivity tests and shell scripts on the remote system. In addition, we have introduced Playbooks as a means to document and run several tasks by one command. Moreover, Inventory files were introduced in order to manage groups of target machines.

Next steps

Following topics are looked at in the next two parts of this series:

  • Part II: Ansible Hello World reloaded will show
    • how to upload files with Ansible
    • how to create dynamic file content with Ansible using jinja2 templates
    • bind it all together by showing a common use case with dynamic shell scripts and data files
  • Part III: Salt Hello World example: same content as part I, but with Salt instead of Ansible
  • Part IV: Ansible Tower Hello World: investigates Ansible Tower, a professional Web Portal for Ansible

Open: Window support of Ansible

Appendix A: Virtualbox Installation (Problems)

  • Download the installer. Easy.
  • When I start the installer, everything seems to be on track until I see “rolling back action” and I finally get this:
    “Oracle VM Virtualbox x.x.x Setup Wizard ended prematurely”

Resolution of the “Setup Wizard ended prematurely” Problem

Let us try to resolve the problem: the installer of Virtualbox downloaded from Oracle shows the exact same error: “…ended prematurely”. This is not a docker bug. Playing with conversion tools from Virtualbox to VMware did not lead to the desired results.

The Solution: Google is your friend: the winner is:https://forums.virtualbox.org/viewtopic.php?f=6&t=61785. After backing up the registry and changing the registry entry

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Network -> MaxFilters from 8 to 20 (decimal)

and a reboot of the Laptop, the installation of Virtualbox is successful.

Note: while this workaround has worked on my Windows 7 notebook, it has not worked on my new Windows 10 machine. However, I have managed to install VirtualBox on Windows 10 by de-selecting the USB support module during the VirtualBox installation process. I remember having seen a forum post pointing to that workaround, with the additional information that the USB drivers were installed automatically at the first time a USB device was added to a host (not yet tested on my side).

Appendix B: HTTP Proxy Configuration

If you need to work behind a HTTP proxy, you need to consider several levels that need to know of it:

  • the physical host for both, your browser as well as your terminal session (http_proxy and https_proxy variables) for successful vagrant init commands and download of the vagrant boxes.
  • the docker host (if it differs from the physical host) for both, in the docker configuration files as well as on the bash. Note that the configuration files differ between CoreOS, boot2docker and Ubuntu.
  • the docker client for the terminal session; needed for apt-get update+install.

Ubuntu Docker:

sudo vi /etc/default/docker

add proxy, if needed like follows (adapt the names and ports, so it fits to your environment):

export http_proxy='http://proxy.example.com:8080'
export https_proxy='http://proxy.example.com:8080'

then:

sudo restart docker

CoreOS:

sudo mkdir /etc/systemd/system/docker.service.d
sudo vi /etc/systemd/system/docker.service.d/http-proxy.conf

and add something like (adapt the names and ports, so it fits to your environment):

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:8080"

Then:

sudo reboot

or try:

sudo sysctl docker restart

Appendix C: How to use putty for accessing Vagrant Boxes

What is to be done:

  1. locate and convert the vagrant private key to ppk format using puTTYgen.
    1. locate the vagrant private key of the box. In my case, this is C:\Users\vo062111\.vagrant.d\boxes\williamyeh-VAGRANTSLASH-ubuntu-trusty64-docker\1.8.1\virtualbox\vagrant_private_key
    2. start puTTYgen -> Conversions -> import -> select the above path.
    3. press “Save private key” and save vagrant_private_key as vagrant_private_key.ppk
  2. In putty,
    1. create a connection to vagrant@127.0.0.1 port 2201 or port 2222 (the port vagrant uses is show in the terminal during “vagrant up”)
    2. specify the ppk key Connection->SSH->Auth->(om the right)Private key file for authentication
    3. Click on Session on the left menu and press Save
    4. Press Open and accept the RSA fingerprint -> you should be able to log in without password prompt. If there is still a password prompt, there is something wrong with the private key.

Appendix D: REST APIs of Ansible, Puppet, Chef and Salt

Here, I have made a quick research on the RESTful interfaces and Web UI Interfaces of Ansible, Puppet, Chef and Salt. I have not found this information on the  Feature comparison table on Wikipedia:

Appendix E: Install the latest Ansible Development Version

The Ansible version in the docker image has the problem that it has a version 1.9.4 (currently), but the Ansible documentation is describing the latest v2.0 features. E.g. in version 1.9.4, variables in the inventory file described in the documentation are ignored (see e.g. the example “jumper ansible_port=5555 ansible_host=192.168.1.50″) and this leads to a “Could not resolve hostname” error ; see also this stackoverflow post).

Here, we will show, how to install the latest Ansible version in the container. For that, run the container:

docker -it williamyeh/ansible:ubuntu14.04-onbuild /bin/bash

ansible --version

will result in an output similar to:

ansible 1.9.4

If you get a version >=2.0, you might not need to upgrade at all. In all other cased, perform the following steps:

If you are behind a proxy, perform sth. like:

export http_proxy=http://proxy.example.com:8080
export https_proxy=http://proxy.example.com:8080
apt-get update; apt-get install git
git clone git://github.com/ansible/ansible.git --recursive
cd ./ansible
source ./hacking/env-setup
ansible --version

should give you some output like:

ansible 2.0.0 (devel 9b9fb51d9d) last updated ...

Now also the v2.0 features should be available. If you want to update the version in future, you will need to perform the git command

git pull

In the /ansible directory.

(chapter added on 2016-04-11)

In order to explore the popularity of the software, we have looked at a google trend analysis of those four tools (puppet/ansible/chef/salt + “automation”) (status November 2015):

2015.11.09-13_52_20-hc_001

Note that the google trends result looks quite differently 5 months later (2016-04-11). Google seems to have changed their source data and/or algorithm. With the same search terms as we had used in Nov. 2015 (puppet/ansible/chef/salt + “automation”; note that the link works only, if you are logged into your google account), in April 2016 we got following non-convincing graph:

2016.04.11-17_59_42-hc_001

Especially the analysis of Salt’s and Chef’s popularity for 2011 and before does not look very convincing.

If we are searching for “Software” instead via this google trends link (works only, if you are logged into your google account), we get something like the following:

2016.04.11-18_32_34-hc_001

Also this data does not look reliable: according to Wikipedia’s Vagrant page, Vagrant’s initial version was March 2010. Why do we see so many search hits before that time? That is not plausible. The same with Puppet, which has started 2005 and has many hits on 2004.

To be honest, google trends analysis used to (at least) look reliable in November 2015, but it does not look reliable anymore. What a pity: I used to work a lot with google trends in the past for finding out, which technology is trending, but looking at the more recent results, I have lost the confidence that I can rely on the data. If you know an alternative to google trends, please add a comment to this blog post.

In any case; for the time after 2013, it looks like the popularity of Ansible is rising quickly (if we believe it).

13

What is Docker? An Introduction into Container Technology.


–Originally posted on this linkedIn post and now moved to WordPress–

Introduction

In this blog, I will give a short Introduction into container technologies and I will give reasons, why I have chosen to start proof of concepts with Docker and no other container technology.

For more blogs in this series on Docker, see here.

Why Containers?

Containers are the the new way how run application in the cloud. It is high time to gain hands on experience with container technologies. However, before I plunge into the installation part, I will investigate the benefits of container technologies over other virtualization technologies and compare the different types of containers.

The VMware Hype: a History

I remember being very much excited about the new virtualization trend in the early 2000s: with the free VMware Server 2.0, it was the first time I had the chance to install and test our cool new IP telephony system for up to 100.000 telephones on my laptop. There even was the prospect of sharing defined test environments by sending around VMs containing all the network and VMs you need for testing.

Okay, that dreams of sharing whole test environments never came true, partly because VMs consume a tremendous amount of hardware resources, especially, if you have several VMs encapsulated inside a VM. Still, Hypervisor virtualization has proven to have tremendous operational advantages over hardware servers: in our lab as well as in our SaaS productive environment, we could spin up a new host whenever needed without waiting for our managers to approve the ordering of new hardware and without waiting for the delivery of the new hardware. Cool stuff.

Containers: are they Game Changers?

So, why do we need containers? Both, containers as well as hypervisors offer a way to provide applications with an encapsulated, defined, and portable environment. However, because of the high resource cost and low performance on my laptop, I (as a hobby-developer) do not develop my applications on a virtual machine. Even if would decide to do so, the environment on my laptop and the one in production will differ: I need to convert the image from VMware Workstation format to ESXi format. Network settings need to be changed. Java code needs to be compiled. Many things can happen until my application has reached its final destination in the production cloud.

Containers promise to improve the situation: in theory, containers are much more lightweight, since you can get rid of the Hypervisor layer, the virtual Hardware layer and the guest OS layer.

containers are light-weight and have a better performance

This does not only promise good performance. I was able to show here that a containerized Rails web application runs 50% than the same web application running on Windows Rails.

And this, although the container is located on a Virtualbox Linux VM that in turn runs on Windows. The 50% performance gain holds true, even if we work with shared folders: i.e. even if both, the Rails code and database are located outside the container on a mapped folder on the Linux VM. Quite surprising.

Note, however, you need to be careful with the auto-share between the Linux VM and Windows C:\Users folder as described in the official Docker documentation, which will cause a performance drop by a factor of ~10 (!).

Container technology increasing the Portability?

Docker evangelists claim that Docker images will work the same way in every environment, be it on the developer’s laptop, be it on a production server. Never say again: “It does not work in production? It works fine on on my development PC, though”. If you are starting a container, you can be sure all libraries in compatible versions are included.

Never again say: “It does not work in production? It works fine on on my development PC, though!”

Okay, along the DDDocker series, you will see that this statement remains a vision to be fulfilled by certain careful measures. One of the topics I stumbled over, is the topic of network connectivity and HTTP proxies. Docker commands as well as Cluster technologies like CoreOS depend on Internet connectivity to public services like Docker Hub and Cluster Discovery Services per default. In case of the discovery services, this is aggravated by the fact that the discovery services protocol does not yet support HTTP proxies.

Container images still might work in one environment, but not in the other. It is still up to the image creator to reduce the dependency of external resources. This can be done by bundling the required resources with the application image.

Even with container technologies, it is still up to the image creator to offer a high degree of portability.

E.g. I have found on this blog post, that CoreOS (a clustered Docker container platform) requires a discovery agent that can be reached without passing any HTTP proxy. As a solution of this cluster discovery problem behind a HTTP proxy, a cluster discovery agent could be included in the image. Or the cluster discovery agent can be automatically installed and configured by automation tools like Vagrant, Chef, Puppet or Ansible. In both cases, the docker (CoreOS) host cluster does not need to access the public cluster discovery service anymore.

Containers are getting social

With the more lightweight images (compared to VMware/KVM/XEN & Co), and the offered public repositories like Docker Hub, the exchange of containerized applications might be a factor that can boost the acceptance of containers in the developer community. Docker Hub still needs to be improved (e.g. the size of the images is not yet visible; the image layers and status of the images often are intransparent to the user), but Docker Hub images can help developers to get more easily started with new, unknown applications, since the image comes along with all software and libraries needed.

Containers helping with Resiliency and horizontal Scalability

Containers help with Resiliency and with scalability. E.g. CoreOS and Atomic offer options to easily build cluster solutions with automatic restart of containers on another cluster node, if the original node fails. Moreover, emerging container orchestration systems like Kubernetes offer possibilities to horizontally scale up the containers and their applications by making sure that the right number of containers of the same type are always up and running. Kubernetes also promises to unify the communications between containers of one type to the rest of the world, so other applications talking to a scaled application does not need to care, how many instances of containers it is talking to.

Why Docker?

Before plunging into the container technology, I want to make sure that I spend my free time on the container technology that has the highest possible potential of becoming the de facto standard. Digging into Google, I found articles about “Docker vs. Rocket vs. Vagrant” and alike. Some examples are here and here.

Vagrant is mentioned often together with docker, but it is not a container technology, but a virtual machine configuration/provisioning tool.

I also find “docker vs LXC” pages on Google. What is LXC? According to stackoverflow, LXC refers to Linux’ capabilities of the Linux kernel (specifically namespaces and control groups) which allow sandboxing processes from one another. Flockport points out that LXC offers more like a lightweight virtual machine while Docker is designed for single applications. Docker will loose all “uncommitted” data after a reboot of the container while LXC will keep the data similar to virtual machines. See here for more details.

The CoreOS project, a lightweight, clustered docker host operating system, has developed a competing container technology called Rocket (say: rock-it). When reading the articles, I still do not know who has won the race: Docker or Rocket. However, the post here states

Yet when you see both Amazon and Google announcing Docker container services within weeks of each other, it’s easy to assume that Docker has won.

When looking into Google Trends, the assumption that docker has won the race, seems to be confirmed:

Let us assume that Docker is a good first choice. It is also supposed to be more mature than rocket, since it is available longer than rocket. Anyway, rocket does not seem to be available for Windows (see its GitHub page).*

* = Okay, later I found out that Docker is supported on Windows only by installing a Linux host VM, which in turn is hosting the containers. A docker client is talking to a docker agent on the Linux host, so it looks as if docker was supported on Windows. Still, I think, Docker is a good choice, considering its popularity.

Summary

This blog has discussed the advantages of container technologies like Docker and rocket when compared to the traditional virtualization techniques like VMware.

  • Docker images are more lightweight and, in a first Ruby on Rails test, have proven a 50% higher performance of a web service compared to running the same service on native Windows.
  • Container images have the potential to improve portability. However, external dependencies limit portability.
  • Container help with resiliency, scalability and ease of operations by making sure that a container image gets restarted on another host, if the original host dies. Moreover, container orchestration systems like Kubernetes by googlemake sure that the administrator-specified number of containers is up and running at all times, and that this set of containers appears to the outside world, as if it was a single application.
  • Google trends shows that docker is much more popular than other container technologies like rocket and LXC.

Those are all reasons why I have started to dig deeper into the container technology, and why I have chosen Docker to do so. An overview of those efforts can be found here.

0

Getting Started with Ansible


This is part 1 of a little “Hello World” example using Ansible, an IT automation tool.

The post has following content:

  • Popularity of Ansible, Salt, Chef and Puppet
  • Installation based on Docker
  • Playbooks (i.e. tasks) and Inventories (i.e. groups of targets)
  • Remote shell script execution

As a convenient and quick way of Ansible installation on Windows (or Mac), we are choosing a Docker Ansible image on an Vagrant Ubuntu Docker Host that has won a java performance competition against CoreOS and boot2docker (see this blog post).

Posts of this series:

  • Part 1: Ansible Hello World (this post) with a comparison of Ansible vs. Salt vs. Chef vs. Puppet and a hello world example with focus on Playbooks (i.e. tasks), Inventories (i.e. groups of targets) and remote shell script execution.
  • Part 2: Ansible Hello World reloaded with focus on templating: create and upload files based on jinja2 templates.
  • Part 3: Salt Hello World example: same content as part I, but with Salt instead of Ansible
  • Part 4: Ansible Tower Hello World: investigates Ansible Tower, a professional Web Portal for Ansible

 

2015.11.18-13_40_14-hc_001

Versions

2015-11-09: initial release
2016-06-08: I have added command line prompts basesystem#, dockerhost# and container#, so the reader can see more easily, on which layer the command is issued.
2017-01-06: Added linked Table of Contents

Table of Contents

Why Ansible?

For the “hello world” tests, I have chosen Ansible. Ansible is a relatively new member of a larger family of IT automation toolsThis InfoWorld article from 2013 compares four popular members, namely Puppet, Chef, Ansible and Salt. Here you and here find more recent comparisons, if not as comprehensive as the InfoWorld article.

In order to explore the popularity of the software, let us look at a google trend analysis of those four tools (puppet/ansible/chef/salt + “automation”) (status November 2015; for a discussion of the somewhat more confusing recent results, please consult Appendix F below):

2015.11.09-13_52_20-hc_001

Okay, in the google trends analysis we can see that Ansible is relatively new and that is does not seem to replace Puppet, Chef or Salt. However, Ansible offers a fully maintained, RESTful API on the Ansible Web application called Ansible Tower (which comes at a cost, though).  Moreover, I have seen another article stating that Ansible is the very popular among docker developers. Since I have learned to love docker (it was not love at first sight), let us dig into Ansible, even though Puppet and Chef seems to be more popular in google searches.

For a discussion of REST and Web UI capabilities of the four tools, see Appendix D.

Ansible “Hello World” Example – the Docker Way

We plan to install Ansible, prepare Linux and Windows targets and perform simple tests like follows:

  1. Install a Docker Host
  2. Create an Ansible Docker Image
    • Download an Ansible Onbuild Image from Docker Hub
    • Start and configure the Ansible Container
    • Locally test the Installation
  3. Remote Access to a Linux System via SSH
    • –> Create a key pair, prepare the Linux target, access the Linux target
    • Note: we will use the Docker Host as Linux Target
  4. Working with Playbooks
    • Ansible “ping” to single system specified on command line
    • Run a shell script on single system specified on command line
  5.  Working with Inventory Files
    • Ansible “ping” to inventory items
    • Run a shell script on inventory items

1. Install a Docker Host

Are you new to Docker? Then you might want to read this blog post.

Installing Docker on Windows and Mac can be a real challenge, but no worries: we will show an easy way here, that is much quicker than the one described in Docker’s official documentation:

Prerequisites:
  • I recommend to have direct access to the Internet: via Firewall, but without HTTP proxy. However, if you cannot get rid of your HTTP proxy, read Appendix B.
  • Administration rights on you computer.
Steps to install a Docker Host VirtualBox VM:

Download and install Virtualbox (if the installation fails with error message “Setup Wizard ended prematurely” see Appendix A: Virtualbox Installation Workaround below)

1. Download and Install Vagrant (requires a reboot)

2. Download Vagrant Box containing an Ubuntu-based Docker Host and create a VirtualBox VM like follows (assumed a Linux-like system or bash on Windows):

(basesystem)$ mkdir ubuntu-trusty64-docker ; cd ubuntu-trusty64-docker
(basesystem)$ vagrant init williamyeh/ubuntu-trusty64-docker
(basesystem)$ vagrant up
(basesystem)$ vagrant ssh

Now you are logged into the Docker host and we are ready for the next step: to create the Ansible Docker image.

Note: I have experienced problems with the vi editor when running vagrant ssh in a Windows terminal. In case of Windows, consider to follow Appendix C and to use putty instead.

2. Create an Ansible Docker Image

1. Download an Ansible Onbuild Image from Docker Hub

In order to check that the docker host has Internet access, we issue following command:

dockerhost# docker search ansible

This command will lead to an error, if the you work behind a HTTP proxy, since we have not (yet) configured the docker host for usage behind a HTTP proxy. I recommend to get direct Internet access without HTTP proxy for now. However, if you cannot get rid of your HTTP proxy, read Appendix B.

Now we download the ansible image:

dockerhost# docker pull williamyeh/ansible:ubuntu14.04-onbuild

2. Start and configure the Ansible Container

dockerhost# docker run -it williamyeh/ansible:ubuntu14.04-onbuild /bin/bash

The -it (interactive terminal) flags starts an interactive session in the container. Now you are logged into the Docker Container and you can prepare the Ansible configuration files, namely the inventory file (hosts file) and the playbook.yml:

3. Locally test the Installation

container# ansible all -i 'localhost,' -u vagrant -c local -m ping

Note that the -i is a CSV list of target hosts and needs to end with a comma. With -u, we define the remote user, and the private key file needs to be specified. The response should look like follows:

localhost | success >> {
 "changed": false,
 "ping": "pong"
}

3. Remote Access to a Linux System via SSH

Now let us test the remote access to a Linux system.

We could perform our tests with any target system with a running SSH service and installed Python >v2.0 on /usr/bin/python (see the FAQs). However, the Ubuntu Docker host is up and running already, so why not use it as target system? In this case, the tested architecture looks like follows:

2015.11.18-15_59_31-hc_001

Ansible is agent-less, but we still need to prepare the target system: Ansible’s default remote access method is to use SSH with public key authentication. The best way is to create an RSA key pair on the Ansible machine (if not already available) and to add the corresponding public key as “authorized key” on the target system.

1. Create an RSA key pair on the Ansible container:
container# ssh-keygen -t rsa

and go through the list of question. For a proof of concept, and if you are not concerned about security, you can just hit <enter> several times. Here is a log from my case:

root@930360e7db68:/etc/ssh# ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
fe:eb:bf:de:30:d6:5a:8c:1b:4a:c0:dd:cb:5f:3b:80 root@930360e7db68
...

On the target machine (i.e. the ubuntu-trusty64-docker system), create a file name /tmp/ansible_id_rsa.pub and copy the content of ansible’s ~/.ssh/id_rsa.pub file into that file. Then:

dockerhost# cat /tmp/ansible_id_rsa.pub >> ~/.ssh/authorized_keys
2. test remote access via SSH:

Now, we should be able to access the target system from the ansible container using this key. To test, try:

container# ssh vagrant@192.168.33.10 -C "echo hello world"

This should echo a “hello world” to your screen. Here 192.168.33.10 is a reachable IP address of the docker host (issue ifconfig on the docker host to check the IP address in your case).  For troubleshooting, you can call ssh with the -vvv option.

3. Remote access via Ansible:

Check that Python version > 2.0 (better: > 2.4) is installed on your target machine. In our case of the ubuntu-trusty64-docker image, this is a pre-installed package and we get:

dockerhost# python --version
Python 2.7.6

Now also a remote Ansible connection should be possible from the container:

container# ansible all -i '192.168.33.10,' -u vagrant -m ping

which results in following output:

192.168.33.10 | success >> {
 "changed": false,
 "ping": "pong"
}

This was your first successful Ansible connection via SSH. Now let us also perform a change on the remote target. For that, we perform a remote shell command:

container# ansible all -i '192.168.33.10,' -u vagrant -m shell -a "echo hello world\! > hello_world"

This time the module is a “shell” and the module’s argument is a echo hello world command. We should get the feedback

192.168.33.10 | success | rc=0 >>

On the target, we can check the result with:

dockerhost# cat hello_world
hello world!

4. Working with Playbooks

This was your first remote shell action via Ansible. Now we want to have a look to playbooks, which are the Ansible way to document and automate the tasks.

1. Create a playbook

On the ansible container terminal, we create a playbook.yml file:

container# vi playbook.yml

and we add and save the following content:

---
# This playbook uses the ping module to test connectivity to Linux hosts
- name: Ping
  hosts: 192.168.33.10
  remote_user: vagrant 

  tasks: 
  - name: ping 
    ping: 
  - name: echo 
    shell: echo Hello World! > hello_world

Note: If you have problems with formatting of the characters in the terminal (I have experienced problems in a Windows terminal), then I recommend to use a putty terminal instead of using vagrant ssh. For that, see Appendix C.

Note also that the number of white spaces are relevant in a yml file. However, note that the ‘!’ does not need to be escaped in the playbook (it was necessare on the command line, though). Now, we perform following command on the Ansible container:

container# ansible-playbook -i '192.168.33.10,' playbook.yml

This time, the -u flag is not needed (it is ignored, if specified), since we have specified the user in the playbook. We get the following feedback:

PLAY [Ping] *******************************************************************

GATHERING FACTS ***************************************************************
ok: [192.168.33.10]

TASK: [ping] ******************************************************************
ok: [192.168.33.10]

TASK: [echo] ******************************************************************
changed: [192.168.33.10]

PLAY RECAP ********************************************************************
192.168.33.10 : ok=3 changed=1 unreachable=0 failed=0

We check the result on the target machine again:

dockerhost# cat hello_world
Hello World!

We see that the file hello_world was overwritten (“Hello World!” with capital letters instead of “hello world!”).

5. Working with Inventory Files

Instead of specifying singular hosts in the playbook.yml, Ansible offers the more elegant way to work with groups of machines. Those are defined in the inventory file.

More information about inventory files can be found the official documentation. However, note that this page describes new 2.0 features that do not work on the Docker image (currently ansible 1.9.4). See Appendix E for more details on the problem and for information on how to upgrade to the latest development build. The features tested in this blog post work on ansible 1.9.4, though; so you do not need to upgrade now.

1. Now we do the same as before, but we use an inventory file to define the target IP addresses:

container# vi /etc/ansible/hosts

and add the following lines:

[vagranthosts]
192.168.33.10

In the playbook.yml we replace 192.168.33.10 by a group name, e.g. vagranthosts

---
# This playbook uses the win_ping module to test connectivity to Windows hosts
- name: Ping
  hosts: vagranthosts
  remote_user: vagranthost

  tasks:
  - name: ping
    ping:
  - name: echo
    shell: echo HELLO WORLD! > hello_world

In order to see the difference, we also have changed the hello world to all capital letters.

Now we perform:

container# ansible-playbook -i /etc/ansible/hosts playbook.yml

Here, we have replaced the list of hosts by a reference to the inventory file.

The output looks like follows:

PLAY [Ping] *******************************************************************

GATHERING FACTS ***************************************************************
ok: [192.168.33.10]

TASK: [ping] ******************************************************************
ok: [192.168.33.10]

TASK: [echo] ******************************************************************
changed: [192.168.33.10]

PLAY RECAP ********************************************************************
192.168.33.10 : ok=3 changed=1 unreachable=0 failed=0

We check the result on the target machine again:

dockerhost# cat hello_world
HELLO WORLD!

We see that the file hello_world was overwritten again (all capital letters). Success!

Summary

We have shown, how you can download and run Ansible as a docker container on any machine (Windows in my case). We have prepared the Ansible container and the target for SSH connections and have shown, how to perform connectivity tests and shell scripts on the remote system. In addition, we have introduced Playbooks as a means to document and run several tasks by one command. Moreover, Inventory files were introduced in order to manage groups of target machines.

Next steps

Following topics are looked at in the next two parts of this series:

  • Part II: Ansible Hello World reloaded will show
    • how to upload files with Ansible
    • how to create dynamic file content with Ansible using jinja2 templates
    • bind it all together by showing a common use case with dynamic shell scripts and data files
  • Part III: Salt Hello World example: same content as part I, but with Salt instead of Ansible
  • Part IV: Ansible Tower Hello World: investigates Ansible Tower, a professional Web Portal for Ansible

Open: Window support of Ansible

Appendix A: Virtualbox Installation (Problems)

  • Download the installer. Easy.
  • When I start the installer, everything seems to be on track until I see “rolling back action” and I finally get this:
    “Oracle VM Virtualbox x.x.x Setup Wizard ended prematurely”

Resolution of the “Setup Wizard ended prematurely” Problem

Let us try to resolve the problem: the installer of Virtualbox downloaded from Oracle shows the exact same error: “…ended prematurely”. This is not a docker bug. Playing with conversion tools from Virtualbox to VMware did not lead to the desired results.

The Solution: Google is your friend: the winner is:https://forums.virtualbox.org/viewtopic.php?f=6&t=61785. After backing up the registry and changing the registry entry

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Network -> MaxFilters from 8 to 20 (decimal)

and a reboot of the Laptop, the installation of Virtualbox is successful.

Note: while this workaround has worked on my Windows 7 notebook, it has not worked on my new Windows 10 machine. However, I have managed to install VirtualBox on Windows 10 by de-selecting the USB support module during the VirtualBox installation process. I remember having seen a forum post pointing to that workaround, with the additional information that the USB drivers were installed automatically at the first time a USB device was added to a host (not yet tested on my side).

Appendix B: HTTP Proxy Configuration

If you need to work behind a HTTP proxy, you need to consider several levels that need to know of it:

  • the physical host for both, your browser as well as your terminal session (http_proxy and https_proxy variables) for successful vagrant init commands and download of the vagrant boxes.
  • the docker host (if it differs from the physical host) for both, in the docker configuration files as well as on the bash. Note that the configuration files differ between CoreOS, boot2docker and Ubuntu.
  • the docker client for the terminal session; needed for apt-get update+install.

Ubuntu Docker:

sudo vi /etc/default/docker

add proxy, if needed like follows (adapt the names and ports, so it fits to your environment):

export http_proxy='http://proxy.example.com:8080'
export https_proxy='http://proxy.example.com:8080'

then:

sudo restart docker

CoreOS:

sudo mkdir /etc/systemd/system/docker.service.d
sudo vi /etc/systemd/system/docker.service.d/http-proxy.conf

and add something like (adapt the names and ports, so it fits to your environment):

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:8080"

Then:

sudo reboot

or try:

sudo sysctl docker restart

Appendix C: How to use putty for accessing Vagrant Boxes

What is to be done:

  1. locate and convert the vagrant private key to ppk format using puTTYgen.
    1. locate the vagrant private key of the box. In my case, this is C:\Users\vo062111\.vagrant.d\boxes\williamyeh-VAGRANTSLASH-ubuntu-trusty64-docker\1.8.1\virtualbox\vagrant_private_key
    2. start puTTYgen -> Conversions -> import -> select the above path.
    3. press “Save private key” and save vagrant_private_key as vagrant_private_key.ppk
  2. In putty,
    1. create a connection to vagrant@127.0.0.1 port 2201 or port 2222 (the port vagrant uses is show in the terminal during “vagrant up”)
    2. specify the ppk key Connection->SSH->Auth->(om the right)Private key file for authentication
    3. Click on Session on the left menu and press Save
    4. Press Open and accept the RSA fingerprint -> you should be able to log in without password prompt. If there is still a password prompt, there is something wrong with the private key.

Appendix D: REST APIs of Ansible, Puppet, Chef and Salt

Here, I have made a quick research on the RESTful interfaces and Web UI Interfaces of Ansible, Puppet, Chef and Salt. I have not found this information on the  Feature comparison table on Wikipedia:

Appendix E: Install the latest Ansible Development Version

The Ansible version in the docker image has the problem that it has a version 1.9.4 (currently), but the Ansible documentation is describing the latest v2.0 features. E.g. in version 1.9.4, variables in the inventory file described in the documentation are ignored (see e.g. the example “jumper ansible_port=5555 ansible_host=192.168.1.50″) and this leads to a “Could not resolve hostname” error ; see also this stackoverflow post).

Here, we will show, how to install the latest Ansible version in the container. For that, run the container:

docker -it williamyeh/ansible:ubuntu14.04-onbuild /bin/bash

ansible --version

will result in an output similar to:

ansible 1.9.4

If you get a version >=2.0, you might not need to upgrade at all. In all other cased, perform the following steps:

If you are behind a proxy, perform sth. like:

export http_proxy=http://proxy.example.com:8080
export https_proxy=http://proxy.example.com:8080
apt-get update; apt-get install git
git clone git://github.com/ansible/ansible.git --recursive
cd ./ansible
source ./hacking/env-setup
ansible --version

should give you some output like:

ansible 2.0.0 (devel 9b9fb51d9d) last updated ...

Now also the v2.0 features should be available. If you want to update the version in future, you will need to perform the git command

git pull

In the /ansible directory.

(chapter added on 2016-04-11)

In order to explore the popularity of the software, we have looked at a google trend analysis of those four tools (puppet/ansible/chef/salt + “automation”) (status November 2015):

2015.11.09-13_52_20-hc_001

Note that the google trends result looks quite differently 5 months later (2016-04-11). Google seems to have changed their source data and/or algorithm. With the same search terms as we had used in Nov. 2015 (puppet/ansible/chef/salt + “automation”; note that the link works only, if you are logged into your google account), in April 2016 we got following non-convincing graph:

2016.04.11-17_59_42-hc_001

Especially the analysis of Salt’s and Chef’s popularity for 2011 and before does not look very convincing.

If we are searching for “Software” instead via this google trends link (works only, if you are logged into your google account), we get something like the following:

2016.04.11-18_32_34-hc_001

Also this data does not look reliable: according to Wikipedia’s Vagrant page, Vagrant’s initial version was March 2010. Why do we see so many search hits before that time? That is not plausible. The same with Puppet, which has started 2005 and has many hits on 2004.

To be honest, google trends analysis used to (at least) look reliable in November 2015, but it does not look reliable anymore. What a pity: I used to work a lot with google trends in the past for finding out, which technology is trending, but looking at the more recent results, I have lost the confidence that I can rely on the data. If you know an alternative to google trends, please add a comment to this blog post.

In any case; for the time after 2013, it looks like the popularity of Ansible is rising quickly (if we believe it).