0

Jenkins Part 3.2: Trigger a downstream Job or Workflow with Hand-over of Parameters


This blog post will lead you through the steps how to trigger a downstream Freestyle job or Pipeline workflow from an upstream Freestyle project. We will also show how to pass a parameter from Freestyle project to the downstream workflow or job.

In the next blog post we will make use of this method and show how it can be used to trigger a Pipeline workflow via a trigger mechanism that is supported for Freestyle projects only.

Tools and Versions used

Tested with Jenkins v2.32.2 and v2.46.2

Step Zero: Access or Install a Docker Host

If you have no Docker host available, you may consider to either

  1. (easiest procedure) start this Katacoda scenario. Step 1 of thei blog post below is identical with step 4 of the Katacoda scenario. After performing this step, the Katacoda terminal and Jenkins web page can be used to perform all other steps described below.
  2. (more work) or install a Docker host by using Vagrant and Virtualbox as described in Step 1 of the blog post “Installation of Jenkins the Docker way“. While this is more work to to, the advantage of this option is that you can keep the Jenkins Home directory persistently on your local PC.

The commands we will use in this blog post are chosen in a way that will work in both environments.

Note: the Katacoda environment does not support the mapping of volumes to the Docker host via -v option. If you wish to keep the Jenkins Home directory for later use, you may consider to opt for option 2 and replace Step 1 and 2 by the steps found in the blog post Installation of Jenkins the Docker way. However, using Katacoda and the steps below is the quickest way to reach our goal to test the Jenkins Pipeline plugin.

Step 1: Run a pre-configured Jenkins Image

In order to skip some steps you might have seen already in part 1 and part 7 of this series, we will start a pre-configured Jenkins Docker image like follows:

docker run -d --rm --name jenkins \
       -p 8080:8080 -p 50000:50000 \
       oveits/jenkins:2.46.2-alpine-nologin-with-maven-git-pipelines

You can load the Jenkins’ dashboard by opening a browser and navigating to http://localhost:8080 (or to the link specified within the Katacoda example).

The used image is prepared to skip any login credentials. Maven, Git and Pipelines Plugins are installed and configured.

Note: if you want to start from scratch, consider to follow part 1 of this series. The installation and configuration of Maven, Git and Pipelines Plugins are described on part 7 of this series.

Step 2: Add Plugin: “Parameterized Trigger plugin”

-> 

-> 

-> 

-> 

-> 

->

Step 3: Create downstream Pipeline Project

Let us now create our a downstream Pipeline workflow job.

-> New Item

->  Triggered Pipeline

-> Pipeline (Jenkins Add New Item)

-> 

-> We scroll down to the Pipeline Script section and choose “try sample pipeline” -> Hello World

-> Save (Jenkins Configure)

Step 4: Create and Configure an upstream Freestyle Project

Why using a Freestyle project to trigger a Pipeline project? The reason is that some trigger plugins do not support pipelines yet. E.g. I was trying to use the Bitbucket Pullrequest Builder Plugin within a pipeline project, but I got a java traceback. The same triggering mechanism works for Freestyle projects, though. The idea now is to use a Freestyle trigger mechanism like the Bitbucket Pullrequest Builder Plugin (or any trigger mechanism supported by Freestyle projects) and to use the Parameterized Trigger plugin to trigger a Pipeline from the Freestyle project. Any Parameter available in the Freestyle project can be transferred and re-used in the Pipeline project, as we will show in our simple example:

-> New Item

-> 

-> Freestyle Project

-> OK

-> 

-> Build-> Trigger/call builds on other projects -> Build Triggers -> Triggered Pipeline

Step 5: Add Parameter

For testing how to pass a parameter from Freestyle to triggered Pipeline project, let us define a myparam parameter like follows:

-> Add parameters -> Predefined parameters

-> now we add “myparam = myvalue” in the Parameters field:

Step 6: Define Parameter on Triggered Pipeline

The parameter we have defined in the Freestyle project needs to be caught on the triggered Pipeline project. For that, the Pipeline Project needs to be configured to be parameterized in the configure section:

-> Configure

-> 

In the Pipeline script section, we replace the “Hello World” echo by following code, which will demonstrate three ways to use the parameter that has been passed between Freestyle and Pipeline project:

node {
 echo "Hello ${env.myparam}"
 echo "Hello " + myparam
 echo "Hello ${myparam}"
}

Step 7: Build Freestyle Project

This yields following result on the “Triggered Pipeline” project, when we “Build now” the Freestyle job:

Freestyle Project -> Build Now

Step 8: Review Results on Pipeline Project

-> Pipeline project

-> 

As we can see, all three syntax samples work.

Excellent! Thump up!

Summary

At this point, we have verified, that

  • A Freestyle job can trigger a Pipeline workflow
  • a parameter can be passed from the Freestyle job to the Pipeline workflow

Next Steps

In the next blog post we will show how to use this method to trigger a Pipeline workflow from a pull request on a BitBucket/Stash Git repository, despite of the fact that the corresponding plugin is not supported for Pipeline workflows, currently (v2.46.2).

 

1

Jenkins Part 7: Pipeline as Code


In this blog post, we will introduce the Jenkins Pipelines, an increasingly popular plugin that helps defining versatile Jenkins workflows as Groovy code.

Traditionally, Jenkins is administered by a graphical Web interface. Now, the Pipeline plugin allows teams and DevOps to manage workflow definitions within source control, giving them the ability to clone, change, review and iterate upon the continuous delivery pipeline. Moreover, the plugin offers teams a nice interface to view the stages of each workflow, like follows:

More advantages of Jenkins Pipelines are discussed in the section “Why is the Jenkins Pipeline plugin so popular?” below.

Tl;dr

If you wish, you can plunge directly on the command line and Jenkins Web interface in this Interactive Hands-on Example powered by Katacoda: Jenkins Pipelines (free sign-up):

Have fun!

Why is the Jenkins Pipeline Plugin so popular?

The popularity of the Pipeline plugin can be seen in its usage chart:

2017-05-20 23_38_04-Pipeline Plugin - Jenkins - Jenkins Wiki

Why am I using the Pipeline plugin? For me, the number one reason is that I can put the workflow under source control. Another reason is: the workflows can be organized in stages, whose results can be observed in the nice graphical interface seen above. However, the Pipeline plugin has more advantages than that (as pointed out here):

  • Code: Pipeline users can put workflows under source control (e.g. git), so they can add edit, review and
  • Durable: a Task that is interrupted by a planned or unplanned restart of the server is handled gracefully by the Durable Task Plugin included in the Pipeline Plugin
  • Interactive/Pausable: manual steps can be added to pipeline workflows
  • Versatile: you can create complex workflows including workflow forks and joins
  • Extensible: Pipeline users can create and add custom extension to the Pipeline domain specific language (DSL)

Enough theory for now. Time to get some hands-on experience.

Tools and  Versions used

Step Zero: Access or Install a Docker Host

If you have no Docker host available, you may consider to either

  1. start the Katacoda scenario of the Tl;dr section above (quick and without any technical challenges)
  2. or install a Docker host by using Vagrant and Virtualbox as described in Step1 of the blog post “Installation of Jenkins the Docker way“. While more work to to, the advantage of this option is that you can keep the Jenkins Home directory persistently on your local PC.

The commands we will use in this blog post are chosen in a way that will work in both environments.

Note: the Katacoda environment does not support the mapping of volumes to the Docker host via -v option. If you wish to keep the Jenkins Home directory for later use, you may consider to opt for option 2 and replace Step 1 and 2 by the steps found in the blog post Installation of Jenkins the Docker way. However, using Katacoda and the steps below is the quickest way to reach our goal to test the Jenkins Pipeline plugin.

Step 1: Start the Jenkins Container

First we start the container in detached mode with a tail to a log file we will use later. This allows us to prepare the Jenkins environment before we start the application:

(dockerhost)$ docker run -d -u root --rm --name jenkins \
    -p 8080:8080 -p 50000:50000 \
    --entrypoint bash \
    jenkins:2.46.2-alpine \
    -c "tail -F /jenkins.log

Step 2: Clone a prepared Jenkins Home Directory and run Jenkins

With the next command, we clone a Jenkins Home directory into the container, before we start the Jenkins application. The cloned Jenkins Home directory has been prepared to allow us using the Jenkins Web Interface without the need to log in:

(dockerhost)$ docker exec -d jenkins \
    bash -c 'git clone https://github.com/oveits/jenkins_home_alpine \
        && export JENKINS_HOME=$(pwd)/jenkins_home_alpine \
        && java -jar /usr/share/jenkins/jenkins.war 2>&1 1>/jenkins.log &

Now you should check, whether jenkins.war was started by issuing the command

(dockerhost)$ docker exec jenkins ps -ef

Katacoda has a good connection

Step 3: Install Pipeline and Git Plugins

In this step, we install the Pipeline and Git plugins like follows:

Manage Jenkins -> Manage Plugins -> Available -> Search Pipeline -> check Pipeline -> Search GitHub Plugin -> check -> Install without Reload

When ready, -> Go back to the top page

Or more elaborated:

  1. Within the Dashboard, select Manage Jenkins on the left.
  2. On the Configuration page, select Manage Plugins.
  3. Manage Plugins page will give you a tabbed interface. Click Available and wait three seconds to view all the Jenkins plugins that can be installed.
  4. Using the search box, search for Github plugin and choose the plugin via checkbox.
  5. While on this page, search for Pipeline and choose the plugin via checkbox.
  6. Click Install without Restart at the bottom.
  7. The plugins will now be downloaded and installed. Once complete, click the link Go back to the top page.

Now is the time to configure Maven and Git.

Step 4: Prepare Maven Usage

Maven must be prepared for first usage like follows:

Manage Jenkins -> Global Tool Configuration -> Maven -> Maven Installations -> Add Maven -> Name M3 -> Install automatically -> Apply

…or more elaborated:

  1. Once again, select Manage Jenkins.
  2. Select Global Tool Configuration.
  3. At the bottom, there is a button called Maven installations…. Click it.
  4. Choose the name M3.
  5. Click the Install automatically checkbox and keep the option Install from Apache with version 3.5.0.
  6. Click Apply at the bottom of the page.

Now Maven is ready for usage.

Step 5: Prepare Git Usage

Also Git needs to be prepared for first usage.

In short:

Manage Jenkins -> Global Tool Configuration -> Git -> Git installations -> keep Name “Default” -> Install automatically -> Add Installer -> Run Shell Command -> no label -> Command: ‘which git || (apk update && apk upgrade &&  apk add –no-cache git)’ -> Tool Home: /usr/bin/git -> Save

…or more elaborated:

  1. On the Manage Jenkins -> Global Tool Configuration page, find the section Git -> Git installations -> Git.
  2. Keep the name Default and check the Install automatically checkbox.
  3. From the Add Installer dropdown menue, choose Run Shell Command.
  4. Copy the command which git || (apk update && apk upgrade && apk add –no-cache git) into the Command box. This command is valid for the chosen alpine image only and must be adapted for other Linux distributions.
  5. In the Tool Home box, enter /usr/bin/git.
  6. Click Save at the bottom of the page.

Now Git is ready for usage.

Step 6: Replacement of Steps 1-5: Start a Pre-installed Jenkins Image

You can replace steps 1 to 5 by the current step. If you already have performed steps 1 to 5, directly to to step 7. Otherwise, follow the instructions how to start a pre-installed, pre-configured Jenkins image.

Step 6.1: Stop and remove any containers named “jenkins”, if required:

For the case you already have started a Docker container named “jenkins”, you need to stop and remove the container first:

docker stop jenkins; docker rm jenkins

Step 6.2: Download and start a pre-configured Jenkins container.

docker run -d --rm --name jenkins \
-p 8081:8080 -p 50000:50000 \
oveits/jenkins:2.46.2-alpine-nologin-with-maven-git-pipelines

If you get an error that a port is occupied already, change the port on the left-hand side of the ‘:’ separator.

You can load the Jenkins’ dashboard via the URL http://localhost:8081 (or different port, if chosen so).

Step 7: Create Pipeline Workflow

Let us now create our first Pipeline workflow job.

-> New Item

->  Enter an item name: Pipeline Hello World

-> Pipeline (Jenkins Add New Item)

-> 

We scroll down to the Pipeline Script section and choose “try sample pipeline” -> GitHub + Maven

-> 

-> Save (Jenkins Configure)

At this point we can review the script:

node {
   def mvnHome
   stage('Preparation') { // for display purposes
      // Get some code from a GitHub repository
      git 'https://github.com/jglick/simple-maven-project-with-tests.git'
      // Get the Maven tool.
      // ** NOTE: This 'M3' Maven tool must be configured
      // **       in the global configuration.           
      mvnHome = tool 'M3'
   }
   stage('Build') {
      // Run the maven build
      if (isUnix()) {
         sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package"
      } else {
         bat(/"${mvnHome}\bin\mvn" -Dmaven.test.failure.ignore clean package/)
      }
   }
   stage('Results') {
      junit '**/target/surefire-reports/TEST-*.xml'
      archive 'target/*.jar'
   }
}

We can see that the script defines three stages ‘Preparation’, ‘Build’ and ‘Results’:

  • In the Preparation stage, a git repository is cloned.
  • In the ‘Build’ stage, Maven performs unit tests and will create a Java archive artifact (JAR).
  • In the last stage, the JUnit test results are collected and reported.

Step 8: Run Pipeline Workflow

Now is the time to run our first pipeline workflow. For that, we click on:

-> Build Now

Jenkins Pipeline Stage View

You can click on the Pipeline field, and you are offered to view the logs:

You will see, that there are some downloads running in the background, when running the build the first time:

-> #1 -> Console

-> Press F5: A Test Result Trend will be shown:

SUCCESS!

Excellent! Thump up!

Summary

In this blog post, we have learned how to use Groovy code in order to define Jenkins workflows with so-called Jenkins Pipelines. Pipelines allow to define workflow stages. Within a simple Hello World example, we have downloaded, compiled and tested Java code. The JUnit stage has helped us to collect historic test reports.

Next Steps

Appendix: Create the Docker Image used in Step 6 to skip Steps 1 to 5

Here we describe how the Docker image was created that allows us to skip steps 1-5:

docker run -d -u root --name jenkins \
 -p 8080:8080 -p 50000:50000 \
 --entrypoint bash \
 jenkins:2.46.2-alpine \
 -c "tail -F /jenkins.log"
docker exec -d jenkins \
 bash -c 'git clone https://github.com/oveits/jenkins_home_alpine \
 && export JENKINS_HOME=$(pwd)/jenkins_home_alpine \
 && java -jar /usr/share/jenkins/jenkins.war 2>&1 1>/jenkins.log &'

Perform the manual steps 1 to 5.

docker stop jenkins
docker commit jenkins newjenkinsimage

Create a new container with the correct entrypoint and CMD:

docker run -d --entrypoint "bash" -p 8080:8080 -p 50000:50000 --name jenkins2 newjenkinsimage -c "JENKINS_HOME=/jenkins_home_alpine java -jar /usr/share/jenkins/jenkins.war"

For commiting the changes, we stop the container, login to Docker Hub and commit + push the changed image:

docker stop jenkins2
docker login

Add your user credentials of Docker Hub here…

Then:

docker commit jenkins2 oveits/jenkins:2.46.2-alpine-nologin-with-maven-git-pipelines
docker push oveits/jenkins:2.46.2-alpine-nologin-with-maven-git-pipelines

Here you need to exchange oveits by your own Docker Hub user name.

Now the image can be used by commands like e.g.

docker run <options> <image> <CMD>
1

Jenkins Part 6: Automated Cross Browser Testing via BrowserStack


With the BrowserStack cloud-based solution, there is no need to buy many different hardware types for testing your web site for many different mobile devices and operating systems. In this blog post, we will learn how to integrate BrowserStack-based automated cross browser tests into a continuous integration workflow controlled by the pupular Jenkins tool.

Jenkins + BrowserStack

First we will demonstrate how to use BrowserStack manually, before we automate the browser tests with the help of a Protractor Github example from BrowserStack. You will need to sign into a BrowserStack trial account with 30 minutes free manual testing and 100 minutes free automated testing. For this tutorial, we will need less than 6 minutes automated testing time.

Moreover, we will integrate the BrowserStack based tests into a Jenkins build job. At the end we will generate individual and trend Jenkins test reports with the help of the Jasmine reporting tool.

Note: The difference to my previous blog post is, that I have concentrated on Protractor without Gulp on Ubuntu this time. More importantly, I have added the Jenkins integration. In addition, you will find the description of many possible errors and their resolution in the appendix.

Table of Contents

Tools and Versions used

  • Vagrant 1.8.6
  • Virtualbox 5.0.20
  • Docker 1.12.1
  • Jenkins 2.32.2
  • Job DSL Plugin 1.58
  • for Windows: GNU bash, version 4.3.42(5)-release (x86_64-pc-msys)

Prerequisites (for the Jenkins part):

  • Free DRAM for the a Docker Host VM >~ 4 GB
  • Docker Host is available. If not, follow the “Prerequisite Step” below.
  • Tested with 2 vCPU (1 vCPU might work as well).

Getting Acquainted with BrowserStack

After signing up for a BrowserStack account, you get a 30 minute free live testing session. Start a local browser, and connect to the BrowserStack start URL. You will be asked to install a Browser plugin, which will take the role of a BrowserStack client.

You can choose any of the many operating systems and browser types

2017-03-04-17_14_58-dashboard

Note that you can interrupt the session any time by clicking Stop on the left (I had overlooked that, so I have wasted most of my 30 minutes time…)

Now you type in the URL you want to load:

Jenkins running on an iOS Simulator on BrowserStack

As you can see, I have typed in localhost:8080 on the remote iOS simulator running on the BrowserStack cloud. However, the browser is loading the Jenkins server page, which is running on my local notebook. The browser does not try to really load localhost (i.e. the iOS the browser is running on). Instead the HTTP request is directed to the locally running Chrome plugin, which is then resolving the DNS name “localhost” locally. This is called local testing, which we will explore in more detail now, before we start our step by step guide.

About BrowserStack Local Testing

Establishing a Tunnel

Local testing means, that Jenkins is connecting to BrowserStack.com via a tunnel the browser is running in the cloud, but all traffic from the browser to the system under test is relayed by the local BrowserStack client running on the Jenkins server:

2017-02-28-18_41_25
Steps to establish a tunnel between BrowserStack client and the repeater/proxy in the BrowserStack Cloud

Local Testing

2017-02-28-18_46_10
Steps to run a browser in the BrowserStack Cloud. All requests from the browser are proxied by the repeater and the BrowserStack client before being sent to the local system under test.

Prerequisite Step: Create a Docker Host via Vagrant

For the case, you do not have a Docker host at hand, you may want to follow Step 1 and 2 of part 1 of my Jenkins tutorial series. After having tried out many options to install Docker, the Vagrant way of installing a Docker is my favorite…

Part 1: Automatic Testing via BrowserStack

As an introduction, this part will show how to perform automated BrowserStack testing from command line without the need to use Jenkins.

Part 1 of this blog post is not a prerequisite to run part 2, which performs similar steps (and more) in the Jenkins way.

Step 1.1: Sign up for BrowserStack

For completing the steps of this tutorial, you need to sign up for a BrowserStack account. Pricing information can be found here. However, for completing the tasks of this tutorial, I did not need to sign up for any of the paid plans.

Step 1.2: Run Ubuntu Docker Container

I have looked for a simple Protractor example and I have found BrowserStack’s Protractor example on GitHub. Let us run it on an Ubuntu 16.04 Docker container, since the official Jenkins Dockerhub image seems to be based on a system that understands apt-get (see the Docker image layer visualizer).

Let us start a recent Ubuntu 16.04 container:

(dockerhost)$ sudo docker run -it ubuntu:16.04 bash 
(container)# mkdir /app; cd /app

Step 1.3: Install Node.js, NPM and GIT

We will need to install Node.js, NPM and GIT:

(container)$ apt-get update && apt-get install -y nodejs npm git
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial/main Sources [1103 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial/restricted Sources [5179 B]
Get:6 http://archive.ubuntu.com/ubuntu xenial/universe Sources [9802 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1558 kB]
Get:8 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9827 kB]
Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/main Sources [296 kB]
Get:11 http://archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [2815 B]
Get:12 http://archive.ubuntu.com/ubuntu xenial-updates/universe Sources [176 kB]
Get:13 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [623 kB]
Get:14 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [12.4 kB]
Get:15 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [546 kB]
Get:16 http://archive.ubuntu.com/ubuntu xenial-security/main Sources [75.0 kB]
Get:17 http://archive.ubuntu.com/ubuntu xenial-security/restricted Sources [2392 B]
Get:18 http://archive.ubuntu.com/ubuntu xenial-security/universe Sources [27.0 kB]
Get:19 http://archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages [282 kB]
Get:20 http://archive.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [12.0 kB]
Get:21 http://archive.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [113 kB]
Fetched 24.9 MB in 3min 13s (129 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
 binutils build-essential bzip2 ca-certificates cpp cpp-5 dpkg-dev fakeroot file g++ g++-5 gcc gcc-5 git-man gyp
 ifupdown iproute2 isc-dhcp-client isc-dhcp-common javascript-common krb5-locales less libalgorithm-diff-perl
 libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan2 libasn1-8-heimdal libatm1 libatomic1 libbsd0 libc-dev-bin
 libc6-dev libcc1-0 libcilkrts5 libcurl3-gnutls libdns-export162 libdpkg-perl libedit2 liberror-perl libexpat1
 libfakeroot libffi6 libfile-fcntllock-perl libgcc-5-dev libgdbm3 libgmp10 libgnutls30 libgomp1 libgssapi-krb5-2
 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhogweed4 libhx509-5-heimdal
 libicu55 libidn11 libisc-export160 libisl15 libitm1 libjs-inherits libjs-jquery libjs-node-uuid libjs-underscore
 libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2 liblsan0 libmagic1 libmnl0
 libmpc3 libmpfr4 libmpx0 libnettle6 libp11-kit0 libperl5.22 libpopt0 libpython-stdlib libpython2.7-minimal
 libpython2.7-stdlib libquadmath0 libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db
 libsqlite3-0 libssl-dev libssl-doc libssl1.0.0 libstdc++-5-dev libtasn1-6 libtsan0 libubsan0 libuv1 libuv1-dev
 libwind0-heimdal libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 libxext6 libxmuu1 libxtables11 linux-libc-dev make
 manpages manpages-dev mime-support netbase node-abbrev node-ansi node-ansi-color-table node-archy node-async
 node-block-stream node-combined-stream node-cookie-jar node-delayed-stream node-forever-agent node-form-data
 node-fstream node-fstream-ignore node-github-url-from-git node-glob node-graceful-fs node-gyp node-inherits node-ini
 node-json-stringify-safe node-lockfile node-lru-cache node-mime node-minimatch node-mkdirp node-mute-stream
 node-node-uuid node-nopt node-normalize-package-data node-npmlog node-once node-osenv node-qs node-read
 node-read-package-json node-request node-retry node-rimraf node-semver node-sha node-sigmund node-slide node-tar
 node-tunnel-agent node-underscore node-which nodejs-dev openssh-client openssl patch perl perl-modules-5.22 python
 python-minimal python-pkg-resources python2.7 python2.7-minimal rename rsync xauth xz-utils zlib1g-dev
Suggested packages:
 binutils-doc bzip2-doc cpp-doc gcc-5-locales debian-keyring g++-multilib g++-5-multilib gcc-5-doc libstdc++6-5-dbg
 gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg
 libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg
 gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-arch git-cvs
 git-mediawiki git-svn ppp rdnssd iproute2-doc resolvconf avahi-autoipd isc-dhcp-client-ddns apparmor apache2
 | lighttpd | httpd glibc-doc gnutls-bin krb5-doc krb5-user libsasl2-modules-otp libsasl2-modules-ldap
 libsasl2-modules-sql libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libstdc++-5-doc make-doc
 man-browser node-hawk node-aws-sign node-oauth-sign node-http-signature debhelper ssh-askpass libpam-ssh keychain
 monkeysphere ed diffutils-doc perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl python-doc python-tk
 python-setuptools python2.7-doc binfmt-support openssh-server
The following NEW packages will be installed:
 binutils build-essential bzip2 ca-certificates cpp cpp-5 dpkg-dev fakeroot file g++ g++-5 gcc gcc-5 git git-man gyp
 ifupdown iproute2 isc-dhcp-client isc-dhcp-common javascript-common krb5-locales less libalgorithm-diff-perl
 libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan2 libasn1-8-heimdal libatm1 libatomic1 libbsd0 libc-dev-bin
 libc6-dev libcc1-0 libcilkrts5 libcurl3-gnutls libdns-export162 libdpkg-perl libedit2 liberror-perl libexpat1
 libfakeroot libffi6 libfile-fcntllock-perl libgcc-5-dev libgdbm3 libgmp10 libgnutls30 libgomp1 libgssapi-krb5-2
 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhogweed4 libhx509-5-heimdal
 libicu55 libidn11 libisc-export160 libisl15 libitm1 libjs-inherits libjs-jquery libjs-node-uuid libjs-underscore
 libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2 liblsan0 libmagic1 libmnl0
 libmpc3 libmpfr4 libmpx0 libnettle6 libp11-kit0 libperl5.22 libpopt0 libpython-stdlib libpython2.7-minimal
 libpython2.7-stdlib libquadmath0 libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db
 libsqlite3-0 libssl-dev libssl-doc libssl1.0.0 libstdc++-5-dev libtasn1-6 libtsan0 libubsan0 libuv1 libuv1-dev
 libwind0-heimdal libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 libxext6 libxmuu1 libxtables11 linux-libc-dev make
 manpages manpages-dev mime-support netbase node-abbrev node-ansi node-ansi-color-table node-archy node-async
 node-block-stream node-combined-stream node-cookie-jar node-delayed-stream node-forever-agent node-form-data
 node-fstream node-fstream-ignore node-github-url-from-git node-glob node-graceful-fs node-gyp node-inherits node-ini
 node-json-stringify-safe node-lockfile node-lru-cache node-mime node-minimatch node-mkdirp node-mute-stream
 node-node-uuid node-nopt node-normalize-package-data node-npmlog node-once node-osenv node-qs node-read
 node-read-package-json node-request node-retry node-rimraf node-semver node-sha node-sigmund node-slide node-tar
 node-tunnel-agent node-underscore node-which nodejs nodejs-dev npm openssh-client openssl patch perl
 perl-modules-5.22 python python-minimal python-pkg-resources python2.7 python2.7-minimal rename rsync xauth xz-utils
 zlib1g-dev
0 upgraded, 179 newly installed, 0 to remove and 2 not upgraded.
Need to get 79.4 MB of archives.
After this operation, 337 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libatm1 amd64 1:2.5.1-1.5 [24.2 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmnl0 amd64 1.0.3-5 [12.0 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 libpopt0 amd64 1.16-10 [26.0 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial/main amd64 libgdbm3 amd64 1.8.3-13.1 [16.9 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxau6 amd64 1:1.0.8-1 [8376 B]
Get:6 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxdmcp6 amd64 1:1.1.2-1.1 [11.0 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxcb1 amd64 1.11.1-1ubuntu1 [40.0 kB]
Get:8 http://archive.ubuntu.com/ubuntu xenial/main amd64 libx11-data all 2:1.6.3-1ubuntu2 [113 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/main amd64 libx11-6 amd64 2:1.6.3-1ubuntu2 [571 kB]
Get:10 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxext6 amd64 2:1.3.3-1 [29.4 kB]
Get:11 http://archive.ubuntu.com/ubuntu xenial/main amd64 perl-modules-5.22 all 5.22.1-9 [2641 kB]
Get:12 http://archive.ubuntu.com/ubuntu xenial/main amd64 libperl5.22 amd64 5.22.1-9 [3371 kB]
Get:13 http://archive.ubuntu.com/ubuntu xenial/main amd64 perl amd64 5.22.1-9 [237 kB]
Get:14 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-minimal amd64 2.7.12-1ubuntu0~16.04.1 [339 kB]
Get:15 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7-minimal amd64 2.7.12-1ubuntu0~16.04.1 [1295 kB]
Get:16 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-minimal amd64 2.7.11-1 [28.2 kB]
Get:17 http://archive.ubuntu.com/ubuntu xenial/main amd64 mime-support all 3.59ubuntu1 [31.0 kB]
Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libexpat1 amd64 2.1.0-7ubuntu0.16.04.2 [71.3 kB]
Get:19 http://archive.ubuntu.com/ubuntu xenial/main amd64 libffi6 amd64 3.2.1-4 [17.8 kB]
Get:20 http://archive.ubuntu.com/ubuntu xenial/main amd64 libsqlite3-0 amd64 3.11.0-1ubuntu1 [396 kB]
Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl1.0.0 amd64 1.0.2g-1ubuntu4.6 [1082 kB]
Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-stdlib amd64 2.7.12-1ubuntu0~16.04.1 [1884 kB]
Get:23 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7 amd64 2.7.12-1ubuntu0~16.04.1 [224 kB]
Get:24 http://archive.ubuntu.com/ubuntu xenial/main amd64 libpython-stdlib amd64 2.7.11-1 [7656 B]
Get:25 http://archive.ubuntu.com/ubuntu xenial/main amd64 python amd64 2.7.11-1 [137 kB]
Get:26 http://archive.ubuntu.com/ubuntu xenial/main amd64 libgmp10 amd64 2:6.1.0+dfsg-2 [240 kB]
Get:27 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmpfr4 amd64 3.1.4-1 [191 kB]
Get:28 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB]
Get:29 http://archive.ubuntu.com/ubuntu xenial/main amd64 bzip2 amd64 1.0.6-8 [32.7 kB]
Get:30 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmagic1 amd64 1:5.25-2ubuntu1 [216 kB]
Get:31 http://archive.ubuntu.com/ubuntu xenial/main amd64 file amd64 1:5.25-2ubuntu1 [21.2 kB]
Get:32 http://archive.ubuntu.com/ubuntu xenial/main amd64 iproute2 amd64 4.3.0-1ubuntu3 [522 kB]
Get:33 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 ifupdown amd64 0.8.10ubuntu1.2 [54.9 kB]
Get:34 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libisc-export160 amd64 1:9.10.3.dfsg.P4-8ubuntu1.5 [153 kB]
Get:35 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libdns-export162 amd64 1:9.10.3.dfsg.P4-8ubuntu1.5 [665 kB]
Get:36 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 isc-dhcp-client amd64 4.3.3-5ubuntu12.6 [223 kB]
Get:37 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 isc-dhcp-common amd64 4.3.3-5ubuntu12.6 [105 kB]
Get:38 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 less amd64 481-2.1ubuntu0.1 [110 kB]
Get:39 http://archive.ubuntu.com/ubuntu xenial/main amd64 libbsd0 amd64 0.8.2-1 [41.7 kB]
Get:40 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libnettle6 amd64 3.2-1ubuntu0.16.04.1 [93.5 kB]
Get:41 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libhogweed4 amd64 3.2-1ubuntu0.16.04.1 [136 kB]
Get:42 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libidn11 amd64 1.32-3ubuntu1.1 [45.6 kB]
Get:43 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libp11-kit0 amd64 0.23.2-5~ubuntu16.04.1 [105 kB]
Get:44 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libtasn1-6 amd64 4.7-3ubuntu0.16.04.1 [43.2 kB]
Get:45 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgnutls30 amd64 3.4.10-4ubuntu1.2 [547 kB]
Get:46 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxtables11 amd64 1.6.0-2ubuntu3 [27.2 kB]
Get:47 http://archive.ubuntu.com/ubuntu xenial/main amd64 netbase all 5.3 [12.9 kB]
Get:48 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 openssl amd64 1.0.2g-1ubuntu4.6 [492 kB]
Get:49 http://archive.ubuntu.com/ubuntu xenial/main amd64 ca-certificates all 20160104ubuntu1 [191 kB]
Get:50 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 krb5-locales all 1.13.2+dfsg-5ubuntu2 [13.2 kB]
Get:51 http://archive.ubuntu.com/ubuntu xenial/main amd64 libroken18-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [41.2 kB]
Get:52 http://archive.ubuntu.com/ubuntu xenial/main amd64 libasn1-8-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [174 kB]
Get:53 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libkrb5support0 amd64 1.13.2+dfsg-5ubuntu2 [30.8 kB]
Get:54 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libk5crypto3 amd64 1.13.2+dfsg-5ubuntu2 [81.2 kB]
Get:55 http://archive.ubuntu.com/ubuntu xenial/main amd64 libkeyutils1 amd64 1.5.9-8ubuntu1 [9904 B]
Get:56 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libkrb5-3 amd64 1.13.2+dfsg-5ubuntu2 [273 kB]
Get:57 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgssapi-krb5-2 amd64 1.13.2+dfsg-5ubuntu2 [120 kB]
Get:58 http://archive.ubuntu.com/ubuntu xenial/main amd64 libhcrypto4-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [84.9 kB]
Get:59 http://archive.ubuntu.com/ubuntu xenial/main amd64 libheimbase1-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [29.2 kB]
Get:60 http://archive.ubuntu.com/ubuntu xenial/main amd64 libwind0-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [48.2 kB]
Get:61 http://archive.ubuntu.com/ubuntu xenial/main amd64 libhx509-5-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [107 kB]
Get:62 http://archive.ubuntu.com/ubuntu xenial/main amd64 libkrb5-26-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [202 kB]
Get:63 http://archive.ubuntu.com/ubuntu xenial/main amd64 libheimntlm0-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [15.1 kB]
Get:64 http://archive.ubuntu.com/ubuntu xenial/main amd64 libgssapi3-heimdal amd64 1.7~git20150920+dfsg-4ubuntu1 [96.1 kB]
Get:65 http://archive.ubuntu.com/ubuntu xenial/main amd64 libsasl2-modules-db amd64 2.1.26.dfsg1-14build1 [14.5 kB]
Get:66 http://archive.ubuntu.com/ubuntu xenial/main amd64 libsasl2-2 amd64 2.1.26.dfsg1-14build1 [48.7 kB]
Get:67 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libldap-2.4-2 amd64 2.4.42+dfsg-2ubuntu3.1 [161 kB]
Get:68 http://archive.ubuntu.com/ubuntu xenial/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d-1build1 [53.9 kB]
Get:69 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libcurl3-gnutls amd64 7.47.0-1ubuntu2.2 [184 kB]
Get:70 http://archive.ubuntu.com/ubuntu xenial/main amd64 libedit2 amd64 3.1-20150325-1ubuntu2 [76.5 kB]
Get:71 http://archive.ubuntu.com/ubuntu xenial/main amd64 libicu55 amd64 55.1-7 [7643 kB]
Get:72 http://archive.ubuntu.com/ubuntu xenial/main amd64 libsasl2-modules amd64 2.1.26.dfsg1-14build1 [47.5 kB]
Get:73 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxmuu1 amd64 2:1.1.2-2 [9674 B]
Get:74 http://archive.ubuntu.com/ubuntu xenial/main amd64 manpages all 4.04-2 [1087 kB]
Get:75 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 openssh-client amd64 1:7.2p2-4ubuntu2.1 [587 kB]
Get:76 http://archive.ubuntu.com/ubuntu xenial/main amd64 rsync amd64 3.1.1-3ubuntu1 [325 kB]
Get:77 http://archive.ubuntu.com/ubuntu xenial/main amd64 xauth amd64 1:1.0.9-1ubuntu2 [22.7 kB]
Get:78 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 binutils amd64 2.26.1-1ubuntu1~16.04.3 [2310 kB]
Get:79 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libc-dev-bin amd64 2.23-0ubuntu5 [68.7 kB]
Get:80 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 linux-libc-dev amd64 4.4.0-66.87 [833 kB]
Get:81 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libc6-dev amd64 2.23-0ubuntu5 [2078 kB]
Get:82 http://archive.ubuntu.com/ubuntu xenial/main amd64 libisl15 amd64 0.16.1-1 [524 kB]
Get:83 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 cpp-5 amd64 5.4.0-6ubuntu1~16.04.4 [7653 kB]
Get:84 http://archive.ubuntu.com/ubuntu xenial/main amd64 cpp amd64 4:5.3.1-1ubuntu1 [27.7 kB]
Get:85 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libcc1-0 amd64 5.4.0-6ubuntu1~16.04.4 [38.8 kB]
Get:86 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgomp1 amd64 5.4.0-6ubuntu1~16.04.4 [55.0 kB]
Get:87 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libitm1 amd64 5.4.0-6ubuntu1~16.04.4 [27.4 kB]
Get:88 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libatomic1 amd64 5.4.0-6ubuntu1~16.04.4 [8912 B]
Get:89 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libasan2 amd64 5.4.0-6ubuntu1~16.04.4 [264 kB]
Get:90 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 liblsan0 amd64 5.4.0-6ubuntu1~16.04.4 [105 kB]
Get:91 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libtsan0 amd64 5.4.0-6ubuntu1~16.04.4 [244 kB]
Get:92 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libubsan0 amd64 5.4.0-6ubuntu1~16.04.4 [95.3 kB]
Get:93 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libcilkrts5 amd64 5.4.0-6ubuntu1~16.04.4 [40.1 kB]
Get:94 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libmpx0 amd64 5.4.0-6ubuntu1~16.04.4 [9766 B]
Get:95 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libquadmath0 amd64 5.4.0-6ubuntu1~16.04.4 [131 kB]
Get:96 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgcc-5-dev amd64 5.4.0-6ubuntu1~16.04.4 [2237 kB]
Get:97 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 gcc-5 amd64 5.4.0-6ubuntu1~16.04.4 [8577 kB]
Get:98 http://archive.ubuntu.com/ubuntu xenial/main amd64 gcc amd64 4:5.3.1-1ubuntu1 [5244 B]
Get:99 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libstdc++-5-dev amd64 5.4.0-6ubuntu1~16.04.4 [1426 kB]
Get:100 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 g++-5 amd64 5.4.0-6ubuntu1~16.04.4 [8300 kB]
Get:101 http://archive.ubuntu.com/ubuntu xenial/main amd64 g++ amd64 4:5.3.1-1ubuntu1 [1504 B]
Get:102 http://archive.ubuntu.com/ubuntu xenial/main amd64 make amd64 4.1-6 [151 kB]
Get:103 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libdpkg-perl all 1.18.4ubuntu1.1 [195 kB]
Get:104 http://archive.ubuntu.com/ubuntu xenial/main amd64 xz-utils amd64 5.1.1alpha+20120614-2ubuntu2 [78.8 kB]
Get:105 http://archive.ubuntu.com/ubuntu xenial/main amd64 patch amd64 2.7.5-1 [90.4 kB]
Get:106 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 dpkg-dev all 1.18.4ubuntu1.1 [584 kB]
Get:107 http://archive.ubuntu.com/ubuntu xenial/main amd64 build-essential amd64 12.1ubuntu2 [4758 B]
Get:108 http://archive.ubuntu.com/ubuntu xenial/main amd64 libfakeroot amd64 1.20.2-1ubuntu1 [25.5 kB]
Get:109 http://archive.ubuntu.com/ubuntu xenial/main amd64 fakeroot amd64 1.20.2-1ubuntu1 [61.8 kB]
Get:110 http://archive.ubuntu.com/ubuntu xenial/main amd64 liberror-perl all 0.17-1.2 [19.6 kB]
Get:111 http://archive.ubuntu.com/ubuntu xenial/main amd64 git-man all 1:2.7.4-0ubuntu1 [735 kB]
Get:112 http://archive.ubuntu.com/ubuntu xenial/main amd64 git amd64 1:2.7.4-0ubuntu1 [3006 kB]
Get:113 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-pkg-resources all 20.7.0-1 [108 kB]
Get:114 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gyp all 0.1+20150913git1f374df9-1ubuntu1 [265 kB]
Get:115 http://archive.ubuntu.com/ubuntu xenial/main amd64 javascript-common all 11 [6066 B]
Get:116 http://archive.ubuntu.com/ubuntu xenial/main amd64 libalgorithm-diff-perl all 1.19.03-1 [47.6 kB]
Get:117 http://archive.ubuntu.com/ubuntu xenial/main amd64 libalgorithm-diff-xs-perl amd64 0.04-4build1 [11.0 kB]
Get:118 http://archive.ubuntu.com/ubuntu xenial/main amd64 libalgorithm-merge-perl all 0.08-3 [12.0 kB]
Get:119 http://archive.ubuntu.com/ubuntu xenial/main amd64 libfile-fcntllock-perl amd64 0.22-3 [32.0 kB]
Get:120 http://archive.ubuntu.com/ubuntu xenial/main amd64 libjs-jquery all 1.11.3+dfsg-4 [161 kB]
Get:121 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libjs-node-uuid all 1.4.0-1 [11.1 kB]
Get:122 http://archive.ubuntu.com/ubuntu xenial/main amd64 libjs-underscore all 1.7.0~dfsg-1ubuntu1 [46.7 kB]
Get:123 http://archive.ubuntu.com/ubuntu xenial/main amd64 zlib1g-dev amd64 1:1.2.8.dfsg-2ubuntu4 [168 kB]
Get:124 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-dev amd64 1.0.2g-1ubuntu4.6 [1344 kB]
Get:125 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-doc all 1.0.2g-1ubuntu4.6 [1079 kB]
Get:126 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libuv1 amd64 1.8.0-1 [57.4 kB]
Get:127 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libuv1-dev amd64 1.8.0-1 [74.7 kB]
Get:128 http://archive.ubuntu.com/ubuntu xenial/main amd64 manpages-dev all 4.04-2 [2048 kB]
Get:129 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 nodejs amd64 4.2.6~dfsg-1ubuntu4.1 [3161 kB]
Get:130 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-async all 0.8.0-1 [22.2 kB]
Get:131 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-node-uuid all 1.4.0-1 [2530 B]
Get:132 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-underscore all 1.7.0~dfsg-1ubuntu1 [3780 B]
Get:133 http://archive.ubuntu.com/ubuntu xenial/main amd64 rename all 0.20-4 [12.0 kB]
Get:134 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libjs-inherits all 2.0.1-3 [2794 B]
Get:135 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-abbrev all 1.0.5-2 [3592 B]
Get:136 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-ansi all 0.3.0-2 [8590 B]
Get:137 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-ansi-color-table all 1.0.0-1 [4478 B]
Get:138 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-archy all 0.0.2-1 [3660 B]
Get:139 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-inherits all 2.0.1-3 [3060 B]
Get:140 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-block-stream all 0.0.7-1 [4832 B]
Get:141 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-delayed-stream all 0.0.5-1 [4750 B]
Get:142 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-combined-stream all 0.0.5-1 [4958 B]
Get:143 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-cookie-jar all 0.3.1-1 [3746 B]
Get:144 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-forever-agent all 0.5.1-1 [3194 B]
Get:145 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-mime all 1.3.4-1 [11.9 kB]
Get:146 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-form-data all 0.1.0-1 [6412 B]
Get:147 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-rimraf all 2.2.8-1 [5702 B]
Get:148 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-mkdirp all 0.5.0-1 [4690 B]
Get:149 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-graceful-fs all 3.0.2-1 [7102 B]
Get:150 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-fstream all 0.1.24-1 [19.5 kB]
Get:151 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-lru-cache all 2.3.1-1 [5674 B]
Get:152 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-sigmund all 1.0.0-1 [3818 B]
Get:153 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-minimatch all 1.0.0-1 [14.0 kB]
Get:154 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-fstream-ignore all 0.0.6-2 [5586 B]
Get:155 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-github-url-from-git all 1.1.1-1 [3138 B]
Get:156 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-once all 1.1.1-1 [2608 B]
Get:157 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-glob all 4.0.5-1 [13.2 kB]
Get:158 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 nodejs-dev amd64 4.2.6~dfsg-1ubuntu4.1 [265 kB]
Get:159 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-nopt all 3.0.1-1 [9544 B]
Get:160 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-npmlog all 0.0.4-1 [5844 B]
Get:161 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-osenv all 0.1.0-1 [3772 B]
Get:162 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-tunnel-agent all 0.3.1-1 [4018 B]
Get:163 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-json-stringify-safe all 5.0.0-1 [3544 B]
Get:164 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-qs all 2.2.4-1 [7574 B]
Get:165 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-request all 2.26.1-1 [14.5 kB]
Get:166 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-semver all 2.1.0-2 [16.2 kB]
Get:167 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-tar all 1.0.3-2 [17.5 kB]
Get:168 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-which all 1.0.5-2 [3678 B]
Get:169 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-gyp all 3.0.3-2ubuntu1 [23.2 kB]
Get:170 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-ini all 1.1.0-1 [4770 B]
Get:171 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-lockfile all 0.4.1-1 [5450 B]
Get:172 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-mute-stream all 0.0.4-1 [4096 B]
Get:173 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-normalize-package-data all 0.2.2-1 [9286 B]
Get:174 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-read all 1.0.5-1 [4314 B]
Get:175 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-read-package-json all 1.2.4-1 [7780 B]
Get:176 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-retry all 0.6.0-1 [6172 B]
Get:177 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-sha all 1.2.3-1 [4272 B]
Get:178 http://archive.ubuntu.com/ubuntu xenial/universe amd64 node-slide all 1.1.4-1 [6118 B]
Get:179 http://archive.ubuntu.com/ubuntu xenial/universe amd64 npm all 3.5.2-0ubuntu4 [1586 kB]
Fetched 79.4 MB in 40s (1962 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libatm1:amd64.
(Reading database ... 7256 files and directories currently installed.)
Preparing to unpack .../libatm1_1%3a2.5.1-1.5_amd64.deb ...
Unpacking libatm1:amd64 (1:2.5.1-1.5) ...
Selecting previously unselected package libmnl0:amd64.
Preparing to unpack .../libmnl0_1.0.3-5_amd64.deb ...
Unpacking libmnl0:amd64 (1.0.3-5) ...
Selecting previously unselected package libpopt0:amd64.
Preparing to unpack .../libpopt0_1.16-10_amd64.deb ...
Unpacking libpopt0:amd64 (1.16-10) ...
Selecting previously unselected package libgdbm3:amd64.
Preparing to unpack .../libgdbm3_1.8.3-13.1_amd64.deb ...
Unpacking libgdbm3:amd64 (1.8.3-13.1) ...
Selecting previously unselected package libxau6:amd64.
Preparing to unpack .../libxau6_1%3a1.0.8-1_amd64.deb ...
Unpacking libxau6:amd64 (1:1.0.8-1) ...
Selecting previously unselected package libxdmcp6:amd64.
Preparing to unpack .../libxdmcp6_1%3a1.1.2-1.1_amd64.deb ...
Unpacking libxdmcp6:amd64 (1:1.1.2-1.1) ...
Selecting previously unselected package libxcb1:amd64.
Preparing to unpack .../libxcb1_1.11.1-1ubuntu1_amd64.deb ...
Unpacking libxcb1:amd64 (1.11.1-1ubuntu1) ...
Selecting previously unselected package libx11-data.
Preparing to unpack .../libx11-data_2%3a1.6.3-1ubuntu2_all.deb ...
Unpacking libx11-data (2:1.6.3-1ubuntu2) ...
Selecting previously unselected package libx11-6:amd64.
Preparing to unpack .../libx11-6_2%3a1.6.3-1ubuntu2_amd64.deb ...
Unpacking libx11-6:amd64 (2:1.6.3-1ubuntu2) ...
Selecting previously unselected package libxext6:amd64.
Preparing to unpack .../libxext6_2%3a1.3.3-1_amd64.deb ...
Unpacking libxext6:amd64 (2:1.3.3-1) ...
Selecting previously unselected package perl-modules-5.22.
Preparing to unpack .../perl-modules-5.22_5.22.1-9_all.deb ...
Unpacking perl-modules-5.22 (5.22.1-9) ...
Selecting previously unselected package libperl5.22:amd64.
Preparing to unpack .../libperl5.22_5.22.1-9_amd64.deb ...
Unpacking libperl5.22:amd64 (5.22.1-9) ...
Selecting previously unselected package perl.
Preparing to unpack .../perl_5.22.1-9_amd64.deb ...
Unpacking perl (5.22.1-9) ...
Selecting previously unselected package libpython2.7-minimal:amd64.
Preparing to unpack .../libpython2.7-minimal_2.7.12-1ubuntu0~16.04.1_amd64.deb ...
Unpacking libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.1) ...
Selecting previously unselected package python2.7-minimal.
Preparing to unpack .../python2.7-minimal_2.7.12-1ubuntu0~16.04.1_amd64.deb ...
Unpacking python2.7-minimal (2.7.12-1ubuntu0~16.04.1) ...
Selecting previously unselected package python-minimal.
Preparing to unpack .../python-minimal_2.7.11-1_amd64.deb ...
Unpacking python-minimal (2.7.11-1) ...
Selecting previously unselected package mime-support.
Preparing to unpack .../mime-support_3.59ubuntu1_all.deb ...
Unpacking mime-support (3.59ubuntu1) ...
Selecting previously unselected package libexpat1:amd64.
Preparing to unpack .../libexpat1_2.1.0-7ubuntu0.16.04.2_amd64.deb ...
Unpacking libexpat1:amd64 (2.1.0-7ubuntu0.16.04.2) ...
Selecting previously unselected package libffi6:amd64.
Preparing to unpack .../libffi6_3.2.1-4_amd64.deb ...
Unpacking libffi6:amd64 (3.2.1-4) ...
Selecting previously unselected package libsqlite3-0:amd64.
Preparing to unpack .../libsqlite3-0_3.11.0-1ubuntu1_amd64.deb ...
Unpacking libsqlite3-0:amd64 (3.11.0-1ubuntu1) ...
Selecting previously unselected package libssl1.0.0:amd64.
Preparing to unpack .../libssl1.0.0_1.0.2g-1ubuntu4.6_amd64.deb ...
Unpacking libssl1.0.0:amd64 (1.0.2g-1ubuntu4.6) ...
Selecting previously unselected package libpython2.7-stdlib:amd64.
Preparing to unpack .../libpython2.7-stdlib_2.7.12-1ubuntu0~16.04.1_amd64.deb ...
Unpacking libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.1) ...
Selecting previously unselected package python2.7.
Preparing to unpack .../python2.7_2.7.12-1ubuntu0~16.04.1_amd64.deb ...
Unpacking python2.7 (2.7.12-1ubuntu0~16.04.1) ...
Selecting previously unselected package libpython-stdlib:amd64.
Preparing to unpack .../libpython-stdlib_2.7.11-1_amd64.deb ...
Unpacking libpython-stdlib:amd64 (2.7.11-1) ...
Processing triggers for libc-bin (2.23-0ubuntu5) ...
Setting up libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.1) ...
Setting up python2.7-minimal (2.7.12-1ubuntu0~16.04.1) ...
Linking and byte-compiling packages for runtime python2.7...
Setting up python-minimal (2.7.11-1) ...
Selecting previously unselected package python.
(Reading database ... 10145 files and directories currently installed.)
Preparing to unpack .../python_2.7.11-1_amd64.deb ...
Unpacking python (2.7.11-1) ...
Selecting previously unselected package libgmp10:amd64.
Preparing to unpack .../libgmp10_2%3a6.1.0+dfsg-2_amd64.deb ...
Unpacking libgmp10:amd64 (2:6.1.0+dfsg-2) ...
Selecting previously unselected package libmpfr4:amd64.
Preparing to unpack .../libmpfr4_3.1.4-1_amd64.deb ...
Unpacking libmpfr4:amd64 (3.1.4-1) ...
Selecting previously unselected package libmpc3:amd64.
Preparing to unpack .../libmpc3_1.0.3-1_amd64.deb ...
Unpacking libmpc3:amd64 (1.0.3-1) ...
Selecting previously unselected package bzip2.
Preparing to unpack .../bzip2_1.0.6-8_amd64.deb ...
Unpacking bzip2 (1.0.6-8) ...
Selecting previously unselected package libmagic1:amd64.
Preparing to unpack .../libmagic1_1%3a5.25-2ubuntu1_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.25-2ubuntu1) ...
Selecting previously unselected package file.
Preparing to unpack .../file_1%3a5.25-2ubuntu1_amd64.deb ...
Unpacking file (1:5.25-2ubuntu1) ...
Selecting previously unselected package iproute2.
Preparing to unpack .../iproute2_4.3.0-1ubuntu3_amd64.deb ...
Unpacking iproute2 (4.3.0-1ubuntu3) ...
Selecting previously unselected package ifupdown.
Preparing to unpack .../ifupdown_0.8.10ubuntu1.2_amd64.deb ...
Unpacking ifupdown (0.8.10ubuntu1.2) ...
Selecting previously unselected package libisc-export160.
Preparing to unpack .../libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.5_amd64.deb ...
Unpacking libisc-export160 (1:9.10.3.dfsg.P4-8ubuntu1.5) ...
Selecting previously unselected package libdns-export162.
Preparing to unpack .../libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.5_amd64.deb ...
Unpacking libdns-export162 (1:9.10.3.dfsg.P4-8ubuntu1.5) ...
Selecting previously unselected package isc-dhcp-client.
Preparing to unpack .../isc-dhcp-client_4.3.3-5ubuntu12.6_amd64.deb ...
Unpacking isc-dhcp-client (4.3.3-5ubuntu12.6) ...
Selecting previously unselected package isc-dhcp-common.
Preparing to unpack .../isc-dhcp-common_4.3.3-5ubuntu12.6_amd64.deb ...
Unpacking isc-dhcp-common (4.3.3-5ubuntu12.6) ...
Selecting previously unselected package less.
Preparing to unpack .../less_481-2.1ubuntu0.1_amd64.deb ...
Unpacking less (481-2.1ubuntu0.1) ...
Selecting previously unselected package libbsd0:amd64.
Preparing to unpack .../libbsd0_0.8.2-1_amd64.deb ...
Unpacking libbsd0:amd64 (0.8.2-1) ...
Selecting previously unselected package libnettle6:amd64.
Preparing to unpack .../libnettle6_3.2-1ubuntu0.16.04.1_amd64.deb ...
Unpacking libnettle6:amd64 (3.2-1ubuntu0.16.04.1) ...
Selecting previously unselected package libhogweed4:amd64.
Preparing to unpack .../libhogweed4_3.2-1ubuntu0.16.04.1_amd64.deb ...
Unpacking libhogweed4:amd64 (3.2-1ubuntu0.16.04.1) ...
Selecting previously unselected package libidn11:amd64.
Preparing to unpack .../libidn11_1.32-3ubuntu1.1_amd64.deb ...
Unpacking libidn11:amd64 (1.32-3ubuntu1.1) ...
Selecting previously unselected package libp11-kit0:amd64.
Preparing to unpack .../libp11-kit0_0.23.2-5~ubuntu16.04.1_amd64.deb ...
Unpacking libp11-kit0:amd64 (0.23.2-5~ubuntu16.04.1) ...
Selecting previously unselected package libtasn1-6:amd64.
Preparing to unpack .../libtasn1-6_4.7-3ubuntu0.16.04.1_amd64.deb ...
Unpacking libtasn1-6:amd64 (4.7-3ubuntu0.16.04.1) ...
Selecting previously unselected package libgnutls30:amd64.
Preparing to unpack .../libgnutls30_3.4.10-4ubuntu1.2_amd64.deb ...
Unpacking libgnutls30:amd64 (3.4.10-4ubuntu1.2) ...
Selecting previously unselected package libxtables11:amd64.
Preparing to unpack .../libxtables11_1.6.0-2ubuntu3_amd64.deb ...
Unpacking libxtables11:amd64 (1.6.0-2ubuntu3) ...
Selecting previously unselected package netbase.
Preparing to unpack .../archives/netbase_5.3_all.deb ...
Unpacking netbase (5.3) ...
Selecting previously unselected package openssl.
Preparing to unpack .../openssl_1.0.2g-1ubuntu4.6_amd64.deb ...
Unpacking openssl (1.0.2g-1ubuntu4.6) ...
Selecting previously unselected package ca-certificates.
Preparing to unpack .../ca-certificates_20160104ubuntu1_all.deb ...
Unpacking ca-certificates (20160104ubuntu1) ...
Selecting previously unselected package krb5-locales.
Preparing to unpack .../krb5-locales_1.13.2+dfsg-5ubuntu2_all.deb ...
Unpacking krb5-locales (1.13.2+dfsg-5ubuntu2) ...
Selecting previously unselected package libroken18-heimdal:amd64.
Preparing to unpack .../libroken18-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libroken18-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libasn1-8-heimdal:amd64.
Preparing to unpack .../libasn1-8-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libasn1-8-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libkrb5support0:amd64.
Preparing to unpack .../libkrb5support0_1.13.2+dfsg-5ubuntu2_amd64.deb ...
Unpacking libkrb5support0:amd64 (1.13.2+dfsg-5ubuntu2) ...
Selecting previously unselected package libk5crypto3:amd64.
Preparing to unpack .../libk5crypto3_1.13.2+dfsg-5ubuntu2_amd64.deb ...
Unpacking libk5crypto3:amd64 (1.13.2+dfsg-5ubuntu2) ...
Selecting previously unselected package libkeyutils1:amd64.
Preparing to unpack .../libkeyutils1_1.5.9-8ubuntu1_amd64.deb ...
Unpacking libkeyutils1:amd64 (1.5.9-8ubuntu1) ...
Selecting previously unselected package libkrb5-3:amd64.
Preparing to unpack .../libkrb5-3_1.13.2+dfsg-5ubuntu2_amd64.deb ...
Unpacking libkrb5-3:amd64 (1.13.2+dfsg-5ubuntu2) ...
Selecting previously unselected package libgssapi-krb5-2:amd64.
Preparing to unpack .../libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2_amd64.deb ...
Unpacking libgssapi-krb5-2:amd64 (1.13.2+dfsg-5ubuntu2) ...
Selecting previously unselected package libhcrypto4-heimdal:amd64.
Preparing to unpack .../libhcrypto4-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libhcrypto4-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libheimbase1-heimdal:amd64.
Preparing to unpack .../libheimbase1-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libheimbase1-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libwind0-heimdal:amd64.
Preparing to unpack .../libwind0-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libwind0-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libhx509-5-heimdal:amd64.
Preparing to unpack .../libhx509-5-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libhx509-5-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libkrb5-26-heimdal:amd64.
Preparing to unpack .../libkrb5-26-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libkrb5-26-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libheimntlm0-heimdal:amd64.
Preparing to unpack .../libheimntlm0-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libheimntlm0-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libgssapi3-heimdal:amd64.
Preparing to unpack .../libgssapi3-heimdal_1.7~git20150920+dfsg-4ubuntu1_amd64.deb ...
Unpacking libgssapi3-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Selecting previously unselected package libsasl2-modules-db:amd64.
Preparing to unpack .../libsasl2-modules-db_2.1.26.dfsg1-14build1_amd64.deb ...
Unpacking libsasl2-modules-db:amd64 (2.1.26.dfsg1-14build1) ...
Selecting previously unselected package libsasl2-2:amd64.
Preparing to unpack .../libsasl2-2_2.1.26.dfsg1-14build1_amd64.deb ...
Unpacking libsasl2-2:amd64 (2.1.26.dfsg1-14build1) ...
Selecting previously unselected package libldap-2.4-2:amd64.
Preparing to unpack .../libldap-2.4-2_2.4.42+dfsg-2ubuntu3.1_amd64.deb ...
Unpacking libldap-2.4-2:amd64 (2.4.42+dfsg-2ubuntu3.1) ...
Selecting previously unselected package librtmp1:amd64.
Preparing to unpack .../librtmp1_2.4+20151223.gitfa8646d-1build1_amd64.deb ...
Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d-1build1) ...
Selecting previously unselected package libcurl3-gnutls:amd64.
Preparing to unpack .../libcurl3-gnutls_7.47.0-1ubuntu2.2_amd64.deb ...
Unpacking libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.2) ...
Selecting previously unselected package libedit2:amd64.
Preparing to unpack .../libedit2_3.1-20150325-1ubuntu2_amd64.deb ...
Unpacking libedit2:amd64 (3.1-20150325-1ubuntu2) ...
Selecting previously unselected package libicu55:amd64.
Preparing to unpack .../libicu55_55.1-7_amd64.deb ...
Unpacking libicu55:amd64 (55.1-7) ...
Selecting previously unselected package libsasl2-modules:amd64.
Preparing to unpack .../libsasl2-modules_2.1.26.dfsg1-14build1_amd64.deb ...
Unpacking libsasl2-modules:amd64 (2.1.26.dfsg1-14build1) ...
Selecting previously unselected package libxmuu1:amd64.
Preparing to unpack .../libxmuu1_2%3a1.1.2-2_amd64.deb ...
Unpacking libxmuu1:amd64 (2:1.1.2-2) ...
Selecting previously unselected package manpages.
Preparing to unpack .../manpages_4.04-2_all.deb ...
Unpacking manpages (4.04-2) ...
Selecting previously unselected package openssh-client.
Preparing to unpack .../openssh-client_1%3a7.2p2-4ubuntu2.1_amd64.deb ...
Unpacking openssh-client (1:7.2p2-4ubuntu2.1) ...
Selecting previously unselected package rsync.
Preparing to unpack .../rsync_3.1.1-3ubuntu1_amd64.deb ...
Unpacking rsync (3.1.1-3ubuntu1) ...
Selecting previously unselected package xauth.
Preparing to unpack .../xauth_1%3a1.0.9-1ubuntu2_amd64.deb ...
Unpacking xauth (1:1.0.9-1ubuntu2) ...
Selecting previously unselected package binutils.
Preparing to unpack .../binutils_2.26.1-1ubuntu1~16.04.3_amd64.deb ...
Unpacking binutils (2.26.1-1ubuntu1~16.04.3) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../libc-dev-bin_2.23-0ubuntu5_amd64.deb ...
Unpacking libc-dev-bin (2.23-0ubuntu5) ...
Selecting previously unselected package linux-libc-dev:amd64.
Preparing to unpack .../linux-libc-dev_4.4.0-66.87_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.4.0-66.87) ...
Selecting previously unselected package libc6-dev:amd64.
Preparing to unpack .../libc6-dev_2.23-0ubuntu5_amd64.deb ...
Unpacking libc6-dev:amd64 (2.23-0ubuntu5) ...
Selecting previously unselected package libisl15:amd64.
Preparing to unpack .../libisl15_0.16.1-1_amd64.deb ...
Unpacking libisl15:amd64 (0.16.1-1) ...
Selecting previously unselected package cpp-5.
Preparing to unpack .../cpp-5_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking cpp-5 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package cpp.
Preparing to unpack .../cpp_4%3a5.3.1-1ubuntu1_amd64.deb ...
Unpacking cpp (4:5.3.1-1ubuntu1) ...
Selecting previously unselected package libcc1-0:amd64.
Preparing to unpack .../libcc1-0_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libcc1-0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libgomp1:amd64.
Preparing to unpack .../libgomp1_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libgomp1:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libitm1:amd64.
Preparing to unpack .../libitm1_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libitm1:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libatomic1:amd64.
Preparing to unpack .../libatomic1_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libatomic1:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libasan2:amd64.
Preparing to unpack .../libasan2_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libasan2:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package liblsan0:amd64.
Preparing to unpack .../liblsan0_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking liblsan0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libtsan0:amd64.
Preparing to unpack .../libtsan0_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libtsan0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libubsan0:amd64.
Preparing to unpack .../libubsan0_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libubsan0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libcilkrts5:amd64.
Preparing to unpack .../libcilkrts5_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libcilkrts5:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libmpx0:amd64.
Preparing to unpack .../libmpx0_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libmpx0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libquadmath0:amd64.
Preparing to unpack .../libquadmath0_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package libgcc-5-dev:amd64.
Preparing to unpack .../libgcc-5-dev_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package gcc-5.
Preparing to unpack .../gcc-5_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking gcc-5 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package gcc.
Preparing to unpack .../gcc_4%3a5.3.1-1ubuntu1_amd64.deb ...
Unpacking gcc (4:5.3.1-1ubuntu1) ...
Selecting previously unselected package libstdc++-5-dev:amd64.
Preparing to unpack .../libstdc++-5-dev_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking libstdc++-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package g++-5.
Preparing to unpack .../g++-5_5.4.0-6ubuntu1~16.04.4_amd64.deb ...
Unpacking g++-5 (5.4.0-6ubuntu1~16.04.4) ...
Selecting previously unselected package g++.
Preparing to unpack .../g++_4%3a5.3.1-1ubuntu1_amd64.deb ...
Unpacking g++ (4:5.3.1-1ubuntu1) ...
Selecting previously unselected package make.
Preparing to unpack .../archives/make_4.1-6_amd64.deb ...
Unpacking make (4.1-6) ...
Selecting previously unselected package libdpkg-perl.
Preparing to unpack .../libdpkg-perl_1.18.4ubuntu1.1_all.deb ...
Unpacking libdpkg-perl (1.18.4ubuntu1.1) ...
Selecting previously unselected package xz-utils.
Preparing to unpack .../xz-utils_5.1.1alpha+20120614-2ubuntu2_amd64.deb ...
Unpacking xz-utils (5.1.1alpha+20120614-2ubuntu2) ...
Selecting previously unselected package patch.
Preparing to unpack .../patch_2.7.5-1_amd64.deb ...
Unpacking patch (2.7.5-1) ...
Selecting previously unselected package dpkg-dev.
Preparing to unpack .../dpkg-dev_1.18.4ubuntu1.1_all.deb ...
Unpacking dpkg-dev (1.18.4ubuntu1.1) ...
Selecting previously unselected package build-essential.
Preparing to unpack .../build-essential_12.1ubuntu2_amd64.deb ...
Unpacking build-essential (12.1ubuntu2) ...
Selecting previously unselected package libfakeroot:amd64.
Preparing to unpack .../libfakeroot_1.20.2-1ubuntu1_amd64.deb ...
Unpacking libfakeroot:amd64 (1.20.2-1ubuntu1) ...
Selecting previously unselected package fakeroot.
Preparing to unpack .../fakeroot_1.20.2-1ubuntu1_amd64.deb ...
Unpacking fakeroot (1.20.2-1ubuntu1) ...
Selecting previously unselected package liberror-perl.
Preparing to unpack .../liberror-perl_0.17-1.2_all.deb ...
Unpacking liberror-perl (0.17-1.2) ...
Selecting previously unselected package git-man.
Preparing to unpack .../git-man_1%3a2.7.4-0ubuntu1_all.deb ...
Unpacking git-man (1:2.7.4-0ubuntu1) ...
Selecting previously unselected package git.
Preparing to unpack .../git_1%3a2.7.4-0ubuntu1_amd64.deb ...
Unpacking git (1:2.7.4-0ubuntu1) ...
Selecting previously unselected package python-pkg-resources.
Preparing to unpack .../python-pkg-resources_20.7.0-1_all.deb ...
Unpacking python-pkg-resources (20.7.0-1) ...
Selecting previously unselected package gyp.
Preparing to unpack .../gyp_0.1+20150913git1f374df9-1ubuntu1_all.deb ...
Unpacking gyp (0.1+20150913git1f374df9-1ubuntu1) ...
Selecting previously unselected package javascript-common.
Preparing to unpack .../javascript-common_11_all.deb ...
Unpacking javascript-common (11) ...
Selecting previously unselected package libalgorithm-diff-perl.
Preparing to unpack .../libalgorithm-diff-perl_1.19.03-1_all.deb ...
Unpacking libalgorithm-diff-perl (1.19.03-1) ...
Selecting previously unselected package libalgorithm-diff-xs-perl.
Preparing to unpack .../libalgorithm-diff-xs-perl_0.04-4build1_amd64.deb ...
Unpacking libalgorithm-diff-xs-perl (0.04-4build1) ...
Selecting previously unselected package libalgorithm-merge-perl.
Preparing to unpack .../libalgorithm-merge-perl_0.08-3_all.deb ...
Unpacking libalgorithm-merge-perl (0.08-3) ...
Selecting previously unselected package libfile-fcntllock-perl.
Preparing to unpack .../libfile-fcntllock-perl_0.22-3_amd64.deb ...
Unpacking libfile-fcntllock-perl (0.22-3) ...
Selecting previously unselected package libjs-jquery.
Preparing to unpack .../libjs-jquery_1.11.3+dfsg-4_all.deb ...
Unpacking libjs-jquery (1.11.3+dfsg-4) ...
Selecting previously unselected package libjs-node-uuid.
Preparing to unpack .../libjs-node-uuid_1.4.0-1_all.deb ...
Unpacking libjs-node-uuid (1.4.0-1) ...
Selecting previously unselected package libjs-underscore.
Preparing to unpack .../libjs-underscore_1.7.0~dfsg-1ubuntu1_all.deb ...
Unpacking libjs-underscore (1.7.0~dfsg-1ubuntu1) ...
Selecting previously unselected package zlib1g-dev:amd64.
Preparing to unpack .../zlib1g-dev_1%3a1.2.8.dfsg-2ubuntu4_amd64.deb ...
Unpacking zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4) ...
Selecting previously unselected package libssl-dev:amd64.
Preparing to unpack .../libssl-dev_1.0.2g-1ubuntu4.6_amd64.deb ...
Unpacking libssl-dev:amd64 (1.0.2g-1ubuntu4.6) ...
Selecting previously unselected package libssl-doc.
Preparing to unpack .../libssl-doc_1.0.2g-1ubuntu4.6_all.deb ...
Unpacking libssl-doc (1.0.2g-1ubuntu4.6) ...
Selecting previously unselected package libuv1:amd64.
Preparing to unpack .../libuv1_1.8.0-1_amd64.deb ...
Unpacking libuv1:amd64 (1.8.0-1) ...
Selecting previously unselected package libuv1-dev:amd64.
Preparing to unpack .../libuv1-dev_1.8.0-1_amd64.deb ...
Unpacking libuv1-dev:amd64 (1.8.0-1) ...
Selecting previously unselected package manpages-dev.
Preparing to unpack .../manpages-dev_4.04-2_all.deb ...
Unpacking manpages-dev (4.04-2) ...
Selecting previously unselected package nodejs.
Preparing to unpack .../nodejs_4.2.6~dfsg-1ubuntu4.1_amd64.deb ...
Unpacking nodejs (4.2.6~dfsg-1ubuntu4.1) ...
Selecting previously unselected package node-async.
Preparing to unpack .../node-async_0.8.0-1_all.deb ...
Unpacking node-async (0.8.0-1) ...
Selecting previously unselected package node-node-uuid.
Preparing to unpack .../node-node-uuid_1.4.0-1_all.deb ...
Unpacking node-node-uuid (1.4.0-1) ...
Selecting previously unselected package node-underscore.
Preparing to unpack .../node-underscore_1.7.0~dfsg-1ubuntu1_all.deb ...
Unpacking node-underscore (1.7.0~dfsg-1ubuntu1) ...
Selecting previously unselected package rename.
Preparing to unpack .../archives/rename_0.20-4_all.deb ...
Unpacking rename (0.20-4) ...
Selecting previously unselected package libjs-inherits.
Preparing to unpack .../libjs-inherits_2.0.1-3_all.deb ...
Unpacking libjs-inherits (2.0.1-3) ...
Selecting previously unselected package node-abbrev.
Preparing to unpack .../node-abbrev_1.0.5-2_all.deb ...
Unpacking node-abbrev (1.0.5-2) ...
Selecting previously unselected package node-ansi.
Preparing to unpack .../node-ansi_0.3.0-2_all.deb ...
Unpacking node-ansi (0.3.0-2) ...
Selecting previously unselected package node-ansi-color-table.
Preparing to unpack .../node-ansi-color-table_1.0.0-1_all.deb ...
Unpacking node-ansi-color-table (1.0.0-1) ...
Selecting previously unselected package node-archy.
Preparing to unpack .../node-archy_0.0.2-1_all.deb ...
Unpacking node-archy (0.0.2-1) ...
Selecting previously unselected package node-inherits.
Preparing to unpack .../node-inherits_2.0.1-3_all.deb ...
Unpacking node-inherits (2.0.1-3) ...
Selecting previously unselected package node-block-stream.
Preparing to unpack .../node-block-stream_0.0.7-1_all.deb ...
Unpacking node-block-stream (0.0.7-1) ...
Selecting previously unselected package node-delayed-stream.
Preparing to unpack .../node-delayed-stream_0.0.5-1_all.deb ...
Unpacking node-delayed-stream (0.0.5-1) ...
Selecting previously unselected package node-combined-stream.
Preparing to unpack .../node-combined-stream_0.0.5-1_all.deb ...
Unpacking node-combined-stream (0.0.5-1) ...
Selecting previously unselected package node-cookie-jar.
Preparing to unpack .../node-cookie-jar_0.3.1-1_all.deb ...
Unpacking node-cookie-jar (0.3.1-1) ...
Selecting previously unselected package node-forever-agent.
Preparing to unpack .../node-forever-agent_0.5.1-1_all.deb ...
Unpacking node-forever-agent (0.5.1-1) ...
Selecting previously unselected package node-mime.
Preparing to unpack .../node-mime_1.3.4-1_all.deb ...
Unpacking node-mime (1.3.4-1) ...
Selecting previously unselected package node-form-data.
Preparing to unpack .../node-form-data_0.1.0-1_all.deb ...
Unpacking node-form-data (0.1.0-1) ...
Selecting previously unselected package node-rimraf.
Preparing to unpack .../node-rimraf_2.2.8-1_all.deb ...
Unpacking node-rimraf (2.2.8-1) ...
Selecting previously unselected package node-mkdirp.
Preparing to unpack .../node-mkdirp_0.5.0-1_all.deb ...
Unpacking node-mkdirp (0.5.0-1) ...
Selecting previously unselected package node-graceful-fs.
Preparing to unpack .../node-graceful-fs_3.0.2-1_all.deb ...
Unpacking node-graceful-fs (3.0.2-1) ...
Selecting previously unselected package node-fstream.
Preparing to unpack .../node-fstream_0.1.24-1_all.deb ...
Unpacking node-fstream (0.1.24-1) ...
Selecting previously unselected package node-lru-cache.
Preparing to unpack .../node-lru-cache_2.3.1-1_all.deb ...
Unpacking node-lru-cache (2.3.1-1) ...
Selecting previously unselected package node-sigmund.
Preparing to unpack .../node-sigmund_1.0.0-1_all.deb ...
Unpacking node-sigmund (1.0.0-1) ...
Selecting previously unselected package node-minimatch.
Preparing to unpack .../node-minimatch_1.0.0-1_all.deb ...
Unpacking node-minimatch (1.0.0-1) ...
Selecting previously unselected package node-fstream-ignore.
Preparing to unpack .../node-fstream-ignore_0.0.6-2_all.deb ...
Unpacking node-fstream-ignore (0.0.6-2) ...
Selecting previously unselected package node-github-url-from-git.
Preparing to unpack .../node-github-url-from-git_1.1.1-1_all.deb ...
Unpacking node-github-url-from-git (1.1.1-1) ...
Selecting previously unselected package node-once.
Preparing to unpack .../node-once_1.1.1-1_all.deb ...
Unpacking node-once (1.1.1-1) ...
Selecting previously unselected package node-glob.
Preparing to unpack .../node-glob_4.0.5-1_all.deb ...
Unpacking node-glob (4.0.5-1) ...
Selecting previously unselected package nodejs-dev.
Preparing to unpack .../nodejs-dev_4.2.6~dfsg-1ubuntu4.1_amd64.deb ...
Unpacking nodejs-dev (4.2.6~dfsg-1ubuntu4.1) ...
Selecting previously unselected package node-nopt.
Preparing to unpack .../node-nopt_3.0.1-1_all.deb ...
Unpacking node-nopt (3.0.1-1) ...
Selecting previously unselected package node-npmlog.
Preparing to unpack .../node-npmlog_0.0.4-1_all.deb ...
Unpacking node-npmlog (0.0.4-1) ...
Selecting previously unselected package node-osenv.
Preparing to unpack .../node-osenv_0.1.0-1_all.deb ...
Unpacking node-osenv (0.1.0-1) ...
Selecting previously unselected package node-tunnel-agent.
Preparing to unpack .../node-tunnel-agent_0.3.1-1_all.deb ...
Unpacking node-tunnel-agent (0.3.1-1) ...
Selecting previously unselected package node-json-stringify-safe.
Preparing to unpack .../node-json-stringify-safe_5.0.0-1_all.deb ...
Unpacking node-json-stringify-safe (5.0.0-1) ...
Selecting previously unselected package node-qs.
Preparing to unpack .../node-qs_2.2.4-1_all.deb ...
Unpacking node-qs (2.2.4-1) ...
Selecting previously unselected package node-request.
Preparing to unpack .../node-request_2.26.1-1_all.deb ...
Unpacking node-request (2.26.1-1) ...
Selecting previously unselected package node-semver.
Preparing to unpack .../node-semver_2.1.0-2_all.deb ...
Unpacking node-semver (2.1.0-2) ...
Selecting previously unselected package node-tar.
Preparing to unpack .../node-tar_1.0.3-2_all.deb ...
Unpacking node-tar (1.0.3-2) ...
Selecting previously unselected package node-which.
Preparing to unpack .../node-which_1.0.5-2_all.deb ...
Unpacking node-which (1.0.5-2) ...
Selecting previously unselected package node-gyp.
Preparing to unpack .../node-gyp_3.0.3-2ubuntu1_all.deb ...
Unpacking node-gyp (3.0.3-2ubuntu1) ...
Selecting previously unselected package node-ini.
Preparing to unpack .../node-ini_1.1.0-1_all.deb ...
Unpacking node-ini (1.1.0-1) ...
Selecting previously unselected package node-lockfile.
Preparing to unpack .../node-lockfile_0.4.1-1_all.deb ...
Unpacking node-lockfile (0.4.1-1) ...
Selecting previously unselected package node-mute-stream.
Preparing to unpack .../node-mute-stream_0.0.4-1_all.deb ...
Unpacking node-mute-stream (0.0.4-1) ...
Selecting previously unselected package node-normalize-package-data.
Preparing to unpack .../node-normalize-package-data_0.2.2-1_all.deb ...
Unpacking node-normalize-package-data (0.2.2-1) ...
Selecting previously unselected package node-read.
Preparing to unpack .../node-read_1.0.5-1_all.deb ...
Unpacking node-read (1.0.5-1) ...
Selecting previously unselected package node-read-package-json.
Preparing to unpack .../node-read-package-json_1.2.4-1_all.deb ...
Unpacking node-read-package-json (1.2.4-1) ...
Selecting previously unselected package node-retry.
Preparing to unpack .../node-retry_0.6.0-1_all.deb ...
Unpacking node-retry (0.6.0-1) ...
Selecting previously unselected package node-sha.
Preparing to unpack .../node-sha_1.2.3-1_all.deb ...
Unpacking node-sha (1.2.3-1) ...
Selecting previously unselected package node-slide.
Preparing to unpack .../node-slide_1.1.4-1_all.deb ...
Unpacking node-slide (1.1.4-1) ...
Selecting previously unselected package npm.
Preparing to unpack .../npm_3.5.2-0ubuntu4_all.deb ...
Unpacking npm (3.5.2-0ubuntu4) ...
Processing triggers for libc-bin (2.23-0ubuntu5) ...
Processing triggers for systemd (229-4ubuntu16) ...
Setting up libatm1:amd64 (1:2.5.1-1.5) ...
Setting up libmnl0:amd64 (1.0.3-5) ...
Setting up libpopt0:amd64 (1.16-10) ...
Setting up libgdbm3:amd64 (1.8.3-13.1) ...
Setting up libxau6:amd64 (1:1.0.8-1) ...
Setting up libxdmcp6:amd64 (1:1.1.2-1.1) ...
Setting up libxcb1:amd64 (1.11.1-1ubuntu1) ...
Setting up libx11-data (2:1.6.3-1ubuntu2) ...
Setting up libx11-6:amd64 (2:1.6.3-1ubuntu2) ...
Setting up libxext6:amd64 (2:1.3.3-1) ...
Setting up perl-modules-5.22 (5.22.1-9) ...
Setting up libperl5.22:amd64 (5.22.1-9) ...
Setting up perl (5.22.1-9) ...
update-alternatives: using /usr/bin/prename to provide /usr/bin/rename (rename) in auto mode
Setting up mime-support (3.59ubuntu1) ...
Setting up libexpat1:amd64 (2.1.0-7ubuntu0.16.04.2) ...
Setting up libffi6:amd64 (3.2.1-4) ...
Setting up libsqlite3-0:amd64 (3.11.0-1ubuntu1) ...
Setting up libssl1.0.0:amd64 (1.0.2g-1ubuntu4.6) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Setting up libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.1) ...
Setting up python2.7 (2.7.12-1ubuntu0~16.04.1) ...
Setting up libpython-stdlib:amd64 (2.7.11-1) ...
Setting up python (2.7.11-1) ...
Setting up libgmp10:amd64 (2:6.1.0+dfsg-2) ...
Setting up libmpfr4:amd64 (3.1.4-1) ...
Setting up libmpc3:amd64 (1.0.3-1) ...
Setting up bzip2 (1.0.6-8) ...
Setting up libmagic1:amd64 (1:5.25-2ubuntu1) ...
Setting up file (1:5.25-2ubuntu1) ...
Setting up iproute2 (4.3.0-1ubuntu3) ...
Setting up ifupdown (0.8.10ubuntu1.2) ...
Creating /etc/network/interfaces.
Setting up libisc-export160 (1:9.10.3.dfsg.P4-8ubuntu1.5) ...
Setting up libdns-export162 (1:9.10.3.dfsg.P4-8ubuntu1.5) ...
Setting up isc-dhcp-client (4.3.3-5ubuntu12.6) ...
Setting up isc-dhcp-common (4.3.3-5ubuntu12.6) ...
Setting up less (481-2.1ubuntu0.1) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Setting up libbsd0:amd64 (0.8.2-1) ...
Setting up libnettle6:amd64 (3.2-1ubuntu0.16.04.1) ...
Setting up libhogweed4:amd64 (3.2-1ubuntu0.16.04.1) ...
Setting up libidn11:amd64 (1.32-3ubuntu1.1) ...
Setting up libp11-kit0:amd64 (0.23.2-5~ubuntu16.04.1) ...
Setting up libtasn1-6:amd64 (4.7-3ubuntu0.16.04.1) ...
Setting up libgnutls30:amd64 (3.4.10-4ubuntu1.2) ...
Setting up libxtables11:amd64 (1.6.0-2ubuntu3) ...
Setting up netbase (5.3) ...
Setting up openssl (1.0.2g-1ubuntu4.6) ...
Setting up ca-certificates (20160104ubuntu1) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Setting up krb5-locales (1.13.2+dfsg-5ubuntu2) ...
Setting up libroken18-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libasn1-8-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libkrb5support0:amd64 (1.13.2+dfsg-5ubuntu2) ...
Setting up libk5crypto3:amd64 (1.13.2+dfsg-5ubuntu2) ...
Setting up libkeyutils1:amd64 (1.5.9-8ubuntu1) ...
Setting up libkrb5-3:amd64 (1.13.2+dfsg-5ubuntu2) ...
Setting up libgssapi-krb5-2:amd64 (1.13.2+dfsg-5ubuntu2) ...
Setting up libhcrypto4-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libheimbase1-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libwind0-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libhx509-5-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libkrb5-26-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libheimntlm0-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libgssapi3-heimdal:amd64 (1.7~git20150920+dfsg-4ubuntu1) ...
Setting up libsasl2-modules-db:amd64 (2.1.26.dfsg1-14build1) ...
Setting up libsasl2-2:amd64 (2.1.26.dfsg1-14build1) ...
Setting up libldap-2.4-2:amd64 (2.4.42+dfsg-2ubuntu3.1) ...
Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d-1build1) ...
Setting up libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.2) ...
Setting up libedit2:amd64 (3.1-20150325-1ubuntu2) ...
Setting up libicu55:amd64 (55.1-7) ...
Setting up libsasl2-modules:amd64 (2.1.26.dfsg1-14build1) ...
Setting up libxmuu1:amd64 (2:1.1.2-2) ...
Setting up manpages (4.04-2) ...
Setting up openssh-client (1:7.2p2-4ubuntu2.1) ...
Setting up rsync (3.1.1-3ubuntu1) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of restart.
Setting up xauth (1:1.0.9-1ubuntu2) ...
Setting up binutils (2.26.1-1ubuntu1~16.04.3) ...
Setting up libc-dev-bin (2.23-0ubuntu5) ...
Setting up linux-libc-dev:amd64 (4.4.0-66.87) ...
Setting up libc6-dev:amd64 (2.23-0ubuntu5) ...
Setting up libisl15:amd64 (0.16.1-1) ...
Setting up cpp-5 (5.4.0-6ubuntu1~16.04.4) ...
Setting up cpp (4:5.3.1-1ubuntu1) ...
Setting up libcc1-0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libgomp1:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libitm1:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libatomic1:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libasan2:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up liblsan0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libtsan0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libubsan0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libcilkrts5:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libmpx0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up gcc-5 (5.4.0-6ubuntu1~16.04.4) ...
Setting up gcc (4:5.3.1-1ubuntu1) ...
Setting up libstdc++-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up g++-5 (5.4.0-6ubuntu1~16.04.4) ...
Setting up g++ (4:5.3.1-1ubuntu1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up make (4.1-6) ...
Setting up libdpkg-perl (1.18.4ubuntu1.1) ...
Setting up xz-utils (5.1.1alpha+20120614-2ubuntu2) ...
update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode
Setting up patch (2.7.5-1) ...
Setting up dpkg-dev (1.18.4ubuntu1.1) ...
Setting up build-essential (12.1ubuntu2) ...
Setting up libfakeroot:amd64 (1.20.2-1ubuntu1) ...
Setting up fakeroot (1.20.2-1ubuntu1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode
Setting up liberror-perl (0.17-1.2) ...
Setting up git-man (1:2.7.4-0ubuntu1) ...
Setting up git (1:2.7.4-0ubuntu1) ...
Setting up python-pkg-resources (20.7.0-1) ...
Setting up gyp (0.1+20150913git1f374df9-1ubuntu1) ...
Setting up javascript-common (11) ...
Setting up libalgorithm-diff-perl (1.19.03-1) ...
Setting up libalgorithm-diff-xs-perl (0.04-4build1) ...
Setting up libalgorithm-merge-perl (0.08-3) ...
Setting up libfile-fcntllock-perl (0.22-3) ...
Setting up libjs-jquery (1.11.3+dfsg-4) ...
Setting up libjs-node-uuid (1.4.0-1) ...
Setting up libjs-underscore (1.7.0~dfsg-1ubuntu1) ...
Setting up zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4) ...
Setting up libssl-dev:amd64 (1.0.2g-1ubuntu4.6) ...
Setting up libssl-doc (1.0.2g-1ubuntu4.6) ...
Setting up libuv1:amd64 (1.8.0-1) ...
Setting up libuv1-dev:amd64 (1.8.0-1) ...
Setting up manpages-dev (4.04-2) ...
Setting up nodejs (4.2.6~dfsg-1ubuntu4.1) ...
update-alternatives: using /usr/bin/nodejs to provide /usr/bin/js (js) in auto mode
Setting up node-async (0.8.0-1) ...
Setting up node-node-uuid (1.4.0-1) ...
Setting up node-underscore (1.7.0~dfsg-1ubuntu1) ...
Setting up rename (0.20-4) ...
update-alternatives: using /usr/bin/file-rename to provide /usr/bin/rename (rename) in auto mode
Setting up libjs-inherits (2.0.1-3) ...
Setting up node-abbrev (1.0.5-2) ...
Setting up node-ansi (0.3.0-2) ...
Setting up node-ansi-color-table (1.0.0-1) ...
Setting up node-archy (0.0.2-1) ...
Setting up node-inherits (2.0.1-3) ...
Setting up node-block-stream (0.0.7-1) ...
Setting up node-delayed-stream (0.0.5-1) ...
Setting up node-combined-stream (0.0.5-1) ...
Setting up node-cookie-jar (0.3.1-1) ...
Setting up node-forever-agent (0.5.1-1) ...
Setting up node-mime (1.3.4-1) ...
Setting up node-form-data (0.1.0-1) ...
Setting up node-rimraf (2.2.8-1) ...
Setting up node-mkdirp (0.5.0-1) ...
Setting up node-graceful-fs (3.0.2-1) ...
Setting up node-fstream (0.1.24-1) ...
Setting up node-lru-cache (2.3.1-1) ...
Setting up node-sigmund (1.0.0-1) ...
Setting up node-minimatch (1.0.0-1) ...
Setting up node-fstream-ignore (0.0.6-2) ...
Setting up node-github-url-from-git (1.1.1-1) ...
Setting up node-once (1.1.1-1) ...
Setting up node-glob (4.0.5-1) ...
Setting up nodejs-dev (4.2.6~dfsg-1ubuntu4.1) ...
Setting up node-nopt (3.0.1-1) ...
Setting up node-npmlog (0.0.4-1) ...
Setting up node-osenv (0.1.0-1) ...
Setting up node-tunnel-agent (0.3.1-1) ...
Setting up node-json-stringify-safe (5.0.0-1) ...
Setting up node-qs (2.2.4-1) ...
Setting up node-request (2.26.1-1) ...
Setting up node-semver (2.1.0-2) ...
Setting up node-tar (1.0.3-2) ...
Setting up node-which (1.0.5-2) ...
Setting up node-gyp (3.0.3-2ubuntu1) ...
Setting up node-ini (1.1.0-1) ...
Setting up node-lockfile (0.4.1-1) ...
Setting up node-mute-stream (0.0.4-1) ...
Setting up node-normalize-package-data (0.2.2-1) ...
Setting up node-read (1.0.5-1) ...
Setting up node-read-package-json (1.2.4-1) ...
Setting up node-retry (0.6.0-1) ...
Setting up node-sha (1.2.3-1) ...
Setting up node-slide (1.1.4-1) ...
Setting up npm (3.5.2-0ubuntu4) ...
Processing triggers for libc-bin (2.23-0ubuntu5) ...
Processing triggers for systemd (229-4ubuntu16) ...
Processing triggers for ca-certificates (20160104ubuntu1) ...
Updating certificates in /etc/ssl/certs...
173 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Step 1.4: Make sure that ‘node’ is found in the PATH

Many NPM packages run ‘node’ commands. For those to work, the system needs to find the ‘node’ executable. In case of Ubuntu, there is an node executable on /usr/sbin/node, but it has nothing to do with Node.js. Ubuntu installs Node.js on /usr/bin/nodejs. For the system to find the correct node command, it is sufficient to create a sybolic link like follows (see also this StackOverflow Q&A):

(container)$ ln -s nodejs /usr/bin/node

If this step is not performed, you will hit an error message

gyp: Call to 'node -e "require('nan')"' returned exit status 127 while in binding.gyp. while trying to load binding.gyp

in the npm install step below; see Appendix A for details.

Step 1.5: Clone BrowserStack Protractor Example from Git

The BrowserStack Protractor Example is cloned like follows:

(container)# git clone https://github.com/browserstack/protractor-browserstack
Cloning into 'protractor-browserstack'...
remote: Counting objects: 185, done.
remote: Total 185 (delta 0), reused 0 (delta 0), pack-reused 185
Receiving objects: 100% (185/185), 28.39 KiB | 0 bytes/s, done.
Resolving deltas: 100% (72/72), done.
Checking connectivity... done.

Step 1.6: Install Dependencies

The next commands is needed for downloading and installing the dependencies:

(container)# cd protractor-browserstack; npm install
> bufferutil@1.2.1 install /app/protractor-browserstack/node_modules/bufferutil
> node-gyp rebuild

make: Entering directory '/app/protractor-browserstack/node_modules/bufferutil/build'
 CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
 SOLINK_MODULE(target) Release/obj.target/bufferutil.node
 COPY Release/bufferutil.node
make: Leaving directory '/app/protractor-browserstack/node_modules/bufferutil/build'

> utf-8-validate@1.2.2 install /app/protractor-browserstack/node_modules/utf-8-validate
> node-gyp rebuild

make: Entering directory '/app/protractor-browserstack/node_modules/utf-8-validate/build'
 CXX(target) Release/obj.target/validation/src/validation.o
 SOLINK_MODULE(target) Release/obj.target/validation.node
 COPY Release/validation.node
make: Leaving directory '/app/protractor-browserstack/node_modules/utf-8-validate/build'
protractor-browserstack@0.1.0 /app/protractor-browserstack
+-- browserstack-local@1.3.0
| +-- https-proxy-agent@1.0.0
| | +-- agent-base@2.0.1
| | | `-- semver@5.0.3
| | +-- debug@2.6.2
| | | `-- ms@0.7.2
| | `-- extend@3.0.0
| +-- is-running@2.1.0
| +-- sinon@1.17.7
| | +-- formatio@1.1.1
| | +-- lolex@1.3.2
| | +-- samsam@1.1.2
| | `-- util@0.10.3
| `-- temp-fs@0.9.9
| `-- rimraf@2.5.4
| `-- glob@7.1.1
| +-- fs.realpath@1.0.0
| +-- inflight@1.0.6
| | `-- wrappy@1.0.2
| +-- minimatch@3.0.3
| | `-- brace-expansion@1.1.6
| | +-- balanced-match@0.4.2
| | `-- concat-map@0.0.1
| +-- once@1.4.0
| `-- path-is-absolute@1.0.1
`-- protractor@2.5.1
 +-- accessibility-developer-tools@2.6.0
 +-- adm-zip@0.4.4
 +-- glob@3.2.11
 | +-- inherits@2.0.1
 | `-- minimatch@0.3.0
 | +-- lru-cache@2.7.3
 | `-- sigmund@1.0.1
 +-- html-entities@1.1.3
 +-- jasmine@2.3.2
 | +-- exit@0.1.2
 | +-- glob@3.2.11
 | | `-- minimatch@0.3.0
 | `-- jasmine-core@2.3.4
 +-- jasminewd@1.1.0
 +-- jasminewd2@0.0.6
 +-- lodash@2.4.2
 +-- minijasminenode@1.1.1
 +-- optimist@0.6.1
 | +-- minimist@0.0.10
 | `-- wordwrap@0.0.3
 +-- q@1.0.0
 +-- request@2.57.0
 | +-- aws-sign2@0.5.0
 | +-- bl@0.9.5
 | | `-- readable-stream@1.0.34
 | | +-- core-util-is@1.0.2
 | | +-- isarray@0.0.1
 | | `-- string_decoder@0.10.31
 | +-- caseless@0.10.0
 | +-- combined-stream@1.0.5
 | | `-- delayed-stream@1.0.0
 | +-- forever-agent@0.6.1
 | +-- form-data@0.2.0
 | | +-- async@0.9.2
 | | `-- combined-stream@0.0.7
 | | `-- delayed-stream@0.0.5
 | +-- har-validator@1.8.0
 | | +-- bluebird@2.11.0
 | | +-- chalk@1.1.3
 | | | +-- ansi-styles@2.2.1
 | | | +-- escape-string-regexp@1.0.5
 | | | +-- has-ansi@2.0.0
 | | | | `-- ansi-regex@2.1.1
 | | | +-- strip-ansi@3.0.1
 | | | `-- supports-color@2.0.0
 | | +-- commander@2.9.0
 | | | `-- graceful-readlink@1.0.1
 | | `-- is-my-json-valid@2.16.0
 | | +-- generate-function@2.0.0
 | | +-- generate-object-property@1.2.0
 | | | `-- is-property@1.0.2
 | | +-- jsonpointer@4.0.1
 | | `-- xtend@4.0.1
 | +-- hawk@2.3.1
 | | +-- boom@2.10.1
 | | +-- cryptiles@2.0.5
 | | +-- hoek@2.16.3
 | | `-- sntp@1.0.9
 | +-- http-signature@0.11.0
 | | +-- asn1@0.1.11
 | | +-- assert-plus@0.1.5
 | | `-- ctype@0.5.3
 | +-- isstream@0.1.2
 | +-- json-stringify-safe@5.0.1
 | +-- mime-types@2.0.14
 | | `-- mime-db@1.12.0
 | +-- node-uuid@1.4.7
 | +-- oauth-sign@0.8.2
 | +-- qs@3.1.0
 | +-- stringstream@0.0.5
 | +-- tough-cookie@2.3.2
 | | `-- punycode@1.4.1
 | `-- tunnel-agent@0.4.3
 +-- saucelabs@1.0.1
 +-- selenium-webdriver@2.47.0
 | +-- tmp@0.0.24
 | +-- ws@0.8.1
 | | +-- bufferutil@1.2.1
 | | | +-- bindings@1.2.1
 | | | `-- nan@2.5.1
 | | +-- options@0.0.6
 | | +-- ultron@1.0.2
 | | `-- utf-8-validate@1.2.2
 | | `-- nan@2.4.0
 | `-- xml2js@0.4.4
 | +-- sax@0.6.1
 | `-- xmlbuilder@8.2.2
 `-- source-map-support@0.2.10
 `-- source-map@0.1.32
 `-- amdefine@1.0.1

Note: this command requires ‘make’ to be installed. In the official Ubuntu 16.04 Docker image, this is the case. However, if it is missing in your case, you need to issue the command apt-get install build-essential as root or with sudo.

Step 1.7: Add BrowserStack Credentials

Now let us specify the BrowserStack credentials you can find in the “Automate” section on your BrowserStack Account Settings page:

(container)# export BROWSERSTACK_USERNAME=your_browserstack_user_id
(container)# export BROWSERSTACK_ACCESS_KEY=your_browserstack_key

Note that the environment variables differ from the the ones we had used in the Gulp examples on my previous blog post: BROWSERSTACK_USERNAME instead of BROWSERSTACK_USER and  BROWSERSTACK_ACCESS_KEY instead of BROWSERSTACK_KEY

Step 1.8: Run the BrowserStack automated Test

Finally, we start the test session via npm run local:

(container)# npm run local

Connecting local
Connected. Now testing...
Using the selenium server at http://hub-cloud.browserstack.com/wd/hub
[launcher] Running 1 instances of WebDriver
.

Finished in 0.763 seconds
1 test, 1 assertion, 0 failures

[launcher] 0 instance(s) of WebDriver still running
[launcher] chrome #1 passed

With that, we have automatically tested a chrome browser using BrowserStack:

Excellent!

Step 1.9: Review the Results on BrowserStack Automate Logs Page

On this BrowserStack link, you can see in detail, which steps were taken during the automated test:

And here are visual logs with screenshot:

Soon after running the first automated test via BrowserStack, I have found an Email from BrowserStack in my Email inbox with the information that they have noticed my first automated test and that I can contact them in case of any questions.

Part 2: Integration of BrowserStack into Jenkins

Step 2.1: Start and Connect to Jenkins

If you have a Jenkins Server that is up and running, you can skip this step, including the sub-steps.

Step 2.1.1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the Jenkins blog series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

50c150e35a774cexxxxxxxxxxxxxxx

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 2.1.2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

Note that this configuration is not permanent, unless you define the port mappings in the Vagrantfile as follows (see official Vagrant documentation):

config.vm.network "forwarded_port", guest: 8080, host: 8080

Step 2.1.3: Initialize Jenkins: Unlock Jenkins

Unlock Jenkins

insert the one-time password found in the log during startup or on /var/jenkins_home/secrets/initialAdminPassword

-> Continue

Step 2.1.4: Initialize Jenkins: Install Plugins

-> Install suggested plugins

Installing suggested plugins

Wait for the installation process to complete.

Step 2.1.5: Initialize Jenkins: Create Admin User

-> Create First Admin User

-> Save and Finish or “Continue as admin”

-> Start using Jenkins

Note: I recommend to log out and log in in order to test the login.

Step 2.2: Prepare Git Usage

As described in this StackOverflow Q&A, we need to add the Git username and email address, since Jenkins tries to tag and commit on the Git repo, which requires those configuration items to be set. For that, we perform:

-> 2017-02-25-18_04_32-job-dsl-hello-world-job-1-console-jenkins

-> Manage Jenkins

-> Configure System

-> scroll down to “Git plugin”

-> Git plugin; user.name = jenkins and user.email = admin@jenkins.org

Step 2.3: Install the Jenkins Node.js Plugin

-> Jenkins Home leading to the Dashboard (left upper corner)

-> Manage Jenkins

-> Manage Plugins

-> Plugins - Available

->  2017-03-04-19_19_34-update-center-jenkins

-> Choose: NodeJS

-> Install without restart

-> observe:

NodeJS Plugin Installation Success

Step 2.4: Prepare Node.js Usage

-> Manage Jenkins

-> Global Tool Configuration

-> NodeJS: specify name: "NodeJS 7.7.1" and add Global npm packages to install: "gulp"

-> Choose NodeJS, check “Install automatically” and specify Global npm packages to install: protractor:

Choose NodeJS, check "Install automatically" and specify Global npm packages to install: protractor

 

-> Save Button (in Global Tool Installer)

Step 2.5: Install the BrowserStack Plugin

The BrowserStack Plugin can be installed like any other Jenkins plugin:

-> Jenkins Home leading to the Dashboard (left upper corner)

-> Manage Jenkins
-> Manage Plugins
-> Plugins - Available

-> Filter: BrowserStack

-> Choose: BrowserStack

-> Install without restart

– > observe:

BrowserStack Installation Success

Step 2.6: Prepare BrowserStack Usage

I am closely following the official documentation:

-> Manage Jenkins

-> Configure System

-> BrowserStack > BrowserStack Credentials > Add > Jenkins

-> Add Username and Access Key as found on your BrowserStack account page:

Insert Username and Key as shown on https://www.browserstack.com/accounts/settings > Aoutomate

-> Add

 

BrowserStack Global Settings shows used credentials after having configured them

-> Save

Step 2.7: Create BrowserStack Jenkins Job

-> New Item

-> Enter Item Name: BrowserStackJob

-> Freestyle Project

-> OK

Step 2.8: Configure BrowserStack Jenkins Job

Step 2.8.1: Configure Git Download

In my case, I have forked the following project from Github: https://github.com/browserstack/protractor-browserstack

-> 

Step 2.8.2: Configure BrowserStack Build Environment

-> Build Environment: check "BrowserStack" and "BrowserStack Local"; keep defaults

Step 2.8.3: Configure Build Step: Set Path

In addition check the checkbox for “Provide Node & npm bin/ folder to PATH”

-> in addition check the checkbox for: Provide Node and; npm bin folder to PATH

If you click on the (?), you will be informed that the BrowserStack user and password are available as environment variables.

Select the BrowserStack credentials (username, access key) to use for this project.
These values will be available as BROWSERSTACK_USER and BROWSERSTACK_ACCESSKEY environment variables.

Step 2.8.4: Configure Build Step: Execute shell: npm run local

-> 

-> we add following shell script:

export BROWSERSTACK_USERNAME=$BROWSERSTACK_USER
export BROWSERSTACK_ACCESS_KEY=$BROWSERSTACK_ACCESSKEY
npm install
npm run local

2017-03-17 22_37_29-BrowserStackJob Config [Jenkins]

This is, because the BrowserStack example expects the username and password to be provided in the variables BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY, which differs from the names the username and password are encapsulated by the Jenkins BrowserStack plugin.

We needed to add npm install to download all dependencies, before we run the test with npm run local. The local script is defined in the package.json file of the project.

Now we need to save the settings:

-> Save

Step 2.9: Add ‘make’ to Jenkins Container

If this step is omitted, we would receive an error message in the next step

> node-gyp rebuild

gyp ERR! build error 
gyp ERR! stack Error: not found: make

With the following commands, we log in as root (with user ID 0) and install the build-essential:

(dockerhost)$ docker exec -u 0 jenkins bash
(container)# apt-get update
(container)# apt-get install build-essential

Step 2.10: Build Project

-> Build Now

-> Click on #1

-> Console Output

The last view lines of the output show, that the test was successful:

Excellent!

Step 2.11: Generate a Jenkins Test Report

Step 2.11.1: Add Jasmine to the Project

For adding a Test report, we add the plugin jasmine-reporters to the repository. For that, I have forked the original BrowserStack Protractor Git repository to /protractor-browserstack. I have made following changes (in bold):

package.json

{
  "name": "protractor-browserstack",
  "version": "0.1.0",
  "readme": "Protractor Integration with [BrowserStack](https://www.browserstack.com)",
  "description": "Selenium examples for Protractor and BrowserStack Automate",
  "scripts": {
    "test": "npm run single && npm run local && npm run parallel",
    "single": "./node_modules/.bin/protractor conf/single.conf.js",
    "local": "./node_modules/.bin/protractor conf/local.conf.js",
    "parallel": "./node_modules/.bin/protractor conf/parallel.conf.js",
    "parallel_local": "./node_modules/.bin/protractor conf/parallel_local.conf.js"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/browserstack/protractor-browserstack"
  },
  "dependencies": {
    "browserstack-local": "^1.0.0",
    "protractor": "^2.5.1",
    "jasmine-reporters": "^1.0.0"
  },
  "license": "MIT"
}

I have added a comma at the end of the protractor dependency line and I have added jasmine 1.0.0 to the dependencies. Note that the current jasmine 2.2.1 version did not generate any reports.

conf/local.conf.js

var browserstack = require('browserstack-local');

exports.config = {
  'specs': [ '../specs/local.js' ],
  'seleniumAddress': 'http://hub-cloud.browserstack.com/wd/hub',

  'capabilities': {
    'browserstack.user': process.env.BROWSERSTACK_USERNAME || 'BROWSERSTACK_USERNAME',
    'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY || 'BROWSERSTACK_ACCESS_KEY',
    'build': 'protractor-browserstack',
    'name': 'local_test',
    'browserName': 'chrome',
    'browserstack.local': true,
    'browserstack.debug': 'true'
  },

  // Add Jasmine JUnit reporter
  onPrepare: function() {
    require('jasmine-reporters');
    jasmine.getEnv().addReporter(
        new jasmine.JUnitXmlReporter('xmloutput', true, true)
    );
  },

  // Code to start browserstack local before start of test
  beforeLaunch: function(){
    console.log("Connecting local");
    return new Promise(function(resolve, reject){
      exports.bs_local = new browserstack.Local();
      exports.bs_local.start({'key': exports.config.capabilities['browserstack.key'] }, function(error) {
        if (error) return reject(error);
        console.log('Connected. Now testing...');

        resolve();
      });
    });
  },

  // Code to stop browserstack local after end of test
  afterLaunch: function(){
    return new Promise(function(resolve, reject){
      exports.bs_local.stop(resolve);
    });
  }
};

Note, that the syntax for Jasmine2 looks different:

var jasmineReporters = require('jasmine-reporters');
    jasmine.getEnv().addReporter(
        new jasmineReporters.JUnitXmlReporter('xmloutput', true, true)
    );

The old syntax is throwing an error JUnitXmlReporter is not a constructor., if you use it with Jasmine2. With the new syntax, Jasmine2 did not throw any errors, but it also did not create any XML reports.

Step 2.11.2 (optional): Run Jenkins Job and Check XML Files

-> Build Now

-> Click on #1 (choose highest number)

-> Console Output

Now the xml reports should have been created. To check this, let us connect to the Jenkins docker container:

(dockerhost)$ docker exec -it jenkins bash
(container)$ cd workspace/BrowserStackJob
(container)$ ls -l xmloutput
total 1
-rwxrwxrwx 1 900 900 311 Mar 18 13:18 TEST-BrowserStackLocalTesting.xml

Step 2.12: Generate individual Jenkins Test report

Step 2.12.1: Specify the Jenkins Report Path

Jenkins dashboard

-> BrowserStackJob

-> 

-> Add post-build action: Publish JUnit test result report

-> in the “Test report XMLs” field, specify the path to the XML reports as defined in conf/local.conf.js (in our case: xmloutput/*.xml):

specify the path to the XML reports as defined in conf/local.conf.js (in our case: xmloutput/*.xml)

-> 

Step 2.12.2: Generate the individual Jenkins Test Report

-> Build Now

Now the build report is showing a link to the Test Result:

-> click on Test Result will reveal a table, which shows, how many tests have failed (0), passed (1), total (1) and diff (+1)

Individual Test Result

For troubleshooting: the console output should show the term “Recording test results”:

Step 2.13: Generate a Test Result Trend Graph

Just rerun the “Build now”, so you have at least two or better three tests with test results. Then the Test Result Graph will show up.

Jenkins Job showing Test Results Trend

Note: do not use the aggregated downstream results:

This had caused a lot of confusion, because the number of the tests is shown to be zero, as can be seen in the graph in build #19 to #23. Then I have found Phil’s answer to this Stackoverflow question, which has helped me to resolve the issue by removing the aggregation. And voila, the number of tests is correct again (build #24).

 

Excellent!

Appendix A: Error : “/bin/sh: 1: node: not found”

Symptoms: Error “node not found”

Full log:

root@6dbba34bf92c:/vagrant/protractor-browserstack# npm install
> bufferutil@1.2.1 install /vagrant/protractor-browserstack/node_modules/bufferutil
> node-gyp rebuild

/bin/sh: 1: node: not found
gyp: Call to 'node -e "require('nan')"' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (/usr/share/node-gyp/lib/configure.js:354:16)
gyp ERR! stack at emitTwo (events.js:87:13)
gyp ERR! stack at ChildProcess.emit (events.js:172:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Linux 4.2.0-42-generic
gyp ERR! command "/usr/bin/nodejs" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /vagrant/protractor-browserstack/node_modules/bufferutil
gyp ERR! node -v v4.2.6
gyp ERR! node-gyp -v v3.0.3
gyp ERR! not ok
npm WARN install:bufferutil@1.2.1 bufferutil@1.2.1 install: `node-gyp rebuild`
npm WARN install:bufferutil@1.2.1 Exit status 1

> utf-8-validate@1.2.2 install /vagrant/protractor-browserstack/node_modules/utf-8-validate
> node-gyp rebuild

/bin/sh: 1: node: not found
gyp: Call to 'node -e "require('nan')"' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (/usr/share/node-gyp/lib/configure.js:354:16)
gyp ERR! stack at emitTwo (events.js:87:13)
gyp ERR! stack at ChildProcess.emit (events.js:172:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Linux 4.2.0-42-generic
gyp ERR! command "/usr/bin/nodejs" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /vagrant/protractor-browserstack/node_modules/utf-8-validate
gyp ERR! node -v v4.2.6
gyp ERR! node-gyp -v v3.0.3
gyp ERR! not ok
npm WARN install:utf-8-validate@1.2.2 utf-8-validate@1.2.2 install: `node-gyp rebuild`
npm WARN install:utf-8-validate@1.2.2 Exit status 1

Resolution: make sure the Node.js “node” is found via the PATH

In case of Ubuntu, there is an executable node on /usr/sbin/node that has nothing to do with Node.js. The problem is, that Node.js is installed as nodejs instead of node and that many NPM commands try to execute node. We need to make sure that the NPM installation commands will find the correct Node.js node. One way of doing so, is to link /usr/bin/node to nodejs in the same folder. This works, since the existing /usr/sbin PATH is found behind the /usr/bin PATH per default.

$ sudo ln -s nodejs /usr/bin/node

or

# ln -s nodejs /usr/bin/node

if you are root (we are root in the container above)..

Then make sure that the correct node is found:

$ which node
/usr/bin/node

If the other /usr/sbin/node executable is hiding /usr/bin/node, then you might need to adapt the PATH with export PATH=/usr/bin:$PATH.

After adding the symbolic link, the full log looks like follows:

root@6dbba34bf92c:/vagrant/protractor-browserstack# npm install
> bufferutil@1.2.1 install /vagrant/protractor-browserstack/node_modules/bufferutil
> node-gyp rebuild

make: Entering directory '/vagrant/protractor-browserstack/node_modules/bufferutil/build'
 CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
 SOLINK_MODULE(target) Release/obj.target/bufferutil.node
 COPY Release/bufferutil.node
make: Leaving directory '/vagrant/protractor-browserstack/node_modules/bufferutil/build'

> utf-8-validate@1.2.2 install /vagrant/protractor-browserstack/node_modules/utf-8-validate
> node-gyp rebuild

make: Entering directory '/vagrant/protractor-browserstack/node_modules/utf-8-validate/build'
 CXX(target) Release/obj.target/validation/src/validation.o
 SOLINK_MODULE(target) Release/obj.target/validation.node
 COPY Release/validation.node
make: Leaving directory '/vagrant/protractor-browserstack/node_modules/utf-8-validate/build'
protractor-browserstack@0.1.0 /vagrant/protractor-browserstack
`-- protractor@2.5.1
  `-- selenium-webdriver@2.47.0
    `-- ws@0.8.1
      +-- bufferutil@1.2.1
      | +-- bindings@1.2.1
      | `-- nan@2.5.1
      `-- utf-8-validate@1.2.2
        `-- nan@2.4.0

Appendix B: Solve Git Problem: “tell me who you are”

Symptoms: Git Error: status code 128

In a new installation of Jenkins, Git does not seem to work out of the box. You can see this by choosing the Jenkins project Job-DSL-Hello-World-Job on the dashboard, then click “build now”, if the build was not already automatically triggered. Then:

-> Build History

-> Last Build (link works only, if Jenkins is running on localhost:8080 and you have chosen the same job name)

-> Console Output

There, we will see:

Caused by: hudson.plugins.git.GitException: Command "git tag -a -f -m Jenkins Build #1 jenkins-Job-DSL-Hello-World-Job-1" returned status code 128:
stdout: 
stderr: 
*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: empty ident name (for <jenkins@61915398735e.(none)>) not allowed

Resolution:

Step 1: Enter Git Username and Email

As described in this StackOverflow Q&A: we can resolve this issue by either suppressing the git tagging, or (I think this is better) by adding your username and email address to git:

-> 2017-02-25-18_04_32-job-dsl-hello-world-job-1-console-jenkins

-> Manage Jenkins

-> Configure System

-> scroll down to “Git plugin”

-> Git plugin; user.name = jenkins and user.email = admin@jenkins.org

Step 2: Re-run “Build Now” on the Project

To test the new configuration, we go to

-> the Job-DSL-Hello-World-Job and press

-> Build Now

Now, we should see a BUILD SUCCESS like follows:

-> Build History

-> #nnn

-> Console Output

If everything went fine, we will a “BUILD SUCCESS”:

Job DSL Hello World Job: BUILD SUCCESS

Appendix C: NPM Error: “Auth Token must be alphanumeric characters only”

Symptoms

After cloning the protractor-browserstack and successfully installing the dependencies, the following command fails with a cryptic message

/vagrant/protractor-browserstack/node_modules/q/q.js:126
 throw e;

But reading further, there is a meaningful error message like follows:

LocalError: Auth Token must be alphanumeric characters only. Please fetch it from Local Testing section of settings page: https://www.browserstack.com/accounts/settings

The full log looks like follows:

# npm run local
> protractor-browserstack@0.1.0 local /vagrant/protractor-browserstack
> protractor conf/local.conf.js

Connecting local

/vagrant/protractor-browserstack/node_modules/q/q.js:126
 throw e;
 ^
LocalError: Auth Token must be alphanumeric characters only. Please fetch it from Local Testing section of settings page: https://www.browserstack.com/accounts/settings
 at /vagrant/protractor-browserstack/node_modules/browserstack-local/lib/Local.js:57:20
 at ChildProcess.exithandler (child_process.js:204:7)
 at emitTwo (events.js:87:13)
 at ChildProcess.emit (events.js:172:7)
 at maybeClose (internal/child_process.js:821:16)
 at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)

npm ERR! Linux 4.2.0-42-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "local"
npm ERR! node v4.2.6
npm ERR! npm v3.5.2
npm ERR! code ELIFECYCLE
npm ERR! protractor-browserstack@0.1.0 local: `protractor conf/local.conf.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the protractor-browserstack@0.1.0 local script 'protractor conf/local.conf.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the protractor-browserstack package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! protractor conf/local.conf.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs protractor-browserstack
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls protractor-browserstack
npm ERR! There is likely additional logging output above.
npm ERR! Linux 4.2.0-42-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "local"
npm ERR! node v4.2.6
npm ERR! npm v3.5.2
npm ERR! path npm-debug.log.705992348
npm ERR! code ETXTBSY
npm ERR! errno -26
npm ERR! syscall rename

npm ERR! ETXTBSY: text file is busy, rename 'npm-debug.log.705992348' -> 'npm-debug.log'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR! /vagrant/protractor-browserstack/npm-debug.log

Resolution: Specify BrowserStack Username and Password in the correct environment variables

The error indicates that the BrowserStack User and/or Password is not set correctly. I had hit that problem, since I was testing another example with Gulp on the same system and the BrowserStack User and Password variables looked similar, but were not exactly the same.

To resolve the issue, let us specify the BrowserStack credentials you can find in the “Automate” section on your BrowserStack Account Settings page:

(container)# export BROWSERSTACK_USERNAME=your_browserstack_user_id
(container)# export BROWSERSTACK_ACCESS_KEY=your_browserstack_key

After that, the full log looks like follows:

# npm run local
> protractor-browserstack@0.1.0 local /vagrant/protractor-browserstack
> protractor conf/local.conf.js

Connecting local
Connected. Now testing...
Using the selenium server at http://hub-cloud.browserstack.com/wd/hub
[launcher] Running 1 instances of WebDriver
.

Finished in 0.925 seconds
1 test, 1 assertion, 0 failures

[launcher] 0 instance(s) of WebDriver still running
[launcher] chrome #1 passed

Appendix D: NPM Error: TypeError: jasmine.JUnitXmlReporter is not a constructor

Symptoms

This error was created with the Protractor Github example from BrowserStack with Jasmine 2.2.1 and Jasmine 1.x.x syntax (see below). This creates an “Error: TypeError: jasmine.JUnitXmlReporter is not a constructor”.

In package.json, jasmine 2.2.1 is defined as dependency:

package.json

{
  "name": "protractor-browserstack",
  "version": "0.1.0",
  "readme": "Protractor Integration with [BrowserStack](https://www.browserstack.com)",
  "description": "Selenium examples for Protractor and BrowserStack Automate",
  "scripts": {
    "test": "npm run single && npm run local && npm run parallel",
    "single": "./node_modules/.bin/protractor conf/single.conf.js",
    "local": "./node_modules/.bin/protractor conf/local.conf.js",
    "parallel": "./node_modules/.bin/protractor conf/parallel.conf.js",
    "parallel_local": "./node_modules/.bin/protractor conf/parallel_local.conf.js"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/browserstack/protractor-browserstack"
  },
  "dependencies": {
    "browserstack-local": "^1.0.0",
    "protractor": "^2.5.1",
    "jasmine-reporters": "^2.2.1"
  },
  "license": "MIT"
}

In conf/local.conf.js, I eronneously had used the Jasmine 1.x.x style syntax:

conf/local.conf.js

var browserstack = require('browserstack-local');

exports.config = {
  'specs': [ '../specs/local.js' ],
  'seleniumAddress': 'http://hub-cloud.browserstack.com/wd/hub',

  'capabilities': {
    'browserstack.user': process.env.BROWSERSTACK_USERNAME || 'BROWSERSTACK_USERNAME',
    'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY || 'BROWSERSTACK_ACCESS_KEY',
    'build': 'protractor-browserstack',
    'name': 'local_test',
    'browserName': 'chrome',
    'browserstack.local': true,
    'browserstack.debug': 'true'
  },

  // Add Jasmine JUnit reporter
  onPrepare: function() {
    require('jasmine-reporters');
    jasmine.getEnv().addReporter(
        new jasmine.JUnitXmlReporter('xmloutput', true, true)
    );
  },

  // Code to start browserstack local before start of test
  beforeLaunch: function(){
    console.log("Connecting local");
    return new Promise(function(resolve, reject){
      exports.bs_local = new browserstack.Local();
      exports.bs_local.start({'key': exports.config.capabilities['browserstack.key'] }, function(error) {
        if (error) return reject(error);
        console.log('Connected. Now testing...');

        resolve();
      });
    });
  },

  // Code to stop browserstack local after end of test
  afterLaunch: function(){
    return new Promise(function(resolve, reject){
      exports.bs_local.stop(resolve);
    });
  }
};

Note, that the syntax for Jasmine2 looks different:

var jasmineReporters = require('jasmine-reporters');
    jasmine.getEnv().addReporter(
        new jasmineReporters.JUnitXmlReporter('xmloutput', true, true)
    );

The old syntax is throwing an error JUnitXmlReporter is not a constructor., if you use it with Jasmine2. With the new syntax, Jasmine2 did not throw any errors, but it also did not create any XML reports.

 

This create the above error. See the full Jenkins log:

Started by user Jenkins Admin
Building in workspace /var/jenkins_home/workspace/BrowserStackJob
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/oveits/protractor-browserstack # timeout=10
Fetching upstream changes from https://github.com/oveits/protractor-browserstack
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/oveits/protractor-browserstack +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision aad0d2c559ba55885c28316776f2053e590b1393 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f aad0d2c559ba55885c28316776f2053e590b1393
 > git rev-list 41afcec2b4b430129f4d83a03837c215c942912e # timeout=10
[BrowserStack] Local: Starting BrowserStack Local...
[BrowserStack] Local: Started
[BrowserStack] BROWSERSTACK_USER=oliverveits1
[BrowserStack] BROWSERSTACK_ACCESSKEY=********************
[BrowserStack] BROWSERSTACK_LOCAL=true
[BrowserStack] BROWSERSTACK_LOCAL_IDENTIFIER=bc7f6106f7f4421e843cd7dca7c039c1
[BrowserStack] BROWSERSTACK_BUILD=jenkins-BrowserStackJob-14
[BrowserStack] BROWSERSTACK_USER=oliverveits1
[BrowserStack] BROWSERSTACK_ACCESSKEY=********************
[BrowserStack] BROWSERSTACK_LOCAL=true
[BrowserStack] BROWSERSTACK_LOCAL_IDENTIFIER=bc7f6106f7f4421e843cd7dca7c039c1
[BrowserStack] BROWSERSTACK_BUILD=jenkins-BrowserStackJob-14
[BrowserStackJob] $ /bin/sh -xe /tmp/hudson3133472361627303547.sh
+ export BROWSERSTACK_USERNAME=<removed manually>
+ export BROWSERSTACK_ACCESS_KEY=<removed manually>
+ npm install
protractor-browserstack@0.1.0 /var/jenkins_home/workspace/BrowserStackJob
└─┬ jasmine-reporters@2.2.1 
  └── xmldom@0.1.27 

+ npm run local

> protractor-browserstack@0.1.0 local /var/jenkins_home/workspace/BrowserStackJob
> protractor conf/local.conf.js

Connecting local
Connected. Now testing...
Using the selenium server at http://hub-cloud.browserstack.com/wd/hub
[launcher] Running 1 instances of WebDriver
[launcher] Error: TypeError: jasmine.JUnitXmlReporter is not a constructor
    at onPrepare (/var/jenkins_home/workspace/BrowserStackJob/conf/local.conf.js:25:7)
    at /var/jenkins_home/workspace/BrowserStackJob/node_modules/protractor/lib/util.js:56:41
    at Function.promise (/var/jenkins_home/workspace/BrowserStackJob/node_modules/q/q.js:650:9)
    at Object.exports.runFilenameOrFn_ (/var/jenkins_home/workspace/BrowserStackJob/node_modules/protractor/lib/util.js:46:12)
    at Runner.runTestPreparer (/var/jenkins_home/workspace/BrowserStackJob/node_modules/protractor/lib/runner.js:76:17)
    at Object.exports.run (/var/jenkins_home/workspace/BrowserStackJob/node_modules/protractor/lib/frameworks/jasmine.js:68:17)
    at /var/jenkins_home/workspace/BrowserStackJob/node_modules/protractor/lib/runner.js:333:35
    at _fulfilled (/var/jenkins_home/workspace/BrowserStackJob/node_modules/q/q.js:797:54)
    at self.promiseDispatch.done (/var/jenkins_home/workspace/BrowserStackJob/node_modules/q/q.js:826:30)
    at Promise.promise.promiseDispatch (/var/jenkins_home/workspace/BrowserStackJob/node_modules/q/q.js:759:13)
[launcher] Process exited with error code 100

npm ERR! Linux 4.2.0-42-generic
npm ERR! argv "/var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/NodeJS_7.7.1/bin/node" "/var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/NodeJS_7.7.1/bin/npm" "run" "local"
npm ERR! node v7.7.1
npm ERR! npm  v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! protractor-browserstack@0.1.0 local: `protractor conf/local.conf.js`
npm ERR! Exit status 100
npm ERR! 
npm ERR! Failed at the protractor-browserstack@0.1.0 local script 'protractor conf/local.conf.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the protractor-browserstack package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     protractor conf/local.conf.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs protractor-browserstack
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls protractor-browserstack
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /var/jenkins_home/workspace/BrowserStackJob/npm-debug.log
Build step 'Execute shell' marked build as failure
[BrowserStack] Local: Stopping BrowserStack Local...
[BrowserStack] Local: Stopped
Finished: FAILURE

Resolution

Either change the Jasmine version in package.json to 1.0.0, or change the syntax in conf/local.conf.js to something similar to:

var jasmineReporters = require('jasmine-reporters');
    jasmine.getEnv().addReporter(
        new jasmineReporters.JUnitXmlReporter('xmloutput', true, true)
    );

See e.g. the accepted answer of this StackOverflow question.

However, version 2.x.x did not create any XML reports in my case. This seems to be a known incompatibility with Protractor. Therefore, if you are using Protractor, I recommend to change the Jasmine version to 1.0.0 and keep the syntax in conf/local.conf.js in the version 1 syntax.

Appendix E: Jenkins Test Trend showing no Tests

Symptoms

The Jenkins Test Trend shows zero tests, even though the XML reports are present in the Workspace. Here in builds #19 to #23:

Jenkins Job showing Test Results Trend

Resolution

Remove the aggregated downstream results under your project > configure:

See also Phil’s answer to this Stackoverflow question, which has helped me to resolve the issue by removing the aggregation. And voila, the number of tests is correct again (build #24).

Appendix F: Updating Jenkins

Updating Jenkins (in my case: from 2.32.1 to 2.32.2) was as simple as following the steps below

Note: you might want to make a backup of your jenkins_home though. Just in case…

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost)$ docker pull jenkins # to update the jenkins image
(dockerhost)$ docker rm jenkins # to make shure the container named jenkins is removed
(dockerhost:jenkins_home)$ sudo docker run -d --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins

However, after that, some data was unreadable:

2017-02-24-20_04_36-manage-old-data-jenkins

I have clicked

-> Manage Jenkins
-> Manage
-> Discard Unreadable Data

to resolve the issue (hopefully…). At least, after that, the warning was gone.

Summary

In this blog post, we

  • got acquainted with BrowserStack
    • performed manual tests
  • learned about BrowserStack local testing that allows to run remote browsers, but show the content of web sites that are available locally only
  • ran BrowserStack tests from command line
    • installed Node.js and NPM and Git
    • cloned a Protractor example with BrowserStack
    • performed automated tests from command line
  • integrated BrowserStack into Jenkins
    • installed a Docker host and a Jenkins Docker Container
    • installed the BrowserStack Plugin, Node.js, NPM and Protractor on Jenkins
    • performed the NPM installation on shell script
    • performed the automated tests as part of the Jenkins pipeline
    • installed Jasmine to made sure that individual and trend test reports are shown
  • At the end, I have recorded several error situations and their resolutions as appendices

We have seen that BrowserStack can help to perform tests with many different browsers on many different operating systems and hardware without the need to buy and install any mobile equipment.

Further Reading

0

Jenkins Part 5.1: Using the Job DSL for automatic Creation of Jenkins Jobs


Today, we will learn how to use the Jenkins Job DSL Plugin to create new Jenkins jobs at a push of a button. We will show how we can use Groovy scripts for defining a “Hello World” Jenkins freestyle project and create such a project by pushing the “Build now” button.

Why creating Jenkins Jobs via Groovy Scripts?

You might say: “Jenkins has such a nice Web interface. Why should I bother about creation of Jenkins Jobs via Groovy Scripts?”

Right: the web interface of Jenkins is simple to use and will lead to quick results for single projects. However, consider a situation with more than 100 projects with more than 400 Jenkins pipelines (e.g. build + test + deploy + release pipelines each). Automating the Jenkins job lifecycle can be a great time-saver here. Automating the Jenkins job lifecycle is best done with the Seed plugin:

From: https://blog.codecentric.de/en/2015/10/using-jenkins-job-dsl-for-job-lifecycle-management/
From: https://blog.codecentric.de/en/2015/10/using-jenkins-job-dsl-for-job-lifecycle-management/ showing a full Jenkins job lifecycle, which is the long term target for us. The current post is a first step towards this target.

The Seed plugin relies on the capabilities of the Job DSL plugin discussed in the current blog post. In one of the next blog posts, we will have a closer look to the Seed plugin.

 

Tools and Versions used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.32.2
      • Job DSL Plugin 1.58

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB.
      • Docker Host is available.
      • Tested with 2 vCPU (1 vCPU might work as well).

Step 1: Connect to Jenkins

Step 1.1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

50c150e35a774cexxxxxxxxxxxxxxx

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 1.2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

Note that this configuration is not permanent, unless you define the port mappings in the Vagrantfile as follows (see official Vagrant documentation):

config.vm.network "forwarded_port", guest: 8080, host: 8080

Step 1.3: Initialize Jenkins: Unlock Jenkins

Unlock Jenkins

insert the one-time password found in the log during startup or on /var/jenkins_home/secrets/initialAdminPassword

-> Continue

Step 1.4: Initialize Jenkins: Install Plugins

-> Install suggested plugins

Installing suggested plugins

Wait for the installation process to complete.

Step 1.4: Initialize Jenkins: Create Admin User

-> Create First Admin User

-> Save and Finish or “Continue as admin”

-> Start using Jenkins

Note: I recommend to log out and log in in order to test the login.

Step 2: Install the Job DSL Plugin

The Job DSL Plugin can be installed like any other Jenkins plugin:

-> Manage Jenkins
-> Manage Plugins
-> Plugins - Available

-> Filter job-dsl (with dash between job and dsl; wait for the filter to become active and do not press enter, otherwise you will get an error message)

-> Install without restart

Choose: Job DSL plugin

Job DSL: Installation Success

Step 3: Create Job DSL Jenkins Project

We create a Job DSL Job like follows:

-> Back to Dashboard

-> New Item

-> Enter Name: JobDSLJob

-> Freestyle Project

-> OK

Step 4: Configure Job DSL Project

-> Add build step: Process Job DSLs

-> if you have got a Github account, fork this open source Java Hello World software (originally created by of LableOrg) that will allow you to see, what happens with your Jenkins job, if you check in changed code. Moreover the hello world software allows you to perform JUnit 4 tests, run PowerMockito Mock services, run JUnit 4 Integration tests and calculate the code coverage using the tool Cobertura.

-> Use the provided DSL script

-> insert:

job('Job-DSL-Hello-World-Job') {
    scm {
        git('git://github.com/oveits/java-maven-junit-helloworld')
    }
    triggers {
        scm('H/15 * * * *')
    }
    steps {
        maven('-e clean test')
    }
}

here, exchange the username oveits by your own Github username.
2017-02-25-11_50_36-jobdsljob-config-jenkins

-> Save

Step 5: Prepare Maven Usage

Goto Jenkins -> Manage Jenkins -> Global Tool Configuration (available for Jenkins >2.0)

2016-12-09-11_34_55-manage-jenkins-jenkins

2016-12-09-11_35_26-global-tool-configuration-jenkins

Scroll down to Maven -> Add Maven

2017-01-02-14_32_46-global-tool-configuration-jenkins

-> choose Version (3.3.9 in my case)

-> Add a name (“Maven 3.3.9” in my case)

-> Save

Since we have checked “Install automatically” above, I expect that it will be installed automatically on first usage.

Step 6: Prepare Git Usage

As described in this StackOverflow Q&A, we need to add the Git username and email address, since Jenkins tries to tag and commit on the Git repo, which requires those configuration items to be set. For that, we perform:

-> 2017-02-25-18_04_32-job-dsl-hello-world-job-1-console-jenkins

-> Manage Jenkins

-> Configure System

-> scroll down to “Git plugin”

-> Git plugin; user.name = jenkins and user.email = admin@jenkins.org

Step 7: Create Jenkins Job from Code

Step 7.1 Build Project

-> Build Now

Step 7.2 (optional): Check Console Output

-> Build History: click on latest #nnn

-> Console Output

JobDSLJob: Console Output showing Added items: GeneratedJob{name='Job-DSL-Hello-World-Job'} and Finished: SUCCESS

->  Back to Project

Step 7.3: Review automatically built Project

-> 2017-02-25-11_48_23-update-center-jenkins

Jenkins Dashboard showing automatically created Job-DSL-Hello-World-Job

-> Job-DSL-Hello-World-Job

Project Job-DSL-Hello-World-Job showing build failure

This is showing a build failure, since I had not performed Step 5 and 6 before. In your case, it should be showing a success (in blue). If you are experiencing problems here, check out the Appendices below.

-> Configure

-> scroll down to Source Code Management

Source Code Management shows correct git URL

-> Scroll down to Build Triggers

Build Triggers showing Poll SCM every 15 minutes

-> Scroll down to Build

-> verify that “Maven 3.3.9” is chosen as defined in Step 5

-> enter “-e clean test” as Maven Goal

Maven Version "Maven 3.3.9" and Maven Goal "-e clean test"

-> Build Now

See, what happens by clicking on:

-> Build History

-> #nnn

-> Console Output

If everything went fine, we will see many downloads and a “BUILD SUCCESS”:

Job DSL Hello World Job: BUILD SUCCESS

Appendix A: Solve Git Problem: “tell me who you are”

Symptoms: Git Error: status code 128

In a new installation of Jenkins, Git does not seem to work out of the box. You can see this by choosing the Jenkins project Job-DSL-Hello-World-Job on the dashboard, then click “build now”, if the build was not already automatically triggered. Then:

-> Build History

-> Last Build (link works only, if Jenkins is running on localhost:8080 and you have chosen the same job name)

-> Console Output

There, we will see:

Caused by: hudson.plugins.git.GitException: Command "git tag -a -f -m Jenkins Build #1 jenkins-Job-DSL-Hello-World-Job-1" returned status code 128:
stdout: 
stderr: 
*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: empty ident name (for <jenkins@61915398735e.(none)>) not allowed

Resolution:

Step 1: Enter Git Username and Email

As described in this StackOverflow Q&A: we can resolve this issue by either suppressing the git tagging, or (I think this is better) by adding your username and email address to git:

-> 2017-02-25-18_04_32-job-dsl-hello-world-job-1-console-jenkins

-> Manage Jenkins

-> Configure System

-> scroll down to “Git plugin”

-> Git plugin; user.name = jenkins and user.email = admin@jenkins.org

Step 2: Re-run “Build Now” on the Project

To test the new configuration, we go to

-> the Job-DSL-Hello-World-Job and press

-> Build Now

Now, we should see a BUILD SUCCESS like follows:

-> Build History

-> #nnn

-> Console Output

If everything went fine, we will a “BUILD SUCCESS”:

Job DSL Hello World Job: BUILD SUCCESS

Appendix B: Maven Error: Cannot run program “mvn”

Symptoms:

When running a Maven Goal, the following error may appear on the Console log:

FATAL: command execution failed
java.io.IOException: Cannot run program "mvn" (in directory "/var/jenkins_home/workspace/Job-DSL-Hello-World-Job"): error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at hudson.Proc$LocalProc.(Proc.java:245)
	at hudson.Proc$LocalProc.(Proc.java:214)
	at hudson.Launcher$LocalLauncher.launch(Launcher.java:846)
	at hudson.Launcher$ProcStarter.start(Launcher.java:384)
	at hudson.Launcher$ProcStarter.join(Launcher.java:395)
	at hudson.tasks.Maven.perform(Maven.java:367)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
	at hudson.model.Build$BuildExecution.build(Build.java:205)
	at hudson.model.Build$BuildExecution.doRun(Build.java:162)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
	at hudson.model.Run.execute(Run.java:1728)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 15 more
Build step 'Invoke top-level Maven targets' marked build as failure
Finished: FAILURE

Resolution:

Perform Step 5

2017-01-02-14_32_46-global-tool-configuration-jenkins

and

For Test, you can test a manual: choose the correct Maven version, when configuring a Maven build step like in Step 7:

Maven Version "Maven 3.3.9" and Maven Goal "-e clean test"

and verify that Build Now does not throw the Maven error anymore.

For our case, we need to correct the Job DSL like follows:

In the Script, we had defined the step:

    steps {
        maven('-e clean test')
    }

However, we need to define the Maven Installation like follows:

    steps {
        maven {
            mavenInstallation("Maven 3.3.9")
            goals('-e clean test')
        }
    }

Here, the mavenInstallation needs to specify the exact same name, as the one we have chosen in Step 5 above.

After correction, we will receive the correct Maven goal

-> JobDSL -> Build Now

Now, we can check the Maven configuration:

-> Jenkins Home leading to the Dashboard

-> Job-DSL-Hello-World-Job

-> Configure

After scrolling down, we will see the correct Maven Version:

Maven Version "Maven 3.3.9" and Maven Goal "-e clean test"

DONE

Appendix C: Updating Jenkins

Updating Jenkins (in my case: from 2.32.1 to 2.32.2) was as simple as following the steps below

Note: you might want to make a backup of your jenkins_home though. Just in case…

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost)$ docker pull jenkins # to update the jenkins image
(dockerhost)$ docker rm jenkins # to make shure the container named jenkins is removed
(dockerhost:jenkins_home)$ sudo docker run -d --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins

However, after that, some data was unreadable:

2017-02-24-20_04_36-manage-old-data-jenkins

I have clicked

-> Manage Jenkins
-> Manage
-> Discard Unreadable Data

to resolve the issue (hopefully…). At least, after that, the warning was gone.

Appendix D: Job DSL Syntax

The reference for the Job DSL syntax can be found on the Job DSL Plugin API pages. As an example, the syntax of Maven within a Freestyle project can be found on this page found via the path

> freeStyleJob > steps > maven:

maven {

  • // Allows direct manipulation of the generated XML.

    configure(Closure configureBlock)

  • // Specifies the goals to execute including other command line options.

    goals(String goals)

  • // Skip injecting build variables as properties into the Maven process.

    injectBuildVariables(boolean injectBuildVariables = true)

  • // Set to use isolated local Maven repositories.

    localRepository(javaposse.jobdsl.dsl.helpers.LocalRepositoryLocation location)

  • // Specifies the Maven installation for executing this step.

    mavenInstallation(String name)

  • // Specifies the JVM options needed when launching Maven as an external process.

    mavenOpts(String mavenOpts)

  • // Adds properties for the Maven build.

    properties(Map props)

  • // Adds a property for the Maven build.

    property(String key, String value)

  • // Specifies the managed global Maven settings to be used.

    providedGlobalSettings(String settingsIdOrName)

  • // Specifies the managed Maven settings to be used.

    providedSettings(String settingsIdOrName)

  • // Specifies the path to the root POM.

    rootPOM(String rootPOM)

}

A Maven example can be found on the same page:

job('example') {
    steps {
        maven('verify')
        maven('clean verify', 'module-a/pom.xml')
        maven {
            goals('clean')
            goals('verify')
            mavenOpts('-Xms256m')
            mavenOpts('-Xmx512m')
            localRepository(LocalRepositoryLocation.LOCAL_TO_WORKSPACE)
            properties(skipTests: true)
            mavenInstallation('Maven 3.1.1')
            providedSettings('central-mirror')
        }
    }
}

Summary

In this blog post, we have learned how to

  1. Start and initialize Jenkins via Docker
  2. Prepare the usage of Git and Maven
  3. Install the Job DSL Plugin
  4. Define a Jenkins Job via Groovy script
  5. Create a Jenkins Job by a push of  the “Build now” button
  6. Review and run the automatically created Jenkins job

We have seen that the usage of the Job DSL is no rocket science. The only topic, we had to take care, is, that Git and Maven need to be prepared for first usage on a Jenkins server.

More sophisticated Jenkins jobs can be created by following the documentation on the Job DSL Plugin API pages.

Now, we are ready to explore the Seed plugin, which relies on the Job DSL plugin and helps to fully automate the Jenkins job lifecycle. Our aim is, that one or more Jenkins job is automatically created, when a new project is created on the Git repository. If this is possible, we will see in one of my next blog posts.

Further Reading

0

Jenkins Part 4.3: Performance Tests via JMeter


2017-01-31-06_43_59

Today, we will learn how to automatically create performance trend analysis reports like the following using Jenkins and JMeter:

Response time after several test runs

First, we will use Apache JMeter’s graphical user interface on the developer’s PC to create a test plan, before we integrate JMeter into Maven for the creation of performance reports on the command line. Finally, we will use Jenkins to aggregate the results into historic reports like the one we have shown above.

This Jenkins blog post series is divided into following parts:

Why do we need Performance Reporting during the Development Process?

In a waterfall process, performance was measured as a one-time effort before the release of the software. However, if you find out that the software performance is poor that late in the process, it is often time-consuming and costly to improve the situation.

If you integrate performance testing into your continuous integration pipeline instead, the developers are always kept informed about the performance of their software. Any time there is a change that affects the software’s response time, the developers get feedback and they have the chance to improve the situation exactly at the code part that has caused the performance issue.

Automated Performance Testing with Jenkins

In this blog post, we will show how to configure Jenkins for automated performance testing using JMeter. The performance tests will be accomplished after test and build of the project was accomplished:

2017-01-31-06_48_00

After this tutorial has been followed, we will have learned how to use JMeter locally on the developer’s PC and to integrate it into Maven and Jenkins for performance trend reporting. The build flow will be comprised of following steps:

  • The developer commits code to the remote repository
  • Jenkins detects the code change and will fetch the changed code
  • Jenkins will perform Unit tests and Checkstyle tests (in our case) and build the executable JAR. Those are the steps I have described in my previous blog posts (see above).
  • Now, we will learn how Jenkins can run the JAR in the background in order to open the local server port
  • While the JAR file is running, we will perform JMeter performance tests. This will create JMeter report files (.jtl)
  • At the end, the running JAR file is stopped automatically
  • As a post-build process, Jenkins will collect and aggregate the information found in the JMeter report files

Tools and Versions used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.32.1
      • JMeter 2.9
      • Maven JMeter Plugin 2.1.0

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB
      • Docker Host is available, Jenkins is installed and a build process for the creation of a JAR is configured. For that, perform the steps in part 1 and part 2 of this blog series
      • Tested with 2 vCPU (1 vCPU might work as well)

Step 1: Download and Install Apache JMeter v2.9 locally

A test plan can be created by downloading Apache JMeter version 2.9 from here (see note below!), starting the JMeter GUI and defining the test plan. For a quick start, I will provide you with an example test plan below you can adapt to your needs.

(!) Note: In order to avoid any incompatibilities with the Maven JMeter plugin 2.1.0 we will use later, we need to use Apache JMeter version 2.9 instead of the latest JMeter version (currently 3.1).

Step 2: Copy Test Plan Example

As a head start, we use the JMeter configuration file shown below. It is a simplistic test plan that is reading a single Web page in 10 threads in a loop 100 times each:

http://localhost:2005/files/JMeterTestFile

The corresponding values of the host, port, path, number of threads and loop count can be seen below in red. Please adapt them as required:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.4" jmeter="2.9 r1437961">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Testplan" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="Benutzer definierte Variablen" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group GET" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">100</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">10</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <longProp name="ThreadGroup.start_time">1486821935000</longProp>
        <longProp name="ThreadGroup.end_time">1486821935000</longProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>
      <hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="HTTP Request" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">localhost</stringProp>
          <stringProp name="HTTPSampler.port">2005</stringProp>
          <stringProp name="HTTPSampler.connect_timeout"></stringProp>
          <stringProp name="HTTPSampler.response_timeout"></stringProp>
          <stringProp name="HTTPSampler.protocol"></stringProp>
          <stringProp name="HTTPSampler.contentEncoding"></stringProp>
          <stringProp name="HTTPSampler.path">/files/JMeterTestFile</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
          <stringProp name="HTTPSampler.implementation">HttpClient4</stringProp>
          <boolProp name="HTTPSampler.monitor">false</boolProp>
          <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
        </HTTPSamplerProxy>
        <hashTree/>
      </hashTree>
      <ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>true</message>
            <threadName>true</threadName>
            <dataType>true</dataType>
            <encoding>false</encoding>
            <assertions>true</assertions>
            <subresults>true</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
          </value>
        </objProp>
        <stringProp name="filename"></stringProp>
      </ResultCollector>
      <hashTree/>
      <ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>true</message>
            <threadName>true</threadName>
            <dataType>true</dataType>
            <encoding>false</encoding>
            <assertions>true</assertions>
            <subresults>true</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
          </value>
        </objProp>
        <stringProp name="filename"></stringProp>
      </ResultCollector>
      <hashTree/>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

Now save this text as a .jmx file on the src/test/jmeter folder on your project’s home, e.g.:

$ ls src/test/jmeter/
GET_loop_single_page_JMeter-2.9.jmx

Step 3: Test and adapt the JMeter Configuration using the graphical Interface

Open the JMX file in the graphical Interface of Apache JMeter 2.9. For that, you can open it via the press -O icon.

2017-02-11-19_18_52-get_loop_single_page_jmeter-2-9-jmx-d__veits_jmeter_simple-restful-file-storage

Start the test by pressing <Ctrl>-R on the JMeter GUI. On the Aggregate Report, you can see the performance statistics.

2017-02-11-19_22_27-get_loop_single_page_jmeter-2-9-jmx-d__veits_jmeter_simple-restful-file-storage

In case you need to troubleshoot reported errors, the “Results Tree” comes in handy:

2017-02-11-19_23_52-get_loop_single_page_jmeter-2-9-jmx-d__veits_jmeter_simple-restful-file-storage

Now you can go to step 4, or you also can run the Software on the command line, if you wish:

jmeter -n -t PerformanceTest.jmx

Note that you can also define the variables at run time: see this Blog post describing how runtime variables can be used on the command line interface: e.g.

${__P(<variable name>,<default value>)}
jmeter -n -t PerformanceTest.jmx -J<variable name>=<value>

As an example, just enter

${__P(loop_count,100)}

instead of just “100” on the loop field.

2017-02-14-05_53_43-get_loop_single_page_jmeter-2-9-jmx-d__veits_jmeter_simple-restful-file-storage-v2

Then you can start the command with

jmeter -n -t PerformanceTest.jmx -Jloop_count=1000

for increasing the loop from 100 to 1000.

Step 4: Install the Maven JMeter Plugin

From this German HowTo blog post, we copy the following plugin configuration into the pom.xml file of the Maven project (the project I am using is both, a Maven project as well as a Gradle project):

<plugin>
  <groupId>com.lazerycode.jmeter</groupId>
  <artifactId>jmeter-maven-plugin</artifactId>
  <version>2.1.0</version>
  <configuration>
    <!--
       Die Ergebnisse werden normalerweise in einer Datei 
       /target/jmeter/results/<TestName>-<TimeStamp>.jtl abgelegt. 
       Für die Weiterverarbeitung ist der Timestamp nur hinderlich.
    -->
    <testResultsTimestamp>false</testResultsTimestamp>
 
    <!--
       Für die Fehlersuche bewährt es sich anfangs das LogLevel hochzuschrauben.
       Die JMeter-Logs werden in die Datei jmeter.log geschrieben.
    -->
    <overrideRootLogLevel>DEBUG</overrideRootLogLevel>
 
    <!--
       Konsolen-Ausgaben des JMeter-Prozesses werden standardmäßig unterdrückt (warum auch 
       immer). Es wird aber explizit der Listener "Create Summary Results" verwendet, damit
       auf der Konsole der aktuelle Testfortschritt mitverfolgt werden kann.
    -->
    <suppressJMeterOutput>false</suppressJMeterOutput>
 
    <!--
       Wenn Tests fehlschlagen (z.B. HTTP-Requests in einen Timeout laufen), wird normalerweise
       auch das entsprechende Maven-Goal als fehlerhaft markiert (und nachfolgende Schritte nicht
       mehr ausgeführt). Im Beispiel sollen aber trotz Fehler Graphen erzeugt werden.
    -->
    <ignoreResultFailures>true</ignoreResultFailures>
  </configuration>
  <dependencies>
    <dependency>
      <groupId>kg.apc</groupId>
      <artifactId>jmeter-plugins</artifactId>
      <version>1.0.0</version>
      <exclusions>
         <!--
            Leider sind einige Abhängigkeiten nicht in mvncentral zu finden,
            deshalb müssen sie hier explizit ausgeschlossen werden.
            Für eine vollständge Liste, siehe https://github.com/mlex/jmeter-maven-example/
        -->
        <exclusion>
            <groupId>kg.apc</groupId>
            <artifactId>perfmon</artifactId>
        </exclusion>
        <!-- ... -->
 
        <!--
            Aufgrund eines Bugs im jmeter-maven-plugin (siehe 
            https://github.com/Ronnie76er/jmeter-maven-plugin/issues/77) müssen 
            JMeter-Abhängigkeiten auch ausgeschlossen werden.
        -->
        <exclusion>
            <groupId>org.apache.jmeter</groupId>
            <artifactId>jorphan</artifactId>
        </exclusion>
        <!-- ... -->
      </exclusions>
    </dependency>
  </dependencies>
</plugin>

Now let us run the server software and the Maven JMeter plugin.

Step 5: Run the Server Software

If you are developing your server software in Eclipse, start the server now. As an alternative, if you have an executable JAR file available, run it via a java -jar command.

Step 6: Run the Maven JMeter Plugin

Now we are ready to run the Maven JMeter plugin manually like follows (the clean goal is not mandatory, but helps to avoid confusion with older JMX files, if this is not the first test run):

$ mvn clean jmeter:jmeter
[INFO] Scanning for projects...
Downloading: https://repository.apache.org/content/repositories/releases/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.pom
Downloading: http://repo.fusesource.com/nexus/content/repositories/releases/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.pom
Downloading: http://repo.fusesource.com/nexus/content/groups/ea/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.pom
Downloading: https://repo.maven.apache.org/maven2/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.pom
Downloaded: https://repo.maven.apache.org/maven2/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.pom (14 KB at 29.0 KB/sec)
Downloading: https://repository.apache.org/content/repositories/releases/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.jar
Downloading: http://repo.fusesource.com/nexus/content/repositories/releases/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.jar
Downloading: http://repo.fusesource.com/nexus/content/groups/ea/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.jar
Downloading: https://repo.maven.apache.org/maven2/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.jar
Downloaded: https://repo.maven.apache.org/maven2/com/lazerycode/jmeter/jmeter-maven-plugin/2.1.0/jmeter-maven-plugin-2.1.0.jar (42 KB at 187.4 KB/sec)
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Velocity Temple 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- jmeter-maven-plugin:2.1.0:jmeter (default-cli) @ camel-spring4 ---
Downloading: https://repository.apache.org/content/repositories/releases/kg/apc/jmeter-plugins/1.0.0/jmeter-plugins-1.0.0.pom
Downloading: http://repo.fusesource.com/nexus/content/repositories/releases/kg/apc/jmeter-plugins/1.0.0/jmeter-plugins-1.0.0.pom
Downloading: http://repo.fusesource.com/nexus/content/groups/ea/kg/apc/jmeter-plugins/1.0.0/jmeter-plugins-1.0.0.pom
Downloading: https://repo.maven.apache.org/maven2/kg/apc/jmeter-plugins/1.0.0/jmeter-plugins-1.0.0.pom
...
Downloading: https://repository.cloudera.com/content/repositories/snapshots/org/apache/thrift/thrift/0.2.0/thrift-0.2.0.jar
Downloading: https://repository.cloudera.com/content/groups/cloudera-repos/org/apache/thrift/thrift/0.2.0/thrift-0.2.0.jar
Downloading: http://people.apache.org/~rawson/repo/org/apache/thrift/thrift/0.2.0/thrift-0.2.0.jar
Downloaded: http://people.apache.org/~rawson/repo/org/apache/thrift/thrift/0.2.0/thrift-0.2.0.jar (184 KB at 610.4 KB/sec)
[INFO]
[INFO] -------------------------------------------------------
[INFO] P E R F O R M A N C E T E S T S
[INFO] -------------------------------------------------------
[INFO] Invalid value detected for <postTestPauseInSeconds>. Setting pause to 0...
[INFO]
[INFO]
[INFO] Executing test: GET_loop_single_page_JMeter-2.9.jmx
[INFO] Writing log file to: C:\Users\olive\git\simple-restful-file-storage\target\jmeter\logs\GET_loop_single_page_JMeter-2.9.jmx.log
[INFO] Creating summariser <summary>
[INFO] Created the tree successfully using C:\Users\olive\git\simple-restful-file-storage\target\jmeter\testFiles\GET_loop_single_page_JMeter-2.9.jmx
[INFO] Starting the test @ Sun Feb 12 23:53:59 CET 2017 (1486940039929)
[INFO] Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
[INFO] summary + 1 in 00:00:00 = 6,0/s Avg: 34 Min: 34 Max: 34 Err: 0 (0,00%) Active: 2 Started: 2 Finished: 0
[INFO] summary + 999 in 00:00:01 = 994,0/s Avg: 3 Min: 1 Max: 79 Err: 0 (0,00%) Active: 0 Started: 10 Finished: 10
[INFO] summary = 1000 in 00:00:01 = 853,2/s Avg: 3 Min: 1 Max: 79 Err: 0 (0,00%)
[INFO] Tidying up ... @ Sun Feb 12 23:54:01 CET 2017 (1486940041154)
[INFO] ... end of run
[INFO] Completed Test: GET_loop_single_page_JMeter-2.9.jmx
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.666 s
[INFO] Finished at: 2017-02-12T23:54:01+01:00
[INFO] Final Memory: 20M/933M
[INFO] ------------------------------------------------------------------------
[INFO] Shutdown detected, destroying JMeter process...

Note that there will be a BUILD SUCCESS, even if there are 100% Errors. Therefore, we need to check the number of errors (0,00%) in our case; see above.

Step 7: Connect to Jenkins

Step 7.1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 7.2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

Note that this configuration is not permanent, unless you define the port mappings in the Vagrantfile as follows (see official Vagrant documentation):

config.vm.network "forwarded_port", guest: 8080, host: 8080

Log in with the admin account we have created in part 1 of the blog post series:

2016-12-09-10_24_00-jenkins

Step 8: Run the Server JAR File on Jenkins

Inour case, the software we are building is a server listening on port 2005. Since we do not want the Jenkins pipeline to depend on any externally started server process, we now will make sure that a server instance is started automatically in the background during the build process. We will

  1. start the server in background
  2. perform the tests in the next step
  3. stop the server (is done by Jenkins automatically at the end of the build process)

For that, we create a shell script that runs the server jar file in the background.

On the Jenkins server, goto:

-> your project
-> Execute shell
-> Execute shell command for running JAR file

See the shell script here:

echo '#!/bin/bash' > runJAR.sh
echo 'echo "Hello JMeterTestFile" > "${WORKSPACE}/src/main/resources/files/JMeterTestFile"' >> runJAR.sh
echo 'nohup java -jar "${WORKSPACE}/build/libs/GitHub Triggered Build-0.0.1-SNAPSHOT.jar" > jar.log &' >> runJAR.sh
echo 'sleep 10' >> runJAR.sh
bash ./runJAR.sh

The second echo command makes sure the JMeterTestFile exists on the server. This file is the one we will retrieve with the GET commands (this is specific for the server software we use).

Then:
-> Save
-> Build Now

After clicking

-> Click on build job
-> Console Output,

we will see following log:

2017-02-13-00_39_01-github-triggered-build-778-console-jenkins

Here we can see, that the runJar.sh command has been started.

Step 9: Run the Maven JMeter Plugin within Jenkins

Let us run JMeter using Maven. For that, we go to

-> Jenkins
-> your project
-> Configure
-> Build -> Add build step
-> Invoke top-level Maven targets
-> Maven Version 3.3.9
-> Maven Goals: clean jmeter:jemeter
-> Save

We have added a “clean” Maven goal in order to remove any previously added JMX files.

Then test the configuration via

-> your project
-> Build Now
-> Build History -> click on running build
-> Console Output

After a long period of downloads, at the end you should see a SUCCESS statement:

2017-02-13-00_50_20-github-triggered-build-779-console-jenkins

Step 10: Collect the Performance Reports

In this step, we will install and configure the Jenkins JMeter plugin in order to collect the JMeter performance reports

Step 10.1: Install the Jenkins JMeter Plugin

Log into Jenkins as admin. Then goto

-> Manage Jenkins
-> Manage Plugins
> Available
> enter “JMeter” in the search field
-> check the Install checkmark
-> Install without restart
-> Choose “Restart Jenkins”

The plugin is visible now on the Installed tab:

2017-02-12-07_59_38-update-center-jenkins

Step 10.2: Configure Jenkins to collect the Performance Reports

Now is the time to trigger the creation of the performance reports. For that, let us choose

-> Your Project
-> Configure
-> Add post-build action
-> Publish Performance test result report
->  Performance Report: Choose JMeter (in case the results are in CSV, choose JMeterCSV instead)
-> Performance Report: **/*.jtl
-> Save

For a manual test, just press

-> Build Now

At the end, you should see Performance Trend graphs like follows:

JMeter Performance Trend Reports: Response Time and Percentage Errors

Okay, but where are the lines showing 90% average and median values? Will they appear after the next build? Let us try:

-> Build Now

Yes, the lines appear only after the second JMeter performance measurement. See here an example after the third measurement:

After the second time JMeter is started, the "Response time" and "Percentage of errors" is shown. In thic case, we see that the JMeter configuration file had specified a wrong target IP address/FQDN, leading to 100% errors initially. After fixing the problem, the percentage of errors decreases to 0%.
After the second time JMeter is started, the “Response time” and “Percentage of errors” is shown. In this case, we see that the JMeter configuration file had specified a wrong target IP address/FQDN “localhost”, leading to 100% errors initially. After fixing the problem in the JMX file, the percentage of errors has decreased to 0%.

After a few test cycles, the test report might look like follows:

Response time after several test runs

thumps_up_3

Appendix A: Error in NonGUIDriver java.lang.NullPointerException

Symptoms:

JMeter plugin v2.1, Jmeter version defined to be 3.1; Jmeter config file (.jmx) created with a JMeter v3.1 GUI

After mvn jmeter:jemeter, we see the error message

Error in NonGUIDriver java.lang.NullPointerException

In the logs we see messages like:

...
2017/02/11 12:27:43 WARN  - jmeter.save.SaveService: Bad _version - expected 2.4, found 3.1.
...
2017/02/11 12:27:43 ERROR - jmeter.save.SaveService: Conversion error com.thoughtworks.xstream.converters.ConversionException: No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration' : No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration'
...

Full logs:

1) Command line:

$ mvn jmeter:jmeter
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Velocity Temple 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> jmeter-maven-plugin:2.1.0:jmeter (default-cli) > :configure @ camel-spring4 >>>
[INFO]
[INFO] --- jmeter-maven-plugin:2.1.0:configure (configure) @ camel-spring4 ---
[INFO] -------------------------------------------------------
[INFO] Configuring JMeter...
[INFO] -------------------------------------------------------
[INFO]
[INFO] <<< jmeter-maven-plugin:2.1.0:jmeter (default-cli) < :configure @ camel-spring4 <<<
[INFO]
[INFO] --- jmeter-maven-plugin:2.1.0:jmeter (default-cli) @ camel-spring4 ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] P E R F O R M A N C E T E S T S
[INFO] -------------------------------------------------------
[INFO] Invalid value detected for <postTestPauseInSeconds>. Setting pause to 0...
[INFO]
[INFO]
[INFO] Executing test: GET Requests.jmx
[INFO] Error in NonGUIDriver java.lang.NullPointerException
[INFO] Completed Test: GET Requests.jmx
[INFO]
[INFO] Executing test: POST Requests.jmx
[INFO] Error in NonGUIDriver java.lang.NullPointerException
[INFO] Completed Test: POST Requests.jmx
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.172 s
[INFO] Finished at: 2017-02-11T12:27:44+01:00
[INFO] Final Memory: 20M/933M
[INFO] ------------------------------------------------------------------------
[INFO] Shutdown detected, destroying JMeter process...
[INFO] Shutdown detected, destroying JMeter process...

Log file:

2017/02/11 12:27:42 INFO - jmeter.util.JMeterUtils: Setting Locale to de_DE
2017/02/11 12:27:43 INFO - jmeter.util.JMeterUtils: Could not find resources for 'de_DE', using 'de'
2017/02/11 12:27:43 INFO - jmeter.JMeter: Loading user properties from: C:\Users\olive\git\simple-restful-file-storage\target\jmeter\bin\user.properties
2017/02/11 12:27:43 INFO - jmeter.JMeter: Loading system properties from: C:\Users\olive\git\simple-restful-file-storage\target\jmeter\bin\system.properties
2017/02/11 12:27:43 WARN - jmeter.JMeter: LogLevel: DEBUG
2017/02/11 12:27:43 INFO - jmeter.JMeter: Copyright (c) 1998-2013 The Apache Software Foundation
2017/02/11 12:27:43 INFO - jmeter.JMeter: Version 2.9 r1437961
2017/02/11 12:27:43 INFO - jmeter.JMeter: java.version=1.8.0_101
2017/02/11 12:27:43 INFO - jmeter.JMeter: java.vm.name=Java HotSpot(TM) 64-Bit Server VM
2017/02/11 12:27:43 INFO - jmeter.JMeter: os.name=Windows 10
2017/02/11 12:27:43 INFO - jmeter.JMeter: os.arch=amd64
2017/02/11 12:27:43 INFO - jmeter.JMeter: os.version=10.0
2017/02/11 12:27:43 INFO - jmeter.JMeter: file.encoding=Cp1252
2017/02/11 12:27:43 INFO - jmeter.JMeter: Default Locale=Deutsch (Deutschland)
2017/02/11 12:27:43 INFO - jmeter.JMeter: JMeter Locale=Deutsch (Deutschland)
2017/02/11 12:27:43 INFO - jmeter.JMeter: JMeterHome=C:\Users\olive\git\simple-restful-file-storage\target\jmeter
2017/02/11 12:27:43 INFO - jmeter.JMeter: user.dir =C:\Users\olive\git\simple-restful-file-storage\target\jmeter\bin
2017/02/11 12:27:43 INFO - jmeter.JMeter: PWD =C:\Users\olive\git\simple-restful-file-storage\target\jmeter\bin
2017/02/11 12:27:43 INFO - jmeter.JMeter: IP: 192.168.178.212 Name: LAPTOP-P5GHOHB7 FullName: LAPTOP-P5GHOHB7.fritz.box
2017/02/11 12:27:43 INFO - jmeter.services.FileServer: Default base='C:\Users\olive\git\simple-restful-file-storage\target\jmeter\bin'
2017/02/11 12:27:43 INFO - jmeter.services.FileServer: Set new base='C:\Users\olive\git\simple-restful-file-storage\target\jmeter\testFiles'
2017/02/11 12:27:43 INFO - jmeter.JMeter: Loading file: C:\Users\olive\git\simple-restful-file-storage\target\jmeter\testFiles\GET Requests.jmx
2017/02/11 12:27:43 INFO - jmeter.save.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2
2017/02/11 12:27:43 INFO - jmeter.save.SaveService: Using SaveService properties file encoding UTF-8
2017/02/11 12:27:43 INFO - jmeter.save.SaveService: Using SaveService properties version 3.1
2017/02/11 12:27:43 WARN - jmeter.save.SaveService: Bad _version - expected 2.4, found 3.1.
2017/02/11 12:27:43 INFO - jmeter.save.SaveService: All converter versions present and correct
2017/02/11 12:27:43 INFO - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/html is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2017/02/11 12:27:43 INFO - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for application/xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2017/02/11 12:27:43 INFO - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2017/02/11 12:27:43 INFO - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/vnd.wap.wml is org.apache.jmeter.protocol.http.parser.RegexpHTMLParser
2017/02/11 12:27:43 INFO - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/css is org.apache.jmeter.protocol.http.parser.CssParser
2017/02/11 12:27:43 INFO - jmeter.protocol.http.sampler.HTTPSamplerBase: Reuse SSL session context on subsequent iterations: true
2017/02/11 12:27:43 ERROR - jmeter.save.SaveService: Conversion error com.thoughtworks.xstream.converters.ConversionException: No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration' : No field 'sentBytes' f
ound in class 'org.apache.jmeter.samplers.SampleSaveConfiguration'
---- Debugging information ----
message : No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration'
cause-exception : com.thoughtworks.xstream.converters.reflection.MissingFieldException
cause-message : No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration'
class : org.apache.jmeter.samplers.SampleSaveConfiguration
required-type : org.apache.jmeter.samplers.SampleSaveConfiguration
converter-type : org.apache.jmeter.save.converters.SampleSaveConfigurationConverter
path : /jmeterTestPlan/hashTree/hashTree/hashTree/ResultCollector/objProp/value/sentBytes
line number : 97
class[1] : org.apache.jmeter.testelement.property.ObjectProperty
converter-type[1] : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
class[2] : org.apache.jmeter.reporters.ResultCollector
converter-type[2] : org.apache.jmeter.save.converters.TestElementConverter
class[3] : org.apache.jorphan.collections.ListedHashTree
converter-type[3] : org.apache.jmeter.save.converters.HashTreeConverter
------------------------------- : No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration' : No field 'sentBytes' found in class 'org.apache.jmeter.samplers.SampleSaveConfiguration'

Resolution:

The problem is caused by the config file (.jmx) being created with JMeter version 3.1, but Maven using JMeter Version 2.9.

Download and extract Apache JMeter version 2.9 from here. Use this version of Apache JMeter to create the JMX file you later use in the Maven JMeter plugin.

Summary

In this blog post, we have learned how to

  1. Installed JMeter locally
  2. Copied and adapted a base test plan
  3. Tested the JMeter performance tests locally
  4. Integrated JMeter into Maven (helps to automatically download and spin up JMeter on Jenkins)
  5. Started the server on Jenkins in the Background
  6. Run JMeter on Jenkins using Maven
  7. Collected the created Performance Reports using the Jenkins JMeter plugin

After I have found out that the latest Maven JMeter plugin version is using an older JMeter version than the one I had used to create the test plan, and that the test plan is not backwards compatible (see appendix A), the installation process was straightforward.

Note that the performance tests put quite a bit of strain to your Jenkins server. In this blog post, the software as well as the JMeter process are run on the same machine. Looking at my own system, ~70% of the CPU was consumed by the server, while JMeter got the rest during a performance measurement. In a professional environment, you will want to run the server on a separate environment with defined resource constraints that are not shared with other processes. Moreover, you need to make sure that the JMeter measurements are not limited by the resources of the measuring device; i.e. you need to carefully monitor the resources consumed by JMeter itself.

Further Reading

 

1

Jenkins Part 4.2: Code Quality Tests via Checkstyle


2017-01-19-23_30_11

Today, we will show how to use Checkstyle for improving the style of Java code. First, we will add Checkstyle to Gradle in order to create XML reports for a single build. Jenkins allows us to visualize the results of more than one test/build run into historic reports. After that, we will show, how a developer can use the Eclipse Checkstyle plugin in order to create better code:

2017-02-01-04_43_16-github-triggered-build-jenkins

This blog post series is divided into following parts:

    • Part 1: Installation and Configuration of Jenkins, loading Plugins
    • Part 2: Creating our first Jenkins job: GitHub download and Software build
    • Part 3: Periodic and automatically triggered Builds
    • Part 4.1: running automated tests: Functional Tests via Java JUnit
    • Part 4.2: running automated tests: Code Quality Test via Checkstyle (this post)
    • Part 4.3: running automated tests: Performance Tests with JMeter (work in progress)

What is Jenkins?

Jenkins is the leading open source automation server mostly used in continuous integration and continuous deployment pipelines. Jenkins provides hundreds of plugins to support building, deploying and automating any project.

 

Jenkins build, test and deployment pipeline
Jenkins build, test and deployment pipeline

A typical workflow is visualized above: a developer checks in the code changes into the repository. Jenkins will detect the change, build (compile) the software, test it and prepare to deploy it on a system. Depending on the configuration, the deployment is triggered by a human person, or automatically performed by Jenkins.

For more information, see the introduction found in part 1 of this blog series.

Checking Code with Checkstyle

In this post, we will show how to configure Jenkins for automated code checking as part of the Post-Build Tests:

2017-01-19-04_15_46

After this tutorial has been followed, we will have learned how to apply standard or custom checks on the code quality using Checkstyle in Eclipse and Jenkins.

Tools & Versions used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.32.1
        • Checkstyle Plug-in 3.47
      • Eclipse Kepler Service Release 2 (Build id: 20140224-0627)
        • Checkstyle Plug-in 7.2.0.201611082205

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB
      • Docker Host is available, Jenkins is installed and a build process is configured. For that, perform all steps in part 1 to part 3 of this blog series (new: you now can skip part 1, if you wish)
      • Tested with 2 vCPU (1 vCPU might work as well)

Step 1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

We have created this entry in part 1 already, but I have seen that the entries were gone again, which seems to be a VirtualBox bug. I have added it again now.

Log in with the admin account we have created in the last session:

2016-12-09-10_24_00-jenkins

Step 3: Code Analysis: Checkstyle

With Gradle, we can invoke the Checkstyle plugin as follows:

Step 3.1: Prepare Gradle for performing Checkstyle

Add to build.gradle:

apply plugin: 'checkstyle'

tasks.withType(Checkstyle) {
 ignoreFailures = true
 reports {
 html.enabled = true
 }
}

We have set ignoreFailures to true, since we do not want the Gradle build to fail for now. We are just interested in the Checkstyle reports for now.

We can download an example Checkstyle configuration file from the Apache Camel repository, for example:

git clone <yourprojectURL>
mkdir -p <yourprojectDir>/config/checkstyle/
curl https://raw.githubusercontent.com/apache/camel/master/buildingtools/src/main/resources/camel-checkstyle.xml > <yourprojectDir>/config/checkstyle/checkstyle.xml

Step 3.2 (optional): Test Checkstyle locally

If you have no Git and/or no Gradle installed, you may want to skip this step and directly proceed to the next step, so Jenkins is performing this task for you.

We can locally invoke CheckStyle as follows:

gradle check

Step 3.3: Configure Jenkins to invoke Checkstyle

Adding Gradle Checkstyle tests to be performed before each build is as simple as performing Step 4.1 and then adding “check” as a goal to the list of Jenkins Build Gradle Tasks:

On Dashboard -> Click on Project name -> Configure -> Build, add “check” before the jar task:

2016-12-28-15_33_58-github-triggered-build-config-jenkins

Click Save.

Now we verify the setting by either checking changed code into the SW repository (now is a good time to commit and push the changes performed in Step 4.1) or by clicking “Build now” -> Click on Build Link in Build History -> Console Output in the Project home:

2016-12-28-15_39_37-github-triggered-build-725-console-jenkins2016-12-28-15_40_08-github-triggered-build-725-console-jenkins

We have received a very long list of CheckStyle Errors, but, as configured, the build does not fail nevertheless.

At the same time, CheckStyle Reports should be available on Jenkins now:

The Links specified in the output are only available on Jenkins, but since Jenkins is running as a Docker container on Vagrant VM residing in

D:\veits\Vagrant\ubuntu-trusty64-docker_openshift-installer\jenkins_home

I need to access the files on

file:///D:/veits/Vagrant/ubuntu-trusty64-docker_openshift-installer/jenkins_home/workspace/GitHub%20Triggered%20Build/build/reports/checkstyle/

2016-12-28-15_48_11-index-von-d__veits_vagrant_ubuntu-trusty64-docker_openshift-installer_jenkins_ho

And on main.html we find:

2016-12-28-15_49_04-main-html

Wow, it seems like I really need to clean the code…

Step 4: Visualize the CheckStyle Warnings and Errors to the Developer

Usually Jenkins is not running as a Docker container on the Developer’s PC or Notebook, so he has no access to the above report files. We need to publish the statistics via the Jenkins portal. For that, we need to install the CheckStyle Jenkins plugin:

Step 4.1 (optional): Install the “Static Analysis Utilities”

Note: I have not tried it out, but I believe that this step is not necessary, since the next step will automatically install all plugins the Checksytle plug-in depends on.

On Jenkins -> Manage Jenkins -> Manage Plugins -> Available

In the Filter field, type “Static Analysis U”

2016-12-28-22_44_53-update-center-jenkins

Check the checkbox of “Static Analysis Utilities” and Install without restart.

2016-12-28-22_47_06-update-center-jenkins

Step 4.2: Install Checkstyle Plugin

On Jenkins -> Manage Jenkins -> Manage Plugins -> Available

In the Filter field, type “Checkstyle ” (with white space at the end; this will limit the number of hits):

2016-12-28-22_56_26-update-center-jenkins

Check the checkbox of “Checkstyle Plug-in” and Install without restart.

2016-12-28-22_58_22-update-center-jenkins

Step 4.3: Configure Checkstyle Reporting in Jenkins

On Dashboard -> <your Project> -> Configure -> Post-build Actions -> Add post-build action, choose

Publish Checkstyle analysis results

Now add the path, where Gradle is placing its result xml files:

**/build/reports/checkstyle/*.xml

2016-12-28-23_10_57-github-triggered-build-config-jenkins

And click Save.

Step 4.4: Manually trigger a new Build

On the Project page, click “Build now”, then click on the build and then “Console output”:

2016-12-28-23_17_16-github-triggered-build-726-console-jenkins

We now can see [CHECKSTYLE] messages after the build, telling us, that the reports were collected. Now, where can we see them?

Step 4.5: Review Checkstyle Statistics

On the Project page, choose Status:

2016-12-28-23_20_31-github-triggered-build-726-jenkins-v2

and click on Checkstyle Warnings on the left, or the warnings link in the center of the page, and we get a graphical representation of the Checkstyle statistics:

2016-12-29-12_27_34-jenkins

When clicking on one of the File Links (MyRouteBuilder.java in this case), we can get an overview of the Warning types for this file:

2016-12-29-12_28_37-jenkins

We choose the category Indentation and get details on the warnings:

2016-12-29-09_03_58-jenkins

and after clicking on one of the links in the Warnings field, we see the java code causing the warning:

2016-12-29-09_05_56-jenkins

Okay, Camel’s Checkstyle configuration does not like my style of grouping each route’s first line with a smaller indent than the rest of the route:

2016-12-29-09_10_26-jenkins

And it does not seem to accept my style of putting the ; in a new line at the end of a route as seen by choosing the Whitespace category and then choosing an occurence:

2016-12-29-12_34_10-jenkins

I either need to change this style, or I need to adapte the checkstyle.xml configuration file to ignore those warnings.

Step 5: Improve Code Style

For the developer, it is very inconvenient to use the Jenkins Checkstyle messages from the console and match them with the code. We need something better than that: the Eclipse Checkstyle plugin.

Step 5.1: Install Eclipse Checkstyle Plugin via local Installation

Since the recommended installation via Marketplace did not work in my case (see Appendix A), I have followed some hints about a local installation found on StackOverflow:

Download Checkstyle from Sourceforge.

2016-12-30-13_54_36-add-repository

2016-12-30-13_55_14-install

In the next window, you are asked to specify some credentials we do not have. However, you can just ignore the window and click Cancel:

2016-12-30-14_01_54-login-required

->Cancel

Then the installation proceeds:

2016-12-30-14_04_17-install

2016-12-30-14_04_26-install

2016-12-30-14_04_33-installing-software

Now I had to klick OK on security warnings twice:

2016-12-29-19_55_50-security-warning

At the end, I had to restart Eclipse:

2016-12-30-19_09_15-software-updates

Now, the Checkstyle plugin is installed on Eclipse.

Step 5.2: Configure Project for Checkstyle Usage

The project in question must be enabled for Checkstyle usage by editing the Project Properties:

2017-01-07-23_14_44

Choosing the Checkstyle style. For now, let us choose the Google Checks in the drop-down list:

2017-01-07-23_18_41-properties-for-simple-restful-file-storage

Then confirm that the project is being re-built:

2017-01-07-23_18_50-rebuild-suggested

Now the code is more yellow than white, with many hints how to improve the code:

2017-01-07-23_28_00-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

However, the hints do not go away, if you correct the code. Do we need to rebuild again? Let us test:

2017-01-07-23_30_36-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Google style does not like that there is no empty line before the package line (sorry, in German):

2017-01-07-23_29_57-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

So, let us add an empty line and save the file. However, the style warning does not change:

2017-01-07-23_31_53-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Let us rebuild the project:

2017-01-07-23_33_05

Yes, after the re-build: the warning has disappeared:

2017-01-07-23_43_01-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Step 5.3: Download and Create Custom Checkstyle Profile in Eclipse

In the Jenkins Checkstyle tests above, we have used following custom Checkstyle configuration file:

$ curl https://raw.githubusercontent.com/apache/camel/master/buildingtools/src/main/resources/camel-checkstyle.xml > <yourprojectDir>/config/checkstyle/checkstyle.xml

I.e. the Checkstyle file is found on <yourprojectDir>/config/checkstyle/checkstyle.xml

Correct:

2017-01-07-23_49_13-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

2017-01-07-23_52_04-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

2017-01-07-23_52_57-preferences

2017-01-07-23_55_39-check-configuration-properties

Step 5.4: Assign Custom Checkstyle Profile to the Project

To assign the new Checkstyle profile to the project, we change the project’s Checkstyle properties by

Project->Properties -> Checkstyle

2017-01-07-23_14_44

-> Choose new Checkstyle profile -> OK

2017-01-08-00_01_13-properties-for-simple-restful-file-storage

On the Rebuild suggested window -> Yes

2017-01-08-00_01_18-rebuild-suggested

This works fine:

2017-01-18-02_29_51-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles-v2

In the code, we can see the Checkstyle warnings. To get more information on the specific Checkstyle warning, the warning text can be retrieved via the mouse over function on the left of the code line, or on the markers tab on the lower pane of Eclipse.

Step 5.5: Improve Code Style

Step 5.5.1: Change Code

In order to test, how the developer can improve the code style, let us replace some of the tabs by four spaces here:

2017-01-18-02_48_39-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Save the file now.

Step 5.5.2: Update Maven

Unfortunately, the Checkstyle warnings update process is a little cumbersome for custom Checkstyle profiles, it seems: we need to

  1. save the changed file,
  2. update Maven and
  3. rebuild the project.

Let us update Maven first:

right-click the project folder in the left pane -> Maven -> Update Project -> OK

2017-01-18-02_54_032017-01-18-02_58_21-update-maven-project

Then all Checkstyle markers are gone (although I have not changed all occurrences of a tab):

2017-01-18-02_59_32-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Step 5.5.3 Rebuild the Project

To get the Checkstyle warnings back, we need to rebuild the project:

Project -> Build Project

2017-01-18-03_02_56

Now we can see that some of the Checkstyle warnings are gone:

2017-01-18-03_04_05-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Next time, you check in the code to the Gir repository, you will see that the number of Checkstyle warnings we get from Jenkins via Gradle will decrease…

Step 6: Verify Jenkins Results

Since we have improved the source code, we expect the Jenkins Checkstyle warnings to decrease. We can verify this by doing the following:

-> save, commit and push the improved code -> log into Jenkins -> check out the build process that is triggered by the code push (or we can manually trigger the build process by clicking project -> Build now)

On the dashboard, we will see, that the Checkstyle statistics have (very) slightly improved:

2017-01-18-04_37_06-github-triggered-build-jenkins-v2

On the upper right edge of the figure, the number of warnings is slightly lower. The code quality is far from being perfect, but we now have all tools and plugins needed to improve the situation.

After changing all tabs by 4 spaces each, the number of Checkstyle violations goes down by ~50%. That is a good start.

2017-01-19-22_51_58-github-triggered-build-jenkins-v2

Perfect, we have learned how to use the Checkstyle plugin for Eclipse in order to produce better code. And the Jenkins Checkstyle plugin allows us to admire the progress we make.

😉

thumps_up_3

Appendix A: Problems with installing Checkstyle Eclipse Plugin via Marketplace

Note: this way of installation is recommended officially, but has failed in my case. If you hit the same problem, try the local installation as described in step 5.1 above.

To improve the style, it would be much too cumbersome to click through all 360 style warnings, edit the Java code, build the Code and check again. It is much better to give the programmer immediate feedback of the warning within the IDE. I am using Eclipse, so I need to install the Checkstyle Eclipse plugin as follows:

Choose Eclipse -> Help -> Eclipse Marketplace

2016-12-29-09_16_30-java-ee-simple-restful-file-storage_src_main_java_de_oveits_simplerestfulfiles

Search for “Checkstyle” and click install:

2016-12-29-09_19_00-eclipse-marketplace

And then “confirm”:

2016-12-29-09_20_01-eclipse-marketplace

What is that?

2016-12-29-09_21_18-proceed-with-installation_

I install it anyway. At this point, it is hanging quite a while:

2016-12-29-09_24_24-eclipse-marketplace

so, let me get a morning coffee…

After approximately two minutes, I can see it to proceed to 4 / 15. Good sign.

After the coffee, I still see 4 / 15. Not a good sign:

2016-12-29-09_44_41-eclipse-marketplace

Meanwhile I am researching the steps needed for performance testing…

After 2 hours or so: 6/15

This will take all day!

2016-12-29-11_31_35-eclipse-marketplace

Some hours later, I checked again, and I have seen the following:

2016-12-29-19_49_10-eclipse-marketplace

I have confirmed, confirmed the license:

2016-12-29-19_50_51-eclipse-marketplace

And have pressed Finish.

Then software gets installed:

2016-12-29-19_52_04-installing-software

I hope I will not break my good old Eclipse installation (it is installed locally, not in a virtual machine or container and it has ever worked better than any new version I have tested…).

After a two or three minutes:

2016-12-29-19_55_57-security-warning

I have confirmed with “OK”…

Then I had been asked to restart Eclipse and I have confirmed.

Problem: however, Checkstyle is still not installed:

Help -> Eclipse Marketplace

2016-12-30-13_20_09-eclipse-marketplace

Let us try again by clicking “Install”:

2016-12-30-13_24_45-eclipse-marketplace

2016-12-30-13_24_59-proceed-with-installation_

This does not work

Workaround

Instead of installing Checkstyle via the Eclipse marketplace, better install the Eclipse Checkstyle Plugin via download (see Step 5.1)

Summary

In this blog post we have performed following tasks:

  1. Started Jenkins in a Docker container
  2. Installed the Checkstyle Gradle Plugin to create Checkstyle plugins as XML files
  3. Installed the Checkstyle Jenkins Plugin to summarize the XML files into graphical historic reports
  4. Installed the Checkstyle Eclipse Plugin to be able to improve the code
  5. Installed custom Checkstyle policies
  6. Visualized the Code Improvement
  7. were happy

All steps apart from the installation of the Eclipse Checkstyle plugin were quite straightforward. For the Eclipse Checkstyle installation, we had to revert back to a local download and installation method described in step 5.1: the installation via Eclipse marketplace had failed. At the end, we could reduce the number of Checkstyle warnings by 50% without much effort.

Further Reading

2

Jenkins Part 4.1: Functional Java Tests via JUnit


2016-11-30-18_19_38

You also think that functional tests are one of the most important ingredients for delivering high quality software? You share my opinion that we should help the developer automating this task in order to get comparable results and to receive meaningful trend reports?

I will cover functional tests here. Instructions on how to perform code quality tests and performance tests are in draft status and will be covered in the next two blog posts.

Any questions and/or comments are highly welcome.

Introduction

As a developer you try hard to deliver high quality software.

You hate searching for this nasty bug that had been introduced unnoticed days ago. Or was it weeks ago? By whom? In which code?

Manual functional and performance testing after each commited code change quickly becomes a NO-GO as the number of features is rising constantly. In this blog post, we will show, how Jenkins can help you with both: delivering high quality software and minimizing the time needed to find the cause of a bug.

How about …

  1. creating automated tests for each functionality and performance at different levels (end to end, and unit tests)
  2. running the automated tests after each code change
  3. keeping track of the test results

… in order to avoid any bad surprises late in the game?

Okay: for 1., the developer needs to create automated functional and perfomance tests; I guess, there is no way around this. Better do this even before writing the actual code. For 2. and 3., however, automation tools like Jenkins step in and can be of great help. The developers checks in the code and Jenkins can do the rest for you.

In the current  blog post, we will show how to integrate automated JUnit functional tests into a Jenkins build pipeline. We will see that JUnit tests can be invoked easily via Gradle (Okay, Maven is more popular than Gradle, I guess, but I like Gradle because of some advantages I have discussed here; However, just give me a hint in a comment to this blog and I will prioritize the creation of a Maven version of this blog post). The Jenkins JUnit plug-in will be used to

  1. display reports on single build runs as well as
  2. display trend analysis graphs like the following one I have borrowed from here:
2016-12-30-18_41_45-jenkins-junit-project-home-jpg-826x707
Source: http://nelsonwells.net/2012/09/how-jenkins-ci-parses-and-displays-junit-output/

In this and the next two blog posts, we plan to cover following quality gate measures:

  • Part 4.1: Functional Tests (this blog post): we will use Java JUnit tests performed before building the executable JAR. Jenkins will report the test trend
  • Part 4.2: Code Quality Tests (coming soon): we will use the Checkstyle Gradle plugin for reporting to which degree the code adheres to the Apache Foundations formal rules
  • Part 4.3: Performance Tests (planned): we will use JMeter for testing and reporting the performance trend performed after the Java build using external performance testers like JMeter

Older blogs of this series:

This blog post series about Jenkins build pipelines is divided into following parts:

    • Part 1: Installation and Configuration of Jenkins, loading Plugins
    • Part 2: Creating our first Jenkins job: GitHub download and Software build
    • Part 3: Periodic and automatically triggered Builds

What is Jenkins?

Jenkins is the leading open source automation server mostly used in continuous integration and continuous deployment pipelines. Jenkins provides hundreds of plugins to support building, deploying and automating any project.

2016-12-30-21_04_46

A typical workflow is visualized above: a developer checks in the code changes into the repository. Jenkins will detect the change, build (compile) the software, test it and prepare to deploy it on a system. Depending on the configuration, the deployment is triggered by a human person, or automatically performed by Jenkins. After each step, the developer is informed depending on the priorites defined.

For more information, see the introduction found in part 1 of this blog series.

Automated Functional Testing based on JUnit

In this blog post, we will show how we need to configure Gradle and Jenkins for automated JUnit testing and reporting. In order to build a quality gate, we will reverse the original order and perform the JUnit tests before we build the executable JAR file (we do not want to create JAR files that are not functional):

2016-12-28-12_50_23

Tools used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.19.3
        • JUnit Plug-in 1.19

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB
      • Docker Host is available, Jenkins is installed and a build process is configured. For that, perform all steps in part 1 to part 3 of this blog series
      • Tested with 2 vCPU (1 vCPU might work as well)

Step 1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

We have created this entry in part 1 already, but I have seen that the entries were gone again, which seems to be a VirtualBox bug. I have added it again now.

Log in with the admin account we have created in the last session:

2016-12-09-10_24_00-jenkins

Step 3: Pre-Build JUnit Tests invoked by Gradle

In this step, we will invoke Gradle Tests before building the JAR. For that, we should verify locally that the Gradle tests are successful and then define a test Gradle task in the build process.

Step 3.1 (optional): Verify that Gradle Tests are successful

You can skip this test and directly let Jenkins do this for you. This may come handy, if you have not installed Git and/or Gradle locally.

Prerequisites

  • Your Java project has successful JUnit tests defined
  • Git is installed
  • The Project is cloned to a local directory
  • Gradle is installed

In order to test, whether the JUnit tests are successful, we can test those on a system with the project cloned (git, java and gradle must be installed):

(basesystem)$ gradle test
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution is an incubating feature.
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
1 warning
:processTestResources
:testClasses
:test

BUILD SUCCESSFUL

Total time: 29.9 secs

With that, we have verified that the command “gradle test”succeeds.

Note that the JUnit test must be designed in a way that they are independent of whether or not the JAR file is run in parallel. No simple way of running the executable JAR file in parallel to the execution of the JUnit tests seems to exist. In my case, I had to alter the JUnit tests to fulfill this prerequisite.

Step 3.2: Add Gradle test Task to Jenkins

As long as JUnit tests are defined in src/test of the project, adding Gradle tests to Jenkiny is as simple as adding “test” as a task to the list of Jenkins Build Gradle Tasks as follows:

On Dashboard -> Click on Project name -> Configure -> Build, add “task ” before the jar task:

2016-12-28-08_50_11-github-triggered-build-config-jenkins

Click Save.

If you have made local code changes on the project, now is the best time to commit and push them to the Git repository. If you have followed the steps in part 3, then this will automatically trigger a build process, so you do not need to click on “Build now” in that case. Otherwise, click on “Build now” on the Jenkins project page (e.g. Dashboard -> click on project name -> “Build now”).

Now we observe the result by clicking on the build process, then -> “Console Output”:

2016-12-28-09_45_34-github-triggered-build-724-console-jenkins

Don’t be confused by the blinking red ball on the upper left of the Console Output page: we see a BUILD SUCCESSFUL message and if we re-enter the same page, the ball is turned to static blue, indicating a successful build.

Step 4: Add JUnit Test Result Reporting to Jenkins

Now we will show how to add the JUnit test reports to the Jenkins build process.

Step 4.1: Install Jenkins JUnit Plugin

For Jenkins JUnit reporting, we need to install the JUnit Plug-in. For that, goto -> Jenkins Dashboard -> Manage Jenkins -> Manage Plugins -> Available -> Enter “JUnit Plugin” to the Find field -> Install

Note: If you do not find the plugin on the Available tab, search for it in the “Installed” tab.

You can install the plugin without reloading Jenkins.

Step 4.2: Configure Jenkins to collect and display the JUnit Test Results

In this step, we will configure Jenkins, so it will display the test results for individual builds as well as trend reporting. For that, navigate to:

Jenkins -> (choose Project) -> Configure -> Post-build Actions -> Publish JUnit test results report

2016-12-30-14_15_25-github-triggered-build-config-jenkins

Add

**/build/test-results/test/TEST-*.xml

to the “Test report XMLs” field, since this is the path, where Gradle is placing its JUnit test result reports (I have found the info here).

2016-12-30-14_18_51-github-triggered-build-config-jenkins

Now click Save.

Step 4.3: Verify JUnit individual Test Reporting

To test the Jenkins JUnit reporting feature, we trigger a clean build by adding “clean” to the Gradle tasks on Project -> Configure -> Build:

2016-12-30-17_43_59-github-triggered-build-config-jenkins

and clicking Save.

Then trigger a new build by clicking on Project -> Build now.

Then click on the Build Process, and then on Console output:

2016-12-30-17_48_38-github-triggered-build-731-console-jenkins

…scrolling down…

2016-12-30-17_50_01-github-triggered-build-731-console-jenkins

Do not be confused that the build process never seems to finish. Just click the Back to Project link:

Back to Project

On the Status page, we see that there were no failed tests:

2016-12-30-17_55_34-github-triggered-build-731-jenkins-v2

When we click on the Tests Result link on the left (or on the lower middle part on the Status page), we will see more details:

2016-12-30-17_58_25-github-triggered-build-731-test-results-jenkins-v2

We can see that we have had four tests (Create/Read/Update/Delete a file) and 100% of them were successful.

Step 4.3: Verify JUnit Test Trend Reporting

On the project’s Status page, a Test Trend graph is automatically added, as soon as there are two or more tests available. For that, click on “Build Now” on the left for a second time and click ENABLE AUTO REFRESH on the upper right. After the second build is complete, the (hopefully) blue Test Result Trend graph is showing up on the project status page:

2016-12-30-18_12_21-github-triggered-build-jenkins

The new blue graph shows that we had 4 successful tests in the last two builds.

Note: disregard the red Checkstyle Trend graph for now. This is something we will cover in the next blog post.

Step 5: Verify failed Test Reporting

Per default, Gradle build will fail, if one of the JUnit tests has failed, so it is building a strict quality gate. Will the test result be collected and reported nevertheless?

Let us test this now by breaking one of the JUnit tests by purpose. We have added an assert message that is expected to fail in one of the tests:

2016-12-30-19_27_46-java-ee-simple-restful-file-storage_src_test_java_de_oveits_simplerestfulfiles-v2

Now we commit and push the change to the SW repository:

$ git clone <Repository-URL>
$ cd <Repository Dir>
<perform the code changes here...>
$ git diff src/test/java/de/oveits/simplerestfulfilestorage/SimpleRestfulFileStorageTests.java
diff --git a/src/test/java/de/oveits/simplerestfulfilestorage/SimpleRestfulFileStorageTests.java b/src/test/java/de/oveits/simplerestfulfilestorage/SimpleRestfulFileStorageTests.java
index 684d30f..10200d5 100644
--- a/src/test/java/de/oveits/simplerestfulfilestorage/SimpleRestfulFileStorageTests.java
+++ b/src/test/java/de/oveits/simplerestfulfilestorage/SimpleRestfulFileStorageTests.java
@@ -115,6 +115,9 @@ public class SimpleRestfulFileStorageTests extends CamelSpringTestSupport {
 // mock expectations need to be specified before sending the message:
 mock.expectedBodiesReceived("File ttt created: href=http://localhost:2005/files/ttt");
 mock.expectedMessageCount(1);
+ ^M
+ // In order to break this test for Jenkins test reporting, we temporarily add a requirement that will fail:^M
+ mock.expectedMessageCount(2);^M

 template.sendBodyAndHeaders("direct:recipientList", body, headers);

$ git add src/test/java/de/oveits/simplerestfulfilestorage/SimpleRestfulFileStorageTests.java
$ git commit -m "Breaking a JUnit test by purpose for Jenkins reporting tests"
[jenkinstest 33655b9] Breaking a JUnit test by purpose for Jenkins reporting tests
 1 file changed, 4 insertions(+), 1 deletion(-)

olive@LAPTOP-P5GHOHB7 /d/veits/eclipseWorkspaceRecent/simple-restful-file-storage (jenkinstest)
$ git push
Counting objects: 9, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (9/9), 744 bytes | 0 bytes/s, done.
Total 9 (delta 4), reused 0 (delta 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To https://github.com/oveits/simple-restful-file-storage.git
 edb49f7..33655b9 jenkinstest -> jenkinstest

This will automatically trigger a new build (if you have followed part 3 of this series; otherwise just press “Build Now” on Jenkin’s project page).

We can see on the dashboard, that the build has failed:

2016-12-30-19_36_12-dashboard-jenkins

This was expected. Now let us click on the Project name, and we will see, what happened:

2016-12-30-19_37_40-github-triggered-build-jenkins

Perfect, that is exactly, what I wanted to achieve: On the Test Result Trend, we can see that we have performed 4 tests, one of which has failed.

Let us fix the failed test by commenting out (or removing) the wrong code again:

2016-12-30-19_40_01-java-ee-simple-restful-file-storage_src_test_java_de_oveits_simplerestfulfiles-v2

After

$ git add <file>
$ git commit -m "Fixed JUnit test again to test Jenkins JUnit trend report"
$ git push

The next build should be successful again and we can see in the trends graph that the failed test is fixed again:

2016-12-30-19_47_48-github-triggered-build-jenkins

thumps_up_3

Summary

In this blog post, we have shown

  1. How to add Java functional tests to the Jenkins build pipeline based on Gradle JUnit Plugin
  2. How to install the JUnit plug-in to Jenkins for report collection
  3. How to display JUnit test results for individual builds on the Jenkins portal
  4. How to display JUnit trend analysis on the Jenkins portal

The only challenge I have encountered is, that I had to re-write my JUnit tests in a way that they were successful when run stand-alone. Before they were successful only, if the executable JAR file was started manually before running the JUnit tests. This was resolved in a way specific to the framework used (Apache Camel in this case).

Coming Soon: Code Analysis Trend Analysis via Jenkins Checkstyle plugin

Further Reading

7

Jenkins Part 3.1: periodic vs triggered Builds


2016-11-30-18_19_38

Today, we will make sure that Jenkins will detect a code change in the software repository without manual intervention. We will show two methods to do so:

  1. Periodic Builds via Schedulers: Jenkins periodically asks the software repository for any code changes
  2. Triggered Builds via Webhooks: Jenkins is triggered by the software repository to perform the build task

We will see that the triggering build processes is more challenging to set up, but has quite some advantages in terms of economics and handling, once it is set up properly. See also the Summary at the end of this post.

This blog post series is divided into following parts:

    • Part 1: Installation and Configuration of Jenkins, loading Plugins
    • Part 2: Creating our first Jenkins job: GitHub download and Software build
    • Part 3 (this blog): Periodic and automatically triggered Builds
    • Part 4 (planned): running automated tests

What is Jenkins?

Jenkins is the leading open source automation server mostly used in continuous integration and continuous deployment pipelines. Jenkins provides hundreds of plugins to support building, deploying and automating any project.

 

Jenkins build, test and deployment pipeline
Jenkins build, test and deployment pipeline

A typical workflow is visualized above: a developer checks in the code changes into the repository. Jenkins will detect the change, build (compile) the software, test it and prepare to deploy it on a system. Depending on the configuration, the deployment is triggered by a human person, or automatically performed by Jenkins.

For more information, see the introduction found in part 1 of this blog series.

Automatic Jenkins Workflow: Periodic Polling

In this chapter, we will show how we need to configure Jenkins for automatic polling of the Software repository and start the build process, if code changes are detected.

2016-12-09-10_12_31

Tools used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.19.3

Prerequisites:

      • Free DRAM for the a Docker Host VM >~ 4 GB
      • Docker Host is available, Jenkins is installed and a build process is configured. For that, perform all steps in part 1 and part 2 of this blog series
      • Tested with 2 vCPU (1 vCPU might work as well)

Step 1: Start Jenkins in interactive Terminal Mode

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

I assume that jenkins_home is already created, all popular plugins are installed and an Admin user has been created as shown in part 1 of the blog series. We start the Jenkins container with the jenkins_home Docker host volume mapped to /var/jenkins_home:

(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /vagrant/jenkins_home/
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

We have created this entry in part 1 already, but I have seen that the entries were gone again, which seems to be a VirtualBox bug. I have added it again now.

Log in with the admin account we have created in the last session:

2016-12-09-10_24_00-jenkins

Step 3: Configure Project for periodic Polling of SW Repository

Step 3.1: Goto Build Trigger Configuration

On the Jenkins Dashboard, find the hidden triangle right of the project name,

2016-12-09-18_17_35-dashboard-jenkins

In the drop-down list, choose “Configure”

2016-12-09-18_18_06-dashboard-jenkins

(also possible: on the Dashboard, click on the project name and then “Configure”).

Step 3.2: Configure a Schedule

We scroll down to “Build Triggers” and check “Build periodically” and specify that it will be done every 10 minutes (H/10 * * * *). I do not recommend to use lower values than that since I have seen that even my monster notebook with i7-6700HQ and 64GB RAM is quite a bit stressed by the build those many build processes.

2016-12-22-23_54_06-github-triggered-build-config-jenkins

Note that this is a very short polling period for our test purposes only; we do not want to wait very long after a code change is detected.

Note also: you can click the Blue Question Markright of the Schedule text box to get help with the scheduler syntax.

Step 3.2: Save

Click Save

Step 4: Change the content of the Software Repository

Now we expect that a change of the SW repository is detected latest 2 minutes after new code is checked in. Let us do so now: In this case, I have changed the content of README.md and commited the change:

(local repository)$ git add README.md
(local repository)$ git commit -m "changed README"
(local repository)$ git push

Within 2 minutes, I see a new job #24 running on the lower left:

2016-12-09-18_35_13-dashboard-jenkins

It seems that the page needs to be reloaded by refreshing the browser, so the dashboard displays the #24 build process as “Last Success”:

The build process was very quick, since we have not changed any relevant source code. The console log can be reached via the Jenkins -> Project Link -> Build History -> click on build number -> Console:

2016-12-11-21_55_22-github-triggered-build-687-console-jenkins

As you can see, after some hours, the git repository is downloaded even if there was no code change at all. However, Gradle will detect that the JAR file is up-to-date and it will not re-build the JAR file, unless there is a code change.

The disadvantage of a scheduled build process with high frequency is that the number of builds in the build history is increasing quickly:

2016-12-11-22_02_24-github-triggered-build-jenkins

Note: The build history is spammed by many successful builds with no code change, and it is not easy to find the interesting build among all those many unnecessary builds. Let us try to improve the situation by replacing periodic, scheduled builds by triggered builds:

Step 5: Triggered Builds

In Step 4, we have seen that periodic builds should not be performed in a very short timeframe, because:

  1. the Jenkins server is stressed quite a bit by configuring a too low build frequency
  2. the build history is polluted by information of many irrelevant build processes with no changed code.

Therefore, it is much better to create a triggered build. The target is to trigger a build process every time the developer is checking in new code to the software repository:

2016-12-21-15_12_25

In this way, a periodic build is not necessary, or can be done much less frequently.

What do we need to do?

  1. Make sure that the Jenkins server is reachable from the SW repository
  2. Configure the SW repository with a web hook for informing Jenkins upon each code change
  3. Configure Jenkins for triggered build

Let us start:

Step 5.1 Configure Jenkins for triggered Build

On the Jenkins Dashboard, click on the project:

2016-12-22-18_56_44-dashboard-jenkins

and then “Configure” on the left pane:

2016-12-22-18_58_28-github-triggered-build-jenkins

Scroll down to Build Triggers and check the “Trigger build remotels (e..g. , from scripts)” checkbox and choose an individual secret token (do not use the one you see here):

2016-12-22-19_03_16-github-triggered-build-config-jenkins

You will be provided with the build trigger URL, which is in my case:

JENKINS_URL/job/GitHub%20Triggered%20Build/build?token=TOKEN_NAME

And the JENKINS_URL is the URL needed to be contacted by the Git Repository. Save the URL above for later use.

Now click Save.

Step 5.2 Test Trigger URL locally

Now we can test the trigger URL locally on the Docker Host as follows (as found on this StackOverflow Q&A):

We need to retrieve a so-called Jenkins-Crumb:

(dockerhost)$ CRUMB=$(curl -s 'http://admin:your_admin_password@localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
(dockerhost)$ echo $CRUMB
Jenkins-Crumb:CCCCCCCCCCCCCCCCCCCCCCCCCC

Please make a note of the returned Jenkins-Crumb, since we will need this value in the next step.

Then we can use the Jenkins-Crumb as header in the build trigger request:

(dockerhost)$ curl -H $CRUMB 'http://admin:your_admin_password@localhost:8080/job/GitHub%20Triggered%20Build/build?token=hdsghewriohwziowrhwsn'

This should trigger a new build on Jenkins:

2016-12-22-21_56_44-dashboard-jenkins-v2

By clicking on the build and then the “Console Output”, we see a successful build with no changed data:

2016-12-22-22_01_36-github-triggered-build-712-console-jenkins

Step 5.3 Make sure that the Jenkins Server is reachable from the SW repository

We are running the Jenkins server as a Docker container within a Vagrant VM as host. In step 2 we have made sure that the Docker container is reachable from the local network by exposing the Docker ports and by configuring port forwarding in VirtualBox. However, the Docker container is not yet reachable from the Git Repository, since the Router will block all requests, as long as no pot forwarding is configured on the router:

2016-12-21-16_02_27

Let us fix that now:

In my case, the (sorry, German) input mask of the router looks like follows:

2016-12-22-19_30_22-fritzbox-7490

I am mapping outside port 8080 to the internal machine running the Docker Host VM.

Now, the routing should work. We will test this in the next step.

2016-12-22-23_59_49

Step 5.4: Add Webhook to Git SW Repository

Now we need to add a Webhook to the Git repository. In my case, the repository is located at https://github.com/oveits/simple-restful-file-storage. On that page, goto

Settings -> Webhooks-> Add webhook -> Confirm password

Then copy&paste the URL of Step 5.1 into the Payload URL with following changes:

  • Change JENKINS_URL by the IP address or DNS name your router is reachable from the Internet.
  • Choose a port that you intend to open for this service (e.g. 8080) in the next step.
  • Add admin:your_admin_password@ before the JENKINS_URL; use your own username and password here
  • append &Jenkins-Crumb=CCCCCCCCCCCCCCCC to the URL with the value of the Jenkins-Crumb we have retrieved in the previous step

Example with the items to change in red:

http://admin:your_admin_password@your_public_ip_or_name:8080/job/GitHub%20Triggered%20Build/build?token=TTTTTTTTTTTTTTTTT&Jenkins-Crumb=CCCCCCCCCCCCCCCCCCCC

2016-12-22-22_08_39-webhook-http___veits-no-ip-biz_8080_job_github%20triggered%20build_build

 

For the other fields, keep the defaults and klick Add Webhook.

If everything works fine, we already should see a successful delivery of the trigger on the lower end of the Github page:

2016-12-22-22_17_57-webhook-http___veits-no-ip-biz_8080_job_github%20triggered%20build_build-v2

If it was not successful, you can see more details by clicking on the request:

2016-12-22-22_20_52-webhook-http___veits-no-ip-biz_8080_job_github%20triggered%20build_build

Step 6: Test triggered Build upon Code Push

This is the final step of this tutorial: we now will test that a build is triggered each time a user pushes new code to the repository.

Step 6.1: Install Git locally

If Git is not installed locally, so do it now.

Step 6.2: Download the Project Repository

We now clone the project by issuing the command

$ git clone https://github.com/oveits/simple-restful-file-storage

Step 6.3: Change Code

You can perform a minor change the content of the README.md in order to test the triggered build.

Step 6.4: Push Code to the Repository

With the commands

$ git commit -am "Minor change of README.md to trigger a Jenkins build"
$ git push

We push the changed code to the SW repository.

If everything works correctly, we will immediately see, that Git has triggered Jenkins to perform a build by reloading the Jenkins Dashboard (32 sec ago, in this screenshot):

2016-12-22-22_50_14-dashboard-jenkins-v2

We can check the build by clicking on the Last Success build and then “Console Output”:

2016-12-22-22_53_08-github-triggered-build-713-console-jenkins

Gradle was clever enough to detect that no relevant code had been changed, so everything is still up to date.

With this procedure we have made sure that the Software repository will trigger a new build process on each and every code change. Moreover, the Jenkins server is not polluted with unnecessary builds anymore, since we have switched off periodic builds.
thumps_up_3

Summary

In this blog post we have performed following tasks:

  1. Started Jenkins in a Docker container
  2. Configured and tested periodic builds
  3. Configured and tested triggered builds
  4. Made sure that the Git Software repository is triggering such a build at every code change

As in the other parts of this series, we have run Jenkins interactively in a Docker container. See below a discussion of the advantages of periodic and triggered builds:

Periodic Builds vs Triggered Builds

When we compare periodic builds with triggered builds, we see following advantages/disadvantages:

Complexity of Setup: periodic builds are much easier to set up. They only need to be configured on Jenkins. Triggered builds requires setup steps on the Jenkins Server, the Software Repository and intermediate Firewalls, if the Jenkins Server is located in a private network.

Economics: Triggered builds are more economic in terms of Jenkins Server load. The build processes run only, when needed.

Handling: Triggered builds have important handling advantages compared to triggered builds: firstly, each and every code change can be tested helping the programmer to get near immediate feedback for every code change. Secondly, the build log is not polluted by hundreds of irrelevant builds.

In my opinion, a clear winner is: triggered builds. Those may be combined with periodic clean builds at certain milestones.

2016-12-22-23_39_41

 

References

 

5

Jenkins Part 2: automated Code Download and Build (Gradle + Maven)


2016-11-30-18_19_38

NEW (2017-01-02): you now can immediately start with part 2 (this post) without going through the steps of part 1. A corresponding pre-installed Docker image is provided.

NEW (2017-01-05): I have added the Maven build path with a fat executable JAR file.

In this blog post, we will perform our first automated job within Jenkins, the most popular open source tool for Continuous Integration and Continuous Deployment. Like in part 1 of this blog series, we will start Jenkins in a Docker container and define and run a first task:

  • download source code from GitHub
  • Create a lean executable JAR file via Gradle
  • Create a fat  executable JAR file via Maven

At the end of this session, we will have learned how to download GitHub code and build a Java program (executable JAR file) on a push of a button.

This blog post series is divided into following parts:

    • Part 1: Installation and Configuration of Jenkins, loading Plugins
    • Part 2 (this blog): Creating our first Jenkins job: GitHub download and Software build
    • Part 3: Periodic and automatically triggered Builds
    • Part 4 (planned): running automated tests

What is Jenkins?

Jenkins is the leading open source automation server mostly used in continuous integration and continuous deployment pipelines. Jenkins provides hundreds of plugins to support building, deploying and automating any project.

 

Jenkins build, test and deployment pipeline
Jenkins build, test and deployment pipeline

A typical workflow is visualized above: a developer checks in the code changes into the repository. Jenkins will detect the change, build (compile) the software, test it and prepare to deploy it on a system. Depending on the configuration, the deployment is triggered by a human person, or automatically performed by Jenkins.

For more information, see the introduction found in part 1 of this blog series.

Our first Jenkins Job

In this hello world, we will perform the first part of the typical build pipeline shown in the previous chapter, applied to a java hello world program:

  1. Upon a push of a button, download code from the GIT repository
  2. Build an executable jar from the java code

2016-12-09-15_29_08

Tools used

      • Vagrant 1.8.6
      • Virtualbox 5.0.20
      • Docker 1.12.1
      • Jenkins 2.32.1

Prerequisites:

      • Free DRAM overall >~ 4 GB
      • A Docker Host is available. Perform Step 1 in Part 1 of this blog series, if you are in need of a Docker host.

Step 1: Start Jenkins in interactive Terminal Mode

Step 1.1: Make sure the TCP Port is unused

Make sure that port 8080 is unused on the Docker host. If you were following all steps in part 1 of the series, you might need to stop cadvisor:

(dockerhost)$ sudo docker stop cadvisor

Alternatively, you can alter the ports option below from -p8080:8080 to -p9090:8080, as an example.

Step 1.2: Alternative (A): If you have followed all steps in part 1 of this blog series, start Jenkins from the official Jenkins image

I assume that you have followed all steps in part 1 of the blog series. In this case, you have created a jenkins_home directory on you Docker host. All popular plugins are installed and an Admin user has been created. If the Jenkins container is not started already, we start it with the jenkins_home Docker host volume mapped to /var/jenkins_home (as we have done in part 1 of this blog series):

(dockerhost)$ cd <path_to_jenkins_home>
(dockerhost:jenkins_home)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Step 1.2: Alternative (B): You prefer to start from a pre-installed Jenkins Image

If you have not followed the steps in part 1, or if you prefer to start from a cleanly installed image, you also can start Jenkins from a cleanly installed Docker image like follows:

(dockerhost)$ sudo docker run -it --name jenkins -p8080:8080 -p50000:50000 oveits/jenkins_tutorial:part2_step1
Running from: /usr/share/jenkins/jenkins.war
...
--> setting agent port for jnlp
--> setting agent port for jnlp... done

Note that in alternative (B), all data is kept within /var/jenkins_home_local directory within the created container. For the case you want to save your work thereafter, I have not used the docker run remove option --rm. This will give you the chance to stop and start the same container later. In addition, you can create your own image from the stopped container in order to retain the work.

Step 2: Open Jenkins in a Browser

Now we want to connect to the Jenkins portal. For that, open a browser and open the URL

<your_jenkins_host>:8080

In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.

localhost:8080

Note: In case of Vagrant with VirtualBox, per default, there is only a NAT-based interface and you need to create port-forwarding for any port you want to reach from outside (also the local machine you are working on is to be considered as outside). In this case, we need to add an entry in the port forwarding list of VirtualBox:
2016-11-30-19_22_22-regel-fur-port-weiterleitung

We have created this entry in part 1 already, but I have seen that the entries were gone again, which seems to be a VirtualBox bug. I have added it again now.

Log in with the admin account we have created in the last session:

2016-12-09-10_24_00-jenkins

Step 3: Alternative (a): Prepare Gradle Usage

If you later prefer to use Gradle instead of (or in addition to) Maven, you need to prepare its first usage. For Maven preparation, see Step 3, Alternative (b) below.

On this wiki page about the Gradle plugin we find that we need to configure Gradle first:

Goto Jenkins -> Manage Jenkins -> Global Tool Configuration (available for Jenkins >2.0)

2016-12-09-11_34_55-manage-jenkins-jenkins

2016-12-09-11_35_26-global-tool-configuration-jenkins

Scroll down to Gradle -> Add Gradle

2017-01-02-14_27_26-global-tool-configuration-jenkins

-> choose Version (Gradle 3.2.1 in my case)

-> Add a name (“Gradle 3.2.1” in my case)

-> Save

Since we have checked “Install automatically” above, I expect that it will be installed automatically on first usage.

Step 3: Alternative (b): Prepare Maven Usage

If you later prefer to use Maven instead of (or in addition to) Gradle, you need to prepare its first usage. For Gradle preparation, see Step 3, Alternative (a) above.

Goto Jenkins -> Manage Jenkins -> Global Tool Configuration (available for Jenkins >2.0)

2016-12-09-11_34_55-manage-jenkins-jenkins

2016-12-09-11_35_26-global-tool-configuration-jenkins

Scroll down to Maven -> Add Maven

2017-01-02-14_32_46-global-tool-configuration-jenkins

-> choose Version (3.3.9 in my case)

-> Add a name (“Maven 3.3.9” in my case)

-> Save

Since we have checked “Install automatically” above, I expect that it will be installed automatically on first usage.

Step 4: Create a Job (Freestyle Project)

Step 4.1 Enter Name and Project Type

Either click on “create new jobs” or on New Item.

Now enter an Item name and click on Freestyle Project and OK:

2016-12-09-10_55_56-new-item-jenkins

Step 4.2: Specify GitHub Project

Check “GitHub project” and add the HTTPS GitHub URL. I have used a small Apache Camel project of mine that provides a simple restful file storage:

https://github.com/oveits/simple-restful-file-storage

2016-12-09-11_02_22-github-triggered-build-config-jenkins

Step 4.3 Configure Source Code Management

Under Source Code Management, we choose “Git” and specify the GitHub repository a second time. If it is public, we do not enter the credentials for now:

2016-12-09-14_53_46-github-triggered-build-config-jenkins

-> Click Apply

Note also that I have chosen a branch different from the master branch (“jenkinstest”). I have created this new branch in order to keep the master branch clean from any changes that might be needed to test Jenkins.

Step 4.4 Configure Build Triggers (postponed to part 3 of this blog post series)

For now, we will test only manual “build now” triggers, so we do not need to specify any build triggers. Build triggers will be tested in the next blog post.

Step 4.5: Alternative (a): Configure Gradle Build

Prerequisite: For creation of an executable JAR, the file build.gradle in the project root directory must be prepared. See e.g.

Here, we show how to build the project via Gradle. If you prefer to build via Maven, seee Step 4.5: Alternative (b): Configure Maven Build below.

Here, we scroll down to “Build” -> click 2017-01-02-14_54_07-github-triggered-build-config-jenkins -> “Invoke Gradle script”

2016-12-09-11_11_47-github-triggered-build-config-jenkins

Choose the Gradle version we have prepared in Step 3 and add the task “jar”:

2017-01-02-14_56_34-github-triggered-build-config-jenkins

Note that keeping the (Default) Gradle version will not work, as long as this Default has not been defined. See Appendix A for details.

The Gradle task “jar” will create our executable JAR file.

-> Click Save at the bottom left.

Step 4.5: Alternative (b): Configure Maven Build

Here, we show how to build the project via Maven. If you prefer to build via Gradle, seee Step 4.5: Alternative (a): Configure Graven Build above.

Here, we scroll down to “Build” -> click 2017-01-02-14_54_07-github-triggered-build-config-jenkins -> “Invoke Maven script”

2017-01-03-13_52_27-github-triggered-build-config-jenkins

Choose the Maven version we have prepared in Step 3 and specify the goal “package”:

2017-01-03-14_00_05-github-triggered-build-config-jenkins

Note that keeping the (Default) Maven version will not work, as long as this Default has not been defined.

The Maven goal “package”will build our JAR file.

-> Click Save at the bottom left.

Step 5: Test manually triggered Build

We can trigger a build manually via Jenkins -> drop-down right of “GitHub Triggered Build” -> Build Now.

2016-12-09-11_46_03-dashboard-jenkins

Click on #1 of the build history:

2017-01-02-14_59_15-github-triggered-build-jenkins-v2

then on Console Output:

2017-01-02-15_02_46-github-triggered-build-1-jenkins-v2

 

We can observe the console output e.g. by clicking on the build link in the build history, then clicking Console:

Output for step 5 in case of Gradle:

2017-01-02-15_04_25-github-triggered-build-1-console-jenkins

This may take a while (~11 min in my case with a 100Mbps Internet connection):