0

GIT remove large binary Files


In this blog post, I will show how to completely remove large files from a GIT repository and its GIT history. I have found many examples on the net, but it was still a challenge to reduce the size of the GIT repository.

This has been tested on a real-world repository with almost 8000 commits and 120 branches. The target was

  • to keep the full (rewritten) history
  • to be able to keep any branch you want
  • usage of external software like BFG was forbidden due to security restrictions. However, I have tested BFG privately in this GIT Repo and I can recommend its usage. See this GIT repo, which will promote a simple example how BFG can be used.

In our case, the GIT history has grown to 223 MB, because a binary of ~40 MB was updated several times:

du -h -d 1
223M    ./.git
...

Step 0: Inform your Developers

First of all, you need to inform everybody, what you are going to do, and when it will be done. The changes are destructive, and even if you fork the repository, there is no easy way to revert all the changes. E.g. existing pull requests will be gone.

Step 1: Disallow any changes to the Repo

Repos like BitBucket allow fine-grained control over who is allowed to do what. I recommend that you disallow any changes to the repo apart the one you are performing.

Step 2: Move and Fork the Repo

As a kind of backup of the repo, I recommend to move the repository and work on a forked copy of the repo.

Step 3 (optional): Replace remote Branches by Tags

In our case, we had about 120 branches. On one hand, cleaning the repo from binaries was a good opportunity to substantially reduce the number of branches. However, I wanted to give everybody the possibility to re-create the branch in question. For that, I have written a little script that helps to replace branches by tags. In a later step, I have sent information to the developers, how they can use the tag to retain their branch.

#!/bin/sh

eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa

# packed-refs might not alwasy work, if the refs are not packed:
#grep -R origin .git/packed-refs | awk '{print $2}' | sed 's/refs\/remotes\/origin\///' | while read BRANCH
# using git branch -a instead:
git branch -a | grep remotes/origin | sed 's/remotes\/origin\///' | while read BRANCH
   do
      echo git checkout $BRANCH
      git checkout $BRANCH
      echo git tag branch/$BRANCH refs/heads/$BRANCH
      git tag branch/$BRANCH refs/heads/$BRANCH
   done

With that, all branches can be re-created from the tags named branch/<branchname>, as seen below.

Step 4: Clean the local GIT History

Step 4.1: Clone the GIT Repo

First of all, we need t clone the forked repository to the local machine. As a reference for later, we also print the number of objects found.

git clone <URL-of-your-large-repo>
cd <your-large-repo>
git count-objects -v

Step 4.2: Remove Binaries from current Commit

Let us that all binaries that you need to remove are located in a directory named binarydir. In that case, let us remove the binaries first:

cd binarydir
rm *.ear *.war" *.jar *.zip*.exe
cd ..

Step 4.3: Filter Binaries from GIT History

Now the binaries are still found in the .git GIT history. To remove them from .git, let us perform the following:

git filter-branch --tag-name-filter 'cat' -f --tree-filter '
    find . -type d -name binarydir | while read dir
      do
        find $dir -type f -name "*.ear" -o -name "*.war" -o -name "*.jar" -o -name "*.zip" -o -name "*.exe" | while read file
          do
             git rm -r -f --ignore-unmatch $file
          done
      done
' -- --all

This will take quite long, if your repo has many commits (in our case, it was ~2 to 3 hrs for >~ 7000 commits).

Note: If you have the possibility to use BFG, you can do so to speed up the process. See my GIT repo git-repo-cleaner for an example.

Step 4.4: Adapt References and perform a Garbage Collection

Now the references need to be updated and we need to perform a garbage collection:

git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref –d
git reflog expire --expire=now –all
git gc --prune=now --aggressive

Step 5: Verify that the Size is Reduced

After this procedure, all specified binaries should be removed from the repository. In our case the git history size has been reduced to 21 MB.

$ du -h -d 1
21M     ./.git
...

If the size is not reduced, you might want to troubleshoot with following command

git count-objects -v

and re-perform step 4.4. If this does not help, you can review some more troubleshooting commands on this blog post.

Step 6: Save the Repository to Overwrite the old Repository

Note: We have experienced a lot of problems, because we tried to retain the old repo names and URL (seen with ‘git remote -v’) furst: there were around 30 developers and testers that had local clones of the repo. Two or three times, a developer or tester was pushing his local clone to the remote repo and all the work was in vain. At the end, we have decided to use a different repo name (and thus a different repo URL) for the cleaned repo. This has helped to prevent any further unintended spoiling of the repo.

On GitHub or BitBucket, create a new repo we will use for the cleaned repo. Then, we change the GIT URL of our local cleaned repo to match the new repo:

git remote -v
git remote remove origin
git remote add origin <new cleaned repo URL>
git remote -v

git push --force

Step 7 (optional): Re-Create Branches

Step 7.1: Re-create Branches from Tags

If you have followed step 3, your developers and testers can re-crate their branches from the tags like follows:

BRANCH=<your branch  name>
git checkout branch/$BRANCH
git checkout -b $BRANCH
git push origin $BRANCH

If the last command does not work, it the error message usually tells you, what to do instead (set the remote branch name…).

Step 7.2: Alternatively, push a Branches directly

Since Step 3 will also have created all branches locally, you also can re-create remote branches from them more easily than other developers and testers:

git checkout $BRANCH
git push --set-upstream origin $BRANCH

Step 7.3: Alternatively, create all Branches

If you want to re-create all branches, you can do that like follows:

eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa

git branch | awk -F '[ *]' '{print $3}' | while read BRANCH
   do
      echo git checkout $BRANCH
      git checkout $BRANCH
      echo "git push --set-upstream origin $BRANCH"
      git push --set-upstream origin $BRANCH
   done

We have chosen, not to do so, since only a few of the branches really were needed.

Step 8 (optional): Clean Tags

Tags that are not needed anymore can be deleted like follows:

TAG=branch/$BRANCH
git tag -d $TAG  # delete tag locally
git push origin :refs/tags/$TAG  # delete tag in repo

Here, I have assumed that the tag you want to remove has been created in Step 3 before with the name branch/$BRANCH.

Step 9: Push Tags to remote Repo

Tags that are still needed, can be pushed to the remote repo:

git push origin <tag_name>

Last Steps

  • The new repo needs to be adapted with respect to all settings, so it matches the original repo, e.g.
    • default branch (=develop in our case)
    • pull request reviewer list
    • whether or not to allow changes without pull request (not allowed for “develo” and “release/vx.x” branches in our case)
    • disallow forced updates

Summary

In a real-world example, we have cleaned a GIT repo from a set of large binary files.

References

0

Getting Started with DC/OS on AWS


In the step-by-step tutorial Getting Started with DC/OS on Vagrant, we have learned how to install a MesosPhere DC/OS data center operating system locally. This time, we will install a DC/OS system on AWS Cloud: existing AWS CloudFormation templates will help us create a fully functional DC/OS data center with a Mesos master and five Mesos slaves within less than two hours. At the end, we will test the environment by starting a “Hello World” service based on Docker from DC/OS’ administration panel and accessing the application from the Internet.

MesoSphere DC/OS is a Data Center Operating System, which is built upon Apache Mesos and Mesosphere Marathon, an open source container orchestration platform. It has the target to hide the complexity of data centers when deploying applications.

AWS, Amazon Web Services is the leading provider offering Infrastructure as a Service and more.

 

Beware that running DC/OS on AWS does not come for free. I am still in the free tier period, so I had to pay only $0.48 for a test duration of less than 45 minutes (measured from the time I have created to the point in time I have terminated the stack). However, the induced cost might be higher in your case. Also, I had to pay a lot more, as the time of usage increased and some of the free usage limits were exceeded.

I recommend to check your current bill before and after the the test on the AWS Billing Home for the region US-West-2.

The guide has been tested for the region us-west-2 and us-east-2. However, it has worked only for us-west-2; probably because the correct image IDs are missing for us-east-2.

We are loosely following https://aws.amazon.com/blogs/apn/announcing-mesosphere-dcos-on-aws/, but we had to add correct some commands and add some instructions on user permissions.

See also

Prerequisites

Step 1: Configure your Credentials

You need to have entered your AWS Access Key and Secret on the ~/.aws/credentials file:

[default]
aws_access_key_id = XXXXXXX
aws_secret_access_key = KKKKKKKK

Step 2: Create an SSH Key for DC/OS

aws --region us-west-2 ec2 create-key-pair --key-name dcos-demo-key --output text --query KeyMaterial > dcos-demo-key_us-west-2.pem
cp dcos-demo-key_us-west-2.pem dcos-demo-key.pem
chmod 600 dcos-demo-key.pem

This will create an additional key pair on region us-west-2 (before, I had no key pair on this region; now it is one key):

Step 3: Find Cloud Formation Template URL

The official DCOS documentation v1.10 on AWS installation offers two options:

For our tests, we will choose the basic variant with one Mesos master and five Mesos slaves.

The corresponding CloudFormation Templates can be found on this page.

We copy the “Launch Stack” link for us-west-2 with Single Master and paste it here:

https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?templateURL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/a5ecc9af5d9ca903f53fa16f6f0ebd597095652e/cloudformation/single-master.cloudformation.json

From the link, we can see that the template URL is as follows. On a Linux shell (e.g. GIT Bash on Windows), we define:

TEMPLATE_URL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/a5ecc9af5d9ca903f53fa16f6f0ebd597095652e/cloudformation/single-master.cloudformation.json

Step 4: Launch the CloudFormation Stack from AWS CLI

Step 4.1: First Attempt to launch the Stack

From our main instructions page, we find something like:

aws --region us-west-2 cloudformation create-stack --stack-name dcos-demo \
    --template-url ${TEMPLATE_URL} \
    --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="dcos-demo-key" \
    --capabilities CAPABILITY_IAM

Note that there were some errors in the instructions page: the line feed formatting was wrong and a comma was missing. This has been corrected above.

If your AWS CLI is using a user without CloudFormation permissions, you will receive the following error message:

A client error (AccessDenied) occurred when calling the CreateStack operation: User: arn:aws:iam::924855196031:user/secadmin is not authorized to perform: cloudformation:CreateStack on resource: arn:aws:cloudformation:us-east-2:924855196031:stack/dcos-demo/*

If you have not encountered this error, you can skip the next three substeps.

Step 4.2: Create Policy for CloudFormation Permissions

On the EC2 Dashboard of the AWS Console for us-west-2 (choose right region in the URL), choose

–> Services
–> IAM
–> Policies
–> Create Policy
–> Select Policy Generator
–> Choose Parameters:
Effect: Allow
AWS Service: AWS CloudFormation
Actions: All
Actions ARN: *

–> Add Statement
–> edit Name, e.g. “CloudFormation”

–> Create Policy

Step 4.3: Attach Policy to User

–> Users
–> Choose your user
–> Add Permission
–> Attach existing policies directly
–> check “CloudFormation”

–> Next Review

–> Add permissions

Step 4.4: Try again: Create Policy for CloudFormation Permissions

TEMPLATE_URL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/a5ecc9af5d9ca903f53fa16f6f0ebd597095652e/cloudformation/single-master.cloudformation.json
aws --region us-west-2 cloudformation create-stack --stack-name dcos-demo \
    --template-url ${TEMPLATE_URL} \
    --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="dcos-demo-key" \
    --capabilities CAPABILITY_IAM

This time we get following Response:

{
“StackId”: “arn:aws:cloudformation:us-west-2:924855196031:stack/dcos-demo/0c90e5c0-c716-11e7-9e0d-50d5ca2e7cd2”
}

After some minutes, we will see CREATE_COMPLETE in the AWS Console of US West 2:

On the EC2 Dashboard, we see:

After clicking the “8 Running Instances” link, we see:

The DC/OS is up and running!

Excellent! Thump up!

If you see other errors like

  • API: s3:CreateBucket Access Denied
  • API: iam:CreateRole User: arn:aws:iam::924855196031:user/secadmin is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::924855196031:role/dcos-demo-SlaveRole-LP582D7P32GZ
  • The following resource(s) failed to create: [Vpc, ExhibitorS3Bucket, SlaveRole, DHCPOptions]. . Rollback requested by user.

then follow the instructions in Appendix A. Those are permissions issues.

Step 5 (recommended): Restrict Admin Access

The default is that the machines are open to the Internet world. I recommend to change the settings, so only you can access your systems.

On the EC2 Dashboard -> Security Groups, check out the security group with the description “Enable admin access to servers” and edit the source IP addresses:

Replace 0.0.0.0/0 (any) to “My IP” for all sources.

–> Save

Note, this step needs to be repeated any time your source IP address changes. See Step B6 of AWS Automation based on Vagrant — Part 2: Installation and Usage of the Vagrant AWS Plugin, if you are interested in an example that shows how to update the security rules to point to “My IP” per shell script based on AWS CLI.

TODO: find a better way to secure the admin interfaces, e.g. by adapting the CloudFoundation templates before starting the stack. This way, the admin interfaces are not open to the world from the beginning on.

Step 6: Access the DC/OS Admin Console

Now let us access our DC/OS Admin Console. For that, let us find the public DNS name of the master:

$ aws cloudformation describe-stacks --region us-west-2 | grep dcos-demo-ElasticL | awk -F '"' '{print $4}'
dcos-demo-ElasticL-XRZ8I3ZZ2BB2-549374334.us-west-2.elb.amazonaws.com

This is the DNS name we can connect to:

In my case, I have signed in with Google.

We reach a nice dashboard:

DCOS Dashboard on AWS

Step 7: Install DCOS CLI

The easiest way to automate application orchestration is to make use of the DCOS CLI. For that, click on your name and then “Install CLI” and follow the instructions. You will find some dcos command examples in my previous blog post on DC/OS.

I have followed the Windows instructions, i.e.

dcos cluster setup http://dcos-demo-elasticl-pu3fgu8047kg-271238338.us-west-2.elb.amazonaws.com

A browser window was started and I have logged into the browser session via Google. Then the token was offered:

I had to Copy and paste the token into the command line:

Enter OpenID Connect ID Token: eyJ0eXAiOiJKV1QiLCJ…

After that you should be able to see the dcos services:

dcos service
NAME              HOST     ACTIVE  TASKS  CPU    MEM    DISK  ID
marathon       10.0.5.242   True     4    2.75  1836.0  0.0   d456c8ce-f0e6-4c61-9974-94e3426f5fe8-0001
metronome      10.0.5.242   True     0    0.0    0.0    0.0   d456c8ce-f0e6-4c61-9974-94e3426f5fe8-0000

Marathon and Metronome are already running.

Step 8: Install Marathon LB

(dcos package describe --config marathon-lb)
dcos package install marathon-lb
By Deploying, you agree to the Terms and Conditions https://mesosphere.com/catalog-terms-conditions/#community-services
We recommend at least 2 CPUs and 1GiB of RAM for each Marathon-LB instance.

*NOTE*: For additional ```Enterprise Edition``` DC/OS instructions, see https://docs.mesosphere.com/administration/id-and-access-mgt/service-auth/mlb-auth/
Continue installing? [yes/no] yes
Installing Marathon app for package [marathon-lb] version [1.11.1]
Marathon-lb DC/OS Service has been successfully installed!
See https://github.com/mesosphere/marathon-lb for documentation.

After clicking on marathon-lb, we the details of the configuration of the marathon load balancer:

 

Step 9: Create a Hello World Application

Similar to the blog post, where we have installed DC/OS locally via Vagrant, let us create a hello world application. We choose a NginX application that is displaying some information on the source and destination IP addresses and ports seen from within the container. For that, let us click

–> Services

–> RUN A SERVICE

–> JSON Configuration

Cut and paste following text into the field:

{
   "id": "nginx-hello-world-service",
   "container": {
     "type": "DOCKER",
     "docker": {
       "image": "nginxdemos/hello",
       "network": "BRIDGE",
       "portMappings": [
         { "hostPort": 0, "containerPort": 80, "servicePort": 10007 }
       ]
     }
   },
   "instances": 3,
   "cpus": 0.1,
   "mem": 100,
   "healthChecks": [{
       "protocol": "HTTP",
       "path": "/",
       "portIndex": 0,
       "timeoutSeconds": 2,
       "gracePeriodSeconds": 15,
       "intervalSeconds": 3,
       "maxConsecutiveFailures": 2
   }],
   "labels":{
     "HAPROXY_DEPLOYMENT_GROUP":"nginx-hostname",
     "HAPROXY_DEPLOYMENT_ALT_PORT":"10007",
     "HAPROXY_GROUP":"external",
     "HAPROXY_0_REDIRECT_TO_HTTPS":"true",
     "HAPROXY_0_VHOST": "dcos-demo-PublicSl-1NSRAFIDG6VZS-267420313.us-west-2.elb.amazonaws.com"
   }
}

As HAPROXY_0_VHOST you need to use the public slave’s load balancer address you can retrieve via AWS CLI via:

$ aws cloudformation describe-stacks --region us-west-2 | grep dcos-demo-PublicSl | awk -F '"' '{print $4}' 
dcos-demo-PublicSl-1NSRAFIDG6VZS-267420313.us-west-2.elb.amazonaws.com

 

Now:

–> REVIEW & RUN

–> RUN SERVICE

You will see that the nginx-hello-world-service is being deployed:

After some seconds, the 3 containers are up&running:

 

After clicking on the name of the service, you will see the three containers:

Note that the column “UPDATED” will disappear, if the browser width is too low. If you have a small screen, you can scale the browser content with CTRL and Minus.

Step 10 (optional): Reach the service from inside

On an internal host, I can reach the NginX server via two ways:

Step 10.1: Access Application Container on a Private Slave

The following command will return the HTML code of the single container running on a private slave:

curl 10.0.2.9:14679 # SlaveServerGroup

Here, we have chosen the Endpoint address we can retrieve from the services details page:

Step 10.2: Access the Load Balancer Address

We can also contact the internal load balancer endpoint for the service. This has the advantage that the access is load balanced among the different containers we have started for the service.

curl 10.0.6.204:10007 # PublicSlaveServerGroup

Here we have combined the Public slave IP address with the HAPROXY port we have configured as a label:

Excellent! Thump up!

In the next step, we will access the load balancer endpoint via the Internet.

Step 11: Connect to the Service via Internet

Step 11.1: Direct Connection to the Public Slave

The CloudFormation stack is configured in a way that allows reaching the public slave via the Internet on port 10007. This allows us to access the hello world application directly:

Step 11.2: Connection via AWS Load Balancer

Consider a case where we have more than one public slave. In those situations, it is better to access the service via AWS load balancer, which will distribute the load among the different public slave marathon load balancers (i.e. HAPROXY load balancers). In our case, we access the service on port 80: http://dcos-demo-PublicSl-1NSRAFIDG6VZS-267420313.us-west-2.elb.amazonaws.com

The load balancer address can be retrieved via

$ aws cloudformation describe-stacks --region us-west-2 | grep dcos-demo-PublicSl | awk -F '"' '{print $4}'
dcos-demo-PublicSl-1NSRAFIDG6VZS-267420313.us-west-2.elb.amazonaws.com

By pasting the return value into the browser, we are redirected to the corresponding https page:

After refreshing the page, we will see that we will get answers from the other two containers as well:

With that, we have learned how to create a service and access it from the Internet.

Excellent! Thump up!

 

Step 12: Explore the Marathon Load Balancer

You can access the marathon load balancer by retrieving the public IP address of the public slave from the AWS console (EC2):

We then access the HA Proxy statistics page and configuration page by entering the public IP address or DNS name into the URL field, and adding one of the following strings:

  • :9090/haproxy?stats
  • :9090/_haproxy_getconfig

Step13: Delete the Stack

Do not forget to delete the stack, since it will induce quite a bit of cost if you fail to do so. The stack can be deleted via AWS CLI as follows:

aws --region us-west-2 cloudformation delete-stack --stack-name dcos-demo

Better you check on the  AWS Console that all resources have been deleted successfully:

Excellent! Thump up!

Summary

In this blog post, we have learned to install a DC/OS Cluster on AWS using an existing CloudFormation template. For that, we have used AWS CLI to spin up a DC/OS environment with a single master, a single public slave, and five private slaves (see Appendix ?? below how to tweak the template to run only two private slaves in order to save some money).

Similar to the tests we had performed on a local machine using Vagrant described in the post Getting Started with DC/OS on Vagrant, we have installed a marathon load balancer, before we have deployed a three-container hello-world application. We have shown how to access this application from the public Internet using the AWS elastic load balancer that has been installed automatically via the CloudFormation stack. Moreover, we have shown how to access the marathon load balancer’s statistics and configuration page.

In the course of this step by step tutorial, we have mastered

  • user permission challenges (see step 4 and Appendix A)
  • networking challenges

We had to figure out that the services are only reachable via the AWS load balancers.

Appendix A: Add required User Permissions

Appendix A1: Remedy S3 Permission Error

Symptoms


If your user lacks the correct S3 permissions, we will get following errors in the  AWS Console, when trying to start the CloudFormation stack:

  • API: s3:CreateBucket Access Denied
  • API: iam:CreateRole User: arn:aws:iam::924855196031:user/secadmin is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::924855196031:role/dcos-demo-SlaveRole-LP582D7P32GZ
  • The following resource(s) failed to create: [Vpc, ExhibitorS3Bucket, SlaveRole, DHCPOptions]. . Rollback requested by user.

Resolution

  1. Add S3 Permissions

2) Add IAM Policy:

Add Permissions -> Create policy

-> Policy Generator -> Select ->

-> Add Statement -> Next Step -> Edit Name “IAM” -> Create Policy

-> Filter: Policy Type: Custom managed

-> Choose “IAM”

Let us delete it via console and try again:

TEMPLATE_URL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/14509fe1e7899f439527fb39867194c7a425c771/cloudformation/single-master.cloudformation.json
aws --region us-west-2 cloudformation create-stack --stack-name dcos-demo \
    --template-url ${TEMPLATE_URL} \
    --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="dcos-demo-key" \
    --capabilities CAPABILITY_IAM

Now we get following success messages on the AWS console:

After some minutes in the EC2 console:

 

 

Appendix B: [AcceptEULA] do not exist in the template

TEMPLATE_URL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/14509fe1e7899f439527fb39867194c7a425c771/cloudformation/single-master.cloudformation.json
aws --region us-east-2 cloudformation create-stack --stack-name dcos-demo \
    --template-url ${TEMPLATE_URL} \
    --parameters ParameterKey=AcceptEULA,ParameterValue="Yes" ParameterKey=KeyName,ParameterValue="dcos-demo-key" \
    --capabilities CAPABILITY_IAM

This time we get:

A client error (ValidationError) occurred when calling the CreateStack operation: Parameters: [AcceptEULA] do not exist in the template

This StackOverflow Q&A has pointed to the right direction: I tried to wrap all parameters in ”, but then I got a syntax error, that a comma is expected. The correct syntax turned out to be:

TEMPLATE_URL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/14509fe1e7899f439527fb39867194c7a425c771/cloudformation/single-master.cloudformation.json
aws --region us-east-2 cloudformation create-stack --stack-name dcos-demo \
    --template-url ${TEMPLATE_URL} \
    --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="dcos-demo-key" \
    --capabilities CAPABILITY_IAM

with commata between all parameters.

Appendix C: “Template error: Unable to get mapping for NATAmi::us-east-2::default”

How to Reproduce:

Get Key for region=us-east-2 from here: copy the link address of the corresponding Launch Stack Link and paste it somewhere:

https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?templateURL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/a5ecc9af5d9ca903f53fa16f6f0ebd597095652e/cloudformation/single-master.cloudformation.json

TEMPLATE_URL=https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/a5ecc9af5d9ca903f53fa16f6f0ebd597095652e/cloudformation/single-master.cloudformation.json;

Create a key for US East:

aws --region us-east-2 ec2 create-key-pair --key-name dcos-demo-key --output text --query KeyMaterial > dcos-demo-key_us-east-2.pem;
cp -i dcos-demo-key_us-east-2.pem dcos-demo-key.pem;
chmod 600 dcos-demo-key.pem;

Try starting the Stack:

aws --region us-east-2 cloudformation create-stack --stack-name dcos-demo \
    --template-url ${TEMPLATE_URL} \
    --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="dcos-demo-key" \
    --capabilities CAPABILITY_IAM;

If the user has all needed permissions (see steps 4.x above), then we get the following error:

A client error (ValidationError) occurred when calling the CreateStack operation: Template error: Unable to get mapping for NATAmi::us-east-2::default

Workaround

I have not investigated this issue. However, I guess that the error has to do with missing mappings for the images (AMI). A workaround is to use region=us-west-2 instead of us-east-2.

Appendix D: ERROR: “parameter value decos-demo-key for parameter name KeyName does not exist”

Reproduce

If you closely follow the instructions on https://aws.amazon.com/blogs/apn/announcing-mesosphere-dcos-on-aws/, correct the syntax errors in the aws commands, but keep the wrong key name “decos-demo-key” instead of “dcos-demo-key”, you will encounter the following problem:

After creation of the stack, we ask for the status:

aws --region us-west-2 cloudformation describe-stacks --stack-name dcos-demo --query Stacks[0].StackStatus

You will get the response:

"ROLLBACK_COMPLETE"

On the AWS Console of US West 2 we get:

The following error message is displayed:

Parameter validation failed: parameter value decos-demo-key for parameter name KeyName does not exist. Rollback requested by user.

Solution:

Correct the demo key name: “dcos-demo-key” instead of “decos-demo-key”

Appendix E: Adapt the CloudFormation Template to your Needs

The CloudFormation template is spinning up one master, one public slave, a NAT machine and five (!) private slaves. For the purpose of hello world testing we are performing, two instead of five private slaves are plenty. For that, I have adapted the CloudFormation template as follows:

Step E.1: Download CloudFormation Template

curl https://s3-us-west-2.amazonaws.com/downloads.dcos.io/dcos/EarlyAccess/commit/a5ecc9af5d9ca903f53fa16f6f0ebd597095652e/cloudformation/single-master.cloudformation.json

Step E.2 Adapt CloudFormation Template

I have added following parameter to the template (in blue):

        "SlaveInstanceCount": {
            "Description": "Required: Specify the number of private agent nodes or accept the default.",
            "Default": "5",
            "Type": "Number"
        },
        "SlaveInstanceCountDesired": {
            "Description": "Required: Specify the number of private agent nodes or accept the default.",
            "Default": "2",
            "Type": "Number"
        },
        "PublicSlaveInstanceCount": {
            "Description": "Required: Specify the number of public agent nodes or accept the default.",
            "Default": "1",
            "Type": "Number"
        },

The default of this parameter is two instead of five.

In the same template, I have changed following parts (in blue)

        "SlaveServerGroup": {
            "CreationPolicy": {
                "ResourceSignal": {
                    "Timeout": {
                        "Fn::FindInMap": [
                            "Parameters",
                            "StackCreationTimeout",
                            "default"
                        ]
                    },
                    "Count": {
                        "Ref": "SlaveInstanceCountDesired"
                    }
                }
            },
            "Properties": {
                "MaxSize": {
                    "Ref": "SlaveInstanceCount"
                },
                "DesiredCapacity": {
                    "Ref": "SlaveInstanceCountDesired"
                },
                "MinSize": {
                    "Ref": "SlaveInstanceCountDesired"
                },

Note that the stack will be stuck in CREATE_IN_PROGRESS if the first Count is not changed from SlaveInstanceCount to SlaveInstanceCountDesired.

Step E.3: Create S3 Bucket

The template is too large to use it directly per file: you will get following error if you try to use the template as file TEMPLATE_FILE=template-file-name:

aws --region us-west-2 cloudformation create-stack --stack-name dcos-demo \
 --template-body ${TEMPLATE_FILE} \
 --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="AWS_SSH_Key" \
 --capabilities CAPABILITY_IAM
An error occurred (ValidationError) when calling the CreateStack operation: 1 validation error detected: Value '<the json cloudformation template is printed>' at 'templateBody' failed to satisfy constraint: Member must have length less than or equal to 51200

The solution is to move the template to an S3 bucket in the same region. Now let us create the bucket:

aws s3api create-bucket --bucket my-us-west-2-bucket --region us-west-2

Step E.4: Copy Template to S3 Bucket

The template file can be copied to the S3 bucket via a command like:

aws s3 cp template_filename s3://my-us-west-2-bucket/

Step E.5: Use Template

Now we are ready to use the S3 bucket URL to create the stack:

TEMPLATE_URL='https://s3.amazonaws.com/my-us-west-2-bucket/template_filename'
SSH_KEY=dcos-demo-key
   aws --region us-west-2 cloudformation create-stack --stack-name dcos-demo \
       --template-url ${TEMPLATE_URL} \
       --parameters ParameterKey=AcceptEULA,ParameterValue="Yes",ParameterKey=KeyName,ParameterValue="AWS_SSH_Key" \
       --capabilities CAPABILITY_IAM

After 15 minutes or so, you should see that the stack is up and running with two private slave instances:

Excellent! Thump up!

Appendix F: Configuration

F.1 Master cloud-config.yml

Is found on /usr/share/oem/cloud-config.yml:

#cloud-config

coreos:
  units:
    - name: etcd.service
      runtime: true
      drop-ins:
        - name: 10-oem.conf
          content: |
            [Service]
            Environment=ETCD_PEER_ELECTION_TIMEOUT=1200

    - name: etcd2.service
      runtime: true
      drop-ins:
        - name: 10-oem.conf
          content: |
            [Service]
            Environment=ETCD_ELECTION_TIMEOUT=1200

    - name: user-configdrive.service
      mask: yes

    - name: user-configvirtfs.service
      mask: yes

    - name: oem-cloudinit.service
      command: restart
      runtime: yes
      content: |
        [Unit]
        Description=Cloudinit from EC2-style metadata

        [Service]
        Type=oneshot
        ExecStart=/usr/bin/coreos-cloudinit --oem=ec2-compat

  oem:
    id: ami
    name: Amazon EC2
    version-id: 0.0.7
    home-url: http://aws.amazon.com/ec2/
    bug-report-url: https://github.com/coreos/bugs/issues

F.2 Public Slave cloud-config.yml

#cloud-config

coreos:
  units:
    - name: etcd.service
      runtime: true
      drop-ins:
        - name: 10-oem.conf
          content: |
            [Service]
            Environment=ETCD_PEER_ELECTION_TIMEOUT=1200

    - name: etcd2.service
      runtime: true
      drop-ins:
        - name: 10-oem.conf
          content: |
            [Service]
            Environment=ETCD_ELECTION_TIMEOUT=1200

    - name: user-configdrive.service
      mask: yes

    - name: user-configvirtfs.service
      mask: yes

    - name: oem-cloudinit.service
      command: restart
      runtime: yes
      content: |
        [Unit]
        Description=Cloudinit from EC2-style metadata

        [Service]
        Type=oneshot
        ExecStart=/usr/bin/coreos-cloudinit --oem=ec2-compat

  oem:
    id: ami
    name: Amazon EC2
    version-id: 0.0.7
    home-url: http://aws.amazon.com/ec2/
    bug-report-url: https://github.com/coreos/bugs/issues

References

 

0

Angular 4 Docker Example – for Angular Universal CLI


In this previous blog post, I have shown how to dockerize an Angular CLI application using NginX. However, the method works without adaptions only for classical (client-side rendered) Angular projects. In the current post, we will show, how an Angular Universal CLI project with server-side rendering and transition to client-side rendering can be dockerized. In this example, we will use the node http-server static server in order to deliver the Angular application to the browser.

We will apply the dockerization on my project I have created in the blog post Angular Universal CLI – Step-by-Step Example with REST Client that has been created based on the angular/universal-starter project.

Tools and Versions used

  • CentOS 7 image installed via Vagrant
  • Docker version 17.10.0-ce, build f4ffd25
  • git is installed (if not: sudo yum install -y git)

Step 1: Clone the Project from GitHub

(dockerhost)$ git clone https://github.com/oveits/universal-starter
(dockerhost)$ cd universal-starter/cli
(dockerhost)$ git checkout d6084ec

This is a project that has been forked from the /universal-starter project on GitHub.  I have added a REST interface towards WordPress API. With the git checkout command, you will start at the same step I have started this blog post.

You might prefer to perform your tests with the original project https://github.com/angular/universal-starter. This should work as well. Only the resulting page should look differently.

Step 2: Review the package.json

The Dockerfile will re-use scripts we find in the package.json. Therefore, we need to make sure that following scripts are defined in package.json:

...
  "scripts": {
    "ng": "ng",
    "start": "ng serve",
    "start:dynamic": "npm run build:dynamic && npm run serve:dynamic",
    "start:static": "npm run build:static && npm run serve:static",
    "build": "ng build",
    "build:client-and-server-bundles": "ng build --prod && ng build --prod --app 1 --output-hashing=false",
    "build:static": "npm run build:client-and-server-bundles && npm run webpack:server && npm run generate:static",
    "build:dynamic": "npm run build:client-and-server-bundles && npm run webpack:server",
    "generate:static": "cd dist && node prerender",
    "webpack:server": "webpack --config webpack.server.config.js --progress --colors",
    "serve:static": "cd dist/browser && http-server",
    "serve:dynamic": "node dist/server"
...

Step 3: Prepare the Dockerfile

The initial Dockerfile has been inspired by /angular4-docker-example on Github. I have forked and slightly changed the file, and we can download it via

(dockerhost)$ curl https://raw.githubusercontent.com/oveits/angular4-docker-example/master/Dockerfile > Dockerfile

For the static HTTP Server, we need to change the content as follows. Here, I have kept the original lines commented out in red, and the added lines in bold blue:

# Dockerfile
### STAGE 1: Build ###

# We label our stage as 'builder'
FROM node:8-alpine as builder

COPY package*.json ./

RUN npm set progress=false && npm config set depth 0 && npm cache clean --force

## Storing node modules on a separate layer will prevent unnecessary npm installs at each build
RUN npm i && mkdir /ng-app && cp -R ./node_modules ./ng-app

WORKDIR /ng-app

COPY . .

## Build the angular app in production mode and store the artifacts in dist folder
#RUN $(npm bin)/ng build --prod --build-optimizer
RUN npm run build:static


### STAGE 2: Setup ###

#FROM nginx:1.13.3-alpine
FROM node:8-alpine

## Install http-server
RUN npm install http-server -g

## Copy our default nginx config
#COPY nginx/default.conf /etc/nginx/conf.d/

## Remove default nginx website
#RUN rm -rf /usr/share/nginx/html/*

## From 'builder' stage copy over the artifacts in dist folder to default nginx public folder
#COPY --from=builder /ng-app/dist /usr/share/nginx/html
COPY --from=builder /ng-app/dist /dist

#CMD ["nginx", "-g", "daemon off;"]

WORKDIR /dist/browser
CMD ["http-server"]

In STAGE 1, we have made use of the script defined in the package.json, which will create all HTML files ahead of time (AOT) and put them in the dist/browser folder. In STAGE 2, we copy the dist folder, and run the http-server in the dist/browser folder, after we have installed the http-server globally.

Step 4: Build, Tag and Upload the Docker Image

The Docker Image can be build with the command similar to:

(dockerhost)$ docker build . --tag oveits/angular_universal-starter_with_cli:v0.1

Here, I have tagged it with a name and version, so I can upload it to Docker Hub on my oveits account. I also set a latest tag like follows:

docker tag oveits/angular_universal-starter_with_cli:v0.1 oveits/angular_universal-starter_with_cli:latest

Step 5: Test the image locally

We can test the image with

$ docker run -it --net=host oveits/angular_universal-starter_with_cli
Starting up http-server, serving ./
Available on:
 http://127.0.0.1:8080
 http://10.0.2.15:8080
 http://192.168.33.12:8080
Hit CTRL-C to stop the server

In this case, we can see that I am using a VirtualBox machine with a host port network interface with static IP address. Since we have chosen the –net=host option, the port should be accessible on this IP address. If your are using Vagrant with the default Vagrant interface only, you might need to map the VM’s port to your Host port within VirtualBox.

The browser content looks as expected. But is server-side rendering working as well?

Step 6: Upload the Image to Docker Hub

Now we can upload the image to Docker Hub:

(dockerhost)$ docker login
Username: <-- answer with your user credentials
Password:
(dockerhost)$ docker push oveits/angular_universal-starter_with_cli:v0.1 
(dockerhost)$ docker push oveits/angular_universal-starter_with_cli:latest

Step 7: Use from any Docker Host

Now we can use the image from any Docker host that can access the Internet. In my case, this is a CentOS machine on AWS. If we want the application to be available on the standard port 80, we can run the application as follows:

(dockerhost)$ docker run --rm --name angular_universal-starter_with_cli -d -p 80:8080 oveits/angular_universal-starter_with_cli

Port 80 was occupied in my case already, so I have mapped port 8080 to 8080 instead:

Also here, the server-side rendering works fine: the innterHTML content is visible in the HTML source:

Summary

General

In this blog post, we have dockerized an Angular Universal CLI with server-side rendering by making use of the ahead-of-time (AOT) compilation scripts given in the /universal-starter seed project. We have covered the case, where the HTML pages are created ahead-of-time (AOT) and the npm http-server is used to serve the pages.

Dynamic vs Static

In Appendix A, you will see, what needs to be changed in the case we would like to use a (dynamic) NodeJS server instead of a (static) npm http-server. The NodeJS server case has the advantage that the user will never get out-of-date content from the server. I have not tested the performance yet, but I expect the latency of the dynamic case to exceed the one we can expect from the AOT compiled case.

In Appendix B, we can see how also NginX can be used to serve the pages that have been created ahead-of-time (AOT) with similar results.

This is no exact science, but in order to get a hint about the experienced latency, I have made a quick comparison of the loading times of the blog post page served by docker images running in detached mode on a CentOS VirtualBox VM as a Docker host on my local machine. The load times have been recorded on Chrome in debug mode (F12):

  • static with http-server: finish times in sec: 1.38, 1.37, 1.30, 1.42, 1.33, 1.33, 1.32, 1.42
  • static with NginX: finish times in sec: 1.85, 1.45, 1.22, 1.49, 1.56, 1.45, 1.70, 1.48
  • dynamic with NodeJS: finish times in sec: 2.75, 1.86, 2.35, 2.24, 2.28, 2.40, 2.65, 2.50

A look at the measured times reveals that the http-server is marginally quicker than the NginX server. The NodeJS server has a higher latency than the other both. This is expected, sind the NginX. The same trend holds, if the server Docker container are running on an AWS CentOS Docker host.

 

Appendix A: Dynamic NodeJS Server instead of static http-server

For the dynamic NodeJS server case, we need to change the following in the Dockerfile

### STAGE 1: Build ###

# We label our stage as 'builder'
FROM node:8-alpine as builder

COPY package*.json ./

RUN npm set progress=false && npm config set depth 0 && npm cache clean --force

## Storing node modules on a separate layer will prevent unnecessary npm installs at each build
RUN npm i && mkdir /ng-app && cp -R ./node_modules ./ng-app

WORKDIR /ng-app

COPY . .

## Build the angular app in production mode and store the artifacts in dist folder
#RUN npm run build:static
RUN npm run build:dynamic


### STAGE 2: Setup ###

FROM node:8-alpine

## Install http-server
#RUN npm install http-server -g

## From 'builder' stage copy over the artifacts in dist folder to default nginx public folder
COPY --from=builder /ng-app/dist /dist

#WORKDIR /dist/browser
#CMD ["http-server"]
CMD ["node", "/dist/server"]

When we run the service, we will notice that the server is accessing the WordPress API any time we click on the blog post link. In the static case, this is done only once at compile time (or more accurate: transpile time).

Appendix B: Static NginX Server instead of static http-server

For the (static) NginX server case, we need to change the following in the Dockerfile:

### STAGE 1: Build ###

# We label our stage as 'builder'
FROM node:8-alpine as builder

COPY package*.json ./

RUN npm set progress=false && npm config set depth 0 && npm cache clean --force

## Storing node modules on a separate layer will prevent unnecessary npm installs at each build
RUN npm i && mkdir /ng-app && cp -R ./node_modules ./ng-app

WORKDIR /ng-app

COPY . .

## Build the angular app in production mode and store the artifacts in dist folder
#RUN npm run build:static
RUN npm run build:dynamic


### STAGE 2: Setup ###

#FROM node:8-alpine
FROM nginx:1.13.3-alpine
## Install http-server
#RUN npm install http-server -g

## Copy our default nginx config
COPY nginx/default.conf /etc/nginx/conf.d/

## Remove default nginx website
RUN rm -rf /usr/share/nginx/html/*

## From 'builder' stage copy over the artifacts in dist folder to default nginx public folder
#COPY --from=builder /ng-app/dist /dist
COPY --from=builder /ng-app/dist/browser /usr/share/nginx/html

#WORKDIR /dist/browser
#CMD ["http-server"]
#CMD ["node", "/dist/server"]
CMD ["nginx", "-g", "daemon off;"]

As with the http-server, the WordPress API is contacted only once by the HTTP Server at compile time (or more accurate: transpile time). After the client (browser) has taken over, the client will contact the WordPress API, though.

 

2

Angular Universal CLI – Step-by-Step Example with REST Client


This time we will learn how to create a small Angular Universal CLI project that is using the WordPress REST service to retrieve and display the title and content of a WordPress blog post.

Angular Universal CLI combines the Universal features like server-side rendering (see Angular 4 Universal: Boosting Performance through Server Side Rendering) with a state-of-the-art handling of Angular projects by Angular CLI.

On a previous blog post, I have given an introduction to server-side rendering via Angular Universal. There, we had cloned a Universal seed file and added a REST client that has retrieved and displayed the content of a WordPress blog post. Later, I have found out that I have used a seed project that does not support Angular CLI. Angular CLI is the more modern way of handling Angular projects. In this blog post, we learn how to port (or create) the end-to-end tests and feature code to a seed project that has been created with Angular Universal CLI. With that, we have access to all ng commands provided by Angular CLI.

Step 0: Get Access to a Docker Host

The instructions will work on any Docker host with 2 GB available RAM. If you do not have access to a Docker host yet, I recommend following the step 0 instructions on my JHipster post.

Step 1: Create Aliases for often used Commands

In this tutorial, we will use following pre-defined aliases and functions for often used commands:

# functions
cli() {
 docker run -it --rm -w /app -v $(pwd):/app --net=host oveits/angular-cli:1.4.3 $@
}
npm() {
 cli npm $@
  if [[ "$@" == "i" ]] || [[ "$@" == "install" ]] ; then
    sudo chown -R $(whoami) .
  fi
}

# aliases
alias ng='cli ng $@'
alias protractor='docker run -it --privileged --rm --net=host -v /dev/shm:/dev/shm -v $(pwd):/protractor webnicer/protractor-headless $@'
alias own='sudo chown -R $(whoami) .'

Step 2: Fork & Clone Universal Starter from GIT

The Github project /universal-starter is a seed project that has been created with Angular CLI. Fork the project and clone it to your local machine (use your own Github name instead of mine, oveits):

$ git clone https://github.com/oveits/universal-starter
$ cd universal-starter

Step 3: Create e2e Test

The universal starter comes with no e2e tests. Let us change that now.

Step 3.1: Add e2e Folder from your existing project

If you have already created specs in an existing project, then copy them into a new e2e folder our project

cd project folder
mkdir e2e
cp <whereever you have your existing specs> e2e/

In my case, I intend to add the functionality I have developed on my blog post Behavior-Driven Angular – Part 2: Inserting REST Data as “innerHTML” into a Web Application. This is, where I have copied the content of the e2e folder from and changed it a little. Namely:

//e2e/app.e2e-spec.ts
import { AppPage } from './app.po';
import { browser, by, element } from 'protractor';

describe('Blog', () => {

  beforeEach(() => {
    browser.get('/2017/06/24/consuming-a-restful-web-service-with-angular-4/');
  });

  const blog_title = element(by.id('blog_title'));
  const blog_content = element(by.id('blog_content'));

  it('should display the blog title as header 1 and id="blog_title"', () => {
    expect(blog_title.getText()).toEqual('Angular 4 Hello World Quickstart');
  });

  it('should display the blog content', () => {
    expect(blog_content.getInnerHtml()).toMatch(/^<p>In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application./);
  });

});

With this end-to-end test, we expect that the page on /2017/06/13/angular-4-hello-world-with-quickstart/ is displaying the title and content of my WordPress blog Angular 4 Hello World Quickstart.

Step 3.2 Run the end-to-end Tests

Let us try to run e2e tests by issuing following command on the root of the project:

$ protractor

However, we are missing some ingredients. Therefore we will get following response:

**you must either specify a configuration file or at least 3 options. See below for the options:

Usage: protractor [configFile] [options]
configFile defaults to protractor.conf.js
The [options] object will override values from the config file.
See the reference config for a full list of options.

Options:
  --help                                 Print Protractor help menu
  --version                              Print Protractor version
  --browser, --capabilities.browserName  Browsername, e.g. chrome or firefox
  --seleniumAddress                      A running selenium address to use
  --seleniumSessionId                    Attaching an existing session id
  --seleniumServerJar                    Location of the standalone selenium jar file
  --seleniumPort                         Optional port for the selenium standalone server
  --baseUrl                              URL to prepend to all relative paths
  --rootElement                          Element housing ng-app, if not html or body
  --specs                                Comma-separated list of files to test
  --exclude                              Comma-separated list of files to exclude
  --verbose                              Print full spec names
  --stackTrace                           Print stack trace on error
  --params                               Param object to be passed to the tests
  --framework                            Test framework to use: jasmine, mocha, or custom
  --resultJsonOutputFile                 Path to save JSON test result
  --troubleshoot                         Turn on troubleshooting output
  --elementExplorer                      Interactively test Protractor commands
  --debuggerServerPort                   Start a debugger server at specified port instead of repl

Step 3.2.1 Add protractor.conf.js File

We need to add a protractor.conf.js file:

// protractor.conf.js
// Protractor configuration file, see link for more information
// https://github.com/angular/protractor/blob/master/lib/config.ts

const { SpecReporter } = require('jasmine-spec-reporter');

exports.config = {
  allScriptsTimeout: 11000,
  specs: [
    './e2e/**/*.e2e-spec.ts'
  ],
  capabilities: {
    'browserName': 'chrome'
  },
  directConnect: true,
  baseUrl: 'http://localhost:8080/',
  useAllAngular2AppRoots: true,
  framework: 'jasmine',
  jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000,
    print: function() {}
  },
  onPrepare() {
    require('ts-node').register({
      project: 'e2e/tsconfig.e2e.json'
    });
    jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
  }
};

This is the default protractor configuration file that comes with Angular CLI (you will find the file in the base folder of a new Angular CLI project created with ‘ng new-project’). However, we have adapted the parts in blue:

  1. Our application is running on port 8080, so we have changed the default port 4200 to 8080
  2. We have added the useAllAngular2AppRoots: true directive, which should fix an issue I had found on a previous blog Behavior-Driven Angular – part 1: Consuming a RESTful Web Service with Angular 4

Now again:

$ protractor
[20:21:07] E/configParser - Error code: 105
[20:21:07] E/configParser - Error message: failed loading configuration file ./protractor.conf.js
[20:21:07] E/configParser - Error: Cannot find module 'jasmine-spec-reporter'
 at Function.Module._resolveFilename (module.js:469:15)
 at Function.Module._load (module.js:417:25)
 at Module.require (module.js:497:17)
 at require (internal/module.js:20:19)
 at Object.<anonymous> (/protractor/protractor.conf.js:4:26)
 at Module._compile (module.js:570:32)
 at Object.Module._extensions..js (module.js:579:10)
 at Module.load (module.js:487:32)
 at tryModuleLoad (module.js:446:12)
 at Function.Module._load (module.js:438:3)

We need to install jasmine reporter package.

Step 3.2.2 Adapt the package.json File

Instead of a lot of trial&error, which package might be missing, I have decided to copy all missing packages found in in the devDependencies section of a new Angular CLI project into my package.json. This has lead to following additions in blue:

{
  "name": "ng-universal-demo",
  "version": "0.0.0",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/angular/universal-starter.git"
  },
  "contributors": [
    "AngularClass <hello@angularclass.com>",
    "PatrickJS <patrick@angularclass.com>",
    "Jeff Whelpley <jeff@gethuman.com>",
    "Jeff Cross <crossj@google.com>",
    "Mark Pieszak <mpieszak84@gmail.com>",
    "Jason Jean <jasonjean1993@gmail.com>",
    "Fabian Wiles <fabian.wiles@gmail.com>"
  ],
  "scripts": {
    "ng": "ng",
    "start": "ng serve",
    "start:dynamic": "npm run build:dynamic && npm run serve:dynamic",
    "start:static": "npm run build:static && npm run serve:static",
    "build": "ng build",
    "build:client-and-server-bundles": "ng build --prod && ng build --prod --app 1 --output-hashing=false",
    "build:static": "npm run build:client-and-server-bundles && npm run webpack:server && npm run generate:static",
    "build:dynamic": "npm run build:client-and-server-bundles && npm run webpack:server",
    "generate:static": "cd dist && node prerender",
    "webpack:server": "webpack --config webpack.server.config.js --progress --colors",
    "serve:static": "cd dist/browser && http-server",
    "serve:dynamic": "node dist/server"
  },
  "private": true,
  "dependencies": {
    "@angular/animations": "^4.2.4",
    "@angular/common": "^4.2.4",
    "@angular/compiler": "^4.2.4",
    "@angular/core": "^4.2.4",
    "@angular/forms": "^4.2.4",
    "@angular/http": "^4.2.4",
    "@angular/platform-browser": "^4.2.4",
    "@angular/platform-browser-dynamic": "^4.2.4",
    "@angular/platform-server": "^4.3.6",
    "@angular/router": "^4.2.4",
    "@nguniversal/express-engine": "^1.0.0-beta.3",
    "@nguniversal/module-map-ngfactory-loader": "^1.0.0-beta.3",
    "core-js": "^2.4.1",
    "rxjs": "^5.4.2",
    "zone.js": "^0.8.14"
  },
  "devDependencies": {
    "@angular/cli": "^1.3.0",
    "@angular/compiler-cli": "^4.2.4",
    "@angular/language-service": "^4.2.4",
    "@types/jasmine": "~2.5.53",
    "@types/jasminewd2": "~2.0.2",
    "@types/node": "^8.0.30",
    "codelyzer": "~3.1.1",
    "jasmine-core": "~2.6.2",
    "jasmine-spec-reporter": "~4.1.0",
    "karma": "~1.7.0",
    "karma-chrome-launcher": "~2.1.1",
    "karma-cli": "~1.0.1",
    "karma-coverage-istanbul-reporter": "^1.2.1",
    "karma-jasmine": "~1.1.0",
    "karma-jasmine-html-reporter": "^0.2.2",
    "protractor": "~5.1.2",
    "ts-node": "~3.2.0",
    "tslint": "~5.3.2",
    "cpy-cli": "^1.0.1",
    "http-server": "^0.10.0",
    "reflect-metadata": "^0.1.10",
    "ts-loader": "^2.3.7",
    "typescript": "~2.3.3"
  }
}

Step 3.2.3 Install the new Modules

We need to re-run the installation:

npm i

Step 3.2.4 Re-run the end-to-end Tests

We re-run the protractor test. We still get a bunch of error messages:

$ protractor
[17:39:28] I/direct - Using ChromeDriver directly...
[17:39:28] I/launcher - Running 1 instances of WebDriver
Jasmine started
[17:39:42] E/protractor - Could not find Angular on page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/ : retries looking for angular exceeded

  Blog
    ✗ should display the blog title as header 1 and id="blog_title"
      - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
          at /usr/local/lib/node_modules/protractor/built/browser.js:506:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)
      From: Task: Run beforeEach in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:7:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:5:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)
      - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"
          at /usr/local/lib/node_modules/protractor/built/browser.js:272:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)Error
          at ElementArrayFinder.applyAction_ (/usr/local/lib/node_modules/protractor/built/element.js:461:27)
          at ElementArrayFinder._this.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:103:30)
          at ElementFinder.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:829:22)
          at Object. (/protractor/e2e/app.e2e-spec.ts:15:34)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2820:25)
      From: Task: Run it("should display the blog title as header 1 and id="blog_title"") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:103:16
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:14:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:5:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)
[17:39:52] E/protractor - Could not find Angular on page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/ : retries looking for angular exceeded
    ✗ should display the blog content
      - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
          at /usr/local/lib/node_modules/protractor/built/browser.js:506:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)
      From: Task: Run beforeEach in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:7:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:5:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)
      - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"
          at /usr/local/lib/node_modules/protractor/built/browser.js:272:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)Error
          at ElementArrayFinder.applyAction_ (/usr/local/lib/node_modules/protractor/built/element.js:461:27)
          at ElementArrayFinder._this.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:103:30)
          at ElementFinder.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:829:22)
          at Object. (/protractor/e2e/app.e2e-spec.ts:20:25)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2820:25)
      From: Task: Run it("should display the blog content") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:103:16
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:19:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:5:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)

**************************************************
*                    Failures                    *
**************************************************

1) Blog should display the blog title as header 1 and id="blog_title"
  - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
  - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"

2) Blog should display the blog content
  - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
  - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"

Executed 2 of 2 specs (2 FAILED) in 21 secs.
[17:39:52] I/launcher - 0 instance(s) of WebDriver still running
[17:39:52] I/launcher - chrome #01 failed 2 test(s)
[17:39:52] I/launcher - overall: 2 failed spec(s)
[17:39:52] E/launcher - Process exited with error code 1

What is it telling us? Okay, I have forgotten to start the application, before we started the test. Let us correct this now.

Step 3.2.5: Run the Application

Let us run the application as a static universal project as described in this Readme:

$ cd my-project-root
$ npm run start:static
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prestart:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~start:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 start:static /app
> npm run build:static && npm run serve:static

npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prebuild:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~build:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 build:static /app
> npm run build:client-and-server-bundles && npm run webpack:server && npm run generate:static

npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prebuild:client-and-server-bundles: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~build:client-and-server-bundles: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 build:client-and-server-bundles /app
> ng build --prod && ng build --prod --app 1 --output-hashing=false

Date: 2017-10-28T19:27:48.583Z
Hash: c5eeae31051a6225ca92
Time: 15759ms
chunk {0} 0.7ce44253311853d97e73.chunk.js () 1.02 kB {2}  [rendered]
chunk {1} polyfills.80bfeb690703af4fafee.bundle.js (polyfills) 66.1 kB {5} [initial] [rendered]
chunk {2} main.2f46e8d1609d5ba758f8.bundle.js (main) 5.04 kB {4} [initial] [rendered]
chunk {3} styles.d41d8cd98f00b204e980.bundle.css (styles) 0 bytes {5} [initial] [rendered]
chunk {4} vendor.74a477af39cd1230db04.bundle.js (vendor) 305 kB [initial] [rendered]
chunk {5} inline.baceea59185918784cfc.bundle.js (inline) 1.47 kB [entry] [rendered]
Date: 2017-10-28T19:27:58.456Z
Hash: bdee05e0c4e2c172ab79
Time: 5226ms
chunk {0} main.bundle.js (main) 13.6 kB [entry] [rendered]
chunk {1} styles.bundle.css (styles) 0 bytes [entry] [rendered]
npm info lifecycle ng-universal-demo@0.0.0~postbuild:client-and-server-bundles: ng-universal-demo@0.0.0
npm info ok
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prewebpack:server: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~webpack:server: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 webpack:server /app
> webpack --config webpack.server.config.js --progress --colors

 10% building modules 0/2 modules 2 active .../ts-loader/index.js!/app/preHash: 7d5e698066b1d16e7805                                                         Version: webpack 3.7.1
Time: 9365ms
       Asset     Size  Chunks                    Chunk Names
   server.js  4.06 MB       0  [emitted]  [big]  server
prerender.js  3.36 MB       1  [emitted]  [big]  prerender
  [55] ./src lazy 160 bytes {0} {1} [built]
 [104] ./dist/server/main.bundle.js 13.6 kB {0} {1} [built]
 [176] ./server.ts 1.94 kB {0} [built]
 [228] ./src 160 bytes {0} [built]
 [234] (webpack)/buildin/module.js 517 bytes {0} [built]
 [251] ./prerender.ts 2.08 kB {1} [built]
 [253] ./static.paths.js 57 bytes {1} [built]
    + 247 hidden modules
npm info lifecycle ng-universal-demo@0.0.0~postwebpack:server: ng-universal-demo@0.0.0
npm info ok
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~pregenerate:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~generate:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 generate:static /app
> cd dist && node prerender

npm info lifecycle ng-universal-demo@0.0.0~postgenerate:static: ng-universal-demo@0.0.0
npm info ok
npm info lifecycle ng-universal-demo@0.0.0~postbuild:static: ng-universal-demo@0.0.0
npm info ok
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~preserve:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~serve:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 serve:static /app
> cd dist/browser && http-server

Starting up http-server, serving ./
Available on:
  http://127.0.0.1:8080
  http://172.31.21.180:8080
  http://172.17.0.1:8080

Step 3.2.6: Repeat the end-to-end Tests

If we then run protractor in the other terminal, we will see that the error messages have not changed:

$ protractor
...
1) Blog should display the blog title as header 1 and id="blog_title"
  - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
  - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"

2) Blog should display the blog content
  - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
  - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"
...

The error messages have not changed compared to the situation before. However, in the other terminal, we can see that the NodeJS server understands the requests and answers with a “404 Not found”, since we have not yet implemented the feature:

...
Available on:
  http://127.0.0.1:8080
  http://172.31.21.180:8080
  http://172.17.0.1:8080
[Sat Oct 28 2017 19:28:45 GMT+0000 (UTC)] "GET /blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36"
[Sat Oct 28 2017 19:28:45 GMT+0000 (UTC)] "GET /blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/" Error (404): "Not found"
[Sat Oct 28 2017 19:28:45 GMT+0000 (UTC)] "GET /favicon.ico" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36"
[Sat Oct 28 2017 19:28:55 GMT+0000 (UTC)] "GET /blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36"
[Sat Oct 28 2017 19:28:55 GMT+0000 (UTC)] "GET /blog/2017/06/24/consuming-a-restful-web-service-with-angular-4/" Error (404): "Not found"

Let us assume that the protractor errors will vanish, once we have implemented the service correctly.

Step 4: Implement the Feature

Step 4.1 Create Link

In src/app/app.component.ts, we add a link to the single blog post as defined in the spec:

On port 8080, we can see that the new link is visible:

However, if we klick the link, nothing happens. When pressing F12 and repeating the click, we get the error message:

ERROR Error: Uncaught (in promise): Error: Cannot match any routes. URL Segment: 'blog/2017/06/13/angular-4-hello-world-with-quickstart'
Error: Cannot match any routes. URL Segment: 'blog/2017/06/13/angular-4-hello-world-with-quickstart'

A route to the link is missing.

Step 4.2 Add Route

Let us add a route now.

If we restart the server, we get the message:

ERROR in Error: Could not resolve "./blog/blog.module" from "/app/src/app/app.module.ts".

This is expected since the file does not exist yet. Let us change that now:

$ cp -R src/app/lazy src/app/blog
$ mv src/app/blog/lazy.module.ts src/app/blog/blog.module.ts
$ sed -r -i "s/lazy/blog/g" src/app/blog/blog.module.ts
$ sed -r -i "s/i'm blog/i'm a blog/g" src/app/blog/blog.module.ts
$ sed -r -i "s/Lazy/Blog/g" src/app/blog/blog.module.ts

This will copy the lazy component to a blog component.

Now the error message is gone and we get the following output in a browser, if we click the link:

However, we the protractor messages do not change at all. We still get the error messages:

$ protractor
...
1) Blog should display the blog title as header 1 and id="blog_title"
  - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/13/angular-4-hello-world-with-quickstart. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
  - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"

2) Blog should display the blog content
  - Failed: Angular could not be found on the page http://localhost:8080/blog/2017/06/13/angular-4-hello-world-with-quickstart. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
  - Failed: Error while waiting for Protractor to sync with the page: "window.getAllAngularTestabilities is not a function"

Why is this the case? I would have expected the error message to tell us that the page is available with the content not being the one expected by the spec. However, if we run the end-to-end tests, we can observe that the link is still returning a “404 Not found” message:

[Sat Oct 28 2017 22:02:19 GMT+0000 (UTC)] "GET /blog/2017/06/13/angular-4-hello-world-with-quickstart" Error (404): "Not found"

This is the case, although the link seems to be reachable within the browser, if we click the link named “Angular 4 Hello World Quickstart:

In debug mode (F12) we see:

However, if we press the reload the page in the browser, we get an empty page:

After finishing the debug mode and reloading the page, we get a “404 Not found” message:

That is interesting: The link /blog/2017/06/13/angular-4-hello-world-with-quickstart is reached by client-side routing by clicking on the link, but a reload of the page fails. Let us fix that now.

Step 4.2.1 Fix the Error: 404 Not found

The error is caused by the fact that we are running the service in static mode, but we have not taken any measures that the path is created in the HTTP server yet.

Fixing the error is as simple as

  • adding the link to static.paths.js link
  • re-starting the server with npm run start:static

The static.paths.js file is located in the project’s root:

// static.paths.js
module.exports = [
  '/',
  '/lazy',
  '/lazy/nested',
  '/blog/2017/06/13/angular-4-hello-world-with-quickstart'
];

Now we create the missing path via:

npm run start:static

This is running the command npm run build:static, which is running the command cd dist && node prerender (among others). This will create the additional path in the directory tree (in blue):

$ yum install -y tree # if not installed already
...
$ tree dist/
dist/
├── browser
│   ├── 0.7ce44253311853d97e73.chunk.js
│   ├── 1.1fd3684dd14207827a93.chunk.js
│   ├── 3rdpartylicenses.txt
│   ├── blog
│   │   └── 2017
│   │       └── 06
│   │           └── 13
│   │               └── angular-4-hello-world-with-quickstart
│   │                   └── index.html
│   ├── favicon.ico
│   ├── index.html
│   ├── inline.a70b6ebe7ee886967e09.bundle.js
│   ├── lazy
│   │   ├── index.html
│   │   └── nested
│   │       └── index.html
│   ├── main.2d468591087d33c2e372.bundle.js
│   ├── polyfills.54dd1bb0dea7bab42697.bundle.js
│   ├── styles.d41d8cd98f00b204e980.bundle.css
│   └── vendor.b2c3f787d02157b98c0e.bundle.js
├── prerender.js
├── server
│   ├── favicon.ico
│   ├── main.bundle.js
│   └── styles.bundle.css
└── server.js

9 directories, 18 files

From now on, we can directly access the path /blog/2017/06/13/angular-4-hello-world-with-quickstart on the server. If we access the path, we see the following line in the server log:

[Sat Oct 28 2017 22:51:25 GMT+0000 (UTC)] "GET /blog/2017/06/13/angular-4-hello-world-with-quickstart" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36"

In the browser, we can directly access the path and we see that the server is serving the path. We not only can click the “Angular 4 Hello World Quickstart link” from the root path, but we also can reload the page. We can see in the browser’s debug window (press F12) that the corresponding path is known to the server:

Finally, as expected, the protractor output changes:

$ protractor
...
1) Blog should display the blog title as header 1 and id="blog_title"
  - Failed: No element found using locator: By(css selector, *[id="blog_title"])

2) Blog should display the blog content
  - Failed: No element found using locator: By(css selector, *[id="blog_content"])

This is expected: we have created a static page with some dummy content, but we have not yet downloaded and displayed the content of the WordPress blog post.

Now let us save the changes on GIT:

git add .
git commit -m'added blog module and added links to blog module on app and static.paths.js'

Step 4.3: Create HTML Template for the Blog

The specs are expecting an HTML template with a blog title in an element with the ID blog_title and a blog content in an element with the ID blog_content. Let us create that now. We create a new file:

To connect the new HTML template with the rest, we need to add a templateURL reference the blog.module.ts file:

// src/app/blog/blog.module.ts
import {NgModule, Component} from '@angular/core'
import {RouterModule} from '@angular/router'

@Component({
  selector: 'blog-view',
  templateUrl: './blog.component.html'
})
export class BlogComponent {}

@NgModule({
  declarations: [BlogComponent],
  imports: [
    RouterModule.forChild([
      { path: '', component: BlogComponent, pathMatch: 'full'}
    ])
  ]
})
export class BlogModule {

}

After a restart of the server, the protractor output changes:

$ protractor
...
1) Blog should display the blog title as header 1 and id="blog_title"
  - Expected 'This is the blog title' to equal 'Angular 4 Hello World Quickstart'.

2) Blog should display the blog content
  - Expected 'This is the blog content' to contain 'In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application.'.
  - Expected 'This is the blog content' to match /^<p>In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application.'.

Now let us save the result:

git add .
git commit -m'added blog HTML template'

Step 4.4: Create Variables

We change the HTML template, so it used variables we will create thereafter:

The variables are not yet defined. Therefore we would get following error message, if we reload the server:

ERROR in /app/src/$$_gendir/app/blog/blog.module.ngfactory.ts (34,31): Property 'title' does not exist on type 'BlogComponent'.
ERROR in ng:///app/src/app/blog/blog.component.html (3,1): Property 'content' does not exist on type 'BlogComponent'.

Let us define the variables:

Step 4.5: Read Variables from the WordPress API

Now we read in the HTTP content into the variables using Observables.

// src/app/blog/blog.module.ts
import {NgModule, Component, OnInit} from '@angular/core'
import {RouterModule} from '@angular/router'
import { Http, Response, Headers } from '@angular/http';
import 'rxjs/add/operator/map'
import { Observable } from 'rxjs/Observable';

@Component({
  selector: 'blog-view',
  templateUrl: './blog.component.html'
})

export class BlogComponent implements OnInit {

  title : String = "Loading..."
  content : String = "Loading..."

  constructor(private _http: Http) {}

  ngOnInit() {
     this._http.get('https://public-api.wordpress.com/rest/v1.1/sites/oliverveits.wordpress.com/posts/3078')
                .map((res: Response) => res.json())
                 .subscribe(data => {
                        this.title = data.title;
                        this.content = data.content;
                        console.log(data);
                });
  }

}

import { HttpModule } from '@angular/http';
@NgModule({
  declarations: [BlogComponent],
  imports: [
    HttpModule,
    RouterModule.forChild([
      { path: '', component: BlogComponent, pathMatch: 'full'}
    ])
  ]
})
export class BlogModule {

}

There, we have defined a private variable for the HTTP service. This service is used to create an observable with the GET function, which we subscribe to read the title and content of a single post into the variable. Moreover, we have defined the provider for HttpModule in the BlogModule part. See this blog post for a quick introduction to this concept. More in-depth step-by-step instructions including end-to-end tests can be found in part 1 and part 2 of the Behavior-driven Angular series.

After restarting the server we already can see the title and content of the blog post retrieved via WordPress API:

npm run start:static

Let us test the result with ‘protractor’:

$ protractor
[17:44:30] I/direct - Using ChromeDriver directly...
[17:44:30] I/launcher - Running 1 instances of WebDriver
Jasmine started

  Blog
    ✓ should display the blog title as header 1 and id="blog_title"
    ✓ should display the blog content

Executed 2 of 2 specs SUCCESS in 1 sec.
[17:44:35] I/launcher - 0 instance(s) of WebDriver still running
[17:44:35] I/launcher - chrome #01 passed

This was successful, this time!

Excellent! Thump up!

Step 5: Verify Server-Side Rendering

The reason why we have chosen Angular Universal is its server-side rendering feature. In situations with low bandwidth to the Internet, server-side rendering helps us to provide the user with the content of the page with a much lower latency.

To be sure that server-side rendering works as expected, we review the HTML source manually:

Unfortunately, server-side rendering does not seem to work the way expected. We still see the “Loading…” directive instead of the innerHTML content. Re-starting the server will reveal some errors in the log:

$ npm run start:static
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prestart:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~start:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 start:static /app
> npm run build:static && npm run serve:static

npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prebuild:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~build:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 build:static /app
> npm run build:client-and-server-bundles && npm run webpack:server && npm run generate:static

npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prebuild:client-and-server-bundles: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~build:client-and-server-bundles: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 build:client-and-server-bundles /app
> ng build --prod && ng build --prod --app 1 --output-hashing=false

Date: 2017-10-29T17:50:51.072Z
Hash: 36c4b63a2e9bf0ceb4c9
Time: 14508ms
chunk {0} 0.7ce44253311853d97e73.chunk.js () 1.02 kB {1} {3}  [rendered]
chunk {1} 1.e11ff4adfdb6d0ed7929.chunk.js () 21.1 kB {0} {3}  [rendered]
chunk {2} polyfills.54dd1bb0dea7bab42697.bundle.js (polyfills) 66.1 kB {6} [initial] [rendered]
chunk {3} main.2d468591087d33c2e372.bundle.js (main) 5.76 kB {5} [initial] [rendered]
chunk {4} styles.d41d8cd98f00b204e980.bundle.css (styles) 0 bytes {6} [initial] [rendered]
chunk {5} vendor.9a16430f75eb51537ae8.bundle.js (vendor) 305 kB [initial] [rendered]
chunk {6} inline.1735ca01d11efd0014a9.bundle.js (inline) 1.5 kB [entry] [rendered]
Date: 2017-10-29T17:50:59.897Z
Hash: cd969720fe342e3bf65d
Time: 5221ms
chunk {0} main.bundle.js (main) 17.1 kB [entry] [rendered]
chunk {1} styles.bundle.css (styles) 0 bytes [entry] [rendered]
npm info lifecycle ng-universal-demo@0.0.0~postbuild:client-and-server-bundles: ng-universal-demo@0.0.0
npm info ok
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~prewebpack:server: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~webpack:server: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 webpack:server /app
> webpack --config webpack.server.config.js --progress --colors

 10% building modules 0/2 modules 2 active .../ts-loader/index.js!/app/preHash: 46749ef5d0e4cdd33844                                                         Version: webpack 3.7.1
Time: 9306ms
       Asset     Size  Chunks                    Chunk Names
   server.js  4.07 MB       0  [emitted]  [big]  server
prerender.js  3.36 MB       1  [emitted]  [big]  prerender
  [56] ./src lazy 160 bytes {0} {1} [built]
 [104] ./dist/server/main.bundle.js 17.1 kB {0} {1} [built]
 [177] ./server.ts 1.94 kB {0} [built]
 [229] ./src 160 bytes {0} [built]
 [235] (webpack)/buildin/module.js 517 bytes {0} [built]
 [252] ./prerender.ts 2.08 kB {1} [built]
 [254] ./static.paths.js 117 bytes {1} [built]
    + 248 hidden modules
npm info lifecycle ng-universal-demo@0.0.0~postwebpack:server: ng-universal-demo@0.0.0
npm info ok
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~pregenerate:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~generate:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 generate:static /app
> cd dist && node prerender

ERROR Error: not implemented
    at Parse5DomAdapter.getCookie (/app/dist/prerender.js:37285:68)
    at CookieXSRFStrategy.configureRequest (/app/dist/prerender.js:41698:119)
    at XHRBackend.createConnection (/app/dist/prerender.js:41747:28)
    at httpRequest (/app/dist/prerender.js:42155:20)
    at Http.request (/app/dist/prerender.js:42265:34)
    at Http.get (/app/dist/prerender.js:42279:21)
    at e.X+Mx.e.ngOnInit (/app/dist/prerender.js:78900:8159)
    at checkAndUpdateDirectiveInline (/app/dist/prerender.js:11698:19)
    at checkAndUpdateNodeInline (/app/dist/prerender.js:13196:20)
    at checkAndUpdateNode (/app/dist/prerender.js:13139:16)
npm info lifecycle ng-universal-demo@0.0.0~postgenerate:static: ng-universal-demo@0.0.0
npm info ok
npm info lifecycle ng-universal-demo@0.0.0~postbuild:static: ng-universal-demo@0.0.0
npm info ok
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info lifecycle ng-universal-demo@0.0.0~preserve:static: ng-universal-demo@0.0.0
npm info lifecycle ng-universal-demo@0.0.0~serve:static: ng-universal-demo@0.0.0

> ng-universal-demo@0.0.0 serve:static /app
> cd dist/browser && http-server

Starting up http-server, serving ./
Available on:
  http://127.0.0.1:8080
  http://172.31.21.180:8080
  http://172.17.0.1:8080
Hit CTRL-C to stop the server

The dynamic server __npm run start:dynamic__ has the same problems. The only difference is that the error message appears every time the link is clicked.

After a lot of googling (in vain) and testing, I finally came up with a workaround: if we move out the HttpModule import from the BlogModule to the AppModule, the client-side-rendering as well as the server-side-rendering work fine:

// src/app/app.module.ts
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { RouterModule } from '@angular/router';

import { AppComponent } from './app.component';
import { HomeComponent } from './home/home.component';

import { HttpModule } from '@angular/http';

@NgModule({
  declarations: [
    AppComponent,
    HomeComponent,
  ],
  imports: [
    HttpModule,
    BrowserModule.withServerTransition({appId: 'my-app'}),
    RouterModule.forRoot([
      { path: '', component: HomeComponent, pathMatch: 'full'},
      { path: 'lazy', loadChildren: './lazy/lazy.module#LazyModule'},
      { path: 'lazy/nested', loadChildren: './lazy/lazy.module#LazyModule'},
      { path: 'blog/2017/06/13/angular-4-hello-world-with-quickstart', loadChildren: './blog/blog.module#BlogModule'}
    ])
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

The corresponding lines need to be removed in the BlogModule. Otherwise, the error messaged do not disappear (I would have preferred to them for portability of the blog component, since I do not want the blog component to depend on imports of an upstream component; however, I am forced to remove it anyway, it seems):

// src/app/blog/blog.module.ts
import {NgModule, Component, OnInit} from '@angular/core'
import {RouterModule} from '@angular/router'
import { Http, Response, Headers } from '@angular/http';
import 'rxjs/add/operator/map'
import { Observable } from 'rxjs/Observable';

@Component({
  selector: 'blog-view',
  templateUrl: './blog.component.html'
})

export class BlogComponent implements OnInit {

  title : String = "Loading..."
  content : String = "Loading..."

  constructor(private _http: Http) {}

  ngOnInit() {
     this._http.get('https://public-api.wordpress.com/rest/v1.1/sites/oliverveits.wordpress.com/posts/3078')
                .map((res: Response) => res.json())
                 .subscribe(data => {
                        this.title = data.title;
                        this.content = data.content;
                        console.log(data);
                });
  }

}

//import { HttpModule }    from '@angular/http'; // moved to src/app/app.module.ts

@NgModule({
  declarations: [BlogComponent],
  imports: [
    //HttpModule, // moved to src/app/app.module.ts
    RouterModule.forChild([
      { path: '', component: BlogComponent, pathMatch: 'full'}
    ])
  ]
})
export class BlogModule {
}

Now, after restarting the server, the blog post page is visible in both ways:

client-side rendering:

For testing client-side rendering, we open the root URL localhost:8080/

server-side rendering:

For testing the server-side rendering, we need to access the URL directly, e.g. by reloading the page we see above (e.g. press F5) or by cutting&pasting the full URL into the browser.

We see the following:

–> the page displays the full content after less than a second

–> the page switches over to client-side rendering and the “Loading…” appears

–> once the page has been retrieved from the WordPress API, the full content is visible again

As the last test before re-running the end-to-end tests, we can see that the blog title and content can be seen in the HTML source:

This is both, SEO-friendly and the content will show up much quicker on mobile devices with a low-bandwidth Internet connection compared to the client-side rendering case.

Excellent! Thump up!

Note that the protractor tests are still successful:

$ protractor
[17:44:30] I/direct - Using ChromeDriver directly...
[17:44:30] I/launcher - Running 1 instances of WebDriver
Jasmine started

  Blog
    ✓ should display the blog title as header 1 and id="blog_title"
    ✓ should display the blog content

Executed 2 of 2 specs SUCCESS in 1 sec.
[17:44:35] I/launcher - 0 instance(s) of WebDriver still running
[17:44:35] I/launcher - chrome #01 passed

This was successful again.

Summary

We have successfully created an Angular Universal application based on Angular CLI with a REST client feeding in a single blog post from the WordPress API. We had experienced some trouble with server-side rendering, which was resolved miraculously by moving the HttpModule import from the BlogModule (the module using HTTP) to the root AppModule. At the end, we have succeeded to create an application that loads the page from the server and hands over the control to the browser thereafter.

ToDo:

  • dynamic path: I would like that all paths /blog/xxx are available and contain the title and content of the corresponding WordPress link https://oliverveits.wordpress.com/xxx/
  • Refactoring:
    • Move the REST client into a separate ‘@Injectable’ service.
    • separate the blog module from the blog component (small effort, but low priority)
  • In order to test server-side rendering, I had to restart the server every time the code has changed. I need to find a better way to handle this in future: e.g. use continuous testing for development with client-side rendering, and let the continuous integration machinery (e.g. TravisCI or CircleCI) perform the full tests in productive mode after each GIT push.
  • In future, I have to find out, how to write tests that will fail if the server-side rendering does not work. This time, I had to manually review, whether the HTML source contains the content.
0

Behavior-Driven Angular – Part 2: Inserting REST Data as “innerHTML” into a Web Application


Today, we will extend the behavior-driven development example of the previous blog post and add the blog content to the document. Like last time, we will retrieve the HTML content from the WordPress API. Sounds easy, right? We will see that the challenge is to display the HTML content correctly, so we do not see escaped HTML like “<p>…” on the page.

As in part 1, we will follow a “test first” strategy: we will create the e2e test specification before we implement the actual code.

Within the Protractor/Jasmine framework, we will learn how to match the text and the inner HTML of browser DOM elements with functions like expect(...).toEqual("..."), .toContain("...") and .toMatch(/regex/) functions. The latter gives us the full flexibility of regular expressions.

Check out this book on Amazon: Angular Test-Driven Development

Plan for Today

Today, we plan to complement the blog title we have shown last time with the blog content, similar to the blog post Angular 4 Hello World Quickstart, which we will uses as our data mine. We will only show the title and the content as follows:

Before we start coding, we will add an e2e test that defines our expectation.

Step 0: Clone the GIT Repository and install the Application

This step can be skipped if you have followed part 1 of this series.

I am assuming that you have a Docker host available with 1.5GB or more RAM, GIT is installed on that host.

alias cli='docker run -it --rm -w /app -v $(pwd):/app -p 4200:4200 oveits/angular-cli:1.4.3 $@'
alias protractor='docker run -it --privileged --rm --net=host -v /dev/shm:/dev/shm -v $(pwd):/protractor webnicer/protractor-headless $@'
git clone https://github.com/oveits/consuming-a-restful-web-service-with-angular.git
cd consuming-a-restful-web-service-with-angular
git checkout -b 320ae88
cli npm i
chown -R $(whoami) .
cli ng serve --host 0.0.0.0

Phase 1: Create an e2e Test

Step 1.1: Create a GIT Feature Branch

As always with a new feature, let us create a feature branch (on a second terminal):

$ cd /vagrant/consuming-a-restful-web-service-with-angular/

$ protractor
[20:24:22] I/direct - Using ChromeDriver directly...
[20:24:22] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display the title

Executed 1 of 1 spec SUCCESS in 0.756 sec.
[20:24:27] I/launcher - 0 instance(s) of WebDriver still running
[20:24:27] I/launcher - chrome #01 passed

$ git checkout -b feature/0004-add-blog-content

You might need to adapt the path to your project. The protractor command is optional, but it will ensure that the e2e tests have worked on your machine before you start changing the code. I have seen some permissions topic described in the Appendices, which have made me cautious.

Step 1.2 (optional): Apply new Test Functions to the Blog Title

We would like to add a test that checks, whether the blog content is showing on the page. There are many Jasmine specification examples out there. Somehow, I have stumbled over this example. In order to verify that the functions I found there work fine, I thought it would be a good idea to write a new test similar to the ones in the example, but apply the test to the blog title before we write a new test for the blog content. This way, we can verify that we apply the correct syntax.

I have kept the original specification code, but I have added following code to the spec:

// e2e/app.e2e-spec.ts
import { browser, by, element } from 'protractor';
import { AppPage } from './app.po';

describe('consuming-a-restful-web-service-with-angular App', () => {
  let page: AppPage;

  beforeEach(() => {
    page = new AppPage();
  });

  it('should display the title', () => {
    page.navigateTo();
    expect(page.getParagraphText()).toContain('Angular 4 Hello World Quickstart');
  });
});

describe('Blog', () => {

  beforeEach(() => {
    browser.get('/');
  });

  it('should display the blog title as header 1 and id="blog_title"', () => {
    expect(element(by.css('h1')).getText()).toEqual('Angular 4 Hello World Quickstart');
  });
});

Both protractor e2e tests are successful without changing the code:

$ protractor
[20:59:51] I/direct - Using ChromeDriver directly...
[20:59:51] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display the title

  Blog
    ✓ should display the blog title as header 1 and id="blog_title"

Executed 2 of 2 specs SUCCESS in 2 secs.
[20:59:57] I/launcher - 0 instance(s) of WebDriver still running
[20:59:57] I/launcher - chrome #01 passed

Save it. Note: for pushing the changes to Github, you will need to fork my project and work with your fork. Otherwise, you can keep the git backups locally only.

git commit -am'1.2 added an addional test for the title looking for the first H1 header (successful test)'
git push

Step 1.3 (optional): Refine the Test

Step 1.3.1 Create a Test looking for a specific Element per ID

Since the blog content will not be a header, we will need to look for something, which is unique on the page. We use an ID for fetching the correct element from the page:

import { browser, by, element } from 'protractor';

...

describe('Blog', () => {

  beforeEach(() => {
    browser.get('/');
  });

  const blog_title = element(by.id('blog_title'));

  it('should display the blog title as header 1 and id="blog_title"', () => {
    expect(element(by.css('h1')).getText()).toEqual('Angular 4 Hello World Quickstart');
    expect(blog_title.getText()).toEqual('Angular 4 Hello World Quickstart');
  });
});

Now the protractor test will fail. This is because we have not set the ID on the HTML template yet:

$ protractor
[21:07:51] I/direct - Using ChromeDriver directly...
[21:07:51] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display the title

  Blog
    ✗ should display the blog title as header 1 and id="blog_title"
      - Failed: No element found using locator: By(css selector, *[id="blog_title"])
          at WebDriverError (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/error.js:27:5)
          at NoSuchElementError (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/error.js:242:5)
          at /usr/local/lib/node_modules/protractor/built/element.js:808:27
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)Error
          at ElementArrayFinder.applyAction_ (/usr/local/lib/node_modules/protractor/built/element.js:461:27)
          at ElementArrayFinder._this.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:103:30)
          at ElementFinder.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:829:22)
          at Object. (/protractor/e2e/app.e2e-spec.ts:28:23)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2820:25)
      From: Task: Run it("should display the blog title as header 1 and id="blog_title"") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:26:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:18:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)

**************************************************
*                    Failures                    *
**************************************************

1) Blog should display the blog title as header 1 and id="blog_title"
  - Failed: No element found using locator: By(css selector, *[id="blog_title"])

Executed 2 of 2 specs (1 FAILED) in 2 secs.
[21:07:58] I/launcher - 0 instance(s) of WebDriver still running
[21:07:58] I/launcher - chrome #01 failed 1 test(s)
[21:07:58] I/launcher - overall: 1 failed spec(s)
[21:07:58] E/launcher - Process exited with error code 1

To save the change:

git commit -am'1.3.1 search title by element id (failed e2e test)'

Step 1.3.2 Fix the Test

Let us fix the failed test like follows: In the HTML template src/app/app.component.html, we specify the element ID:

<h1 id="blog_title">{{title}}</h1>

Now the protractor test is successful again:

$ protractor
[21:14:27] I/direct - Using ChromeDriver directly...
[21:14:27] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display the title

  Blog
    ✓ should display the blog title as header 1 and id="blog_title"

Executed 2 of 2 specs SUCCESS in 2 secs.
[21:14:34] I/launcher - 0 instance(s) of WebDriver still running
[21:14:34] I/launcher - chrome #01 passed

That was simple. Now let us apply our learnings to the blog content.

To save the change:

git commit -am'1.3.2 add ID to HTML template (success)'; git push

Phase 2: Create the Test for the Blog Content

The content of the blog can be seen on WordPress:

The content starts with: In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application.

Let us search for that on our application.

Step 2.1 Add Blog Content e2e Tests

Similar to what we have done for the Blog Title, let us create an e2e test for the blog content. We add the parts in blue to e2e/app.e2e-spec.ts:

// e2e/app.e2e-spec.ts
...
describe('Blog', () => {

  beforeEach(() => {
    browser.get('/');
  });

  const blog_title = element(by.id('blog_title'));
  const blog_content = element(by.id('blog_content'));

  it('should display the blog title as header 1 and id="blog_title"', () => {
    expect(element(by.css('h1')).getText()).toEqual('Angular 4 Hello World Quickstart');
    expect(blog_title.getText()).toEqual('Angular 4 Hello World Quickstart');
  });

  it('should display the blog content', () => {
    expect(blog_content.getText()).toContain('In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application.');
  });
});

Since the content is quite large, we did not compare it with the equality operator, but we have used the ‘toContain’ function instead.

The new protractor test fails as expected:

$ protractor
[21:23:04] I/direct - Using ChromeDriver directly...
[21:23:04] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display the title

  Blog
    ✓ should display the blog title as header 1 and id="blog_title"
    ✗ should display the blog content
      - Failed: No element found using locator: By(css selector, *[id="blog_content"])
          at WebDriverError (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/error.js:27:5)
          at NoSuchElementError (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/error.js:242:5)
          at /usr/local/lib/node_modules/protractor/built/element.js:808:27
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)Error
          at ElementArrayFinder.applyAction_ (/usr/local/lib/node_modules/protractor/built/element.js:461:27)
          at ElementArrayFinder._this.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:103:30)
          at ElementFinder.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:829:22)
          at Object. (/protractor/e2e/app.e2e-spec.ts:33:25)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2820:25)
      From: Task: Run it("should display the blog content") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:32:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:18:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)

**************************************************
*                    Failures                    *
**************************************************

1) Blog should display the blog content
  - Failed: No element found using locator: By(css selector, *[id="blog_content"])

Executed 3 of 3 specs (1 FAILED) in 3 secs.
[21:23:12] I/launcher - 0 instance(s) of WebDriver still running
[21:23:12] I/launcher - chrome #01 failed 1 test(s)
[21:23:12] I/launcher - overall: 1 failed spec(s)
[21:23:12] E/launcher - Process exited with error code 1

To save the change:

git commit -am'2.1 add test for blog content (failed)'; git push

Step 2.2 Fix the Blog Content Test

Let us fix the test now.

Step 2.2.1 Add the Blog Content to the HTML Template

In order to display the blog content, we need to add the following to the HTML template src/app/app.component.html:

Step 2.2.2 Define the Variable ‘content’ in the Component

However, as long as the variable ‘content’ is not defined, we will have added an empty div. To define the variable, we must change the component src/app/app.component.ts

import { Component, OnInit } from '@angular/core';
import { Http } from '@angular/http';
import { Response } from '@angular/http';
import 'rxjs/add/operator/map'

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent implements OnInit {

  title : any = null
  content : any = null

  constructor(private _http: Http) {}

  ngOnInit() {
     this._http.get('https://public-api.wordpress.com/rest/v1.1/sites/oliverveits.wordpress.com/posts/3078')
                .map((res: Response) => res.json())
                 .subscribe(data => {
                        this.title = data.title;
                        this.content = data.content;
                        console.log(data);
                });
  }
}

That’s it: the e2e tests are successful:

$ protractor
[21:30:12] I/direct - Using ChromeDriver directly...
[21:30:12] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display the title

  Blog
    ✓ should display the blog title as header 1 and id="blog_title"
    ✓ should display the blog content

Executed 3 of 3 specs SUCCESS in 3 secs.
[21:30:19] I/launcher - 0 instance(s) of WebDriver still running
[21:30:19] I/launcher - chrome #01 passed

To save the change:

git commit -am'2.2.2 Added content to HTML template and component (success)'; git push

Step 2.3 Explore the Result

Now let us have a look at what we have accomplished and let us open the browser on http://localhost:4200:

The good news is: the content is there.

😉

The bad news it: it is not readable because the HTML code in the blog content variable has been HTML escaped.

😦

This is the standard behavior in Angular. So what can we do now? The solution to the problem can be found in Step 2.3 of my original post: we need to set the innerHTML of the div instead of adding the content as text. But, as we are performing a “behavior-driven” approach, let us try to write the tests first.

Step 2.4 Improve the e2e Test Spec

Let us add an additional line to the test specification in order to make sure, we will see the HTML in the correct format:

import { browser, by, element } from 'protractor';

describe('Blog', () => {

  beforeEach(() => {
    browser.get('/');
  });

  const blog_title = element(by.id('blog_title'));
  const blog_content = element(by.id('blog_content'));

  it('should display the blog title as header 1 and id="blog_title"', () => {
    expect(element(by.css('h1')).getText()).toEqual('Angular 4 Hello World Quickstart');
    expect(blog_title.getText()).toEqual('Angular 4 Hello World Quickstart');
  });

  it('should display the blog content', () => {
    expect(blog_content.getText()).toContain('In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application.');
    
  });
});

With that, we test, whether the innerHTML of the div element starts with the correct HTML code. For that, we have made use of two functionalities of Jasmine:

  1. reading the innerHTML of an element with the getInnerHtml() function
  2. matching against a regular expression with toMatch(/regexp/)

As expected, the protractor test fails with the message

To save the change:

git commit -am'2.4 added innerHTML test for content with regular expression (fail)'; git push

Step 2.5 Fulfill the improved e2e Test

We can see that the content is escaped (e.g.  instead of ). Let us fix that by specifying the innerHTML like follows:

As soon as the content is loaded, the innerHTML ‘Loading…’ will be replaced by the content retrieved from WordPress.

Let us run the test:

$ protractor
[20:55:50] I/direct - Using ChromeDriver directly...
[20:55:50] I/launcher - Running 1 instances of WebDriver
Jasmine started
[20:55:56] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  consuming-a-restful-web-service-with-angular App
    ✓ should display blog title

[20:55:57] W/element - more than one element found for locator By(css selector, h1) - the first result will be used
  Blog
    ✓ should display the blog title as header 1 and id="blog_title"
    ✓ should display the blog content

Executed 3 of 3 specs SUCCESS in 3 secs.
[20:55:57] I/launcher - 0 instance(s) of WebDriver still running
[20:55:57] I/launcher - chrome #01 passed

That was easy, again.

To save the change:

git commit -am'2.5 Fix the content innerHTML test (success)'; git push

Step 3: Explore the Final Result

Now let us head over to the browser on URL http://localhost:4200 again:

Even though there is no styling implemented yet, that looks much better now. This is, what we had in mind to implement today.

Excellent! Thump up!

 

As a wrap-up, the changes can be merged into the develop branch: the tests are successful and also the explorative “tests” have shown a correct result.


git checkout develop
git merge feature/0004-add-blog-content
git push

Summary

In this blog post, we have shown how to retrieve HTML-formated data from the WordPress API and display it in a correct format. In a “test-driven” approach, we have created Protractor e2e test specifications, before we have implemented the function.

Appendix: Error message: failed loading configuration file ./protractor.conf.js

After successfully cloning and installing the repo, I had seen following error message, when trying to perform the e2e tests:

$ protractor
[19:23:16] E/configParser - Error code: 105
[19:23:16] E/configParser - Error message: failed loading configuration file ./protractor.conf.js
[19:23:16] E/configParser - Error: Cannot find module 'jasmine-spec-reporter'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object. (/protractor/protractor.conf.js:4:26)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)

Resolution:

I have seen that the cli command was creating all files as user root. This was because I had defined

alias cli='docker run -it --rm -w /app -v $(pwd):/app oveits/angular-cli:1.4.3 $@'

After changing this to

alias cli='docker run -it --rm -w /app -v $(pwd):/app -p 4200:4200 -u $(id -u $(whoami)) oveits/angular-cli:1.4.3 $@'

and re-performing the cli npm i after the clone, the problem was resolved. However, this has caused the next ‘npm i’ issue described below, and it is better to perform following workaround:

Better:

  1. Keep the first version of the alias
  2. After applying ‘cli npm i’, perform the command sudo chown -R $(whoami) PROJECT_ROOT_DIR .

Appendix npm i: Error: EACCES: permission denied, mkdir ‘/.npm’

npm ERR! Linux 4.2.0-42-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "i"
npm ERR! node v6.11.2
npm ERR! npm  v3.10.10
npm ERR! path /.npm
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir

npm ERR! Error: EACCES: permission denied, mkdir '/.npm'
npm ERR!     at Error (native)
npm ERR!  { Error: EACCES: permission denied, mkdir '/.npm'
npm ERR!     at Error (native)
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/.npm',
npm ERR!   parent: 'consuming-a-restful-web-service-with-angular' }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

npm ERR! Please include the following file with any support request:
npm ERR!     /app/npm-debug.log

The reason is, that I had defined

alias cli='docker run -it --rm -w /app -v $(pwd):/app -p 4200:4200 -u $(id -u $(whoami)) oveits/angular-cli:1.4.3 $@'

With that, npm i is run as the vagrant user with ID=900. However, inside the container, neither the user “vagrant” nor the user ID 900 is defined. This seems to cause the problem that the cli npm i command wants to create a directory /$HOME/.npm, but $HOME is not set. Therefore, the user with id=900 wants to create the file /.npm, but only root is allowed to do so.

The better workaround is to define

alias cli='docker run -it --rm -w /app -v $(pwd):/app -p 4200:4200 oveits/angular-cli:1.4.3 $@'

without the -u option and perform a command

chown -R $(whoami) .

where needed (e.g. after each npm i command).

2

Angular 4 Docker Example – for Angular CLI projects


In this Hello-World-like tutorial, we will show how to run Angular 4 applications that have been created with Angular CLI in a Docker container. As an introduction, we will start running an existing, already dockerized example from Github. As a second step, we will create and dockerize our own Hello World app. We will verify our installation by testing the remote accessibility of the application on a CentOS Docker host on AWS.

Next time I plan to show how to tweak the Dockerfile, so it works also with Angular projects that have not been created with Angular CLI.

tl;dr

The fastest way I have found to dockerize an Angular application is to copy the Dockerfile and nginx/default.conf, into the project folder, remove the package-lock.json reference from Dockerfile, perform a docker build and a docker run. After that, the application running in a Docker container can be reached via the browser.

Tools and Versions used

  • CentOS 7 image on AWS (t2.micro with 1 CPU, 1 GB RAM, and a manually added 2GB additional swap, because 1GB RAM is not sufficient for the build job. See the appendix)
  • Docker version 17.09.0-ce, build afdb6d4
  • git is installed (if not: sudo yum install -y git)

Phase 1 (optional): Docker Installation via Github Example

In this chapter, we make use of an existing Docker example /angular4-docker-example  from Github. We will be able to create and run the example with following few commands:

Step 1.1: Clone Example Project from Github

We clone the project from Github:

$ git clone https://github.com/avatsaev/angular4-docker-example
$ cd angular4-docker-example

Step 1.2: Create Docker Image

Then we create the docker image:

$ docker build . --tag angular4-docker-example:v0.1

Step 1.3: Run the App

Now is the time to run the app:

$ docker run --rm --name angular4-docker-example -d -p 80:80 angular4-docker-example:v0.1

Step 1.4: Test it locally

We can test it locally:

$ curl localhost:4200
<!doctype html><html><head><meta charset="utf-8"><title>Myapp</title><base href="/"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" type="image/x-icon" href="favicon.ico"><link href="styles.d41d8cd98f00b204e980.bundle.css" rel="stylesheet"/></head><body><app-root>Loading...</app-root>http://inline.5a1918ffa059c81333bd.bundle.jshttp://polyfills.c551d6b1e1d6192ebe54.bundle.jshttp://main.a7b2dfb807be711d1780.bundle.js</body></html>

Step 1.5: Test it remotely

And the app works also when we access it remotely. For that, I have retrieved the public IPv4 DNS name from the EC2 AWS console

and copied it into the URL bar of the browser, appending the port 4200:

For this to work, I have chosen the security rules within AWS to allow accessing the port 4200 from my Home network:

That was easy. Thanks a lot to  

😉

Excellent! Thump up!

Phase 2: Create Angular 4 Hello World Project

Okay, creating the app from a great example by  was easy. As far as I have seen, he has used stages in Dockerfile, a new feature my old Docker host did not understand. However, the used version is working fine.

Step 2.1: Create Angular Hello World App

Let us create a hello world application using Angular CLI so we can perform all necessary changes needed to run Angular in a Docker container:

We are closely following the steps of Phase 1 of my blog post Behavior-Driven Development Example: Consuming a RESTful Web Service with Angular 4:

(dockerfile)$ alias cli='docker run -it --rm -w /app -v $(pwd):/app -p 4200:4200 -u $(id -u $(whoami)) oveits/angular-cli:1.4.3 $@'

Why this complicated user option -u $(id -u $(whoami))? The reason is that

  • if we omit it, then all new files will be created as root, so we will get permissions problems later on
  • If we use ‘centos’, then the container will complain that he does not find the user ‘centos’ in its passwd file
  • If we use the ID of centos, then it works. However, it might not work in all cases. This time, the ID of centos user is 1000, and by chance, a user (named ‘node’) exists on the container as well. But let us live with this uncertainty for now.
$ cli ng new angular-cli-hello-world-with-docker

Step 2.2: Install the Packages

$ cli npm i

Step 2.3: Run and Test the Service

The service can be started with an ng serve command:

cli ng serve --host 0.0.0.0

Step 2.4: Local Test

The local test is successful:

$ curl localhost:4200
<!doctype html><html lang="en"><head><meta charset="utf-8"><title>AngularCliHelloWorldWithDocker</title><base href="/"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" type="image/x-icon" href="favicon.ico"><link href="styles.d41d8cd98f00b204e980.bundle.css" rel="stylesheet"/></head><body><app-root></app-root>http://inline.c1122970c69d0e49cb86.bundle.jshttp://polyfills.d8d3d78a4deb2ab66856.bundle.jshttp://main.3ff2f2cccb753b0aef42.bundle.js</body></html>

Step 2.5: Remote Test

After trying to open the application in a remote browser, we see:

There might be possibilities to resolve this, but our aim is to run the application in a Docker container, not via ng serve. So let us ignore this issue for now since it might not be relevant for the application if it runs in a container.

Phase 3: Dockerize the Angular Hello World App

Step 3.1: Add Dockerfile

Add Dockerfile from the example project above, i.e. you can download it from here:

### STAGE 1: Build ###

# We label our stage as 'builder'
FROM node:8-alpine as builder

COPY package.json package-lock.json ./

RUN npm set progress=false && npm config set depth 0 && npm cache clean --force

## Storing node modules on a separate layer will prevent unnecessary npm installs at each build
RUN npm i && mkdir /ng-app && cp -R ./node_modules ./ng-app

WORKDIR /ng-app

COPY . .

## Build the angular app in production mode and store the artifacts in dist folder
RUN $(npm bin)/ng build --prod --build-optimizer


### STAGE 2: Setup ###

FROM nginx:1.13.3-alpine

## Copy our default nginx config
COPY nginx/default.conf /etc/nginx/conf.d/

## Remove default nginx website
RUN rm -rf /usr/share/nginx/html/*

## From 'builder' stage copy over the artifacts in dist folder to default nginx public folder
COPY --from=builder /ng-app/dist /usr/share/nginx/html

CMD ["nginx", "-g", "daemon off;"]

We can see that the Docker build is designed as a two-stage process; a Docker feature that requires a newer version of Docker than the one I have used for months now. With the version described above, it works, though.

Step 3.2: Adapt Dockerfile

We have to adapt the Dockerfile a little bit: package-lock.json does not exist in our project, so we need to remove it from the COPY command in stage 1. This is the only change, we need to perform.

Step 3.3: Add NginX Config

In Stage 2, we can see that the default config is copied from an NginX directory, which does not exist in our case. Let us change that now and add the default config file:

$ mkdir nginx
$ vi nginx/default.conf

The content of nginx/default.conf is:

server {

  listen 80;

  sendfile on;

  default_type application/octet-stream;


  gzip on;
  gzip_http_version 1.1;
  gzip_disable      "MSIE [1-6]\.";
  gzip_min_length   256;
  gzip_vary         on;
  gzip_proxied      expired no-cache no-store private auth;
  gzip_types        text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
  gzip_comp_level   9;


  root /usr/share/nginx/html;


  location / {
    try_files $uri $uri/ /index.html =404;
  }

}

Step 3.3: Build Docker image

With those changes, the docker build is successful:

$ docker build --tag oveits/angular-cli-hello-world-with-docker:v0.1 .
Sending build context to Docker daemon  237.6MB
Step 1/12 : FROM node:8-alpine as builder
 ---> b7e15c83cdaf
...
Step 12/12 : CMD nginx -g daemon off;
 ---> Using cache
 ---> 2fa5d6c8e5da
Successfully built 2fa5d6c8e5da
Successfully tagged oveits/angular-cli-hello-world-with-docker:v0.1

Step 3.4: Push the Docker Image to Docker Hub

If the development machine is not the same machine you run the container on, we now need to push the image to Docker Hub. For that, we need to login first:

$ docker login
Username: oveits
Password: <my password>

For convenience, we have tagged the image also with the latest tag:

$ docker tag oveits/angular-cli-hello-world-with-docker:v0.1 oveits/angular-cli-hello-world-with-docker:latest

Then we can push the image, once for the tag v0.1, and a second time with the latest tag

$ docker push oveits/angular-cli-hello-world-with-docker:v0.1
$ docker push oveits/angular-cli-hello-world-with-docker

Step 3.4: Run the App

Now is the time to run the app. This can be done on any Docker host. The image will be downloaded automatically.

$ alias webapp='docker run --rm --name angular-cli-hello-world-with-docker-example -d -p 80:80 oveits/angular-cli-hello-world-with-docker $@'
$ webapp
1bc275d1d906fc5b63ace92ea20ec7b27c7de0ba3663c186122b30ead3a076d5

I have chosen port 80 to map on container port 80, since, at the end of the day, I do not want my customers to be forced to use custom ports, as long as I have not implemented proper load balancing on AWS.

We can verify that the container is running:

$ docker ps
CONTAINER ID        IMAGE                                        COMMAND                  CREATED             STATUS              PORTS                  NAMES
91481fcec020        oveits/angular-cli-hello-world-with-docker   "nginx -g 'daemon ..."   5 seconds ago       Up 3 seconds        0.0.0.0:80->80/tcp     angular-cli-hello-world-with-docker-example

Step 3.5: Local test

The local test is successful:

$ curl localhost
<!doctype html><html lang="en"><head><meta charset="utf-8"><title>AngularCliHelloWorldWithDocker</title><base href="/"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" type="image/x-icon" href="favicon.ico"><link href="styles.d41d8cd98f00b204e980.bundle.css" rel="stylesheet"/></head><body><app-root></app-root>http://inline.c1122970c69d0e49cb86.bundle.jshttp://polyfills.d8d3d78a4deb2ab66856.bundle.jshttp://main.3ff2f2cccb753b0aef42.bundle.js</body></html>

Step 3.6: Remote Test

As with the example in phase 1, we expect the application to be available remotely:

Perfect! Different from running the application with ng serve --host 0.0.0.0 the application is also available remotely. The issue with  “Invalid Host Header” was fixed by dockerizing our application.

Angular is running in a Docker container.

Excellent! Thump up!

Summary

We have created and run a dockerized Angular 4 example by . This has worked like a charm without any adaptations.

As a second step, we have created our own Angular Hello World application and we have applied the necessary changes to create a Docker image. We have found that the only measures needed to dockerize the application was to copy and slightly adapt the Dockerfile and the NginX default configuration file from avatsaev great example before we build the image.

Note, that we had got an “Invalid Host Header” response in the local browser when we were running the application with the ng serve --host 0.0.0.0 command on a Docker host on AWS. Dockerizing our application has fixed that issue. Our application is fully accessible from the outside world when running on a Docker host on AWS.

0

Adding Swap to CentOS


I describe how to add swap to CentOS and why using dd instead of fallocate for this.

Why?

In my case, I wanted to take the opportunity of an AWS promotion, which limits the DRAM to 1 GB. The CentOS Docker Host I am using for Angular CLI topics needs slightly more than 1 GB or RAM. Instead of upgrading the AWS plan, I have decided to try it with adding a swap file.

How?

First, I have followed this Ubuntu instructions, because I wanted to test the fallocate function on CentOS. This looked less hardware-related than the dd commands I have found on the CentOS instructions.

Fallocate seemed to work at first sight, but when it came to the activation as swap, I got an error message also discussed on this StackExchange page.:

$ sudo swapon /myswap
swapon: /myswap: swapon failed: Invalid argument

I had to revert back to the CentOS instructions:

Step 1: Create File with dd

Multiply the number of MB by 1024 (in my case: 2 GB = 2014 MB; multiplied with 1024, this is 2097152):

$ sudo dd if=/dev/zero of=/swapfile bs=1024 count=2097152

This takes some time; it is zeroing out 2 GB of disk space…

Step 2: Correct Permissions

We correct the permissions of the swapfile:

$ sudo chmod 600 /swapfile
$ ls -lh /swapfile
-rw-------. 1 root root 2.0G Oct  1 09:48 /swapfile

Step 3: Declare as Swap

We mark the file as swap:

$ sudo mkswap /swapfile
Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=e57fba27-064a-456b-be56-32fb230eede0

Step 4: Activate Swap

Now the swapfile can be activated:

$ sudo swapon /swapfile

Step 5: Verify

To verify:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           991M         71M         70M         12M        849M        738M
Swap:          2.0G          0B        2.0G

Step 6: Make the Change Reboot-Save

To make the change survive a reboot, we edit /etc/fstab as follows:

$ sudo vi /etc/fstab
...
/swapfile swap swap defaults 0 0 # append this line

Instead of using an editor, the step can be automated as follows:

$ cat /etc/fstab | grep -q swapfile || echo "/swapfile swap swap defaults 0 0" | sudo tee -a /etc/fstab

Step 7: Verify after Reboot

Verify that the swap is still enabled after reboot:

$ sudo reboot

Reconnect after some minutes and:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           991M         81M        626M         12M        283M        737M
Swap:          2.0G          0B        2.0G

Works!

Excellent! Thump up!

0

Resolving the VirtualBox code E_FAIL Problem caused by Hyper-V


Shoot! The Microsoft update this morning has caused trouble with my VirtualBox installation. Again! This is the second or third time I had to search for my blog post that is describing the resolution in an appendix. Since the number of blog posts containing the word “Hyper-V” is increasing on my blog, I now publish the information in a separate blog post. This way, I hope to find it more quickly next time a Microsoft update is messing up with my configuration.

Error message: “VT-x is not available” | code E_FAIL

Seen with:

  • VirtualBox-5.1.10-112026-Win.exe
  • Microsoft Windows 10 Pro, version 1703, OS build 15063.483 or older

If you get an error message during vagrant up or when starting a VirtualBox machine, telling you that VT-x is not available, a reason may be that you have enabled Hyper-V on your Windows 10 machine: VirtualBox and Hyper-V cannot share the VT-x CPU:

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'thesteve0/openshift-origin' is up to date...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
 default: Adapter 1: nat
 default: Adapter 2: hostonly
==> default: Forwarding ports...
 default: 8443 (guest) => 8443 (host) (adapter 1)
 default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "8ec20c4c-d017-4dcf-8224-6cf530ee530e", "--type", "headless"]

Stderr: VBoxManage.exe: error: VT-x is not available (VERR_VMX_NO_VMX)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole

Note: after the latest Windows 10 Pro update, the error message has slightly changed. It does not tell anymore, that VT-x is not avialable (an anti-improvement). However, the code E_FAIL remains the same.

Resolution:

found here

Step 1: prepare your Windows machine for dual boot with and without Hyper-V

As Administrator, open a CMD and issue the commands

bcdedit /copy "{current}" /d "Hyper-V" 
bcdedit /set "{current}" hypervisorlaunchtype off
bcdedit /set "{current}" description "non Hyper-V"

Step 2: Reboot the machine and choose the “non-Hyper-V” option.

Now, the vagrant up command should not show the “VT-x is not available” error message anymore.

The problem with some Microsoft updates is, that they tend to set the “hypervisorlaunchtype” to “auto” in the default profile.

This time it did not help: Trying a Re-installation of VirtualBox…

I was sure the procedure would help, as it had helped several times before, but this time, it was different.

The exact error message looked like follows:

Command: ["startvm", "8646a1bd-9371-4eb4-ad7c-9517419fb7ea", "--type", "headless"]

Stderr: VBoxManage.exe: error: The virtual machine 'ubuntu-trusty64-docker_2017-02_default_1496852974046_93388' has terminated unexpectedly during startup with exit code -1073741819 (0xc0000005)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component MachineWrap, interface IMachine

Setting the hypervisorlaunchtype to off did not help this time. Instead, a download of the latest VirtualBox software (VirtualBox-5.1.28-117968-Win.exe) and re-installation of VirtualBox has improved the behavior in the sense, that VirtualBox GUI is working again:

Do not forget to reload your system at this point! If you do not reload the system, you will see error message similar to before.

After a reload of the system, vagrant should work fine:

D:\veits\Vagrant\ubuntu-trusty64-docker_2017-02>vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'williamyeh/ubuntu-trusty64-docker' is up to date...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
    default: Adapter 3: hostonly
==> default: Forwarding ports...
    default: 4200 (guest) => 4200 (host) (adapter 1)
    default: 8000 (guest) => 8000 (host) (adapter 1)
    default: 8001 (guest) => 8001 (host) (adapter 1)
    default: 8002 (guest) => 8002 (host) (adapter 1)
    default: 8080 (guest) => 8080 (host) (adapter 1)
    default: 8081 (guest) => 8081 (host) (adapter 1)
    default: 8082 (guest) => 8082 (host) (adapter 1)
    default: 8083 (guest) => 8083 (host) (adapter 1)
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default:
    default: Guest Additions Version: 5.0.24
    default: VirtualBox Version: 5.1
==> default: Setting hostname...
==> default: Configuring and enabling network interfaces...
==> default: Mounting shared folders...
    default: /vagrant => D:/veits/Vagrant/ubuntu-trusty64-docker_2017-02
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.

Puh, that has taken 2 hours of my precious time again. Oh, Microsoft!

At least, it is working again.
🙂

 

0

Behavior-Driven Angular – part 1: Consuming a RESTful Web Service with Angular 4


In this step-by-step tutorial, we will follow a behavior-driven development approach to create an Angular 4 application from Angular CLI. The hello-world-like application will consume the WordPress REST API and it will display a blog post title. We will create and run end-to-end test scripts that simulate the customer behavior on a Chrome browser within a Protractor headless Docker container.

As a side feature of this tutorial, we will demonstrate basic Git handling: we will learn how to create a GIT Repository, create a feature branch, commit code changes, and merge the tested and fully functional feature branch into the main development branch.

Check out this book on Amazon: Angular Test-Driven Development

Introduction

My post Consuming a RESTful Web Service with Angular 4 has grown much more popular than expected. Thanks a lot for your views! The September ist still not finished, and the article has achieved more than 3000 views in its fourth month. I hope the trend will keep on:

😉

So, why would I want to rework a blog post that seemingly plays a chord in the developer’s community? The reasons I have started to rework the example are:

  • I have come to a point, where I had the need to refactor the code. However, I do not like refactoring before I do not have a good coverage of end-to-end tests. This was fixed easily in my previous blog post Angular end-to-end Testing.
  • The next topic was not so easy to be resolved: I had created a working example, but when I have created a GIT repository from it, Angular CLI had a problem with new clones of that code. An Angular problem I could not resolve easily and it looked like I had to start from scratch. This is, what I am doing now, committing many snapshots to GIT. If I so, why not explaining to my audience, what I am doing and why? This way, the current post has become an example that demonstrates basic GIT handling.

This blog post will fix those two issues, I deem.

Even if I am tempted to automate many of the development process steps, we will keep it simple without the usage of DevOps tools like Jenkins with BitBucket, Sonar, BrowserStack, JMeter Integration and Docker data center integration you would find in real-world agile projects. Some of such topics can be explored in more detail on my other blog posts on Jenkins (explore the “Jenkins Tutorial” drop-down menu of my blog).

Why behavior driven development?

I have made a very good experience with behavior driven development (BDD), or “test first” development. Some years ago, I have applied this principle on a ProvisioningEngine I had developed based on Ruby on Rails and java (Apache Camel). The advantages of BDD I see are:

  • better customer view: if you follow the behavior driven principle, your first thought is, how the web pages look like and how the web pages behave with respect to customer actions — in detail. This helps me to always start with the customer view in mind.
  • higher motivation: as a developer, I find it rewarding to start with test development with “red” test cases that become green over time
  • higher quality: I often challenge myself to optimize my code (e.g. make it DRYer of more versatile). In this process, I do not want to sacrifice and previous achievements. A large set of unit tests and e2e test help me to keep the set of features intact in phases of code restructuring

Okay, as an Angular beginner, I deem I am far from being an ideal behavior driven Angular developer. However, at some point in future, I believe that I can increase my hobby development productivity by applying principles like BDD together with build&deployment automation based on TravisCI, CircleCI or a local Jenkins system to my development approach.

Overview

Along the way, we will get acquainted with a set of typical error messages and we will learn how to cope with them.

So, if you are ready for a quick ride into a simple “test first” strategy example with GIT repo handling, buckle up and start coding with me in four phases:

😉

  • Phase 1: Create a Hello World App based on Angular CLI
  • Phase 2: Adapt the end-to-end Tests
  • Phase 3: Adapt the Code
  • Phase 4: Verify the successful e2e Tests

If you do not care about BDD and GIT, then you might also want head over to the post Consuming a RESTful Web Service with Angular 4. Or better, follow the instructions you find here, but omit the steps related to e2e testing (protractor) and/or GIT.

Phase 1: Create a Hello World App based on Angular CLI

In this phase, we will

  • use an Angular CLI Docker image to create a new application
  • fix some problems with the end to end testing inherent in the standard hello world app
  • save and upload the changes to GIT

Step 1.0: Get access to a Docker Host with enough Resources

If you do not have access to a Docker host yet, I recommend following the step 0 instructions on my JHipster post. I recommend to use a Docker host with at least 1.5 GB RAM. To be honest, this is a guess. I always test on a 4 GB Docker Host Virtualbox VM, but I know that 750 MB RAM is not sufficient.

Step 1.1: Prepare an alias for later use

Let us first define an alias that helps us to shorten the commands thereafter.

(dockerhost)$ alias cli='docker run -it --rm -w /app -v $(pwd):/app -p 4200:4200 -u $(id -u $(whoami)) oveits/angular-cli:1.4.3 $@'

Why this complicated user option -u $(id -u $(whoami))? The reason is that

  • if we omit it, then all new files will be created as root, so we will get permissions problems later on
  • If we use ‘centos’, then the container will complain that he does not find the user ‘centos’ in its passwd file
  • If we use the ID of centos, then it works. However, it might not work in all cases. This time, the ID of centos user is 1000, and by chance, a user (named ‘node’) exists on the container as well. But let us live with this uncertainty for now.

With each cli something command, we will start a something command on an Angular CLI @ Alpine container originally created by Alex Such and enriched with git and bash by me.

Consider appending the alias command to your Docker host’s ~/.bashrc file, so the alias is persistent.

Step 1.2: Create a Project and install required Modules

Now let us create a new project and install the node modules via npm:

(dockerhost)$ cli ng new consuming-a-restful-web-service-with-angular
(dockerhost)$ cd consuming-a-restful-web-service-with-angular
(dockerhost)$ cli npm install
npm info it worked if it ends with ok
npm info using npm@3.10.10
npm info using node@v6.11.2
npm info attempt registry request try #1 at 7:54:24 PM
npm http request GET https://registry.npmjs.org/fsevents
npm http 200 https://registry.npmjs.org/fsevents
npm info lifecycle consuming-a-restful-web-service-with-angular@0.0.0~preinstall: consuming-a-restful-web-service-with-angular@0.0.0
npm info linkStuff consuming-a-restful-web-service-with-angular@0.0.0
npm info lifecycle consuming-a-restful-web-service-with-angular@0.0.0~install: consuming-a-restful-web-service-with-angular@0.0.0
npm info lifecycle consuming-a-restful-web-service-with-angular@0.0.0~postinstall: consuming-a-restful-web-service-with-angular@0.0.0
npm info lifecycle consuming-a-restful-web-service-with-angular@0.0.0~prepublish: consuming-a-restful-web-service-with-angular@0.0.0
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm info ok

Step 1.3 (optional): Create a local GIT Repository

Now is a good time to create a git repository and to commit the initial code.

If you have not installed GIT on your Docker host, depending on the operating system of your Docker host, you might need to install it first (e.g. apt-get update; apt-get install -y git in case of Ubuntu, or yum install -y git in case of CentOS). Alternatively, you may want to use the git I have installed in the container. In that case, prepend a “do” before the git command, e.g. try cli git --version. However, a git diff does not look nice in a container, so I recommend to install GIT on your Docker host instead.

Now let us initialize the git repo, add all files and commit the changes:

(dockerhost)$ git init
(dockerhost)$ git add .
(dockerhost)$ git commit -m'initial commit'

Now let us start the service in a container:

(dockerhost)$ cli ng serve --host 0.0.0.0
** NG Live Development Server is listening on 0.0.0.0:4200, open your browser on http://localhost:4200/ **
Date: 2017-09-26T20:04:45.036Z
Hash: 24fe32460222f3b3faf2
Time: 15376ms
chunk {inline} inline.bundle.js, inline.bundle.js.map (inline) 5.83 kB [entry] [rendered]
chunk {main} main.bundle.js, main.bundle.js.map (main) 8.88 kB {vendor} [initial] [rendered]
chunk {polyfills} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 209 kB {inline} [initial] [rendered]
chunk {styles} styles.bundle.js, styles.bundle.js.map (styles) 11.3 kB {inline} [initial] [rendered]
chunk {vendor} vendor.bundle.js, vendor.bundle.js.map (vendor) 2.29 MB [initial] [rendered]

webpack: Compiled successfully.

Step 1.4: Perform end-to-end Tests

Step 1.4.1: Use a Protractor Docker Image to perform the Tests

In the spirit of “test first” strategies of “behavior-driven development”, let us check the end-to-end tests that come with Angular CLI 1.4.3. We will see that they are broken and need to be adapted.

Like above, we will use a Docker container for the task. This time we will use the Docker image protractor-headless from webnicer. In a second terminal, we first define an alias, enter the project root folder and run protractor.

(dockerhost)$ alias protractor='docker run -it --privileged --rm --net=host -v /dev/shm:/dev/shm -v $(pwd):/protractor webnicer/protractor-headless $@'
(dockerhost)$ cd consuming-a-restful-web-service-with-angular
(dockerhost)$ protractor

[20:20:34] I/direct - Using ChromeDriver directly...
[20:20:34] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✗ should display welcome message
      - Failed: Error while waiting for Protractor to sync with the page: "Could not find testability for element."
          at /usr/local/lib/node_modules/protractor/built/browser.js:272:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)Error
          at ElementArrayFinder.applyAction_ (/usr/local/lib/node_modules/protractor/built/element.js:461:27)
          at ElementArrayFinder._this.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:103:30)
          at ElementFinder.(anonymous function) [as getText] (/usr/local/lib/node_modules/protractor/built/element.js:829:22)
          at AppPage.getParagraphText (/protractor/e2e/app.po.ts:9:43)
          at Object. (/protractor/e2e/app.e2e-spec.ts:12:17)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
      From: Task: Run it("should display welcome message") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:10:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:3:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:392:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:395:12)

**************************************************
*                    Failures                    *
**************************************************

1) consuming-a-restful-web-service-with-angular App should display welcome message
  - Failed: Error while waiting for Protractor to sync with the page: "Could not find testability for element."

Executed 1 of 1 spec (1 FAILED) in 0.878 sec.
[20:20:41] I/launcher - 0 instance(s) of WebDriver still running
[20:20:41] I/launcher - chrome #01 failed 1 test(s)
[20:20:41] I/launcher - overall: 1 failed spec(s)
[20:20:41] E/launcher - Process exited with error code 1

Even though my application is listening on port 4200,  we can see that the e2e tests have a problem.

Step 1.4.2: Correct the Protractor sync Issue

As already pointed out in this blog post, we need to add the option

useAllAngular2AppRoots: true

to our protractor.conf.js file. At the end, the file has following content (correction in blue):

// protractor.conf.js
// Protractor configuration file, see link for more information
// https://github.com/angular/protractor/blob/master/lib/config.ts

const { SpecReporter } = require('jasmine-spec-reporter');

exports.config = {
  allScriptsTimeout: 11000,
  specs: [
    './e2e/**/*.e2e-spec.ts'
  ],
  capabilities: {
    'browserName': 'chrome'
  },
  directConnect: true,
  baseUrl: 'http://localhost:4200/',
  useAllAngular2AppRoots: true,
  framework: 'jasmine',
  jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000,
    print: function() {}
  },
  onPrepare() {
    require('ts-node').register({
      project: 'e2e/tsconfig.e2e.json'
    });
    jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
  }
};

After that, the e2e test is still not successful:

$ protractor
[20:30:32] I/direct - Using ChromeDriver directly...
[20:30:32] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✗ should display welcome message
      - Expected 'Welcome to !' to equal 'Welcome to app!'.
          at Object. (/protractor/e2e/app.e2e-spec.ts:12:37)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2820:25)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)

**************************************************
*                    Failures                    *
**************************************************

1) consuming-a-restful-web-service-with-angular App should display welcome message
  - Expected 'Welcome to !' to equal 'Welcome to app!'.

Executed 1 of 1 spec (1 FAILED) in 0.848 sec.
[20:30:40] I/launcher - 0 instance(s) of WebDriver still running
[20:30:40] I/launcher - chrome #01 failed 1 test(s)
[20:30:40] I/launcher - overall: 1 failed spec(s)
[20:30:40] E/launcher - Process exited with error code 1

Step 1.4.3: Correct the e2e Test Script

The reason is that the app.component.ts file is not correct. In the HTML template, we find a line

Welcome to {{title}}!

but in the component file, the title is missing:

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent implements OnInit {

  constructor() { }

  ngOnInit() {
  }

}

This is leading to following corrupt web page:

Let us correct this now (in blue):

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent implements OnInit {

  title : any = null

  constructor() { }

  ngOnInit() {
     this.title = "app";
  }

}

Now the Web page looks better:

Now the e2e tests are successful:

$ protractor
[20:53:42] I/direct - Using ChromeDriver directly...
[20:53:42] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display welcome message

Executed 1 of 1 spec SUCCESS in 0.956 sec.
[20:53:50] I/launcher - 0 instance(s) of WebDriver still running
[20:53:50] I/launcher - chrome #01 passed

The angular CLI installation works as expected now.

Excellent! Thump up!

Let us save the changes:

(dockerhost)$ git add protractor.conf.js
(dockerhost)$ git commit -m'protractor.conf.js: added useAllAngular2AppRoots: true for avoiding sync problems'
(dockerhost)$ git add src/app/app.component.ts
(dockerhost)$ git commit -m'app component: defined missing title'

Now is the time to sign up with Github and save the project. In my case, I have created following project: a project like follows:

https://github.com/oveits/consuming-a-restful-web-service-with-angular

Once this is done, we can upload the changes as follows:

(dockerhost)$ git remote add origin https://github.com/oveits/consuming-a-restful-web-service-with-angular.git
(dockerhost)$ git push -u origin master

Phase 2: Adapt the end-to-end Tests

In this phase, we will

  • based on the input from the WordPress API, we will plan, how the web page should look like from a customer’s point of view.
  • We will adapt the e2e tests, so they reflect the (assumed) customer’s expectations.
  • We will save the changed code to the remote GIT repository.

Step 2.1: Planning

In an attempt to follow a behavior driven development process, we will write/adapt the end to end tests first, before we perform the changes. For this, let us outline our plan:

  • We would like to create a Web page that displays the title and content of a WordPress Article
  • the WordPress article of our choice is the first angular blog post I have written: the Angular 4 Hello World Quickstart blog post
  • The article will be retrieved dynamically from the WordPress API, a REST API.

Step 2.2: Explore the WordPress REST API

Let us have a look at the WordPress API. The WordPress API can be explored via the WordPress.com REST API console. We can display a list of blog posts like so:

We can see that the blog post we would like to display has the ID 3078 and the title and content star like follows:

  • title: “Angular 4 Hello World Quickstart”
  • content: “<p>In this hello world style tutorial, we will follow a step by step guide to a working Angular 4 application. We will also …

The  single blog post can be retrieved with the URL

https://public-api.wordpress.com/rest/v1.1/sites/oliverveits.wordpress.com/posts/3078

We can verify this by copying the URL into a Browser:

Step 2.3: Adapt the end-to-end Tests

With the knowledge about the title and content of the blog post, we can re-write the end-to-end (e2e) test. The e2e test is found in the e2e folder:

ls e2e/
app.e2e-spec.ts app.po.ts tsconfig.e2e.json

$ cat e2e/app.e2e-spec.ts
import { AppPage } from './app.po';

describe('consuming-a-restful-web-service-with-angular App', () => {
  let page: AppPage;

  beforeEach(() => {
    page = new AppPage();
  });

  it('should display welcome message', () => {
    page.navigateTo();
    expect(page.getParagraphText()).toEqual('Welcome to app!');
  });
});

Instead of searching for the text ‘Welcome to app’, let us search for the title “Angular 4 Hello World Quickstart”:

$ cat e2e/app.e2e-spec.ts
import { AppPage } from './app.po';

describe('consuming-a-restful-web-service-with-angular App', () => {
  let page: AppPage;

  beforeEach(() => {
    page = new AppPage();
  });

  it('should display the title', () => {
    page.navigateTo();
    expect(page.getParagraphText()).toContain('Angular 4 Hello World Quickstart');
  });
});

The e2e test should fail now with the message Expected 'Welcome to app!' to contain 'Angular 4 Hello World Quickstart'

$ protractor
[20:46:02] I/direct - Using ChromeDriver directly...
[20:46:02] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✗ should display welcome message
      - Expected 'Welcome to app!' to contain 'Angular 4 Hello World Quickstart'.
          at Object. (/protractor/e2e/app.e2e-spec.ts:12:37)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:94:23
          at new ManagedPromise (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1082:7)
          at controlFlowExecute (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:80:18)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2820:25)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)

**************************************************
*                    Failures                    *
**************************************************

1) consuming-a-restful-web-service-with-angular App should display welcome message
  - Expected 'Welcome to app!' to contain 'Angular 4 Hello World Quickstart'.

Executed 1 of 1 spec (1 FAILED) in 0.907 sec.
[20:46:21] I/launcher - 0 instance(s) of WebDriver still running
[20:46:21] I/launcher - chrome #01 failed 1 test(s)
[20:46:21] I/launcher - overall: 1 failed spec(s)
[20:46:21] E/launcher - Process exited with error code 1

Step 2.4: Save the Changes on a separate GIT Branch

We believe that the e2e tests are correct now, so it is a good time to create a new git feature branch and commit the code:

git checkout -b feature/0001-retrieve-and-display-WordPress-title-from-API
git add .
git commit -m'adapted e2e tests to display WordPress blog title'
git push

Phase 3: Adapt the Code

Now, after having written the e2e tests, let us change the code, so our app fulfills the expectations.

Step 3.1: Define the HTML View

In the spirit of a behavior driven approach, let us define the view first. For that we replace the content of the app’s template file:

$ cat src/app/app.component.html
<h1>{{title}}</h1>

The output of the application now is:

This is because, in the Hello World app, we have set the title to the static value ‘app’. The e2e tests are not successful and the error ‘Expected ‘app’ to contain ‘Angular 4 Hello World Quickstart’.’ is thrown when we run protractor.

Step 3.2: Subscribe an Observable

As can be seen in many tutorials, we now subscribe to an observable like follows:

$ cat src/app/app.component.ts
import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent implements OnInit {

  title : any = null

  constructor() { }

  ngOnInit() {
     //this.title = "app";
     this._http.get('https://public-api.wordpress.com/rest/v1.1/sites/oliverveits.wordpress.com/posts/3078')
                .map((res: Response) => res.json())
                 .subscribe(data => {
                        this.title = data.title;
                        console.log(data);
                });
  }

}

We perform an HTTP GET on the WordPress API’s URL, map the response to a JSON object and subscribe the retrieved data. The data should contain a title, which we assign to the local title variable.

However, we will see in the log:

ERROR in /app/src/app/app.component.ts (16,11): Property '_http' does not exist on type 'AppComponent'.

And in the browser, we see:

Let us fix that now.

Step 3.3: Define private local _http Variable

In angular, we can define the private local _http variable in the constructor:

constructor(private _http: Http) {}

Once, this is done, the error message is changed to:

ERROR in /app/src/app/app.component.ts (12,30): Cannot find name 'Http'.

Step 3.4: Import Http Components

The used Http module is not known to our app component. Let us change this now. We add the following line

import { Http } from '@angular/http';

to the file src/app/app.component.ts. The error message changes to:

ERROR in /app/src/app/app.component.ts (18,18): Property 'map' does not exist on type 'Observable<Response>'.

Step 3.5: Import map

The map function needs to be imported as well:

import 'rxjs/add/operator/map'

Now we get an illegible error like follows:

ERROR in /app/src/app/app.component.ts (18,6): The type argument for type parameter 'T' cannot be inferred from the usage. Consider specifying the type arguments explicitly.
  Type argument candidate 'Response' is not a valid type argument because it is not a supertype of candidate 'Response'.
    Types of property 'type' are incompatible.
      Type 'ResponseType' is not assignable to type 'ResponseType'. Two different types with this name exist, but they are unrelated.
        Type '"basic"' is not assignable to type 'ResponseType'.

Step 3.6: Import Response Type

We finally can get rid of the quite illegible error message by adding another import:

import { Response } from '@angular/http';

However, this still does not lead to the desired result. In the browser we see an empty page:

and the e2e tests fail with the following message:

$ protractor
...
Failed: Angular could not be found on the page http://localhost:4200/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load

Step 3.7: Add HttpModule in the app Module

The solution of the above error lies in the src/app/app.module.ts (added parts in blue). We first need to add the HttpModule to the imports, which alters the error message to

$ cat src/app/app.module.ts
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { HttpModule }    from '@angular/http';

import { AppComponent } from './app.component';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    HttpModule,
    BrowserModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

This seems to have been the last stepping stone towards success:

Phase 4: Verify the successful e2e Tests

Now the e2e tests are successful:

$ protractor
[22:16:06] I/direct - Using ChromeDriver directly...
[22:16:06] I/launcher - Running 1 instances of WebDriver
Jasmine started

  consuming-a-restful-web-service-with-angular App
    ✓ should display welcome message

Executed 1 of 1 spec SUCCESS in 1 sec.
[22:16:14] I/launcher - 0 instance(s) of WebDriver still running
[22:16:14] I/launcher - chrome #01 passed

That is, how the e2e Tests should look like. Success!

Excellent! Thump up!

Step 4.2: Save the changes to the develop branch on GIT

Since our new feature “retrieve and display a blog title from WordPress API” has been verified to work fine, it is time to commit the change and push it to the remote repository:

git add .
git commit -m'added all code needed for successful e2e tests'
git push
git checkout -b develop
git push

In addition to that, we can create a new “develop” branch, if it does not exist yet:

git checkout -b develop
git push

In case the develop branch exist already, you need to merge the code to the instead of creating the develop branch:

git checkout develop
git merge feature/0001-retrieve-and-display-WordPress-title-from-API
git push

It makes sense to allow a merge to the develop branch only if the code is fully tested. This way, we will never break the code in the develop branch.

For large teams, several measures can be taken to make sure that only high quality code enters the develop branch: e.g. on BitBucket GIT, you can allow a merge only, if code has been reviewed and acknowledged by a certain number of team members. Moreover, you can integrate the repository with a Jenkins system: with the correct plugins, you can make sure that a merge is allowed only in case all quality gates (e2e test, unit tests, style, performance, …) in the Jenkins pipeline are met.

However, if you are a hobby developer working on a , it is probably sufficient if you run the tests manually before you merge the changed code into the develop or master branch.

Summary

In this hello world style step-by-step guide, we have learned

  • How to create a new Hello World project using Angular CLI, repair the e2e tests and save the changes on GIT.
  • How to create/adapt the e2e tests in advance a “test first” manner.
  • How to consume a REST service using Angular 4 and verify the result using the e2e test scripts we have created before.

Next Steps

In part 2 of this series, we will learn how to add and display HTML content to the body of our application. We will see that we cannot just use the method we have used for the title. If we do so, we will see escaped HTML code like follows:

<p>In this hello world style tutorial,…

We will show how to make Angular accept the HTML code and display it correctly.

References:

Appendix A: Adding Docker Support

This is, how I have added Docker support for the application, following my tl:dr of the blog post Angular 4 Docker Example.

A.1 Add Dockerfile and NginX config

git clone https://github.com/oveits/consuming-a-restful-web-service-with-angular
git checkout -b feature/0002-docker-support
curl -O https://raw.githubusercontent.com/avatsaev/angular4-docker-example/master/Dockerfile
curl -O https://raw.githubusercontent.com/avatsaev/angular4-docker-example/master/nginx/default.conf
mkdir nginx
mv default.conf nginx/

remove ‘package-log.json’ from Dockerfile

git add .
git commit -m 'added Dockerfile and nginx config file'
git push

A.2 Build the Docker  Image

On a docker host, I have issued following commands:

docker build . --tag oveits/consuming-a-restful-web-service-with-angular:v0.2
docker push oveits/consuming-a-restful-web-service-with-angular:v0.2
docker tag oveits/consuming-a-restful-web-service-with-angular:v0.2 oveits/consuming-a-restful-web-service-with-angular:latest
docker push oveits/consuming-a-restful-web-service-with-angular:v0.2
docker push oveits/consuming-a-restful-web-service-with-angular:latest

A.3 Run the Service

$ alias consuming='docker run --rm --name consuming-a-restful-web-service-with-angular -d -p 80:80 oveits/consuming-a-restful-web-service-with-angular $@'
$ consuming

A.4 Access the Service

In a browser, head to the public DNS of the image:

Works!

Excellent! Thump up!

 

 

0

Angular end to end Testing


– Angular e2e Protractor Tests on Systems without GUI applied to static and dynamic Web Pages –

In this blog post, we will show how to perform end-to-end (e2e) tests with Angular 4:

  • first, we will apply Protractor end to end tests on a little Angular CLI hello world application with static HTML content
  • second, we will perform end to end tests for a dynamic web application, a REST client application that retrieves and displays content received via the WordPress API.

For that, we will use a Docker Protractor Image that can be used on systems with or without graphical user interface (e.g. a Jenkins CI system).

 

Protractor + Docker

In a classical installation situation, you often use a machine with a graphical window system (e.g. X11 for Linux) so you can run a real Chrome or Firefox Browser on the system. In such a situation, you often install Jasmine, Protractor, Selenium and a Chrome or Firefox browser on the test machine. However, in this blog post, we prefer to use a pre-installed Docker image provided by webnicer instead, which allows us to run the e2e tests on Docker hosts without a graphical system. The missing need for a graphical interface makes this an ideal deployment option for continuous integration purposes, e.g. for Jenkins systems.

Why End to End Tests?

My motivation to write a blog about end-to-end tests is, that I have made a very good experience with “tests first” and more specifically “behavior driven development” (BDD) principles with previous projects. And end-to-end tests are the main ingredient needed for BDD. In my experience, projects that follow “test first” or BDD principles benefit from a better customer view and higher quality, together with higher motivated developers, who start their work with a set of failed (red) tests and are rewarded for their work with successful (green) tests.

Even if we are far from following behavior driven principles yet, let us perform our first tiny steps towards this principle by looking more closely at end-to-end testing now:

Step 1: Install and start an Angular CLI Hello World Application

We are closely following the phase 1 instructions on a previous popular blog post of mine Consuming a RESTful Web Service with Angular 4. Since I do not want to copy and paste that part of the other blog post, let me just summarize the commands after you have got access to a Docker host (e.g. by following the instructions found here; search for the term “Install a Docker Host”):

  • start Docker container
docker run -it -p 4200:4200 -v $(pwd):/localdir centos bash
  • install Angular CLI
(container)# yum install -y epel-release
(container)# yum install -y https://kojipkgs.fedoraproject.org//packages/http-parser/2.7.1/3.el7/x86_64/http-parser-2.7.1-3.el7.x86_64.rpm
(container)# yum install -y nodejs 
(container)# npm install -g @angular/cli
  • create a project
(container)# cd /localdir
(container)# ng new my-project-name
(container)# cd my-project-name
  • we do not yet start the service in order to see a certain error message, but in step 3, we will start the service with this command:
(container)# ng serve --host 0.0.0.0

For more detailed information about Step 1, see phase 1 of this blog post.

Step 2: Prepare Docker Host for Protractor Usage

For a convenient handling of the protractor test, let us open a new terminal and create a shell script on the Docker host like follows (see the readme of the Docker image page webnicer/protractor-headless):

If your Docker host does not allow the usage of sudo, then try the same commands without sudo.

(dockerhost)$ sudo cat - << END | sudo tee /usr/local/bin/protractor-headless
#!/bin/bash
docker run -it --privileged --rm --net=host -v /dev/shm:/dev/shm -v \$(pwd):/protractor webnicer/protractor-headless \$@
END

Then we make sure the file is executable:

(dockerhost)$ sudo chmod +x /usr/local/bin/protractor-headless
(dockerhost)$ which /usr/local/bin/protractor-headless
/usr/local/bin/protractor-headless

Step 3: Download and run Protractor in a Docker Container

Let us download the protractor Docker image webnicer/protractor-headless so the container will be started immediately from the image later:

(dockerhost)$ docker pull webnicer/protractor-headless 

Then we enter the project root folder we have created in step 1:

(dockerhost)$ cd my-project-name

and run protractor via:

(dockerhost)$ protractor-headless
[23:43:08] I/direct - Using ChromeDriver directly...
[23:43:08] I/launcher - Running 1 instances of WebDriver
Spec started
[23:43:24] E/protractor - Could not find Angular on page http://localhost:4200/ : retries looking for angular exceeded

  my-project-name App
    ✗ should display welcome message
      - Failed: Angular could not be found on the page http://localhost:4200/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
          at /usr/local/lib/node_modules/protractor/built/browser.js:506:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)
      From: Task: Run it("should display welcome message") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:10:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:3:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:382:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:385:12)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Failed: Angular could not be found on the page http://localhost:4200/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load

Executed 1 of 1 spec (1 FAILED) in 10 secs.
[23:43:24] I/launcher - 0 instance(s) of WebDriver still running
[23:43:24] I/launcher - chrome #01 failed 1 test(s)
[23:43:24] I/launcher - overall: 1 failed spec(s)
[23:43:24] E/launcher - Process exited with error code 1

Okay, this is expected, because no angular is running on port 4200 yet.

Step 4: Start Angular CLI Application and run Protractor again

As anticipated, let us start the Angular application as describes in step 1:

(container)# ng serve --host 0.0.0.0

Then we get following output, if we try to run the e2e tests:

(dockerhost)$ protractor-headless
[00:19:21] I/direct - Using ChromeDriver directly...
[00:19:21] I/launcher - Running 1 instances of WebDriver
Spec started

  my-project-name App
    ✗ should display welcome message
      - Failed: Error: Error while waiting for Protractor to sync with the page: "Could not find testability for element."
          at proxyDone.fail (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:87:34)
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)
      From: Task: Run it("should display welcome message") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:10:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:3:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:382:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:385:12)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Failed: Error: Error while waiting for Protractor to sync with the page: "Could not find testability for element."

Executed 1 of 1 spec (1 FAILED) in 0.565 sec.
[00:19:26] I/launcher - 0 instance(s) of WebDriver still running
[00:19:26] I/launcher - chrome #01 failed 1 test(s)
[00:19:26] I/launcher - overall: 1 failed spec(s)
[00:19:26] E/launcher - Process exited with error code 1

Step 5: Fix the “waiting for Protractor to sync” Problem and run Protractor again

I have found a solution on this StackOverflow Q&A: we need to add the additional configuration line into protractor.conf.js:

useAllAngular2AppRoots: true

In my case, the protractor configuration file looks like follows:

// Protractor configuration file, see link for more information
// https://github.com/angular/protractor/blob/master/lib/config.ts

const { SpecReporter } = require('jasmine-spec-reporter');

exports.config = {
  allScriptsTimeout: 11000,
  specs: [
    './e2e/**/*.e2e-spec.ts'
  ],
  capabilities: {
    'browserName': 'chrome'
  },
  directConnect: true,
  baseUrl: 'http://localhost:4200/',
  useAllAngular2AppRoots: true,
  framework: 'jasmine',
  jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000,
    print: function() {}
  },
  onPrepare() {
    require('ts-node').register({
      project: 'e2e/tsconfig.e2e.json'
    });
    jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
  }
};

After that, the e2e test via Protractor Docker Container is successful:

protractor-headless
[00:46:32] I/direct - Using ChromeDriver directly...
[00:46:32] I/launcher - Running 1 instances of WebDriver
Spec started
[00:46:38] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ✓ should display welcme message

Executed 1 of 1 spec SUCCESS in 1 sec.
[00:46:38] I/launcher - 0 instance(s) of WebDriver still running
[00:46:38] I/launcher - chrome #01 passed

Excellent! Thump up!

Step 6: Review the Protractor Spec File

Now, we want to understand in more detail, what happened. For that, let us analyze the e2e specification file (e2e/app.e2e-spec.ts):

import { MyProjectNamePage } from './app.po';

describe('my-project-name App', () => {
  let page: MyProjectNamePage;

  beforeEach(() => {
    page = new MyProjectNamePage();
  });

  it('should display welcome message', done => {
    page.navigateTo();
    page.getParagraphText()
      .then(msg => expect(msg).toEqual('Welcome to app!!'))
      .then(done, done.fail);
  });
});

Now let us compare that with the browser content on http://localhost:4200:

There, it is: the e2e test is loading the page, retrieving the first paragraph and will compare it with the text “Welcome to app!!”. Since the text of the first paragraph matches this text, the test is successful.

Step 7 (optional): Review the Error Message of a failed Test

Now let us see, what happens, if we change the expected text in e2e/app.e2e-spec.ts to “Welcome to ppa!!” instead:

$ protractor-headless
[00:32:30] I/direct - Using ChromeDriver directly...
[00:32:30] I/launcher - Running 1 instances of WebDriver
Spec started
[00:32:36] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ✗ should display welcome message
      - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to ppa!!'.
          at /protractor/e2e/app.e2e-spec.ts:13:32
          at /usr/local/lib/node_modules/protractor/built/element.js:798:32
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to ppa!!'.

Executed 1 of 1 spec (1 FAILED) in 1 sec.
[00:32:36] I/launcher - 0 instance(s) of WebDriver still running
[00:32:36] I/launcher - chrome #01 failed 1 test(s)
[00:32:36] I/launcher - overall: 1 failed spec(s)
[00:32:36] E/launcher - Process exited with error code 1

So, this is, how a failed test looks like.

Step 8 (advanced): Apply the e2e Test to a dynamic Web Page

Up to now, we have seen a simple test, which is comparing a static pattern with a static web page. We now want to apply the principle to a dynamic page as we have created in my blog post Consuming a RESTful Web Service with Angular 4. For that, we reverse the change in Step 7, so we get a successful e2e test again. Then we need to follow the steps in the corresponding blog post and run the resulting service on localhost port 4200. After that, the e2e test will fail:

(dockerhost)$ protractor-headless
[00:32:30] I/direct - Using ChromeDriver directly...
[00:32:30] I/launcher - Running 1 instances of WebDriver
Spec started
[00:32:36] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ✗ should display welcome message
      - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to app!!'.
          at /protractor/e2e/app.e2e-spec.ts:13:32
          at /usr/local/lib/node_modules/protractor/built/element.js:798:32
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to app!!'.

Executed 1 of 1 spec (1 FAILED) in 1 sec.
[00:32:36] I/launcher - 0 instance(s) of WebDriver still running
[00:32:36] I/launcher - chrome #01 failed 1 test(s)
[00:32:36] I/launcher - overall: 1 failed spec(s)
[00:32:36] E/launcher - Process exited with error code 1

But what is the expected result we want to see? Let us head to http://localhost:4200 again, and we will see:

The blue part is static content again. However, the Title “Angular 4 Hello World Quickstart” is dynamic content retrieved from the WordPress API. We can easily see that this pattern is missing in the source HTML code:

Nevertheless, we want to use our Protractor Docker Container to test, whether the dynamic content is visible in a browser. For that, let us adapt the e2e test file on (e2e/app.e2e-spec.ts) to the new situation:

import { MyProjectNamePage } from './app.po';

describe('my-project-name App', () => {
  let page: MyProjectNamePage;

  beforeEach(() => {
    page = new MyProjectNamePage();
  });

  it('should display title', done => {
    page.navigateTo();
    page.getParagraphText()
      .then(msg => expect(msg).toContain('Angular 4 Hello World Quickstart'))
      .then(done, done.fail);
  });
});

the e2e test is successful:

(dockerhost)$ protractor-headless
[00:46:32] I/direct - Using ChromeDriver directly...
[00:46:32] I/launcher - Running 1 instances of WebDriver
Spec started
[00:46:38] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ✓ should display title

Executed 1 of 1 spec SUCCESS in 1 sec.
[00:46:38] I/launcher - 0 instance(s) of WebDriver still running
[00:46:38] I/launcher - chrome #01 passed

Excellent! Thump up!

We have achieved our goal: we had intended to perform a successful end to end test for a dynamic web page that retrieves its content from an external resource like the WordPress API.

Summary

In this blog post, we

  • have applied a Protractor Docker Container on an existing Angular CLI Hello World.
    For this to work, we had to adapt the Protractor configuration file to circumvent an “Error while waiting for Protractor to sync with the page”
  • have applied a Protractor Docker Container with the patched Protractor configuration on an app that is dynamically downloading and displaying content retrieved from the WordPress API. This has worked as expected although the content we have looked for was not visible in the HTML source code.

With the latter, we have successfully verified that the Protractor Docker container can handle dynamic content retrieved and altered via javascript.

Next Steps

  • Write more complete test cases for the dynamic web page
  • Refactor the dynamic web page application by dividing the HTTP GET into a separate service. The Protractor tests will help me to verify that the refactored application will keep its previously achieved features.

References: