1

Rails Active Admin: from config File to Admin Portal


This blog post is a continuation of part 1 that had focused on the the installation process of Active Admin. In the current post, we show how Active Admin can be used to migrate from a configuration file based approach by an admin portal based approach. We start with a configuration file a la figaro, and migrate a boolean-like variable to Active Admin Portal, allowing admins to manage this variable. We will follow a DRY (don’t repeat yourself) approach by help of a “catch all” method that can handle any future variable of the same kind without the need to change any Ruby code.

Versions

  1. first published version
  2. changed model name ‘Config’ by ‘SystemSetting’, since RbConfig used by travisCI has a problem with the ‘Config’ name. Note, that the remaining error of v1 has disappeared with this measure as well: it seems that also Active Admin itself had a problem with the name ‘Config’.

How to use Active Admin to define System Settings

Now let us use go to work and prepare the Active Admin console for allowing to manage system wide variables:

Prerequisites

  • We assume that Active Admin is already installed. See part 1 for step by step installation instructions.
  • Rails server is started, e.g. with the command rails s

Step 1: create a model

Let us generate a model that will hold the configuration variables I am managing via figaro today. I want to support Strings and Booleans in a single class, so I have added a value_type column:

$ rails g model SystemSetting name:string value_type:string value_default:string value:string short_description:string description:text
      invoke  active_record
      create    db/migrate/20160205102214_create_system_settings.rb
      create    app/models/system_setting.rb
      invoke    rspec
      create      spec/models/system_setting_spec.rb

We need to migrate the database changes:

rake db:migrate

If you have forgotten to add a column, you can add it later by issuing commands similar to:

rails g migration AddNewColumnToSystemSetting new_column:text
rake db:migrate

Step 2: register the model with Active Admin:

Now let us add this model to Active Admin:

$ rails generate active_admin:resource SystemSetting
      create  app/admin/system_setting.rb

Step 3: log into the Active Admin console and add a SystemSetting entry:

After refreshing the Active Admin console, a new menu item SystemSettings shows up in the Active Admin page (log in; default user:pass = admin@example.com:password). When we follow the link, we see:

2016-02-05_113246_capture_018

No let us click on the Create one link in the SystemSettings page above and we can enter our first variable:

2016-02-05_113619_capture_019

Oups?

2016.02.03-21_28_39-hc_001

Okay, we have not yet specified in a controller, which attributes are allowed. In a normal model, this would be done in the controller by a command like

def user_params
 params.require(:user).permit(:username, :email, :password, :salt, :encrypted_password)
end

For Active Admin, it is similar, but different: it is described in the Active Admin documentation (found via this stackoverflow answer):

In app/admin/system_setting.rb add the permit_params line:

ActiveAdmin.register SystemSetting do
   permit_params :name, :value_type, :value_default, :value, :short_description, :description ## add this line

Now, press the browser’s back button and retry to save the entry. We will get:

2016-02-05_114100_capture_020

Step 4: use the new variable in the project source code

Let us make use of the new variable:

The following code snippet shows, how I use the system variable WEBPORTAL_SIMULATION_MODE today:

if ENV[WEBPORTAL_SIMULATION_MODE] == "true"
...
end

I would like to replace that snippet by something like:

if SystemSetting.webportal_simulation_mode
...
end

This is simpler than the more obvious standard handling of database searches like follows:

values =SystemSetting.where(name: "WEBPORTAL_SIMULATION_MODE")
value = values[0] if values.count == 1 
if value == "true" 
... 
end

In addition, the SystemSetting.webportal_simulation_mode approach will allow for more functionality as we will see below. The value of the SystemSetting.webportal_simulation_mode method shall be read from the SystemSetting database entry with name=”WEBPORTAL_SIMULATION_MODE”.

Step 4.1: create a catch all method in the model

Since we do not want to create such a method for each and every variable, I have made use of a cool Ruby feature: the method_missing method, which is a kind of catch all method. Any method that is not defined upfront will be handled by the method_missing method.

The code I have created looks like follows:

class SystemSetting < ActiveRecord::Base
    
    # catch any method like SystemSetting.whatever and do something with it instead of raising an error
    # found on http://stackoverflow.com/questions/185947/ruby-define-method-vs-def?rq=1
    def self.method_missing(*args)
        # assumption:
        # if we call SystemSetting.webportal_simulation_mode, we assume that the associated environment variable reads WEBPORTAL_SIMULATION_MODE (all capitals)
        environment_variable = args[0].to_s.upcase
        
        # look for database entries matching the method name, but with all capitals:
        foundlist =  self.where(name: environment_variable)
        
        # return value, if non-ambiguous entry was found; else return environment variable, if it exists:
        if foundlist.count == 1
            # found in the database: return its value as boolean
            foundlist[0].value == "true"
        elsif foundlist.count == 0 && !ENV[environment_variable].nil?
            # not found in the database: try to find corresponding environment variable as a fallback. 
            value = ENV[environment_variable]
            
            # If found, auto-create a database entry
            self.new(name: environment_variable, value: value, value_type: :boolean).save!
            
            # return its value as boolean
            value == "true"
        elsif foundlist.count > 1
            # error handling: variable found more than once (should never happen with the right validation)
            abort "Oups, this looks like a bug: Configuration.variable with name #{environment_variable} found more than once in the database."
        else
            # error handling: variable not found:
            message = "#{environment_variable} not found: neither in the database nor as system environment variable." +
                      " As administrator, please create a SystemSetting variable with name #{environment_variable} " +
                      " and the proper value (in most situations: 'true' or 'false') on the Active Admin Console on https://localhost:3000/admin/system_settings " +
                      "(please adapt the host and port to your environment). Alternatively, restart the server. " +
                      "This should reset the environment variable to its default value and the SystemSetting variable will be auto-created."
            abort message
        end # if foundlist.count == 1
    end # def self.method_missing(*args)
    
end

The method_missing catch all trick works like follows:

  1. the method_missing catch all method will handle any occurrence of e.g. SystemSetting.webportal_simulation_mode used anywhere in the project.
  2. The first argument handed over to method_missing (i.e. arg[0]) will be a symbol like :webportal_simulation_mode.
  3. We convert this argument to “WEBPORTAL_SIMULATION_MODE” and search for this name pattern in the SystemSetting database.
  4. If there is one and only one matching entry in the database, we return its value as a boolean. If there is no matching entry in the database, but the environment variable WEBPORTAL_SIMULATION_MODE is set, we auto-generate a corresponding entry in the database.

Now we can replace all occurrences of ENV[WEBPORTAL_SIMULATION_MODE] == "true" within the project by SystemSetting.webportal_simulation_mode.

Step 4.2: correct the destroy behavior

If we now want to destroy a SystemSetting that is being used in the project code, it will be auto-created, when SystemSetting.webportal_simulation_mode is called. This might not be the desired behavior. If I delete a SystemSetting, I do not want it to pop up again. Let us fix that now:

 

class SystemSetting < ActiveRecord::Base
    def self.method_missing(*args)
    ...
        # If found, auto-create a database entry
        @@autocreate = {} unless defined?(@@autocreate)
        @@autocreate[environment_variable] = true if @@autocreate[environment_variable].nil?
        self.new(name: environment_variable, value: value, value_type: :boolean).save! if @@autocreate[environment_variable]
    ...  
    def destroy
        # if we destroy a database entry, we do not want it to be auto-created again:
        @@autocreate = {} unless defined?(@@autocreate)
        @@autocreate[name] = false
        
        super
    end
    ...
end

Note, that I had defined a fallback, i.e. the SystemSetting.webportal_simulation_mode will still return a meaningful value, if the environment variable WEBPORTAL_SIMULATION_MODE is defined via configuration file (not visible to the Active Admin portal user). Therefore, the code  SystemSetting.webportal_simulation_mode within the project will not raise an exception in this case.

Step 4.3 add validations

A SystemSetting myvar will be mapped to an environment variable MYVAR and another SystemSetting MyVar will be mapped to the same environment variable MYVAR. Therefore we need to make sure we do not allow for two SystemSettings with the same meaning. We will allow for SystemSetting names with all capital letters only. For that we add following validations:

class SystemSetting < ActiveRecord::Base
    ...  
    # prevent that a name can exist twice:
    validates :name, uniqueness: true
 
    # allow only variables with capital letters and underscores:
    validates_format_of :name, :with => /\A[A-Z0-9_\.]{1,255}\Z/, message: "needs to consist of 1 to 255 characters: A-Z, 0-9, . and/or _"
end

We have achieved our goal: our first configuration variable WEBPORTAL_SIMULATION_MODE has been moved from a configuration file to the database and can be controlled by the administrator via the Active Admin console.

thumps_up_3

Summary

We have migrated a boolean configuration setting from a configuration file to the database in order to allow the manipulation of its setting from Active Admin console. We have encountered a Active Admin problem with the view of a single entry, but this did not prevent us to reach our goal (troubleshooting and/or error report to active Admin is postponed…).

Instead of requiring a standard database reading syntax like

values = SystemSetting.where(name: "WEBPORTAL_SIMULATION_MODE")
value = values[0] if values.count == 1
if value == "true"
...
end

we allow for a simpler syntax like

if SystemSetting.webportal_simulation_mode
...
end

In order to prepare the solution for more upcoming variables of the same kind, we use the  method_missing method as a catch all method. With this approach, any new variable like

SystemSetting.whatever

can be handled with the same piece of code. An administrator can add new variables in the Active Admin portal without the need to write a single line of ruby code. Okay, the SystemSetting.whatever must be used somewhere in the project’s ruby code to have a function. 😉

For convenience and backwards compatibility with the configuration file, I had added an auto-create function (the line starting with self.new in the  method_missing method): for any already defined environment variable, a database entry is created automatically from the environment variable. This will help for a quick migration of all variables from the configuration file to the database. Moreover, it allows us to keep the configuration file as a fallback, e.g. if the database is destroyed. However, in future, it will be better to automatically export the system_setting table from the database and re-import it upon server start if the system_setting table is found empty (or allow to retrieve it from some SW repository).

The results of the measure can be reviewed in the open source on Github, e.g. on this specific commit reflecting the work status at the end of the blog post.

Next steps: TODO: allow for non-boolean-like variables

Idea:

  • re-write the code that methods like SystemSetting.what_is_the_weather_today? with a question mark are handled as booleans.
  • handle methods without question mark (e.g. SystemSetting.weather_report) as strings.
  • convert such string values to a numbers, it the string values are convertible.
1

Rails Active Admin Installation Experiences / HowTo


In this post, I have recorded my experiences with ActiveAdmin, a Rails Engine that has following goals (copied from their readme file):

  1. Enable developers to quickly create good-looking administration interfaces.
  2. Build a DSL for developers and an interface for businesses.
  3. Ensure that developers can easily customize every nook and cranny.

The path to success is described step by step

while the troubleshooting and resolution steps you should not need are indented to the right and kept as reference only.

I am applying Active Admin to my ProvisioningEngine Frontend proof of concept, because I want to move some configurations from a figaro based config file to the database, and I want to make sure that only authorized administrators can change those settings. Moreover, Active Admin offers nice tables with pagination, sorting and filtering. And an already present model can be migrated to active Admin simply by a single rails g command (not part of this blog post).

I am closely following the officially installation documentation on http://activeadmin.info/docs/0-installation.html. However, because I have run into a rails generator bug (falsely reported as activeadmin issue#4295), I had to roll back (git stash) my changes, which in turn did not restore the database (is ignored with git) and that had caused additional challenges. But first things first:

Step 1: install activeadmin

For installation of Active, we need to add following lines to the Gemfile

#Gemfile
gem 'devise'
gem 'activeadmin', '1.0.0.pre2'

Devise is a requirement, and is not automatically installed as dependency. The activeadmin version statement '1.0.0.pre2' is important as well: I was running into several additional issues, when I first tried without version statement, and later with '1.0.0.pre1'. Only '1.0.0.pre2' worked, apart from issue#4295, but this is an issue that is unlikely to hit.
Then we perform:

bundle

Step 2: generate the base admin user model and routing

The next command

rails generate active_admin:install

should add the following files (among others):

app/admin/dashboard.rb
app/assets/javascripts/active_admin.js.coffee
app/assets/stylesheets/active_admin.css.scss
config/initializers/active_admin.rb

and add routes to the existing config/routes.rb file.

Note: it did not work in my case, see activeadmin issue#4295). However, this was caused  by following statement at the end of my  config/routes.rb file:

# config/routes.rb
Rails.application.routes.draw do
...(many statements)...
end # Rails.application.routes.draw do

I often insert such comments after end statements to better see, what this is the end of. In this case, this comment has caused the rails generator to insert ActiveAdmin statments twice, like follows:

Rails.application.routes.draw do
 
 devise_for :admin_users, ActiveAdmin::Devise.config
 ActiveAdmin.routes(self)
 ...(many statements)...
end # Rails.application.routes.draw do
 
 devise_for :admin_users, ActiveAdmin::Devise.config
 ActiveAdmin.routes(self)

It seem to be looking for a pattern similar to /Rails.application.routes.draw do$/ and inserts the three lines thereafter. It does not recognize that it should ignore the second occurence of the pattern, since it is a comment only.

Okay, I do not recommend adding comments like # Rails.application.routes.draw do after the end statment in the routes.rb file for now.

After rolling back with git stash, removing the empty lines after the end statement and starting from scratch, the problem disappeared. However, through the rollback process, I have introduced (minor) other problems, as we will see.

 

Step 3: prepare the database and start the Web Server

Now, we migrate our database and start the server:

rake db:migrate
rails server

Note: In my case, since I had to roll back and the database changes are ignored and thus not rolled back properly, the rake db:migrate command did not work anymore: the corresponding object(s) had already been created in the database the first attempt.

To resolve this, I had done it the hard way and removed db/development.sqlite3 file and tried to  rake db:migrate again. That worked, seemingly.

Then we can visit http://localhost:3000/admin and log in as the default user (in my case, I was using a cloud9 URL instead, since I recently moved part of my development to that online IDE). The initial credentials are:

  • User: admin@example.com
  • Password: password

I guess, because of the fact that I had to reset the database, I ran into an error “invalid email or password”. In my case, the following workaround has worked fine:

$ rails console
AdminUser.create :email => 'admin@example.com', :password => 'password', :password_confirmation => 'password'

After successful login, we see following Dashboard:

2016-02-02_012243_capture_003

Finally, ActiveAdmin is installed properly.

thumps_up_3

Installation Summary

I have shown step by step how ActiveAdmin can be installed. If you circumvent all the trip hazards I have run into, it can be done in 30 minutes or less.

In my case, the installation process was very much like trial and error. First, you need to add the ‘devise’ dependency not described in the official installation guide. Then, the right version has to be picked; the pre-release: '1.0.0.pre2'. Unfortunately I ran into a rails bug, but it is unlikely to run into the same one. Everybody has the right of his own bugs ;-).

The rollback process I was enforced to perform was incomplete, since the database is not saved in the repository. This has caused additional minor problems, but those could be worked around with the help of google and stackoverflow.

Happy Coding!

Next post: Active Admin: from config File to Admin Portal

0

Upgrading Rails to 4.2.x and Ruby to 2.2.x — an end to end Example


In this post, I have recorded my experiences when upgrading my hobby Web Application, a ProvisioningEngine front end from Rails 4.1.x to Rails 4.2.x and ruby from 2.1.x to 2.2.x. The upgrade went relatively smoothly, apart from an additional gem needed (‘test-unit’) and a test failure due to changes in the Rails Url helpers.

v1: 2016-01-26: original post
v2: 2016-02-13: added a caveats section at the end (just before the summary) and added two rspec caveats with respect to the exception handling

Upgrading Rails and Ruby

Prerequisites

  • In this post, I assume that rbenv is installed. If you need to install rbenv, find the instructions here. Other options are possible (e.g. RVM), but they are out of scope for this blog post.

Preparation steps:

  • Better do not upgrade unless you have good coverage of automated tests of your application which all pass.
    • In my case, I have a small app with about ~200 rspec tests and an in-built mock that allows me to perform the tests independently of the backend systems I integrate with. Perform the tests after each major step.
  • Create a backup of your system.

Upgrade steps:

  • (recommended) Update rails to latest minor version and perform the automated tests. In my case I have updated rails from 4.1.4 to 4.1.14, and check that the tests are still O.K.
  • (optional, depending on the current ruby version and your target rails version) Upgrade ruby to the latest stable version and perform the automated tests. In my case, I have updated ruby from 2.1.3 to 2.2.4. Since our target rails version 4.2.x supports older versions of ruby, this step was optional. However, it will be mandatory later on, when we upgrade rails to 5.0, so I have performed it now.
  • (mandatory) Now we can upgrade rails to 4.2.x (you can find all rails versions here). I have decided to specify version >~ 4.2.4 in the Gemfile and we will see that the latest available 4.2.x version is fetched (4.2.5.1). Also, it is not clear yet, whether this version is supported by travisCI.
  • (mandatory) resolve issues and handle deprecation warnings

Cleanup steps:

  • (mandatory) save and commit changes to the SW repository
  • (mandatory) integration tests and merge to master

Step 0: take backup and perform automated tests:

Perform your automated tests and confirm that they are successful. In my case (I am using rspec), this means:

$ bundle exec rspec -f d spec/requests/provisioningobjects_spec.rb
...
Finished in 40.04 seconds
196 examples, 0 failures

I am a lazy guy, so I have created a VMware Snapshot in stead of performing a backup, which helps me to go back within two minutes. Once the upgrade has proven to be successful, I will remove the snapshot in order to retain the full performance. In addition, I have made sure the SW is committed and pushed to my GitHub repository.

Step 1: update rails to the latest stable minor version:

If you are using git, I recommend to create a new branch, e.g.:

$ git checkout -b upgrade_branch

You may need to save your work before with git add/git commit or save away current changes for later use with git stash.

In the application’s root folder, edit the Gemfile and change the rails version (in my case from 4.1.4 to 4.1.14):

# Gemfile
gem 'rails', '4.1.14'

Now repeat the automated tests and check the results. If successful, commit and save your work e.g. with git:

git add .; git commit -am "updated rails 4.1.4 -> 4.1.14"; git push

Step 2: update ruby

Step 2.1 update rbenv, so you get the full list of ruby versions:

cd ~/.rbenv/
git pull
rbenv install --list

Step 2.2: install new ruby version

Install an available ruby version:

rbenv install 2.2.4
ruby -v

The last command will still show the old version. We still need to tell rbenv to use the new version. Since I am running a productive rails server on the host, I do not want to change the global setting, so I am specifying an app-local version, which will be written into .ruby-version:

rbenv versions
rbenv local 2.2.4
rbenv versions
ruby -v

Now ruby is updated.

Step 2.3: re-install rails and other gems

For this version of ruby, rails and the other gems are not yet installed. Therefore the following error message is expected:

$ rails -v
rbenv: rails: command not found

The `rails' command exists in these Ruby versions:
 2.1.3

Since I have a working project with a Gemfile, we can install bundle first and install rails by performing a bundle update within the project’s root folder:

gem install bundle
bundle update

The latter command will install all gems specified in the Gemfile, inclulding the rails gem. In my case, I had following output:

provisioningengine@ProvisioningEngine:~/ProvisioningEnginev0.5.15_testfolder$ gem install bundle
Fetching: bundler-1.11.2.gem (100%)
Successfully installed bundler-1.11.2
Fetching: bundle-0.0.1.gem (100%)
Successfully installed bundle-0.0.1
Parsing documentation for bundler-1.11.2
Installing ri documentation for bundler-1.11.2
Parsing documentation for bundle-0.0.1
Installing ri documentation for bundle-0.0.1
Done installing documentation for bundler, bundle after 6 seconds
2 gems installed
provisioningengine@ProvisioningEngine:~/ProvisioningEnginev0.5.15_testfolder$ bundle install
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Installing rake 10.5.0
Installing i18n 0.7.0
Installing json 1.8.3 with native extensions
Installing minitest 5.8.4
Installing thread_safe 0.3.5
Installing builder 3.2.2
Installing erubis 2.7.0
Installing rack 1.5.5
Installing mime-types 2.99
Installing arel 5.0.1.20140414130214
Installing addressable 2.4.0
Installing execjs 2.6.0
Installing sass 3.2.19
Installing mini_portile2 2.0.0
Installing ffi 1.9.10 with native extensions
Installing choice 0.2.0
Installing coffee-script-source 1.10.0
Installing thor 0.19.1
Installing cookiejar 0.3.0
Installing daemons 1.2.3
Installing tilt 1.4.1
Installing diff-lcs 1.2.5
Installing eventmachine 1.0.9.1 with native extensions
Installing http_parser.rb 0.6.0 with native extensions
Installing hike 1.2.3
Installing multi_json 1.11.2
Installing pg 0.15.1 with native extensions
Using bundler 1.11.2
Installing ruby-graphviz 1.2.2
Installing rails_serve_static_assets 0.0.4
Installing rails_stdout_logging 0.0.4
Installing rspec-core 2.13.1
Installing rspec-mocks 2.13.1
Installing rubyzip 0.9.9
Installing websocket 1.0.7
Installing spork 1.0.0rc4
Installing sqlite3 1.3.11 with native extensions
Installing rdoc 4.2.1
Installing tzinfo 1.2.2
Installing rack-test 0.6.3
Installing rack-protection 1.5.3
Installing mail 2.6.3
Installing autoprefixer-rails 6.3.1
Installing uglifier 2.7.2
Installing nokogiri 1.6.7.2 with native extensions
Installing childprocess 0.5.9
Installing coffee-script 2.4.1
Installing figaro 1.1.1
Installing rspec-expectations 2.13.0
Installing em-socksify 0.3.1
Installing sprockets 2.12.4
Installing rails_12factor 0.0.2
Installing sdoc 0.4.1
Installing activesupport 4.1.14
Installing sinatra 1.4.7
Installing bootstrap-sass 3.3.5
Installing xpath 2.0.0
Installing selenium-webdriver 2.35.1
Installing em-http-request 1.1.3
Installing actionview 4.1.14
Installing activemodel 4.1.14
Installing delayed_job 4.1.1
Installing factory_girl 4.2.0
Installing jbuilder 2.4.0
Installing capybara 2.1.0
Installing actionpack 4.1.14
Installing activerecord 4.1.14
Installing actionmailer 4.1.14
Installing railties 4.1.14
Installing kaminari 0.16.3
Installing sprockets-rails 2.3.3
Installing delayed_job_active_record 4.1.0
Installing delayed_job_web 1.2.5
Installing rails-erd 1.4.5
Installing seed_dump 3.2.4
Installing coffee-rails 4.0.1
Installing factory_girl_rails 4.2.0
Installing jquery-rails 3.1.4
Installing respond-rails 1.0.1
Installing rspec-rails 2.13.1
Installing rails 4.1.14
Installing sass-rails 4.0.5
Installing turbolinks 2.5.3
Installing spork-rails 4.0.0
Installing jquery-turbolinks 2.1.0
Bundle complete! 30 Gemfile dependencies, 85 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
  Post-install message from ruby-graphviz:

Depending on your version of ruby, you may need to install ruby rdoc/ri data:

<= 1.8.6 : unsupported
 = 1.8.7 : gem install rdoc-data; rdoc-data --install
 = 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!

Post-install message from capybara:
IMPORTANT! Some of the defaults have changed in Capybara 2.1. If you're experiencing failures,
please revert to the old behaviour by setting:

    Capybara.configure do |config|
      config.match = :one
      config.exact_options = true
      config.ignore_hidden_elements = true
      config.visible_text_only = true
    end

If you're migrating from Capybara 1.x, try:

    Capybara.configure do |config|
      config.match = :prefer_exact
      config.ignore_hidden_elements = false
    end

Details here: http://www.elabs.se/blog/60-introducing-capybara-2-1

Step 2.4: check ruby installation

The I tried to perform the automated tests, but there are some missing dependencies:

$ bundle exec rspec -f d spec/requests/provisioningobjects_spec.rb
/home/provisioningengine/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:in `require': cannot load such file -- test/unit/assertions (LoadError)
        from /home/provisioningengine/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:in `block in require'

8-((

What the heck is going on here?

Following the hints in the issue descriptions here and here, I have added the gem ‘test-unit’ to the test group of the Gemfile:

# Gemfile
group :test do
  # ...
  gem 'minitest'
  gem 'test-unit'
end

and performed

bundle install

This has done the job:

$ bundle exec rspec -f d spec/requests/provisioningobjects_spec.rb
...(lots of output and debugs omitted here)...
Finished in 40.04 seconds
196 examples, 0 failures

The procedure might also work without the gem ‘minitest’. In my case it was already installed, so I have not tried without.

Now is a good time to commit the change in git:

$ git add.; git commit -a

And I have specified:

Updated ruby to 2.2.4, re-installed all gems; rails v4.1.14

all 196 tests successful in simulation mode

Step 3: upgrade rails to v 4.2.x

In the Gemfile, I have exchanged the rails line by:

# Gemfile
gem 'rails', '~> 4.2.4'

Then:

$ bundle update
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies............................................................................
Using rake 10.5.0
Using i18n 0.7.0
Using json 1.8.3
Using minitest 5.8.4
Using thread_safe 0.3.5
Using builder 3.2.2
Using erubis 2.7.0
Using mini_portile2 2.0.0
Installing rack 1.6.4 (was 1.5.5)
Using mime-types 2.99
Installing arel 6.0.3 (was 5.0.1.20140414130214)
Using addressable 2.4.0
Using execjs 2.6.0
Using sass 3.2.19
Using bundler 1.11.2
Using ffi 1.9.10
Using choice 0.2.0
Using coffee-script-source 1.10.0
Using thor 0.19.1
Using cookiejar 0.3.0
Using daemons 1.2.3
Using tilt 1.4.1
Using diff-lcs 1.2.5
Using eventmachine 1.0.9.1
Using http_parser.rb 0.6.0
Using hike 1.2.3
Using multi_json 1.11.2
Using pg 0.15.1
Installing power_assert 0.2.7 (was 0.2.2)
Using ruby-graphviz 1.2.2
Using rails_serve_static_assets 0.0.4
Using rails_stdout_logging 0.0.4
Using rspec-core 2.13.1
Using rspec-mocks 2.13.1
Using rubyzip 0.9.9
Using websocket 1.0.7
Using spork 1.0.0rc4
Using sqlite3 1.3.11
Using rdoc 4.2.1
Using tzinfo 1.2.2
Using nokogiri 1.6.7.2
Using rack-test 0.6.3
Using rack-protection 1.5.3
Using mail 2.6.3
Using autoprefixer-rails 6.3.1
Using uglifier 2.7.2
Using childprocess 0.5.9
Using coffee-script 2.4.1
Using figaro 1.1.1
Using rspec-expectations 2.13.0
Using em-socksify 0.3.1
Using sprockets 2.12.4
Installing test-unit 3.1.7 (was 3.0.8)
Using rails_12factor 0.0.2
Using sdoc 0.4.1
Installing activesupport 4.2.5.1 (was 4.1.14)
Installing loofah 2.0.3
Using xpath 2.0.0
Using sinatra 1.4.7
Using bootstrap-sass 3.3.5
Using selenium-webdriver 2.35.1
Using em-http-request 1.1.3
Installing rails-deprecated_sanitizer 1.0.3
Installing globalid 0.3.6
Installing activemodel 4.2.5.1 (was 4.1.14)
Using delayed_job 4.1.1
Using factory_girl 4.2.0
Using jbuilder 2.4.0
Installing rails-html-sanitizer 1.0.3
Using capybara 2.1.0
Installing rails-dom-testing 1.0.7
Installing activejob 4.2.5.1
Installing activerecord 4.2.5.1 (was 4.1.14)
Installing actionview 4.2.5.1 (was 4.1.14)
Using delayed_job_active_record 4.1.0
Using delayed_job_web 1.2.5
Using rails-erd 1.4.5
Using seed_dump 3.2.4
Installing actionpack 4.2.5.1 (was 4.1.14)
Installing actionmailer 4.2.5.1 (was 4.1.14)
Installing railties 4.2.5.1 (was 4.1.14)
Using kaminari 0.16.3
Using sprockets-rails 2.3.3
Using coffee-rails 4.0.1
Using factory_girl_rails 4.2.0
Installing jquery-rails 4.1.0 (was 3.1.4)
Using respond-rails 1.0.1
Using rspec-rails 2.13.1
Installing rails 4.2.5.1 (was 4.1.14)
Using sass-rails 4.0.5
Using turbolinks 2.5.3
Using spork-rails 4.0.0
Using jquery-turbolinks 2.1.0
Bundle updated!

We can see in the log that rails has been upgraded to the latest available 4.2 version at time of writing (4.2.5.1). Now let us test again. This time we get lots of DEPRECATION WARNINGs and an error:

bundle exec rspec -f d spec/requests/provisioningobjects_spec.rb
...
DEPRECATION WARNING: Calling URL helpers with string keys controller, action is deprecated. Use symbols instead. (called from _app_views_customers__sidebar_html_erb__545434719134816655_70018526941280 at /home/provisioningengine/ProvisioningEnginev0.5.15_testfolder/app/views/customers/_sidebar.html.erb:8)
 should have link to 'New Customer' (FAILED - 1)
Failures:

  1) On target solution 'Environment2_V7R1' index should have link to 'New Customer'
     Failure/Error: expect(page).to have_link( "New #{obj}", href: new_provisioningobject_path(obj) + '?per_page=all') #unless obj == "Site"
       expected #has_link?("New Customer", {:href=>"/customers/new?per_page=all"}) to return true, got false
     # ./spec/requests/provisioningobjects_spec.rb:1246:in `block (5 levels) in <top (required)>'

Finished in 17.13 seconds
68 examples, 1 failure

Failed examples:

rspec ./spec/requests/provisioningobjects_spec.rb:1241 # On target solution 'Environment2_V7R1' index should have link to 'New Customer'

Step 4.1: resolving the introduced Url-Helper issue

Okay, there is more work to be done. Maybe I should have tried with ‘bundle install’ first, not with ‘bundle update’, which was updating many gems at once? We could go back with a git stash and try again with bundle install. However, let us quickly see, if there is a quick solution:

The DEPRECATION WARNING is related with the problem. It points to

/home/provisioningengine/ProvisioningEnginev0.5.15_testfolder/app/views/customers/_sidebar.html.erb:8

and there we find:

<li><%= link_to 'New Customer', new_customer_path(@params) %></li>

That is causing the errored test:

expected #has_link?("New Customer" ...

So let us try to understand the DEPRECATION WARNING. By guessing from the error message, I have replaced the @params assignment in

class ProvisioningobjectsController < ApplicationController
def index
  @params = params

by

  @params = {"per_page"=>"all", "controller"=>"customers", "action"=>"index", "target_id"=>nil}

after I had seen that a puts @params.inspect had returned {“per_page”=>...}. Surprisingly, after this change the test was successful with the the DEPRECATION WARNING still present. The reasons seems to be that params is an ActionController::Parameters while the literal {...} is a Hash.

The remaining warning was removed by replacing the keys by symbols:

  @params = {:per_page=>"all", :controller=>"customers", :action=>"index", :target_id=>nil}

Since I do not want to care about the actual parameters, I replaced this by the more general code:

class ProvisioningobjectsController < ApplicationController
def index
  @params = {}
  params.each do |key,value|
     @params[key.parameterize.underscore.to_sym] = value
  end

This will convert any ActionController::Parameters parameters by a Hash with symbol keys.

Even more surprisingly, now all 196 test cases are successful:

$ bundle exec rspec -f d spec/requests/provisioningobjects_spec.rb
...
Finished in 42.56 seconds
196 examples, 0 failures

Step 4.2: resolving DEPRECATION WARNING: You are passing an instance of ActiveRecord::Base to `find`

During the test, we have seen other deprecation warnings like follows:

DEPRECATION WARNING: You are passing an instance of ActiveRecord::Base to `find`. Please pass the id of the object by calling `.id`. (called from validate at /home/provisioningengine/ProvisioningEnginev0.5.15_testfolder/app/models/user.rb:15)

Let us resolve this as well. The waring is quite explicit and points to a line containing:

@site = Site.find(record.site)

This can easily be resolved by exchanging it by:

@site = Site.find(record.site.id)

Done and works.

Step 4.3: resolving DEPRECATION WARNING: renamed configuration option

Now let us tackle the last remaining deprecation warning:

DEPRECATION WARNING: The configuration option `config.serve_static_assets` has been renamed to `config.serve_static_files` to clarify its role (it merely enables serving everything in the `public` folder and is unrelated to the asset pipeline). The `serve_static_assets` alias will be removed in Rails 5.0. Please migrate your configuration files accordingly. (called from block in <top (required)> at /home/provisioningengine/ProvisioningEnginev0.5.15_testfolder/config/environments/test.rb:16)

We just need to exchange the line 16 in config/environments/test.rb

config.serve_static_assets  = true

by

config.serve_static_files  = true

and we are done with all deprecation warnings.

Step 5: cleanup: save changes, additional test with travisCI and remove snapshot

Finally, this is a good time to commit the change to git and to merge the branch to the original branch (“development” branch) and push it to GitHub. In my case, this has triggered an additional test on travisCI (if you have no access, try travisCI home).

2016.01.26-23_59_14-hc_001

Success.

Once real live tests with real provisioned legacy end systems have proven to be successful as well, we will merge the development branch to the master branch and remove the VMware snapshot we had taken before we started the upgrade.

Caveats

  • In the new rspec version (rspec-core: 3.4.2), the exception handling seems to have changed: if there is an exception/abort in the rspec script or in the called code, the rspec just stops, but it does not indicate an error. A short google session did not lead to any results. I need to find a solution or a workaround for this.
  • rspec: begin/rescue block does not work as (I had) expected: if you write a begin/rescue block like
begin
  something_raising_an_exception
rescue
  some_other_code
end

then the rescue is ignored, the abort message is printed and the rspec test suite is stopped.

Summary

For my hobby Web Application, a ProvisioningEngine front end,  I could upgrade Rails and Ruby relatively smoothly within an afternoon+evening session (including the documentation on my blog). There only was a missing dependency ‘test-unit’ and a changed behavior of the url helpers, which has forced me to convert the Url helper parameter from ActionController::Parameters class ) to Hash {...} class with the same content. In addition, the Hash keys had to be converted from "String" to :symbol in order to get rid of the corresponding DEPRECATION WARNING. After that, all ~200 rspec tests were successful.

3

Docker Performance Tests for Ruby on Rails


Abstract

In a series of performance tests, we could show that a (Rails) web application has 50% higher performance on docker (installed on Windows/boot2docker) than the same application run natively on Windows, no matter whether or not the database and program code is located on a shared volume. However, if such a shared volume is auto-mounted to one of the Windows C:\Users folders, the performance drops by a factor of almost ten.

Preface

This is the 5th blog of my Dummy’s Diary on Docker (= DDDocker), which originally was hosted on LinkedIn. See the DDDocker table of contents to find more.

Introduction

What is the performance of a dockerized application compared to the same application running natively on Windows or Linux or on a Windows or Linux VM? This is, what I have asked myself when starting with Docker. However, my Internet search on that subject has not given me the desired results. I cannot believe that there are no such articles in the Internet, and if you happen to find some, please point them to me. In any case, I have decided to perform some quick&dirty tests myself. I call them “quick&dirty”, because I did not care to separate the measurement tool from the server: everything is performed on my Windows Laptop. Also, I did not care to document the exact test setup (Rails version, Laptop data sheet, etc.). For now, I have deemed it sufficient to get an idea on the trend, rather more than the absolute numbers.

I have received some unexpected results…

Test Scenarios

In DDDocker (4): Persistence via shared Volumes (sorry, this is a LinkedIn blog, which has a limited lifetime; tell me, if you need access) we have experienced a major decrease of performance of a web server, when the server’s code and data base is located on a shared volume on a Windows host.

In the current blog, we perform performance measurements of a Rails web server for following scenarios:

  1. web server run natively on Windows
  2. web server run within a docker container with all source code and database in the container. The docker container runs on a VirtualBox VM with 2 GB DRAM (boot2docker image).
  3. web server run within a docker container with all source code and and database in a shared volume. The docker container runs on a VirtualBox VM with 2 GB DRAM (boot2docker VM). The shared volume is located on the Linux VM (docker host, i.e. the boot2docker VM).
  4. web server run within a docker container with all source code and and database in a shared volume. The docker container runs on a VirtualBox VM with 2 GB DRAM (boot2docker image). The shared volume is located on the C: partition on Windows and is mapped to the VM and from there it is mapped to the docker container.

The quick and dirty tests are performed using Apache Bench tool “ab.exe” on my Windows Laptop, which is also hosting the Web Server and/or the boot2docker host. This is not a scientific setup, but we still will get meaningful results, since we are interested in relative performance numbers only.

1. Windows Scenario

In this scenario the rails application runs natively on Windows. All Ruby on Rails code as well as the local database reside on a local Windows folder.

2. Docker on Linux VM Scenario

In this scenario, Docker is installed on Windows like specified on the Docker documentation. A Linux VM is running on Windows and hosts the container. All Ruby on Rails code as well as the local database reside on a folder within the container.

3. Docker on Linux VM Scenario with shared Volume

The difference between this scenario and scenario 2 is that all Ruby on Rails code as well as the local database reside on a folder in the Linux VM and are mounted as so-called virtual volume into the container.

4. Docker on Linux VM Scenario with shared Volume mounted on Windows

The difference between this scenario and scenario 3 is that all Ruby on Rails code as well as the local database reside on a Windows folder on C:\Users\. The rails executable within the container is accessing a local folder which maps to the Linux’s hosts folder via the shared volume feature, which in turn maps to the auto-mounted Windows folder on C:\Users\<username>\<foldername>. Because of the extra-hops, the lowest performance is expected here. We will see below that this performance impact for this scenario is dramatic.

Test Environment / Equipment

All tests will be run on a Windows 7 Dell i5 notebook with 8 GB of RAM. During the native Windows test, the docker VM will run in parallel, so the CPU and RAM resources available for the app are similar in all 4 tests.

As the performance test tool, Apache Bench (ab.exe) is used, which ships with XAMPP installer (xampp-win32-5.6.8-0-VC11-installer.exe).

We perform 1000 read requests, with a concurrency factor 100. Each request will cause a SQL query and returns ~245 kB text.

For the tests, I have intentionally chosen an operation that is quite slow: a GET command that searches and reads 240 kB text from an SQLite database on an (already overloaded) laptop.

The total performance numbers are not meaningful for other web applications, different databases etc. However, the relative numbers are meaningful for the purpose to compare the different setups.

Results

From the overhead point of view, I would have expected a performance relation like follows:

Expectation:

  1. native Windows performance is greater than
  2. Docker performance on Linux VM on Windows is greater than 
  3. Docker performance on Linux VM on Windows with shared volume is greater than
  4. Docker performance on Linux VM on Windows with shared volume auto-mounted on Windows

However, I have found the surprising results that native Windows performance ~33% lower than Docker performance:

Finding 1: Rails on docker outperforms native Rails on Windows by 50%, even if the Docker Linux host is a VM, which is hosted on Windows.

In addition, we have seen that:

Finding 2: the performance of a dockerized (Rails) web application is not degraded by locating the database and (Rails) program code on a shared volume.

The next finding was already expected from our experience in the last blog:

Finding 3: the performance of a dockerized (Rails) web application drops dramatically (by factor of ~10), if the shared volume is auto-mounted on a C:\Users folder.

The last statement was expected from the DDDocker (4) blog. Here the results in detail:

This graph shows the number of GET requests per second the web portal can handle. Docker outperforms Windows by ~50%, no matter whether docker is run with local folders or with shared volumes pointing to the Linux host. The performance drops by the factor of ~9, if the shared folder is used to access an auto-mounted Windows folder.

The same picture is seen with respect to the throughput, since the transfer rate is proportional to the requests/sec rate:

A corresponding inverse picture is seen with respect to the average time an end user has to wait for each request: because of the overload situation with 100 parallel requests, each requests takes 5.88, 3.89 and 34.59 (!) seconds for the application running on Windows, on a docker container within the VirtualBox VM and on a docker image again, but with all data located on a shared Volume.

Since the shared volume is located on Windows and is auto-mounted to the VM, and then is mounted to the container, there is no surprise that the performance is lower. A response time increase of a factor 8.9 is still an unexpected high factor you have to take into consideration when working with shared volumes on Windows machines.

Appendix

Preparation of Scenario 3

The preparation of scenarios 1, 2 and 4 have been described in the last blog already. Here is a description on how I have built scenario 3:

We need to copy the folder from Windows into the Linux VM. This way the performance test would also have relevance for docker users that are running docker on Linux. Let us see:

(Linux VM)# sudo cp -R /c/Users/vo062111/dockersharedfolder2 /home/rails/

will copy the auto-mounted volume to /home/rails/dockersharedfolder2 inside the VM. Now let us start a container, which maps to this folder instead of the /c/Users/vo062111/dockersharedfolder2, which is volume that is mounted from Windows.

docker@boot2docker:~$ JOB=$(docker run -d -p 8080:3000 -v /home/rails/dockersharedfolder2:/home/rails/ProvisioningEngine oveits/rails_provisioningengine:latest /bin/bash -c "cd /home/rails/ProvisioningEngine; rails s")

With that we get rid of the auto-mount-hop, when we compare it to scenaro 4.

Scenario 1: Test Protocol Rails Server on native Windows

bash-3.1$ ./ab -n 1000 -c 100 "http://127.0.0.1:3000/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 127.0.0.1
 Server Port: 300
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 58.774 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 17.01 [#/sec] (mean)
 Time per request: 5877.425 [ms] (mean)
 Time per request: 58.774 [ms] (mean, across all concurrent requests)
 Transfer rate: 4077.01 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 0 0 0.3 1 4
 Processing: 84 5630 1067.3 5723 12428
 Waiting: 79 5623 1067.7 5718 12424
 Total: 85 5630 1067.3 5723 12428
 ERROR: The median and mean for the initial connection time are more than twice the standard
 deviation apart. These results are NOT reliable.
Percentage of the requests served within a certain time (ms)
50% 5723
66% 6072
75% 6156
80% 6205
90% 6418
95% 6496
98% 6549
99% 6566
100% 12428 (longest request)
 bash-3.1$ ./ab -n 1000 -c 100 "http://127.0.0.1:3000/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 127.0.0.1
 Server Port: 3000
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 58.761 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 17.02 [#/sec] (mean)
 Time per request: 5876.075 [ms] (mean)
 Time per request: 58.761 [ms] (mean, across all concurrent requests)
 Transfer rate: 4077.94 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 0 0 0.3 1 4
 Processing: 55 5590 1117.4 5779 11605
 Waiting: 51 5584 1117.5 5773 11601
 Total: 56 5590 1117.5 5780 11605
 ERROR: The median and mean for the initial connection time are more than twice the standard
 deviation apart. These results are NOT reliable.
Percentage of the requests served within a certain time (ms)
 50% 5780
 66% 5929
 75% 5999
 80% 6052
 90% 6173
 95% 6276
 98% 6411
 99% 6443
 100% 11605 (longest request)
 bash-3.1$

Server Logs

...
Started GET "/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config" for 127.0.0.1 at 2015-07-09 08:57:43 +0200
 Processing by TextDocumentsController#index as TEXT
 Parameters: {"filter"=>{"systemType"=>"OSV_V7R1", "action"=>"Add Customer", "templateType"=>"config"}}
 TextDocument Load (0.5ms) SELECT "text_documents".* FROM "text_documents"
 Rendered text_documents/index.text.erb (0.5ms)
 Completed 200 OK in 18ms (Views: 15.0ms | ActiveRecord: 0.5ms)
...

Summary

  • 15.4 requests/sec and 6.5 sec/requests at 100 concurrent requests and a total of 1000 requests with ~245 kB answer per request.

Open Issue:

  • ab.exe tells us that the responses are non-2xx, but the server log shows 200 OK.

2) Test Protocol Rails Server on Docker

bash-3.1$ ./ab -n 1000 -c 100 "http://192.168.56.101:8080/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.56.101 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 192.168.56.101
 Server Port: 8080
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 32.550 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 30.72 [#/sec] (mean)
 Time per request: 3254.951 [ms] (mean)
 Time per request: 32.550 [ms] (mean, across all concurrent requests)
 Transfer rate: 7361.80 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 1 4 3.1 3 28
 Processing: 66 3103 697.2 3186 6725
 Waiting: 22 3049 694.9 3135 6665
 Total: 73 3107 697.3 3190 6728
Percentage of the requests served within a certain time (ms)
 50% 3190
 66% 3245
 75% 3297
 80% 3349
 90% 3693
 95% 3783
 98% 3888
 99% 6004
 100% 6728 (longest request)
 bash-3.1$

 bash-3.1$ ./ab -n 1000 -c 100 "http://192.168.56.101:8080/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.56.101 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 192.168.56.101
 Server Port: 8080
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 45.330 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 22.06 [#/sec] (mean)
 Time per request: 4533.006 [ms] (mean)
 Time per request: 45.330 [ms] (mean, across all concurrent requests)
 Transfer rate: 5286.18 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 1 6 8.0 4 127
 Processing: 76 4290 1588.3 3969 14826
 Waiting: 29 4210 1587.7 3901 14647
 Total: 84 4296 1589.0 3976 14846
Percentage of the requests served within a certain time (ms)
 50% 3976
 66% 4130
 75% 4530
 80% 5083
 90% 5699
 95% 7520
 98% 9635
 99% 10546
 100% 14846 (longest request)
 bash-3.1$

Summary

  • 26 requests/sec and 3,9 sec/requests at 100 concurrent requests and a total of 1000 requests with ~245 kB answer per request.

Open Issue:

  • ab.exe tells us that the responses are non-2xx, but the server log shows 200 OK.

3) Test Protocol Rails Server on Docker with Source Code and DB on a shared Volume residing on the Linux VM

Docker command:

docker@boot2docker:~$ JOB=$(docker run -d -p 8080:3000 -v /c/Users/vo062111/dockersharedfolder2:/home/rails/ProvisioningEngine oveits/rails_provisioningengine:latest /bin/bash -c "cd /home/rails/ProvisioningEngine; rails s")

ab.exe log Test 1

bash-3.1$ ./ab -n 1000 -c 100 "http://192.168.56.101:8080/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.56.101 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 192.168.56.101
 Server Port: 8080
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 34.173 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 29.26 [#/sec] (mean)
 Time per request: 3417.283 [ms] (mean)
 Time per request: 34.173 [ms] (mean, across all concurrent requests)
 Transfer rate: 7012.09 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 1 4 4.4 3 65
 Processing: 103 3232 726.1 3303 7013
 Waiting: 58 3164 725.8 3235 6900
 Total: 107 3236 726.1 3308 7015
Percentage of the requests served within a certain time (ms)
 50% 3308
 66% 3448
 75% 3552
 80% 3619
 90% 3768
 95% 3948
 98% 4148
 99% 4346
 100% 7015 (longest request)

ab.exe log Test 2

bash-3.1$ ./ab -n 1000 -c 100 "http://192.168.56.101:8080/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.56.101 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 192.168.56.101
 Server Port: 8080
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 36.615 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 27.31 [#/sec] (mean)
 Time per request: 3661.482 [ms] (mean)
 Time per request: 36.615 [ms] (mean, across all concurrent requests)
 Transfer rate: 6544.43 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 1 4 4.1 3 30
 Processing: 83 3458 1077.6 3531 8051
 Waiting: 38 3378 1070.1 3452 7886
 Total: 84 3462 1077.7 3532 8055
Percentage of the requests served within a certain time (ms)
 50% 3532
 66% 3776
 75% 3895
 80% 3985
 90% 4430
 95% 4680
 98% 6623
 99% 7259
 100% 8055 (longest request)
 bash-3.1$

4) Test Protocol Rails Server on Docker with Source Code and DB on a shared Volume residing on Windows

Docker command:

docker@boot2docker:~$ JOB=$(docker run -d -p 8080:3000 -v /c/Users/vo062111/dockersharedfolder2:/home/rails/ProvisioningEngine oveits/rails_provisioningengine:latest /bin/bash -c "cd /home/rails/ProvisioningEngine; rails s")

ab.exe log Test 1

bash-3.1$ ./ab -n 1000 -c 100 "http://192.168.56.101:8080/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.56.101 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 192.168.56.101
 Server Port: 8080
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 348.463 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 2.87 [#/sec] (mean)
 Time per request: 34846.268 [ms] (mean)
 Time per request: 348.463 [ms] (mean, across all concurrent requests)
 Transfer rate: 687.66 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 1 3 4.8 2 112
 Processing: 598 33224 6430.7 34965 57297
 Waiting: 468 33076 6427.7 34808 57177
 Total: 600 33227 6430.6 34966 57299
Percentage of the requests served within a certain time (ms)
 50% 34966
 66% 35521
 75% 35781
 80% 35961
 90% 37069
 95% 37879
 98% 38203
 99% 38340
 100% 57299 (longest request)

ab.exe log Test 2

bash-3.1$ ./ab -n 1000 -c 100 "http://192.168.56.101:8080/text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config"
 This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.56.101 (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Completed 1000 requests
 Finished 1000 requests
 Server Software: WEBrick/1.3.1
 Server Hostname: 192.168.56.101
 Server Port: 8080
Document Path: /text_documents.txt/?filter[systemType]=OSV_V7R1&filter[action]=Add%20Customer&filter[templateType]=config
 Document Length: 244884 bytes
Concurrency Level: 100
 Time taken for tests: 343.399 seconds
 Complete requests: 1000
 Failed requests: 0
 Total transferred: 245374000 bytes
 HTML transferred: 244884000 bytes
 Requests per second: 2.91 [#/sec] (mean)
 Time per request: 34339.862 [ms] (mean)
 Time per request: 343.399 [ms] (mean, across all concurrent requests)
 Transfer rate: 697.80 [Kbytes/sec] received
Connection Times (ms)
 min mean[+/-sd] median max
 Connect: 0 2 2.2 2 23
 Processing: 639 32505 5954.2 33841 37156
 Waiting: 594 32356 5953.0 33699 37114
 Total: 643 32508 5954.2 33848 37157
Percentage of the requests served within a certain time (ms)
 50% 33848
 66% 34555
 75% 34970
 80% 35247
 90% 35872
 95% 36190
 98% 36382
 99% 36774
 100% 37157 (longest request)
 bash-3.1$

Summary

  • 2.9 requests/sec and 34.8 sec/requests at 100 concurrent requests and a total of 1000 requests with ~245 kB answer per request (i.e. 687 kB/sec)

Open Issue:

  • ab.exe tells us that the responses are non-2xx, but the server log shows 200 OK.