Micgo.net

Serverspec, Guard and Test Kitchen: Testing Servers Like a Boss

Jan 10, 2014 in GUARD • SERVERSPEC • TEST-KITCHEN • TESTING
test-kitchen guard testing serverspec
8 min read

Up until now, we have been automating testing and linting of our cookbook code without actually converging on a node. Test Kitchen has been dutifully converging a virtual machine for us as we’ve written our cookbook code, but we haven’t been validating the end product. For the last portion of this series, we’ll be covering automating Serverspec tests with Guard and Test Kitchen to cover post-convergence (or “integration”) testing.

Serverspec is a tool that allows you to write RSpec tests for your servers. These tests can be executed against most systems, regardless of whether they are configured automatically or not. There are two modes of executing tests through Serverspec, via SSH or locally. This lends itself to a fairly robust tool for pre-validating tests against existing systems as well as providing excellent integration testing for configuration code.

The primary difference between ChefSpec and Serverspec tests is when they are executed. If you recall from my earlier post, ChefSpec tests are run in memory on your local workstation. No actual convergence takes place. This makes them extremely fast and excellent for unit testing your cookbook code, however it’s not a complete validation of a fully configured system.

While Serverspec tests can be executed manually, we will be leveraging Test Kitchen to manage the execution for us. Test Kitchen is able to execute tests located in the test/integration directory structure of a cookbook for various testing frameworks. The most awesome part of this feature is that we don’t need to add the serverspec gem to our Gemfile to get it to work with our cookbook!

Test Kitchen uses the appropriate Busser plugin based on a specific directory structure. In our case, we will be placing tests in test/integration/default/serverspec to instruct Test Kitchen to execute Serverspec tests against the default suite. Test Kitchen will automatically install the busser-serverspec gem on our VM and then execute our tests after each Chef run.

###Step 1: Setup Serverspec Serverspec requires a little bit of configuration prior to execution, similar to what we did to configure ChefSpec. First, we’ll need to create the test/integration/default/serverspec directory and then add a spec_helper.rb file inside.

require 'serverspec'

include SpecInfra::Helper::Exec
include SpecInfra::Helper::DetectOS

RSpec.configure do |c|
  if ENV['ASK_SUDO_PASSWORD']
    require 'highline/import'
    c.sudo_password = ask("Enter sudo password: ") { |q| q.echo = false }
  else
    c.sudo_password = ENV['SUDO_PASSWORD']
  end
end

This file does a few things to setup and configure Serverspec. Most importantly, it includes helpers for auto-detecting the operating system and configuring a sudo password. Since we will be using a Vagrant instance, the sudo portion isn’t necessary. However, this is the default spec_helper.rb file generated by the serverspec-init command if you were to configure using the built-in toolset.

NOTE: It is important to remember that you are using Serverspec in local mode if you decide to generate your own spec_helper.rb file using the serverspec-init command. Test Kitchen transfers all tests to the instance and runs them locally, even though you would have to SSH into the VM manually.

Now that we have the basic plumbing done, we can validate that Serverspec is working properly by manually executing bundle exec kitchen verify.

$~/chef-repo/cookbooks/strider-cd: bundle exec kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Setting up <default-centos-65>...
Fetching: thor-0.18.1.gem (100%)
Fetching: busser-0.6.0.gem (100%)
       Successfully installed thor-0.18.1
       Successfully installed busser-0.6.0
       2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin serverspec installed (version 0.2.5)
-----> Running postinstall for serverspec plugin
       Finished setting up <default-centos-65> (0m54.14s).
-----> Verifying <default-centos-65>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
       Uploading /tmp/busser/suites/serverspec/localhost/mongodb_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/mongodb_spec.rb --color --format documentation
       No examples found.

       Finished in 0.00005 seconds
       0 examples, 0 failures
       Finished verifying <default-centos-65> (0m1.63s).
-----> Kitchen is finished. (0m1.93s)

NOTE: If you have already a Test Kitchen VM running, you will need to execute bundle exec kitchen converge before attempting to verify Serverspec tests. This is because the new files and directories have not been copied over to your existing VM.

Awesome! Now we need to write some tests to make this thing useful. Before we do that though, we should take a second to talk about what we should be testing.

As we discussed in the ChefSpec post, there are four major levels of testing: Unit, Integration, System and Acceptance. There are many opinions on where post-convergence testing lives, however I prefer to include them in the Integration level of testing. This is mainly due to the fact that we are validating the integration of our cookbook code with the system configuration.

Serverspec comes with many assertions that can be made against the system that may seem similar to some of the ChefSpec tests we have written. It’s important to remember when writing integration tests not to duplicate your unit tests. My personal rule of thumb is to write as much as I can in ChefSpec and anything that requires a fully converged system to validate is left to Serverspec. This post isn’t about testing methodologies though, so let’s get on with the coding.

###Step 2: Write Tests There are many things to validate in our Strider CD cookbook, but for brevity, I’ll only show one set of tests for MongoDB. Serverspec checks for a directory that matches the machine hostname that it will execute against. In our case, that will be localhost. So let’s create a mongodb_spec.rb file in the test/integration/default/serverspec/localhost directory to contain our tests. You could contain all of your tests in one file, however I prefer to break them up into the major dependencies for my cookbook.

require 'spec_helper'

describe service('mongod') do
  it { should be_enabled   }
  it { should be_running   }
end

The syntax for Serverpec tests is fairly easy to read. Here we are validating that the mongod service is running. We could have chosen to write a ChefSpec test for the service checks, however the mongodb cookbook is coming from somewhere else and the logic that starts that service may change. So it’s best to simply check for what is necessary for the application to function and leave the rest to the dependent cookbook authors to validate. No need to write tests against code we don’t control!

Let’s go ahead and manually validate our new tests.

$~/chef-repo/cookbooks/strider-cd: bundle exec kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Verifying <default-centos-65>...
       Removing /tmp/busser/suites/serverspec
       Uploading /tmp/busser/suites/serverspec/localhost/mongodb_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/mongodb_spec.rb --color --format documentation

       Service "mongod"
         should be enabled
         should be running

       Finished in 0.23318 seconds
       2 examples, 0 failures
       Finished verifying <default-centos-65> (0m1.94s).
-----> Kitchen is finished. (0m2.24s)

Good news everybody! The mongodb cookbook we are including appears to be configuring the MongoDB service correctly for our application. Now let’s see about automating our Serverspec tests with Guard.

###Step 3: Automate Serverspec Testing Notice that early we mentioned that Test Kitchen was able to manage the Serverspec gem installation for us? Well, it happily manages the automation of running Serverspec test for us as well. The following line in our kitchen guard statement watches for changes in test and then executes a kitchen verify.

  watch(%r{test/.+})

Go ahead and restart Guard if it’s not already running in the background.

~/chef-repo/cookbooks/strider-cd: bundle exec guard
14:04:52 - INFO - Guard is using GNTP to send notifications.
14:04:52 - INFO - Guard is using TerminalTitle to send notifications.
14:04:52 - INFO - Guard::Kitchen is starting
-----> Starting Kitchen (v1.1.1)
-----> Creating <default-centos-65>...
       Bringing machine 'default' up with 'virtualbox' provider...
...
14:04:59 - INFO - Guard::RSpec is running
14:04:59 - INFO - Guard is now watching at '/Users/michaelgoetz/Development/teamplatypus.net/cookbooks/strider-cd'
[1] guard(main)>

Now if we add another test to check if MongoDB is listening on the correct port, Guard should pick up the changes and re-run Test Kitchen.

require 'spec_helper'

describe service('mongod') do
  it { should be_enabled   }
  it { should be_running   }
end

describe port(27017) do
  it { should be_listening }
end

You should see something similar in your terminal that’s running Guard.

10:28:51 - INFO - Guard::Kitchen is running suites: default
-----> Starting Kitchen (v1.1.1)
-----> Converging <default-centos-65>...
       Preparing files for transfer
...
       Finished converging <default-centos-65> (0m19.91s).
-----> Setting up <default-centos-65>...
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin serverspec already installed
       Finished setting up <default-centos-65> (0m12.93s).
-----> Verifying <default-centos-65>...
       Removing /tmp/busser/suites/serverspec
       Uploading /tmp/busser/suites/serverspec/localhost/mongodb_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/mongodb_spec.rb --color --format documentation

       Service "mongod"
         should be enabled
         should be running

       Port "27017"
         should be listening

       Finished in 0.25392 seconds
       3 examples, 0 failures
       Finished verifying <default-centos-65> (0m2.01s).
-----> Kitchen is finished. (0m35.19s)

10:29:26 - INFO - Kitchen verify succeeded for: default
[1] guard(main)>

That’s it! If you have followed all of the previous posts, you should have a completely automated development environment that runs Foodcritic, ChefSpec and Serverspec tests whenever you make local changes. You can continue to write tests against your Strider CD cookbook using the other Serverspec matchers or grab the complete set here.

This blog is the fourth post in a series covering automated cookbook development for Chef. You can find links to all current posts below.

  1. Automating Cookbook Testing with Test-Kitchen, Berkshelf, Vagrant and Guard
  2. Check Yo Self Before You Wreck Yo Self with Foodcritic & Guard
  3. Continuous ChefSpec Validation with Guard
  4. Serverspec, Guard and Test Kitchen: Testing Servers Like a Boss
Continuous ChefSpec Validation with Guard
comments powered by Disqus