Continuous ChefSpec Validation with Guard

Now that we are doing some basic linting of our cookbook code, we need to move onto some actual testing. But before all that, let’s make sure we are all on the same page when we mention the “T” word.

The official Software Body of Knowledge defines four major levels of testing: Unit, Integration, System and Acceptance. For purpose of this blog post, we will be writing tests at the Unit level. There are several ways to unit test cookbook code, but by far the best tool available today is ChefSpec.

I could write an entire post about why you should be writing unit tests for your cookbooks, but Seth Vargo says it best: >So why unit test at all? >Answer: Regression

###Step 1: Install ChefSpec The first thing we need to do is update our Gemfile to include the chefspec gem. We could simply add the gem and let Bundler figure it out the appropriate version, but we might not get the version we want. While Bundler is an amazing tool, it seems to prefer lower versions of gems when deciding how to resolve dependencies. So let’s go ahead and pin our ChefSpec gem version to 3.1 or higher. While we’re at it, let’s make sure we are getting the latest Foodcritic and guard-foodcritic gems as well.

Note: If you would like to learn more about how Bundler resolves dependency conflicts, I would suggest reading this post: How does Bundler bundle?

source 'https://rubygems.org'

gem 'berkshelf'

group :development do
  gem 'test-kitchen'
  gem 'kitchen-vagrant'
  gem 'guard'
  gem 'guard-kitchen'
  gem 'guard-foodcritic', '>= 1.0'
  gem 'foodcritic', '>= 3.0'
  gem 'chefspec', '>= 3.1'
end

Next we’ll install ChefSpec into our vendor/bundle directory using Bundler.

$~/chef-repo/cookbooks/strider-cd: bundle update
Fetching gem metadata from https://rubygems.org/.......
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using rake (10.1.1)
Using i18n (0.6.9)
Using multi_json (1.8.2)
...
Installing chefspec (3.1.2)
Your bundle is complete!
It was installed into ./vendor/bundle

NOTE: You can run bundle update to update your dependencies now that more gems are added. Similar to deleting the Gemfile and re-running bundle install

Let’s go ahead and run ChefSpec manually once to verify everything is setup properly.

$~/chef-repo/cookbooks/strider-cd: bundle exec rspec
No examples found.

Finished in 0.00005 seconds
0 examples, 0 failures

Perfect! However we need to do a little configuration to help tune ChefSpec to our liking.

###Step 2: Setup the spec_helper.rb file Before we start writing tests (I know, I’m getting impatient too!), we’ll need to do a little plumbing to “help” out our specs so they have access to all the tools they’ll need to execute properly. First we’ll need to create a spec directory to hold our helper file. Second, we will need to create our file at spec/spec_helper.rb

require 'chefspec'
require 'chefspec/berkshelf'

RSpec.configure do |config|
  config.platform = 'centos'
  config.version = '6.5'
end

This sets up the integration with Berkshelf for ChefSpec, as well as defining our testing platform OS and version. There are many more configuration options you can setup with ChefSpec, but these will do nicely for us right now.

We could go ahead and write some tests at this point and run them manually, but that will get tiresome quickly. You came here to automate things!

###Step 3: Automate ChefSpec Similar to how we automated our Foodcritic linting, we will need a Guard plugin to enable watching and executing of our ChefSpec tests. So let’s go ahead and add the guard-rspec gem to our Gemfile, pinning it to version 4.2 or higher.

There is currently a gem dependency conflict between the latest available version of Berkshelf and ChefSpec. Given that Berkshelf 3.0 is due to be released any day, we’ll go ahead and use the pre-release beta version from here on out as well.

source 'https://rubygems.org'

gem 'berkshelf', '~> 3.0.0.beta4'

group :development do
  gem 'test-kitchen'
  gem 'kitchen-vagrant'
  gem 'guard'
  gem 'guard-kitchen'
  gem 'guard-foodcritic', '>= 1.0'
  gem 'guard-rspec', '>= 4.2'
  gem 'foodcritic', '>= 3.0'
  gem 'chefspec', '>= 3.1'
end

Again, we’ll use Bundler to install our new gem.

$~/chef-repo/cookbooks/strider-cd: bundle update
Fetching gem metadata from https://rubygems.org/.......
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using rake (10.1.1)
Using i18n (0.6.9)
Using multi_json (1.8.2)
...
Installing guard-rspec (4.2.3)
Your bundle is complete!
It was installed into ./vendor/bundle

With our gems properly installed, we can setup our ChefSpec watchers in our Guardfile to wire up all the automation. The snippet below is a slightly modified version of the default watcher syntax auto-generated by the guard-rspec gem.

guard :rspec, cmd: 'bundle exec rspec', :all_on_start => false do
  watch(%r{^spec/(.+)_spec\.rb$})
  watch(%r{^(recipes)/(.+)\.rb$})   { |m| "spec/#{m[1]}_spec.rb" }
  watch('spec/spec_helper.rb')      { 'spec' }
end

Let’s talk about these new lines so we can understand what’s going on here. The first line sets up the watcher with a few options that help streamline our automation. We are specifying the command to be run as bundle exec rspec because we are managing all of our gems with Bundler. Next we tell Guard to only test the changed spec files instead of running all tests on each change.

guard :rspec, cmd: 'bundle exec rspec', :all_on_start => false, notification: false do

Then we proceed to write a few watch statements to cover changes to our tests. The first line simply watches for changes to any files in the spec directory that end with _spec.rb.

  watch(%r{^spec/(.+)_spec\.rb$})

Our second watch statement looks for changes to any files in our recipes folder and then executes the matching spec file in the spec directory. For example, if you modified recipes/default.rb, then Guard will execute the tests found in spec/default_spec.rb. You can read more about this syntax here.

  watch(%r{^(recipes)/(.+)\.rb$})   { |m| "spec/#{m[1]}_spec.rb" }

The last of our new watch statements monitors the spec/spec_helper.rb file and then executes all tests in the spec directory if any changes are detected.

  watch('spec/spec_helper.rb')      { 'spec' }

With our Guardfile updated, we can let Guard take over from here on out and see our test results as we move on to writing some tests.

$~/chef-repo/cookbooks/strider-cd: bundle exec guard
21:36:03 - INFO - Guard is using GNTP to send notifications.
21:36:03 - INFO - Guard is using TerminalTitle to send notifications.
21:36:03 - INFO - Guard::Kitchen is starting
-----> Starting Kitchen (v1.1.1)
-----> Creating <default-centos-65>...
       Bringing machine 'default' up with 'virtualbox' provider...
       [default] VirtualBox VM is already running.
       Vagrant instance <default-centos-65> created.
       Finished creating <default-centos-65> (0m7.18s).
-----> Kitchen is finished. (0m7.52s)

21:36:11 - INFO - Guard::RSpec is running
21:36:11 - INFO - Guard is now watching at '/Users/YourName/chef-repo/cookbooks/strider-cd'
[1] guard(main)>

###Step 4: Write Some Tests Finally! We get to write some tests. Since this post is really about automating ChefSpec testing with Guard, we’ll go over a pretty simple example and leave the test-writing advice to others.

Let’s create a new spec/default_spec.rb file and write our first test. If we peruse the Strider-CD requirements, we’ll notice that the first dependency we have is installing npm, so let’s write a test checking whether we have included the npm recipe from the nodejs cookbook available on the Chef Community site.

require 'chefspec'
require_relative 'spec_helper'

describe 'strider-cd::default' do
  let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }
  
  it 'includes the nodejs::npm recipe' do
    expect(chef_run).to include_recipe('nodejs::npm')
  end  
end

Now if we look, our tests are failing. Yay! However, they aren’t failing because we didn’t include the nodejs::npm recipe. What’s going on?

16:10:51 - INFO - Running: ./spec/default_spec.rb:7
Run options: include {:locations=>{"./spec/default_spec.rb"=>[7]}}
F

Failures:

  1) strider-cd::default includes the nodejs::npm recipe
     Failure/Error: let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }
     ChefSpec::Error::CommandNotStubbed:
       Executing a real command is disabled. Unregistered command: `command("/usr/local/bin/npm -v 2>&1 | grep '1.3.5'")`"

       You can stub this command with:

         stub_command("/usr/local/bin/npm -v 2>&1 | grep '1.3.5'").and_return(...)
     # ./spec/default_spec.rb:5:in `block (2 levels) in <top (required)>'
     # ./spec/default_spec.rb:12:in `block (2 levels) in <top (required)>'

Finished in 10.56 seconds
1 example, 1 failure

Failed examples:

rspec ./spec/default_spec.rb:7 # strider_cd::default includes the nodejs::npm recipe

So what gives? Luckily ChefSpec gives you some advice on what to do. We need to stub out the command used in a guard parameter in the nodejs cookbook. Doubley awesome is that ChefSpec tells you the command that needs to be stubbed, so we don’t have to go digging into other people’s cookbooks.

Using the information from the failure above, we can update our spec/default_spec.rb file with the necessary stubs.

describe 'strider-cd::default' do
  let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }  

  before do
    stub_command("/usr/local/bin/npm -v 2>&1 | grep '1.3.5'").and_return(false)
  end

Hey, that’s pretty cool! Guard automatically reexecuted our tests and they are failing for all the right reasons now!

22:26:33 - INFO - Running: spec/default_spec.rb
[2014-01-04T22:26:46-05:00] WARN: CentOS doesn't provide mongodb, forcing use of 10gen repo
F

Failures:

  1) strider-cd::default includes the nodejs::npm recipe
     Failure/Error: expect(chef_run).to include_recipe('nodejs::npm')
       expected ["strider-cd::default"] to include "nodejs::npm"
     # ./spec/default_spec.rb:12:in `block (2 levels) in <top (required)>'

Finished in 11.09 seconds
1 example, 1 failure

Failed examples:

rspec ./spec/default_spec.rb:11 # strider-cd::default includes the nodejs::npm recipe

###Step 5: Write some code If we implement our cookbook code correctly now, we should get to a passing status. Our first step is to update the dependency in the metadata.rb file.

name             'strider-cd'
maintainer       'Jane Doe'
maintainer_email 'you@example.com'
license          'All rights reserved'
description      'Installs/Configures strider-cd'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'

depends 'nodejs'

Next we’ll add the proper code to our recipes/default.rb file.

include_recipe "nodejs::npm"

Guard should detect the changes to our recipe and execute the matching spec/default_spec.rb tests. If this is a fresh run, you may also notice Guard telling Test-Kitchen to execute a kitchen converge, which will install Chef and execute any recipes in our cookbook.

22:39:11 - INFO - Running: ./spec/default_spec.rb:11
Run options: include {:locations=>{"./spec/default_spec.rb"=>[11]}}
[2014-01-04T22:39:24-05:00] WARN: CentOS doesn't provide mongodb, forcing use of 10gen repo
.

Finished in 11.56 seconds
1 example, 0 failures

Awesome! Our test passes and we have our first requirement for Strider-CD handled in our cookbook. You can continue in this red/green/refactor TDD cycle for the remaining requirements on your own. If you would like to cheat, you can view the completed specs here.

Next up in the series on automating your Chef development workflow, I’ll be covering Serverspec integration with Guard to handle integration/system testing.

This blog is the third post in a series covering automated cookbook development for Chef. You can find links to all current posts below.

1. Automating Cookbook Testing with Test-Kitchen, Berkshelf, Vagrant and Guard
2. Check Yo Self Before You Wreck Yo Self with Foodcritic & Guard
3. Continuous ChefSpec Validation with Guard
4. Serverspec, Guard and Test Kitchen: Testing Servers Like a Boss