Dispatches from the Bubble
  • May 4, 2016

    RSpec For Ops Part 3: Test driven design with rspec-puppet

    This is the third part in a series of blog posts on RSpec for Ops. See the first post here and the second post here.

    Designing for Testability

    Once you get more comfortable with TDD for Puppet, you’ll start to realize that there’s a particular way to design your code to make it easier to test. I often hear complaints from newcomers to TDD to the effect of “what’s the point of TDD if I have to change how I write my code just for these tests?” This is a very good question. To answer this, let’s take a look at what makes code easier to test.

    Interfaces

    As I mentioned in my last post, the key to writing good tests is to focus on your module’s interfaces. Your module should have a well defined interface. When I say interface, I’m talking about an Application Programming Interface (or API) which, in Puppet, is created using parameters. Your goal should be to validate your module’s interface, ensuring that all branch conditions are covered by your tests. It’s much easier to test code that has an interface rather than trying to test conditions that are hardcoded inside the module. Let’s look at an example:

    class ssh {
  $package_name = $osfamily ? {
    'gentoo' => 'openssh',
    default  => 'openssh-server'
  }

  package { $package_name:
    ensure => installed
  }
}

    The example above installs an ssh server. We want to support more than one platform, so we’ve used a selector to determine the package name based on the osfamily fact. Let’s see what the test looks like for this:

    describe 'ssh' do
  context 'on gentoo' do
    let(:facts) { { :osfamily => 'gentoo' } }
    it { is_expected.to contain_package 'openssh' }
  end

  context 'on redhat' do
    let(:facts) { { :osfamily => 'redhat' } }
    it { is_expected.to contain_package 'openssh-server' }
  end
end

    This test looks simple enough, right? It’s testing all of our branch conditions and gets pretty good coverage. What happens when you want to add a new platform? What happens when the package name changes? You’ll have to write new tests to ensure that your branch conditions are covered. Let’s see what this looks like with an interface:

    class ssh (
  $package_name = $ssh::params::package_name,
) inherits ssh::params {

  package { $package_name:
    ensure => installed
  }
}

    This example uses the $package_name parameter to set the package name and gets its default value from the params class. Let’s see what the test would look like for this class:

    describe 'ssh' do
  let(:params) { { :package_name => 'openssh-server' } }
  it { is_expected.to contain_package 'openssh-server' }
end

    This test validates our interface and makes sure that the value we set at $package_name is passed to the package resource as we expected. If some part of the code changes the way this parameter is passed to the package resource, the tests will fail. To be sure, you should also validate your branching logic from your params class, but focusing on the interface here has forced us to redesign our code in a way that has some nice corollary benefits. This class is no longer dependent on embedded logic, so it is more resilient to unforeseen use cases. Our tests needs to know even less about our implementation code, which makes them more resilient and easier to maintain.

    Composition and Single Responsibility

    Composition is a practice where you combine discrete bits of code to achieve a desired behavior. Using composition along with classes that have a single responsibility, you can achieve complex behaviors that are easy to test, read, and extend. To illustrate, let’s take a look at a bit of code with many responsibilities.

    class ssh {
  package { 'openssh-server':
    ensure => installed,
  }
  file { '/etc/ssh/sshd_config':
    ensure  => file,
    content => template('ssh/sshd_config.erb'),
    require => Package['openssh-server'],
    notify  => Service['sshd'],
  }
  service { 'sshd':
    ensure  => running,
    enabled => true,
  }
}

    This example does three things: it installs a package, places a configuration file, and ensures a service. This class has three separate responsibilities. This means it has three behaviors that we’d need to test. Let’s look at the tests for this module.

    describe 'ssh' do
  it { is_expected.to contain_package 'openssh-server' }

  it 'should configure ssh' do
    is_expected.to contain_file('/etc/ssh/sshd_config').that_requires('Package[openssh-server]')
    is_expected.to contain_file('/etc/ssh/sshd_config').that_notifies('Service[sshd]')
  end

  it 'should start sshd' do
    is_expected.to contain_service('sshd').with { :ensure => 'running', :enabled => true }
  end
end

    This is not such a burden right now, but once this module gets big enough, it’ll be much easier to test these behaviors individually rather than together. In fact, there’s an established pattern to handle this exact example: the package, file, service pattern. Let’s refactor this module to use the package, file, service pattern and see how the tests look.

    class ssh {
  include ssh::install
  include ssh::config
  include ssh::service

  Class['ssh::install'] ->
  Class['ssh::config'] ~>
  Class['ssh::service']
}

class ssh::install {
  package { 'openssh-server': ensure => installed }
}

class ssh::config {
  file { '/etc/ssh/sshd_config':
    ensure  => file,
    content => template('ssh/sshd_config.erb'),
  }
}

class ssh::service {
  service { 'sshd': ensure => running, enabled => true }
}

    Each of these classes has a singular responsibility. The ssh class is responsible for including the subclasses that make up the desired behavior for the module, and ensuring that the resource ordering is correct. The ssh::install class is responsible for installing the ‘openssh-server’ package. The ssh::config class is responsible for configuring the sshd service. Finally, the ssh::service class is responsible for managing the sshd service.

    describe 'ssh' do
  it { is_expected.to contain_class('ssh::install').that_comes_before('Class[ssh::config]') }
  it { is_expected.to contain_class('ssh::config').that_notifies('Class[ssh::service]') }
  it { is_expected.to contain_class('ssh::service') }
end

describe 'ssh::install' do
  it { is_expected.to contain_package('openssh-server') }
end

describe 'ssh::config' do
  it { is_expected.to contain_file('/etc/ssh/sshd_config') }
end

describe 'ssh::service' do
  is_expected.to contain_service('sshd').with({ :ensure => 'running', :enabled => true })
end

    Writing classes with a single responsibility makes writing tests easier because you can focus on a single behavior at a time. You can group your tests based on the behaviors they’re testing, and you can test different conditions that may affect your code’s behavior discretely. Once you’ve refactored this code into discrete classes with a single responsibility, you can achieve the same overall behavior with composition.

    Designing for Testability is Good Overall Design

    Design for testability just so happens to be good overall design. These design principles include programming to an interface, composition, and single responsibility. If you encounter a class that’s hard to test, think about why it’s hard. Chances are you’ve identified a good opportunity to redesign your code because test driven development fleshes out hidden design flaws. At first it may seem like you’re massaging your code to fit your testing strategy but, in reality, you’re fixing design flaws that make it harder to read, maintain, and change your code.

    Tests should be designed to be largely independent of your code’s internals. If you find yourself battling fragile tests, you’ve probably coupled your tests to your code too tightly. Take a look at your test code, as well as your implementation code, and try and identify missing or poorly defined interfaces. Think about the assumptions you’re making in your code. Make sure you’re not hard coding those assumptions into your tests to make up for missing interfaces.

    TDD may seem counterintuitive at first, but that will quickly pass. The benefits of TDD are proven, and become apparent rather quickly. After a while you’ll notice yourself writing code that’s simpler, more modular, more readily able to handle unforeseen use cases, and easier to maintain.


  • May 3, 2016

    RSpec For Ops Part 2: Diving in with rspec-puppet

    This is the second part in a series of blog posts on RSpec for Ops. See the first post here.

    Now that we’ve got a good understanding of some of the RSpec primitives, we’ll dive into how these can be used when testing infrastructure. I’m going to use rspec-puppet for my examples here. In a forthcoming post I’ll use serverspec for a more agnostic approach. If you’re not a Puppet user, don’t worry, the two main Infracode testing tools (Beaker and Kitchen) both make heavy use of Serverspec.

    There are two general types of tests that you’ll write as an Infracoder: acceptance and unit tests. We’ll use rspec-puppet for unit tests.

    Unit tests

    Unit tests are written to test a specific bit of code. With rspec-puppet we test our Puppet classes, defined types, and resources. We’ll use rspec-puppet to define a specification for a Puppet class. This class will install a package, place a configuration file, and start a service. Because we’re using test driven development, we’ll write our tests before we write the code. Let’s get started:

    require 'spec_helper'

describe 'ntp' do
  it { is_expected.to compile }
end

    Here we have written a test for the ntp class. We have a single example with the expectation that the catalog will compile with the ntp class included. There’s some rspec-puppet magic going on here, so let’s delve into this. First, you’ll notice that I’m not using an explicit subject here, we’re relying on the implicit subject using the first argument passed to the describe method. The rspec-puppet helpers assume the first argument to the describe method is the name of a Puppet class. It will automatically generate a manifest with include ntp as the content and will use that manifest to compile a catalog. The resulting catalog object then gets stored as the subject for use in our examples. This test has a single expectation. Using the compile matcher, we set the expectation that the catalog will successfully compile. When we run this, we’ll get an error:

    ntp
  should compile into a catalogue without dependency cycles (FAILED - 1)

Failures:

  1) ntp should compile into a catalogue without dependency cycles
     Failure/Error: it { is_expected.to compile }
       error during compilation: Evaluation Error: Error while evaluating a Function Call, Could not find class ::ntp for mbp.danzil.io at line 1:1 on node mbp.danzil.io
     # ./spec/classes/ntp_spec.rb:4:in `block (2 levels) in <top (required)>'

Finished in 0.17856 seconds (files took 0.69616 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/classes/ntp_spec.rb:4 # ntp should compile into a catalogue without dependency cycles

    This error is expected, since we never actually wrote the code to implement the ntp class. Let’s do that now:

    class ntp { }

    The goal is to write the minimum amount of code to make the tests pass. All our tests are looking for is that the ntp class exists and that it compiles. The code above is all we need to meet that requirement. If we run the tests again, we’ll see:

    ntp
  should compile into a catalogue without dependency cycles

Finished in 0.18003 seconds (files took 0.69723 seconds to load)
1 example, 0 failures

    Now, this class doesn’t do anything, so it’s not particularly useful. Let’s install the ntp package:

    require 'spec_helper'

describe 'ntp' do
  it { is_expected.to compile }
  it { is_expected.to contain_package('ntpd') }
end

    We’ve added a new example to our ntp group with the expectation that the catalog will contain the ntpd package. We’re using the contain_package matcher to assert this expectation. When we run our new tests we’ll have a new failure:

    ntp
  should compile into a catalogue without dependency cycles
  should contain Package[ntpd] (FAILED - 1)

Failures:

  1) ntp should contain Package[ntpd]
     Failure/Error: it { is_expected.to contain_package('ntpd') }
       expected that the catalogue would contain Package[ntpd]
     # ./spec/classes/ntp_spec.rb:5:in `block (2 levels) in <top (required)>'

Finished in 1.47 seconds (files took 0.79995 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/classes/ntp_spec.rb:5 # ntp should contain Package[ntpd]

    The output is complaining that the catalog is missing the expected ntpd package. Let’s add the code to make that test pass:

    class ntp {
  package { 'ntpd': }
}

    Now our tests should pass:

    ntp
  should compile into a catalogue without dependency cycles
  should contain Package[ntpd]

Finished in 1.41 seconds (files took 0.77791 seconds to load)
2 examples, 0 failures

    You’ll continue with this until you have a functioning module. I’m going to skip ahead a bit so we can move on to more interesting things. Here’s the rest of the spec I’m going to write for this module:

    require 'spec_helper'

describe 'ntp' do
  it { is_expected.to compile }
  it { is_expected.to contain_package('ntpd') }

  it 'should contain the ntp configuration file' do
    is_expected.to contain_file('/etc/ntpd.conf')
    is_expected.to contain_file('/etc/ntpd.conf').that_requires('Package[ntpd]')
    is_expected.to contain_file('/etc/ntpd.conf').that_notifies('Service[ntpd]')
  end

  it { is_expected.to contain_service('ntpd') }
end

    When I run this test, we’ll see the expected failures:

    ntp
  should compile into a catalogue without dependency cycles
  should contain Package[ntpd]
  should contain the ntp configuration file (FAILED - 1)
  should contain Service[ntpd] (FAILED - 2)

Failures:

  1) ntp should contain the ntp configuration file
     Failure/Error: is_expected.to contain_file('/etc/ntpd.conf')
       expected that the catalogue would contain File[/etc/ntpd.conf]
     # ./spec/classes/ntp_spec.rb:8:in `block (2 levels) in <top (required)>'

  2) ntp should contain Service[ntpd]
     Failure/Error: it { is_expected.to contain_service('ntpd') }
       expected that the catalogue would contain Service[ntpd]
     # ./spec/classes/ntp_spec.rb:13:in `block (2 levels) in <top (required)>'

Finished in 1.39 seconds (files took 0.66577 seconds to load)
4 examples, 2 failures

Failed examples:

rspec ./spec/classes/ntp_spec.rb:7 # ntp should contain the ntp configuration file
rspec ./spec/classes/ntp_spec.rb:13 # ntp should contain Service[ntpd]

    I’ll implement those features in the code:

    class ntp {
  package { 'ntpd': }
  file { '/etc/ntpd.conf':
    ensure  => file,
    content => file('ntp/ntpd.conf'),
    require => Package['ntpd'],
    notify  => Service['ntpd'],
  }
  service { 'ntpd':
    ensure => running,
    enable => true,
  }
}

    When I run my tests, all of my examples pass:

    ntp
  should compile into a catalogue without dependency cycles
  should contain Package[ntpd]
  should contain the ntp configuration file
  should contain Service[ntpd]

Finished in 2.03 seconds (files took 0.73826 seconds to load)
4 examples, 0 failures

    Great! We have a functioning Puppet module, but not a particularly useful one. This is an admittedly contrived example. It only works on platforms that have a package named ntpd and call that service ntpd and look for a configuration file at /etc/ntpd.conf. Let’s make this code a little more useful by adding some parameters:

    require 'spec_helper'

describe 'ntp' do
  let(:params) do
    {
      :package_name => 'ntpd',
      :config_file  => '/etc/ntpd.conf',
      :service_name => 'ntpd'
    }
  end

  it { is_expected.to compile }
  it { is_expected.to contain_package('ntpd') }

  it 'should contain the ntp configuration file' do
    is_expected.to contain_file('/etc/ntpd.conf')
    is_expected.to contain_file('/etc/ntpd.conf').that_requires('Package[ntpd]')
    is_expected.to contain_file('/etc/ntpd.conf').that_notifies('Service[ntpd]')
  end

  it { is_expected.to contain_service('ntpd') }
end

    The test above uses the let method to declare params. rspec-puppet looks at params for a hash of parameters to pass to the Puppet class. Let’s run this test:

    ntp
  should compile into a catalogue without dependency cycles (FAILED - 1)
  should contain Package[ntpd] (FAILED - 2)
  should contain the ntp configuration file (FAILED - 3)
  should contain Service[ntpd] (FAILED - 4)

Failures:

  1) ntp should compile into a catalogue without dependency cycles
     Failure/Error: it { is_expected.to compile }

       error during compilation: Evaluation Error: Error while evaluating a Resource Statement, Class[Ntp]:
         has no parameter named 'package_name'
         has no parameter named 'config_file'
         has no parameter named 'service_name' at line 1:1 on node mbp.danzil.io
     # ./spec/classes/ntp_spec.rb:10:in `block (2 levels) in <top (required)>'

  2) ntp should contain Package[ntpd]
     Failure/Error: it { is_expected.to contain_package('ntpd') }

     Puppet::PreformattedError:
       Evaluation Error: Error while evaluating a Resource Statement, Class[Ntp]:
         has no parameter named 'package_name'
         has no parameter named 'config_file'
         has no parameter named 'service_name' at line 1:1 on node mbp.danzil.io

  3) ntp should contain the ntp configuration file
     Failure/Error: is_expected.to contain_file('/etc/ntpd.conf')

     Puppet::PreformattedError:
       Evaluation Error: Error while evaluating a Resource Statement, Class[Ntp]:
         has no parameter named 'package_name'
         has no parameter named 'config_file'
         has no parameter named 'service_name' at line 1:1 on node mbp.danzil.io

  4) ntp should contain Service[ntpd]
     Failure/Error: it { is_expected.to contain_service('ntpd') }

     Puppet::PreformattedError:
       Evaluation Error: Error while evaluating a Resource Statement, Class[Ntp]:
         has no parameter named 'package_name'
         has no parameter named 'config_file'
         has no parameter named 'service_name' at line 1:1 on node mbp.danzil.io

Finished in 0.18966 seconds (files took 0.69537 seconds to load)
4 examples, 4 failures

Failed examples:

rspec ./spec/classes/ntp_spec.rb:10 # ntp should compile into a catalogue without dependency cycles
rspec ./spec/classes/ntp_spec.rb:11 # ntp should contain Package[ntpd]
rspec ./spec/classes/ntp_spec.rb:13 # ntp should contain the ntp configuration file
rspec ./spec/classes/ntp_spec.rb:19 # ntp should contain Service[ntpd]

    I’ve truncated the output here for clarity. You’ll see that everything blew up because I’m trying to pass parameters to a class that doesn’t accept parameters! Let’s add those parameters to my Puppet class:

    class ntp (
  $package_name = 'ntpd',
  $config_file  = '/etc/ntpd.conf',
  $service_name = 'ntpd',
){
  package { $package_name: }
  file { $config_file:
    ensure  => file,
    content => file('ntp/ntpd.conf'),
    require => Package[$package_name],
    notify  => Service[$service_name],
  }
  service { $service_name:
    ensure => running,
    enable => true,
  }
}

    Now when we run our tests, we’ll see a much greener (and more familiar) output:

    ntp
  should compile into a catalogue without dependency cycles
  should contain Package[ntpd]
  should contain the ntp configuration file
  should contain Service[ntpd]

Finished in 1.76 seconds (files took 0.66553 seconds to load)
4 examples, 0 failures

    Interfaces are Key

    We’ve just refactored this code to accept parameters. The most important testing practice (IMHO) is to focus on the interface, not the implementation. By adding parameters to this class, we’ve created an interface. Our parameters are our module’s API. Well written tests should validate our interfaces and ensure that the data we’re passing to our interface is making it to our resources the way we expect it to. When we focus on the implementation we end up just rewriting our code in another language, focusing on the interface allows us to focus on the bits of our code that we really care about.

    Now that we have some parameters, we can start testing different contexts. Contexts are helpful when testing your module’s interface under different conditions. We’ll want to validate our interface in a number of different ways. I’m going to restructure our tests to focus on the interface:

    require 'spec_helper'

describe 'ntp' do
  context 'with the default parameters' do
    it { is_expected.to compile }
    it { is_expected.to contain_package('ntpd') }

    it 'should contain the ntp configuration file' do
      is_expected.to contain_file('/etc/ntpd.conf')
      is_expected.to contain_file('/etc/ntpd.conf').that_requires('Package[ntpd]')
      is_expected.to contain_file('/etc/ntpd.conf').that_notifies('Service[ntpd]')
    end

    it { is_expected.to contain_service('ntpd') }
  end

  context 'when specifying the package_name parameter' do
    let(:params) { { :package_name => 'ntp' } }
    it { is_expected.to contain_package('ntp') }
    it { is_expected.to contain_file('/etc/ntpd.conf').that_requires('Package[ntp]') }
  end

  context 'when specifying the config_file parameter' do
    let(:params) { { :config_file => '/etc/ntp.conf' } }
    it { is_expected.to contain_file('/etc/ntp.conf') }
  end

  context 'when specifying the service_name parameter' do
    let(:params) { { :service_name => 'ntp' } }
    it { is_expected.to contain_service('ntp') }
    it { is_expected.to contain_file('/etc/ntpd.conf').that_notifies('Service[ntp]') }
  end
end

    As you can see, I’ve added contexts for each of my individual parameters, as well as a context that covers the default parameter values. I am testing to make sure that the parameter values are being consumed correctly in my code. These tests will already pass since we’re not implementing any new features here just improving the tests.

    ntp
  with the default parameters
    should compile into a catalogue without dependency cycles
    should contain Package[ntpd]
    should contain the ntp configuration file
    should contain Service[ntpd]
  when specifying the package_name parameter
    should contain Package[ntp]
    should contain File[/etc/ntpd.conf] that requires Package[ntp]
  when specifying the config_file parameter
    should contain File[/etc/ntp.conf]
  when specifying the service_name parameter
    should contain Service[ntp]
    should contain File[/etc/ntpd.conf] that notifies Service[ntp]

Finished in 2.11 seconds (files took 0.78148 seconds to load)
9 examples, 0 failures

    Remember, the goal is not to rewrite your Puppet code in RSpec, it’s to ensure that your code is behaving the way you expect it to.

    Stay Tuned

    In the next post in this series, I’ll dive a little deeper into module development and rspec-puppet best practices. I’ll also follow up with a post where I’ll use serverspec to write some acceptance tests for this example code.


  • May 2, 2016

    Changes

    There have been a few big changes in my life over the past few months. In March we bought a house, which has been an interesting learning experience for me. But I promised I won’t talk too much about my homeownership adventures, so that’s not what this post is about. This post is about another adventure, a new job.

    In February of 2014 I moved from New Mexico back to my home state of Massachusetts. My husband had just been accepted to Boston University Law School, and I had been dying to get back to my roots. I had been living in New Mexico for almost a decade, which turned out to be much longer than I originally planned. I took a job at Constant Contact as a Systems Engineer and Puppet Evangelist.

    The past two-ish years at Constant Contact have been great. This was my first experience in a real web-scale SaaS shop. SaaS is really the holy-grail of tech ops, and I was quite lucky to land a job like this one. Constant Contact has taught me a lot about myself, about scaling, about succeeding in an enterprise, and about what DevOps really means in practice. I’ve picked up a lot of hard skills, and even more soft skills. But, all good things must come to an end. This coming Friday will be my last day at Constant Contact. It is bittersweet, as leaving a job often is. I’ve made some great friends, and grown quite a bit as a professional during my tenure.

    I’m going to be joining the Professional Services team at Kovarus. Kovarus is based in Silicon Valley, but I’ll be working remotely from Boston. My focus as a consultant will be on Puppet and continuous delivery. I’m really looking forward to joining this team and being able to share my experiences with others. If you know me, or follow me, you probably know that I’m really passionate about teaching and sharing. I think this will be a great way to engage that passion in combination with my other, more engineering oriented skills. Cue the consultant jokes…you know you want to…

    So stay tuned for updates on the transition. I’ll probably be writing about my adventures as a remote employee and learning how to work from home full time. I’ll still be active in the community, hopefully even more so.

    Also, if you’re interested in some consulting, let me know :)


  • May 1, 2016

    Welcome to Blog Month!

    Today marks the beginning of a new experiment for me. I, along few others, will be taking part in blog month. Blog month is a commitment to write 30 posts over the course of a month. You can blog about anything you want, it doesn’t have to be technical in nature. The goal is to get into the habit of writing regularly.

    I first heard about this idea from Rob Nelson (rnelson0 on the interwebz). A few months ago, Rob tossed around the idea of a mid-year blog month in the #voxpupuli channel on Freenode. The traditional blog month is in November (to coincide with NaNoWriMo), but that often clashes with the holidays, so it’s not particularly convenient for some of us. At the time, May sounded great. Since then, I’ve taken a new job (see tomorrow’s post), bought a house, and am slated to speak and attend a couple conferences this month. I’ve basically come to the realization that there’s no good time for this, so I’m just going to dig in.

    I have a few posts in mind. I want to finish up a couple blog series that I started and never finished (Seven Habits, and RSpec for Ops). I’m also going to start fleshing out some topics I’ll be speaking on starting this month. I’d love to get some feedback on these things, so please, don’t hold back. If there’s anything you might be interested in reading about, please let me know!

    I’ll be announcing these posts on Twitter. Rob has put together a Twitter list tracking those of us participating in blog month. You can find that list here. If you’re interested in participating, or just want to learn more about the genesis of this idea, I highly encourage you to read Rob’s post about it back in March. You can also track the hashtags #blogmonth or #vDM30in30.

    I’m really looking forward to writing and reading some great content this month! Good luck to all of those participating! Happy blogging!


  • Feb 5, 2016

    RSpec For Ops Part 1: Essentials

    RSpec For Ops

    I’ve noticed a lot of interest in RSpec from ops folks lately, likely driven by infracode testing tools like Kitchen, Beaker, and Serverspec. I’ve also noticed a lot of ops folks using rspec without really understanding some of the fundamentals. This results in some suboptimal rspec code, and generally lots of headaches for newcomers to rspec. This blog post is the first in a series attempting to outline some of the basics of test driven development with rspec from the perspective of an Op :) I’ll start by outlining rspec fundamentals, another post will follow with some (hopefully) relatable examples.

    RSpec Essentials

    RSpec is a behavior driven development (BDD) framework for Ruby. BDD focuses on the behavior and expected outcomes of your code. RSpec provides a simple domain-specific language that employs natural-lanaguage constructs for describing tests. As a result, BDD code is often very easy to read with inputs and expectations being very easy to identify. RSpec offers several primitives to model the behavior of your code including groups, examples, contexts, expectations, and matchers.

    Examples and Groups

    One of the most fundamental primitives in rspec is the example. An example is the description of a behavior you expect your code to exhibit. Examples are declared using the it method and are often grouped together logically inside an (very creatively named) example group. Examples are comprised of a subject and an expectation. Example groups are declared with the describe method. Example groups make up your application’s specification; they often end up looking a lot like lists of acceptance criteria. Here’s a simple example:

    it { is_expected.to be true }

    This code can be read as “it is expected to be true” and literally means that the subject of the test (we’ll go into more detail in a minute) should return true. I didn’t set a subject in the example above, but we can do that with the let method:

    let(:subject) { true }

it { is_expected.to be true }

    The let method creates a memoized helper method set to the value passed in the block. A memoized helper method is basically an attr_reader on an instance variable. Here’s what the above let method effectively does under the hood:

    def subject
  @subject ||= true
end

    The let method creates a new method based on the name you passed it as an argument. The code passed to the let method’s block is then lazily evaluated and cached in the instance variable @subject. Lazy evaluation means that the code associated with this method is not evaluated until it is called the first time. This is in contrast to eager evaluation where the code is evaluated as soon as it is bound to a variable. This allows you to set some state that persists inside an example group, this also means that you can only use the let method inside an example group. Let’s fix our example code:

    describe 'Truthiness' do
  let(:subject) { true }

  it { is_expected.to be true }
end

    Here we have an example group named Truthiness, we’ve set the subject to true, and we’ve created an example with the expectation that our subject is true. Let’s run this code and see what the output is:

    Truthiness
  should equal true

Finished in 0.00107 seconds (files took 0.08412 seconds to load)
1 example, 0 failures

    Expectations and Matchers

    An example is just a list of behaviors expected of our subject. Our example has one expectation: that the subject should be true. Our example above could also be expressed using the expect method:

    describe 'Truthiness' do
  let(:subject) { true }

  it { expect(subject).to be true }
end

    This doesn’t read as smoothly as it did with the is_expected method, which is just expect with an implicit subject. We can make this example more readable by passing a descriptive title to the it method:

    describe 'Truthiness' do
  let(:subject) { true }

  it 'should behave truthy' do
    expect(subject).to be true
  end
end

    Now this example is easier to read. We can see that our subject should behave truthy and we have a list of expectations that need to be met for our example to pass. Let’s add an expectation that our subject is not false:

    describe 'Truthiness' do
  let(:subject) { true }

  it 'should behave truthy' do
    expect(subject).to be true
    expect(subject).not_to be false
  end
end

    These code examples are available on my GitHub here. The example above can be found in spec/truthiness_spec.rb

    As you can see, examples can contain any number of expectations. You should use as many expectations as necessary to validate the behavior you’re testing in an example. One element of an expectation that we haven’t yet discussed is the matcher, which is really the heart of an expectation. Our example above only uses the be matcher but there are many others. Matchers describe the behavior you expect your subject to exhibit. In our example, we are using the be matcher to compare our subject’s identity with our matcher.

    There are lots of built-in matchers; common matchers include be, eq, exist, include, match, start_with, and respond_to. RSpec has a robust matcher API that makes it very easy to write custom matchers. Infracode frameworks built on RSpec like Serverspec include a number of domain-specific matchers. Let’s take a look at some different kinds of matchers:

    describe 'Hello world!' do
  it 'should say hello' do
    expect(subject).to match /Hello world/
    expect(subject).not_to match /Goodbye world/
    expect(subject).to eq 'Hello world!'
  end
end

    In this example group we are using the match and eq matchers. Notice I didn’t explicitly set a subject here, we’re relying on the implicit subject set by the argument we passed to the describe method. The match matcher compares your subject with a regular expression, in this case /Hello world/. You can set a negative expectation by using the not_to method on our expectation. In this example, we want to make sure that we’re saying “hello” so we want to make sure our subject does not match /Goodbye world/. Finally, we’re using the eq matcher to compare our subject with the string Hello world!. Let’s see what the output of this test would look like:

    Hello world!
  should say hello

Finished in 0.00203 seconds (files took 0.07591 seconds to load)
1 examples, 0 failures

    Notice we did not use the be matcher in our example. This is because the be matcher tests object identity while the eq matcher tests equality. Here’s what the output would look like if we used the be matcher:

    Hello world!
  should say hello (FAILED - 1)

Failures:

describe 'Hello world!' do
  1) Hello world! should say hello
     Failure/Error: expect(subject).to be 'Hello world!'

       expected #<String:70291920962560> => "Hello world!"
            got #<String:70291943850520> => "Hello world!"

       Compared using equal?, which compares object identity,
       but expected and actual are not the same object. Use
       `expect(actual).to eq(expected)` if you don't care about
       object identity in this example.
     # ./spec/blog_examples/hello_world_spec.rb:6:in `block (2 levels) in <top (required)>'

Finished in 0.01117 seconds (files took 0.07584 seconds to load)
1 examples, 1 failure

Failed examples:

rspec ./spec/blog_examples/hello_world_spec.rb:2 # Hello world! should say hello

    As you can see, the be matcher actually compares the identity of each object instance, but this example uses two different instances of String. We’re really only interested in comparing the values of the String instances so we use the eq matcher.

    Contexts

    RSpec offers two primary methods of grouping tests together. We’ve already looked at one way, by creating example groups with the describe method. The other way is to use the context method. The context method is really just an alias for the describe method. The describe method is generally used to create example groups that describe an object, while the context method is used to create example groups that describe the state of an object that can vary. Here’s an example describing an Array object with multiple contexts:

    describe Array do
  context 'with no elements' do
    let(:subject) { [] }
    it 'should be empty' do
      expect(subject.count).to eq 0
    end
  end

  context 'with elements' do
    let(:subject) { ['foo', 'bar', 'baz'] }
    it 'should not be empty' do
      expect(subject.count).not_to eq 0
    end
  end
end

    Here we’ve grouped our examples by the class of object we’re testing (Array), but we’ve segmented our examples based on the context we’re looking to describe. Each context introduces a new scope. Examples grouped by context can have their own subjects, or they can inherit their subject from their parent. Here’s an example where we inherit our subject from our parent and test for different contexts:

    describe Hash do
  let(:baz_val) { nil }
  let(:subject) do
    { :foo => 'bar', :baz => baz_val }
  end

  it 'should have the foo key set to bar' do
    expect(subject[:foo]).to eq 'bar'
  end

  it 'should have the baz key set to nil' do
    expect(subject[:baz]).to be nil
  end

  context 'with baz_val set to qux' do
    let(:baz_val) { 'qux' }
    it 'should have the baz key set to qux' do
      expect(subject[:baz]).to eq 'qux'
    end
  end

  context 'with baz_val set to quux' do
    let(:baz_val) { 'quux' }
    it 'should have the baz key set to quux' do
      expect(subject[:baz]).to eq 'quux'
    end
  end
end

    Here we’re testing the same Hash object, but we’re testing that object under different contexts. We’re overriding the baz_val method inside our context blocks and ensuring that our object behaves the way we expect it to under different conditions. This allows us to test parts of our code that vary with context while avoiding duplicating our tests that focus on the parts of our code that remain stable. Segmenting our examples into context blocks makes our test code and output more readable. Here’s the output of the above example:

    Hash
  should have the foo key set to bar
  should have the baz key set to nil
  with baz_val set to qux
    should have the baz key set to qux
  with baz_val set to quux
    should have the baz key set to quux

Finished in 0.00189 seconds (files took 0.10379 seconds to load)
4 examples, 0 failures

    You can declare a context outside of a describe block using the shared_context method. A shared context is a context that can be shared across many example groups. It’s a way to describe common behavior without having to resort to duplicating code. We can break out the common behavior in our example above into a shared context:

    shared_context 'foo should always be set to bar' do
  it 'should have the foo key set to bar' do
    expect(subject[:foo]).to eq 'bar'
  end
end

describe Hash do
  let(:baz_val) { nil }
  let(:subject) do
    { :foo => 'bar', :baz => baz_val }
  end

  include_context 'foo should always be set to bar'

  it 'should have the baz key set to nil' do
    expect(subject[:baz]).to be nil
  end

  context 'with baz_val set to qux' do
    let(:baz_val) { 'qux' }
    include_context 'foo should always be set to bar'
    it 'should have the baz key set to qux' do
      expect(subject[:baz]).to eq 'qux'
    end
  end

  context 'with baz_val set to quux' do
    let(:baz_val) { 'quux' }
    include_context 'foo should always be set to bar'
    it 'should have the baz key set to quux' do
      expect(subject[:baz]).to eq 'quux'
    end
  end
end

    Since the foo key should always be set to bar, we can include this shared context to ensure that our Hash meets our expectations in all of our contexts. Here’s the output of that test:

    Hash
  should have the foo key set to bar
  should have the baz key set to nil
  with baz_val set to qux
    should have the foo key set to bar
    should have the baz key set to qux
  with baz_val set to quux
    should have the foo key set to bar
    should have the baz key set to quux

Finished in 0.00221 seconds (files took 0.08721 seconds to load)
6 examples, 0 failures

    Shared contexts are a powerful way to describe common behavior without resorting to boilerplate. Shared contexts can be stored in separate files and loaded using the require method at the beginning of your spec file.

    Making the connection

    This post focused mostly on RSpec itself. In a future post I’ll bring these elements together with some examples using rspec-puppet and serverspec.