Bonus Recipe: Cucumber + Codeship for Continuous Integration

In Recipe 10 of the book, we show you how to set up your own Jenkins continuous integration server to run Cucumber every time you push your latest code changes. Jenkins is of course not the only CI option out there. The CruiseControl family, just to name one, can do a lot of the same things.

You also have the option of using a hosted service such as Travis CI or Codeship, where the vendor takes care of maintaining the server for you. These are a nice, low-hassle way to get started with CI.

In this recipe, we’re going to show you how to get started with Cucumber on the Codeship hosted Continuous Integration service.

The App

We’ll use a toy time-tracking app called Out the Door as the guinea pig for this experiment. OTD is a simple web service that helps you track your punctuality on the Beeminder goal-tracking website.

Here’s how the app works: when you leave the house in the morning, you ping OTD from your smartphone (preferably automatically), and OTD in turn notifies Beeminder how many minutes early or late you are. If you start to miss your target repeatedly, Beeminder’s incentives kick in.

The Feature

With such a simple app, the Cucumber test practically writes itself. In your project directory, create a file called features/otd.feature with the following contents:

Feature: Time logging

  Scenario: Leaving
    Given I want to leave at 8:00
    When I leave at 8:17
    Then I am 17 minutes late

Eventually, we’ll test this app through its web interface. But just for the purposes of getting started with Codeship, let’s start with simple math on the departure times.

The Step Definitions

Create features/step_definitions/otd_steps.rb, and add the following step definitions to it:

require 'time'

Given(/^I want to leave at (\d+:\d+)$/) do |time|
  @desired = Time.parse time

When(/^I leave at (\d+:\d+)$/) do |time|
  @achieved = Time.parse time

Then(/^I am (\d+) minutes late$/) do |minutes|
  late = (@achieved - @desired) / 60.0
  late.should be_within(0.001).of(minutes.to_f)

This code is Ruby-agnostic, but we’ll be giving the examples here using JRuby. Install Cucumber and make sure the feature passes:

$ jruby -S gem install cucumber rspec-expectations
$ jruby -S cucumber
1 scenario (1 passed)
3 steps (3 passed)

Once that’s working, it’s time to create your project on Codeship.

Getting Started with Codeship

You can create an account on Codeship with your existing GitHub or Bitbucket credentials, or with an e-mail address and password. Once you’ve done so, they’ll lead you to the new project page:

New project

Codeship needs to know where to get the source code for your app. Head over to GitHub or Bitbucket and create a blank repository. Then, click the corresponding button on the Codeship new project page and choose the repo you just created.

Once you’ve created your project, you need to tell Codeship how to test it. Codeship uses the familiar Ruby Version Manager to choose Ruby implementations, so all you need to do is tell their server to make sure JRuby is installed.

In the Test tab of the user interface, leave the default technology stack set to “I want to create my own custom commands.” Then, enter the following text into the Setup Commands window:

rvm install jruby-1.7.8
rvm use jruby-1.7.8
jruby -S gem install cucumber rspec-expectations

Further down the page, in the Test Commands, window, type:

jruby -S cucumber

Now, when you save your project, Codeship will prompt you to complete the setup by adding a hook to your project:


Follow the on-screen instructions to configure Github or Bitbucket to use the Codeship hook.

On your local machine, commit your project to Git and push it up to the server, so that Codeship can see it:

git init
git add features
git commit -m "Initial commit"
git remote add origin git@...:username/otd.git

As soon as you push the project, Codeship will grab the latest version and run your Cucumber features:

New build

Congratulations! You’ve created your first passing Codeship integration.

What’s Next?

Now that we have a running project, what do we do? There are several ways we could improve this project:

  • Switch to Bundler for managing our gems
  • Add the actual app code to the project
  • Replace the pure-math step definitions with real ones that drive our app over HTTP
  • Use Heroku to deploy the app

In a future post, we’ll tackle a few of these tasks.

Beta 5 Release

This release includes two new recipes and a new appendix:

  1. “Test Android Apps With Calabash” demonstrates using Cucumber to drive an Android application.
  2. “Define Steps as Regular Ruby Methods” shows you a way to declutter your step definitions and make your tests more maintainable.
  3. “RSpec Expectations” gives you a quick overview of writing good pass/fail assertions using RSpec Expectations, the assertion framework we use most heavily in the book.

We’ve also substantially updated “Test .NET Code Using Cucumber Syntax” and “Drive a Windows App Using White” to use Visual Studio Professional and the NuGet package manager.

You can download the update from your bookshelf or buy the book.

Beta 4 Release

This release includes two new recipes:

  1. “Drive a Flash App Using Cuke4AS3” walks you through setting up the environment for testing Flash apps.
  2. “Manipulate Time” shows you a couple of different ways to make your tests faster by speeding up the clock.

We’ve also substantially updated one recipe:

  1. “Test On iOS Using Frank” now covers the new version of the Frank toolkit, which is now much more automated and primarily command-line-driven. Thanks to Pete Hodgson, the creator of Frank, for taking the time to walk us through what’s new.

You can download the update from your bookshelf or buy the book.

Any Requests?

Hi, everyone.

Are there any Cucumber techniques you’d like to see covered in the book? These could be tricks you’ve discovered, tasks you’ve always wondered how to perform, or follow-ons for specific recipes.

For example:

  • A follow-on to the JavaScript/CoffeeScript recipe that shows the next step of firing up a headless browser such as Zombie.js
  • A discussion of the Page Objects technique as it applies to web testing

We’d love to hear from you, either in the Pragmatic forums or on the book’s Twitter account.

Beta 3 Release

This release includes two new recipes:

  1. “Test a Grails App Using grails-cucumber” shows you how to get started testing a Grails app by plugging Cucumber into Groovy
  2. “Test Scala Code” demonstrates driving a Scala library from Cucumber using Cucumber-JVM

Beta 2 Release

This new release includes two new recipes:

  1. “Test Through Multiple Interfaces Using Worlds” explains how to drive both your GUI and your API from the same set of Cucumber features
  2. “Drive JavaScript / CoffeeScript Using Cucumber-JS” shows you how to use the full power of Cucumber in a JavaScript environment

We’ve also added a little real-world advice to our recipes on HTML tables and Sikuli. Thanks to readers Dean Cornish and Chuck van der Linden for weighing in on these topics in the forums.

Announcing Cucumber Recipes

You can test just about anything with Cucumber—we certainly have! See how to test desktop, web, mobile, and server applications across a variety of platforms. We’ll arm you with ready-rolled solutions to real-world problems: your tests will run faster, read more clearly, and work in any environment.

See below for the table of contents, or read the Pragmatic Programmers release announcement for more information.

Table of Contents

Cucumber Techniques

  1. Compare and Transform Tables of Data
  2. Generate an RTF Report With a Custom Formatter
  3. Run Slow Setup/Teardown Code With Global Hooks excerpt
  4. Refactor to Extract Your Own Application Driver DSL
  5. Compare Images
  6. Test Across Multiple Cores
  7. Test Across Multiple Machines With SSH
  8. Run Your Features Automatically With Guard and Growl
  9. Add Cucumber To Your Continuous Integration Server
  10. Publish Your Documentation on Relish
  11. Drive Cucumber’s Wire Protocol
  12. Implement a Wire Protocol Listener


  1. Use Cucumber Directly With JRuby
  2. Use Cucumber With Java Via Cucumber-JVM excerpt
  3. Drive a Spring Model
  4. Drive Hibernate Persistence
  5. Drive a Grails Appa
  6. Test Scala Code
  7. Test Clojure Code
  8. Drive a Swing Interface With Marathon

.NET and Windows

  1. Get Good Text Output On Windows
  2. Test .NET Code Using Cucumber Syntax
  3. Drive a Windows App Using win_gui
  4. Drive a Windows App Using White

Mobile and Web

  1. Test On iOS Using Frank
  2. Test On Android
  3. Test Across Multiple Mobile OSes With Calabash
  4. Test On Windows Phone
  5. Parse HTML Tables excerpt
  6. Drive JavaScript / CoffeeScript Using Cucumber-JS
  7. Test a Web App Using Watir
  8. Test a PHP App With cuke4php
  9. Play Back Canned Network Data Using VCR
  10. Drive a Flash App Using Cuke4AS3
  11. Monitor a Web Service Using Nagios and Cucumber

Other Languages and Platforms

  1. Drive a Mac GUI Using AppleScript and System Events
  2. Drive a Mac GUI Using MacRuby and AXElements
  3. Test Python Code Using Lettuce
  4. Test Erlang Code
  5. Test Lua Code Using cucumber-lua
  6. Test a GUI on Linux, Mac, or Windows With Sikuli
  7. Test an Arduino Project Using Serial