Behavior Driven Testing

To automate executing a .feature file, you need to define step implementations that associate a piece of script code with a pattern. Every time a step is encountered in a .feature file, Squish will try to find a pattern matching the step text. If a match is found, the associated code is executed.

Defining Step Implementations Using Step

A step implementation is defined by calling a pre-defined Step function. A simple definition would look like this:

@Step("user starts the addressbook application")
def step(context):
    startApplication("addressbook")

Step("user starts the addressbook application", function(context) {
    startApplication("addressbook");
});

Step {user starts the addressbook application} {context} {
    startApplication addressbook
}

Step("user starts the addressbook application") do |context|
    Squish::startApplication("addressbook")
end

use Squish::BDD;

Step "user starts the addressbook application", sub {
    startApplication("addressbook");
};

In this example, a step matching the pattern "user starts the addressbook application" will cause Squish to execute

startApplication("addressbook")

startApplication("addressbook");

startApplication addressbook

Squish::startApplication("addressbook")

startApplication("addressbook");

Step Text

The step text, after stripping any keywords from the text, such as Given, has to match the pattern exactly. The match is anchored to the start and end of the text.

The table below shows which step texts would match the pattern "user starts the addressbook application":

Limitations

Defining a step via Step will always succeed, except in the following cases:

• The pattern given is empty or otherwise malformed — the latter may occur when using regular expressions.
• There is already an existing definition for the given pattern.
• For script languages in which Step takes a signature argument: the signature must at least contain one argument for the context.

Using Step Patterns with Placeholders

Patterns can make use of certain placeholders to extract data from the step text and pass it to the script code. This allows making step-definitions more reusable since it avoids having to hardcode specific values. For instance, here's an improvement to the above step definition which avoids hardcoding the name of the application to start:

@Step("user starts the |word| application")
def step(context, appName):
    startApplication(appName)

Step("user starts the |word| application", function(context, appName) {
    startApplication(appName);
});

Step {user starts the |word| application} {context appName} {
    startApplication $appName
}

Step("user starts the |word| application") do |context, appName|
    Squish::startApplication(appName)
end

use Squish::BDD;

Step "user starts the |word| application", sub {
    my %context = %{shift()};
    my $appName = shift();
    startApplication($appName);
};

The application name addressbook is no longer hardcoded. Instead, the |word| will match the word and pass it via the new appName argument.

The following placeholders can be used:

Using Step Patterns with Regular Expressions

Sometimes, you may find that the above mentioned placeholders are insufficient, that you need more control over which portions of a step text gets extracted. This can be achieved by not passing a plain string to the Step function but rather a regular expression. Regular expressions are vastly more powerful, but also require being more careful - they can easily become somewhat cryptic to the reader.

Regular expressions not only allow much finer control over the text being matched. They can make use of capturing groups to extract data from the step text and pass it to the step code.

Python-specific

In Python scripts, you can pass a regular expression object directly. That is, the object returned by Python's re.compile function. If you don't want to import the re module, you can also pass a string and the optional argument regexp set to True, as in the following snippet:

@Step(r"user starts the (\w+) application", regexp=True)
def step(context, appName):
    ...

We also recommend that you use Python's raw string syntax (r"..." resp. r'...') for specifying the strings to avoid escaping of the backslash character in the regular expression.

Tcl-specific

In Tcl scripts, there are no regular expression values. Instead, the Step command is invoked with the -rx option, as in the following snippet:

Step -rx "This is a reg.lar expression$" {context} { ... }

Examples of Regular Expressions

For example, the above examples can be expressed using regular expressions as follows:

@Step(r"user starts the (\w+) application", regexp=True)
def step(context, appName):
    startApplication(appName)

Step(/user starts the (\w+) application/, function(context, appName) {
    startApplication(appName);
});

Step -rx {user starts the (\w+) application} {context appName} {
    startApplication $appName
}

Step(/user starts the (\w+) application/) do |context, appName|
    Squish::startApplication(appName)
end

use Squish::BDD;

Step qr/user starts the (\w+) application/, sub {
    my %context = %{shift()};
    my $appName = shift();
    startApplication($appName);
};

The BDD context Object

Each BDD implementation function has a context singleton object passed in as the first argument. The context objects of scenario hooks, step hooks and step implementations provide different properties. (See Hooks API for details on hooks.)

Context Object of Scenario Hooks (@OnScenarioStart, @OnScenarioEnd)

Context Object of Step Hooks (@OnStepStart, @OnStepEnd)

Context Object of Step Implementations

Examples using userData, table and multiLineText are in the following sections.

context.userData: Passing Data Between Steps

The context argument passed to all steps (and hooks) features a userData field which can be used to pass data between steps. This property is initially empty but can be written to from within a step. Consider e.g. a feature file like:

Feature: Shopping Cart

  Scenario: Adding three items to the shopping cart

    Given I have an empty shopping cart
    When I add bananas to the shopping cart
    And I add apples to the shopping cart
    And I add milk to the shopping cart
    Then I have 3 items in the shopping cart

The implicit state in this scenario (the current contents of the shopping cart) can be modeled using userData:

@Given("I have an empty shopping cart")
def step(context):
    context.userData = []

@When("I add |word| to the shopping cart")
def step(context, itemName):
    context.userData.append(itemName)

@Then("I have |integer| items in the shopping cart")
def step(context, expectedAmount):
    test.compare(len(context.userData), expectedAmount)

Given("I have an empty shopping cart", function(context) {
    context.userData = [];
});

When("I add |word| to the shopping cart", function(context, itemName) {
    context.userData.push(itemName);
});

Then("I have |integer| items in the shopping cart", function(context, expectedAmount) {
    test.compare(context.userData.length, expectedAmount);
});

Given("I have an empty shopping cart", sub {
    my $context = shift();
    $context->{userData} = [];
});

When("I add |word| to the shopping cart", sub {
    my($context, $itemName) = @_;
    push @{$context->{userData}}, $itemName;
});

Then("I have |integer| items in the shopping cart", sub {
    my($context, $expectedAmount) = @_;
    test::compare(scalar(@{$context->{userData}}), $expectedAmount);
});

Given "I have an empty shopping cart" {context} {
    $context userData {}
}

When "I add |word| to the shopping cart" {context itemName} {
    set items [$context userData]
    lappend items $itemName
    $context userData $items
}

Then "I have |integer| items in the shopping cart" {context expectedAmount} {
    set items [$context userData]
    test compare [llength $items] $expectedAmount
}

Given("I have an empty shopping cart") do |context|
  context.userData = []
end

When("I add |word| to the shopping cart") do |context, itemName|
  context.userData.push itemName
end

Then("I have |integer| items in the shopping cart") do |context, expectedAmount|
  Test.compare context.userData.count, expectedAmount
end

Here, the userData field keeps track of the items added to the shopping cart, because this is extra data related to the context of our test. It could also be used to keep track of active Application Context objects in multiple-AUT test cases, for example.

Accessing Tables and Multi-Line Text

The context object exposes optional extra arguments to the step with the multiLineText and table properties.

The multiLineText property returns an optional multi-line text argument to a step. The text is returned as a list of strings, each of which being a single line. For example, in the following step:

Given I create a file containing
"""
[General]
Mode=Advanced
"""

The multi-line text can be accessed via

@Step("I create a file containing")
def step(context):
    text = context.multiLineText
    f = open("somefile.txt", "w")
    f.write("\n".join(text))
    f.close()

Step("I create a file containing", function(context) {
  var text = context.multiLineText;
  var file = File.open("somefile.txt", "w");
  file.write(text.join("\n"));
  file.close();
});

Step {I create a file containing} {context} {
  set text [$context multiLineText]
  set f [open "somefile.txt" "w"]
  puts $f [join $text "\n"]
  close $f
}

Step("I create a file containing") do |context|
    File.open("somefile.txt", 'w') { |file|
        file.write(context.multiLineText.join("\n"))
    }
end

use Squish::BDD;

Step "I create a file containing", sub {
    my %context = %{shift()};
    my @multiLineText = @{$context{'multiLineText'}};

    my $text = join("\n", @multiLineText);

    open(my $fh, '>', 'somefile.txt') or die;
    print $fh $text;
    close $fh;
};

The table property returns an optional text argument to a step. The table is returned as a list of lists: each inner list represents a row, and individual list elements represent cells. For example, in the following step:

Given I enter the records
  | firstName | lastName | age |
  | Bob       | Smith    | 42  |
  | Alice     | Thomson  | 27  |
  | John      | Martin   | 33  |

The table argument can be accessed via

@Step("I enter the records")
def step(context):
    table = context.table

    # Drop initial row with column headers
    for row in table[1:]:
        first = row[0]
        last = row[1]
        age = row[2]
        # ... handle first/last/age

Step("I enter the records", function(context) {
    var table = context.table;

    // Skip initial row with column headers by starting at index 1
    for (var i = 1; i < table.length; ++i) {
        var first = table[i][0];
        var last = table[i][1];
        var age = table[i][2];
        // ... handle first/last/age
    }
});

Step {I enter the records} {context} {
  set table [$context table]

  # Drop initial row with column headers
  foreach row [lreplace $table 0 0] {
    foreach {first last age} $row break
    # ... handle $first/$last/$age
  }
}

Step("I enter the records") do |context|
    table = context.table

    # Drop initial row with column headers
    table.shift

    for first,last,age in table do
        # ... handle first/last/age
    end
end

use Squish::BDD;

Step "I enter the records", sub {
    my %context = %{shift()};
    my @table = @{$context{'table'}};

    # Drop initial row with column headers
    shift(@table);

    for my $row (@table) {
        my ($first, $last, $age) = @{$row};
        # ... handle $first/$last/$age
    }
};

Defining Step Implementations using Given/When/Then

In addition to Step, Squish supports defining step implementations using three more functions, called Given, When and Then. These three functions follow the same syntax as Step but cause a slightly different behavior when executing steps since step implementations registered using Step match more step texts than those registered with any of the other three functions. The following table illustrates the difference:

Patterns registered via Step always match, no matter what keyword the step at hand starts with. Given, When and Then only match if the step starts with the matching keyword or a synonym such as And or But.

Whether patterns registered via Given/When/Then match steps starting with And or But depends on the keyword preceding the And/But. Consider:

Feature: Some Feature
  Scenario: Some Scenario
    Given I am hungry
      And I'm in the mood for 20000 of something
      But I do not like peas
     Then I will enjoy rice.

In this feature description, the lines starting with "And" and "But" succeed a line starting with "Given", they are synonymous. This means that only patterns registered via Step or Given would match the step "And I'm in the mood for 20000 of something".

Step Lookup Order & Overriding Shared Step Implementations

When executing a BDD test, Squish will respect a specific order when deciding which step implementation matches a given step text. Step implementations are considered based on the order in which source files containing step implementations are loaded when starting a BDD test. By default, this order is as follows:

1. steps subdirectory of current test case.
2. shared/scripts/steps directory of current test suite.

This order causes steps defined in the local test case to be preferred over steps defined in the shared scripts directory of the test suite.

Actually, Squish will not only load step implementations stored in above-mentioned directories but also those stored in subdirectories. For instance, you can register step implementations in files like steps/basic/basicsteps.py or steps/uisteps/mainwindowsteps.js.

Even though registering two step implementations with the same pattern will yield an error, registering two step implementations which are defined in different directories will not trigger an error. This means that you can 'override' shared steps by using the exact same pattern in a testcase-specific file.

Influencing Scenario Execution from Within Step Implementations

It is possible to define step implementations such that the current scenario is aborted and all subsequent steps (if any) are skipped. This is useful in case subsequent steps rely on certain conditions (e.g. some file exists) but a preceding step which is supposed to establish/verify that condition fails.

To skip subsequent steps in the current scenario, step implementations can return a special value - the exact name of which depending on the respective scripting language:

@Step("I create a file containing")
def step(context):
    try:
        text = context.multiLineText
        f = open("somefile.txt", "w")
        f.write("\n".join(text))
        f.close()
    except:
        # Failed to create file; skip subsequent steps in current scenario
        return AbortScenario

Step("I create a file containing", function(context) {
  try {
    var text = context.multiLineText;
    var file = File.open("somefile.txt", "w");
    file.write(text);
    file.close();
  } catch (e) {
    // Failed to create file; skip subsequent steps in current scenario
    return AbortScenario;
  }
});

Step {I create a file containing} {context} {
  if {[catch {
    set text [$context multiLineText]
    set f [open "somefile.txt" "w"]
    puts $f [join $text "\n"]
    close $f
  }]} then {
    # Failed to create file; skip subsequent steps in current scenario
    return AbortScenario
  }
}

Step("I create a file containing") do |context|
    begin
        File.open("somefile.txt", 'w') { |file|
            file.write(context.multiLineText.join("\n"))
        }
    rescue Exception => e
        # Failed to create file; skip subsequent steps in current scenario
        next ABORT_SCENARIO
    end
end

use Squish::BDD;

Step "I create a file containing", sub {
    my %context = %{shift()};
    my @multiLineText = @{$context{'multiLineText'}};

    my $text = join("\n", @multiLineText);

    if (open(my $fh, '>', 'somefile.txt')) {
        print $fh $text;
        close $fh;
    } else {
        # Failed to create file; skip subsequent steps in current scenario
        return ABORT_SCENARIO;
    }
};

Performing Actions During Test Execution Via Hooks

It is common that some actions need to be performed before or after some event occurs. This need arises in cases like

• Global variables should be initialized before test execution starts.
• An application needs to be started before a Feature is executed.
• Temporary files should be removed after a Scenario is executed.

Squish allows defining functions which are hooked into the test execution sequence. The functions can be registered with certain events and are executed before or after an event occurs. You can register as many functions for an event as you like: the hook functions will be called in the same order as they were defined. Functions can be associated with any of the following events:

• OnFeatureStart/OnFeatureEnd: These events are raised before/after the first/last scenario in a specific Feature is executed. The context argument passed to functions associated with either of these events provides a title field which yields the title of the feature which is about to get executed (resp. just finished executing).
• OnScenarioStart/OnScenarioEnd: These events are raised before/after the first/last step in a specific Scenario is executed. The OnScenarioStart event is also raised before any Background is executed. The context argument passed to functions associated with either of these events provides a title field which yields the title of the scenario which is about to get executed (resp. just finished executing).
• OnStepStart/OnStepEnd: These events are raised before/after the code in a specific Step is executed. The context argument passed to functions associated with either of these events provides a text field which yields the text of the step which is about to get executed (resp. just finished executing).

You can associate code to be executed in case any of these events occur using language-specific API. In general, the name of the function to sign up for an event equals the name of the event except for Python, where the name of the function doesn't matter since a decorator is used instead. These functions are registered in the same script files in which you would use the Step function to register step implementations.

Here are a few examples:

Setting up a OnFeatureStart hook to setup global variables:

OnFeatureStart(function(context) {
    counter = 0;
    inputFileName = "sample.txt";
});

my $inputFileName;
my $counter;

OnFeatureStart sub {
    $counter = 0;
    $inputFileName = 'sample.txt';
};

@OnFeatureStart
def hook(context):
    global counter
    global inputFileName
    counter = 0
    inputFileName = "sample.txt"

OnFeatureStart do
    @counter = 0
    @inputFileName = 'sample.txt'
end

OnFeatureStart {context} {
  global counter
  global inputFileName
  set $counter 0
  set $inputFileName "sample.txt"
}

Registering OnScenarioStart & OnScenarioEnd event handlers to start & stop an AUT:

OnScenarioStart(function(context) {
    startApplication("addressbook");
});

OnScenarioEnd(function(context) {
    currentApplicationContext().detach();
});

OnScenarioStart sub {
    ::startApplication('addressbook');
};

OnScenarioEnd sub {
    ::currentApplicationContext.detach();
};

@OnScenarioStart
def hook(context):
    startApplication("addressbook")

@OnScenarioEnd
def hook(context):
    currentApplicationContext().detach()

OnScenarioStart do |context|
    Squish::startApplication 'addressbook'
end

OnScenarioEnd do |context|
    Squish::currentApplicationContext.detach
end

OnScenarioStart {context} {
  startApplication "addressbook"
}

OnScenarioEnd {context} {
    applicationContext [currentApplicationContext] detach
}

Generating extra warning log output whenever a step which mentions the word delete is about to get executed:

OnStepStart(function(context) {
    var text = context["text"];
    if (text.search("delete") > -1) {
        test.warning("About to execute dangerous step: " + text);
    }
});

OnStepStart sub {
    my %context = %{shift()};
    if (index( $context{text}, 'delete') != -1) {
        ::test::warning("About to execute dangerous step: $context{text}");
    }
};

@OnStepStart
def hook(context):
    text = context.text
    if text.find("delete") > -1:
        test.warning("About to execute dangerous step: %s" % text)

OnStepStart do |context|
    if context['text'].include? 'delete'
        Squish::Test.warning "About to execute dangerous step: #{context['text']}"
    end
end

OnStepStart {context} {
  set text [$context text]
  if {[string first "delete" $text] > -1} {
    test warning "About to execute dangerous step: $text"
  }
}

The Anatomy of a BDD Test Case

All BDD test cases start by running a regular script in your chosen scripting language. This script is auto-generated, can be found in the same directory as the corresponding test.feature file, and is called test.xy, where xy = (pl|py|rb|js|tcl). It looks like this:

source(findFile('scripts', 'python/bdd.py'))

setupHooks('../shared/scripts/bdd_hooks.py')
collectStepDefinitions('./steps', '../shared/steps')

def main():
    testSettings.throwOnFailure = True
    runFeatureFile('test.feature')

source(findFile('scripts', 'javascript/bdd.js'));

setupHooks(['../shared/scripts/bdd_hooks.js']);
collectStepDefinitions(['./steps', '../shared/steps']);

function main()
{
    testSettings.throwOnFailure = true;
    runFeatureFile("test.feature");
}

use warnings;
use strict;
use Squish::BDD;

setupHooks("../shared/scripts/bdd_hooks.pl");
collectStepDefinitions("./steps", "../shared/steps");

sub main {
    testSettings->throwOnFailure(1);
    runFeatureFile("test.feature");
}

# encoding: UTF-8
require 'squish'
require 'squish/bdd'

include Squish::BDD
setupHooks "../shared/scripts/bdd_hooks.rb"
collectStepDefinitions "./steps", "../shared/steps"

def main
    Squish::TestSettings.throwOnFailure = true
    Squish::runFeatureFile "test.feature"
end

source [findFile "scripts" "tcl/bdd.tcl"]

Squish::BDD::setupHooks "../shared/scripts/bdd_hooks.tcl"
Squish::BDD::collectStepDefinitions "./steps" "../shared/steps"

proc main {} {
    testSettings set throwOnFailure true
    runFeatureFile "test.feature"
}

There are some functions that are called from this script which are part of the Squish BDD API, but are rarely used in regular testcases. This section will explain what those functions do, in case you wish to reuse BDD steps or other BDD features in script-based tests.

Setting Up Hooks

First, setupHooks() is called at the start of each BDD test case to scan for and set up the hook functions described in Performing Actions During Test Execution Via Hooks.

Collecting Step Definitions

Next, collectStepDefinitions() is used to scan for and import step definitions found in the directories specified as arguments to the function. One or more arguments can be provided, and the earlier ones have priority over the later ones. Step implementations are typically located in files called steps.xy, where xy = (pl|py|rb|js|tcl).

collectStepDefinitions('./steps', '../shared/steps')

collectStepDefinitions('./steps', '../shared/steps');

use Squish::BDD;
collectStepDefinitions("./steps", "../shared/steps");

include Squish::BDD
collectStepDefinitions "./steps", "../shared/steps"

source [findFile "scripts" "tcl/bdd.tcl"]
Squish::BDD::collectStepDefinitions "./steps" "../shared/steps"

Hence in above example, the step definition from the current Test Case Resources will be used in favor of one with the same name, if it is also found in Test Suite Resources.

Finally, runFeatureFile() can be called on a BDD .feature file after hooks are set up and steps are collected, to run all the Scenarios in the specified Gherkin feature file.

Reading Tables from External Files using the From Keyword

Sometimes it may be more practical to store table data for Scenario Outline placeholders or the context table in an external file instead of having the table directly in the feature file. Examples for this are large data sets or use of the same data set in multiple BDD test cases.

In order to read a table from an external file, we use the From keyword followed by the file path relative to the .feature file. For example:

Scenario: Register orders
    Given orderbook is running
      And orderbook is empty
     When I register orders
        From testdata/order_list.txt
     Then orderbook is not empty

In this case, we store the order_list.txt file in the testdata directory next to our feature file. It contains a table in the familiar Gherkin format.

# testdata/order_list.txt
| name | date       | type   | amount |
| Amy  | 04.02.2017 | apple  |      2 |
| Josh | 04.02.2017 | peach  |      3 |
| Marc | 05.02.2017 | cheese |     14 |
| Lea  | 07.02.2017 | soda   |      1 |

Squish accepts table files in the following formats:

• Raw text (.txt or no extension)
• Comma-separated values (.csv)
• Tab-separated values (.tsv)
• Microsoft Excel Sheet (.xls)

When reading tables from raw text files (.txt or no extension), Squish simply jumps into the file and reads it just like it was written inline in the feature file (thus we could also use further From statements inside of a text table file). For all other formats, Squish parses the records directly.

From uses the header (first row) in the table file to map the table columns to script variables or Scenario Outline placeholders by default, but it is possible to define a custom header to be used instead before reading the table file:

| customer | date | item | count |
From testdata/order_list.txt

This makes it easier to use table files with our test scripts.