Features for Testing CSL

CSL tests are written in definitions that have the test modifier. The value of a val test definition must be a test.

Tests are run from the JVM-based CSL interaction APIs.

Test

unitTest : String -> (Unit -> UnitTest) -> Test

The simplest form of a test is a unit test, which is intended to test one small part of a CSL program. The string argument is a human-readable name to be used in test reports. When a unit test is run, the function is invoked, and it will either pass or fail.

Unit tests are built from three operations:

pass : UnitTest

A test that passes.

fail : a -> UnitTest

A test that fails, reporting its argument as the reason for the failure.

expected : a -> b -> UnitTest

A test that fails, reporting a mismatch between its first and second arguments as the reason for the failure. The first argument is interpreted as the expected result, while the second is what was provided.

In addition, a few helpers are provided:

assert : Bool -> UnitTest

Fails when its argument is False, passes otherwise.

assertEqual : a -> a -> UnitTest

Uses the default equal function to check the two arguments for equality, failing if they are not equal. The first value is interpreted as the expected value, the second as the actual one.

assertEqualBy : (a -> b -> Bool) -> a -> b -> UnitTest

Uses the provided function to check the other two arguments for equality, failing if they are not equal. The first value is interpreted as the expected value, the second as the actual one.

assertEqualEpsilon : Float -> Float -> UnitTest

Asserts that two values differ by at most 0.0000001.

suite : String -> List Test -> Test

Collect a list of tests into a single test suite to be run together. The first argument is a human-readable name for the test suite.

withData : String -> (a -> Test) -> Test

States that a test depends on externally-provided data. The string argument is a name used in the JVM test runner to provide data, while the function argument computes a new test from the provided data.

suiteWithData : String -> (a -> Tuple String (Unit -> UnitTest)) -> Test

Construct a test suite from externally-provided data. The external data is expected to have type List a, and the test suite is constructed by applying the function argument to each element of the list, with the tuple being used for the name and the individual test. The string argument is both the name of the suite and the name of the data source.

runScenario : String -> Scenario -> Test

Run a test scenario with the provided name. A scenario consists of a contract together with a sequence of test steps, which describe an execution path for the contract. Test steps may apply events to the contract or make assertions about its current state. Test scenarios are constructed using the scenario operator, which takes a contract and a list of states as arguments.

withAgent : String -> (Agent -> Test) -> Test

Given an agent name as a string, construct an actual agent value to test with and invoke the provided test on the agent. If the same agent name is used twice in a test suite, the same underlying agent value is used.

with2Agents : Tuple String String -> (Tuple Agent Agent -> Test) -> Test

Given two agent names as strings, construct two actual agent values to test with and invoke the provided test on the agents. If the same agent name is used twice in a test suite, the same underlying agent value is used.

Scenario

Scenarios consist of a contract and a list of steps. When run as part of a test suite, the contract is instantiated and then each step is applied in order. If no step fails, then the test passes.

event : Event -> Scenario::Step

Applies the given event to the contract under test. If the event is successfully applied, the contract is replaced with the resulting residual contract. If the event cannot be applied, then the test fails.

observation : String -> a -> b -> Scenario::Step

Adds a new observed value of type b associated with the observation input of type a for the observable denoted by the given String. The observable is denoted by a string value such as "Prices" and in that case it must be the case that a corresponding observable is in scope:

observable Prices : Date -> Float

If instead the observable declaration is in a module Obs, the string-value to pass in the scenario test would be "Obs::Prices".

isFulfilled : Scenario::Step

Asserts that the contract is in a state in which no further events are required. For instance, both success and success or <*> Event then success are fulfilled, although only the latter permits subsequent events. The test fails if the contract under test is not fulfilled.

isNotFulfilled : Scenario::Step

Asserts that the contract is in a state in which further events are required. For instance, <*> Event then success is not fulfilled. The test fails if the contract under test is fulfilled.

steps : List Scenario::Step -> Scenario::Step

“Bundle up” a list of steps into a single step. Running this new step is the same as running the steps in the list. This operator is useful when defining steps that can be re-used in a variety of scenarios, as well as with withRollback or expectFailure.

withRollback : Scenario::Step -> Scenario::Step

Run a scenario step, and then undo any changes to the contract under test. This can be used to explore small alternatives to a scenario without having to copy it. For instance, there might be a series of events to bring the contract into an interesting state, after which withRollback can be used to assert that a variety of events can be applied, but then the test can continue down the main path.

expectFailure : String -> Scenario::Step -> Scenario::Step

Demand that a step causes a failure (such as an event that cannot be applied). If the step does cause a failure, then the contract under test is rolled back and testing may continue. If the step does not cause a failure, then the test fails with the provided string as a reason.

eventsSatisfy : String -> (List Event -> Bool) -> Scenario::Step

Demand that the series of events that have been applied thus far in the current scenario satisfy some predicate. For instance, this could be used to ensure a neutral economic outcome after a number of payments. If the function returns false, the test fails, with the provided string as the reason.