ct_hooks

MODULE

ct_hooks

MODULE SUMMARY

A callback interface on top of Common Test

DESCRIPTION

Warning

This feature is in alpha release right now. This means that the interface may change in the future and that there may be bugs. We encourage you to use this feature, but be prepared that there might be bugs and that the interface might change inbetween releases.

The Common Test Hook (henceforth called CTH) framework allows extensions of the default behaviour of Common Test by means of callbacks before and after all test suite calls. It is meant for advanced users of Common Test which want to abstract out behaviour which is common to multiple test suites.

In brief, Common Test Hooks allows you to:

Manipulate the runtime config before each suite configuration call
Manipulate the return of all suite configuration calls and in extension the result of the test themselves.

The following sections describe the mandatory and optional CTH functions Common Test will call during test execution. For more details see Common Test Hooks in the User's Guide.

For information about how to add a CTH to your suite see Installing a CTH in the User's Guide.

Note

See the Example CTH in the User's Guide for a minimal example of a CTH.

CALLBACK FUNCTIONS

The following functions define the callback interface for a Common Test Hook.

EXPORTS

Module:init(Id, Opts) -> State

Types:

Id = reference() | term()

Opts = term()

State = term()

MANDATORY

Always called before any other callback function. Use this to initiate any common state. It should return a state for this CTH.

Id is the return value of id/1, or a reference (created using make_ref/0) if id/1 is not implemented.

For details about when init is called see scope in the User's Guide.

Module:pre_init_per_suite(SuiteName, Config, CTHState) -> Result

Types:

SuiteName = atom()

Config = NewConfig = [{Key,Value}]

CTHState = NewCTHState = term()

Result = {Return, NewCTHState}

Return = NewConfig | SkipOrFail

SkipOrFail = {fail, Reason} | {skip, Reason}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called before init_per_suite if it exists. It typically contains initialization/logging which needs to be done before init_per_suite is called. If {skip,Reason} or {fail,Reason} is returned, init_per_suite and all test cases of the suite will be skipped and Reason printed in the overview log of the suite.

SuiteName is the name of the suite to be run.

Config is the original config list of the test suite.

CTHState is the current internal state of the CTH.

Return is the result of the init_per_suite function. If it is {skip,Reason} or {fail,Reason} init_per_suite will never be called, instead the initiation is considered to be skipped/failed respectively. If a NewConfig list is returned, init_per_suite will be called with that NewConfig list. See Pre Hooks in the User's Guide for more details.

Note that this function is only called if the CTH has been added before init_per_suite is run, see CTH Scoping in the User's Guide for details.

Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -> Result

Types:

SuiteName = atom()

Config = [{Key,Value}]

Return = NewReturn = Config | SkipOrFail | term()

SkipOrFail = {fail, Reason} | {skip, Reason} | term()

CTHState = NewCTHState = term()

Result = {NewReturn, NewCTHState}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called after init_per_suite if it exists. It typically contains extra checks to make sure that all the correct dependencies have been started correctly.

Return is what init_per_suite returned, i.e. {fail,Reason}, {skip,Reason}, a Config list or a term describing how init_per_suite failed.

NewReturn is the possibly modified return value of init_per_suite . It is here possible to recover from a failure in init_per_suite by returning the ConfigList with the tc_status element removed. See Post Hooks in the User's Guide for more details.

CTHState is the current internal state of the CTH.

Note that this function is only called if the CTH has been added before or in init_per_suite, see CTH Scoping in the User's Guide for details.

Module:pre_init_per_group(GroupName, Config, CTHState) -> Result

Types:

GroupName = atom()

Config = NewConfig = [{Key,Value}]

CTHState = NewCTHState = term()

Result = {NewConfig | SkipOrFail, NewCTHState}

SkipOrFail = {fail,Reason} | {skip, Reason}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called before init_per_group if it exists. It behaves the same way as pre_init_per_suite, but for the init_per_group instead.

Module:post_init_per_group(GroupName, Config, Return, CTHState) -> Result

Types:

GroupName = atom()

Config = [{Key,Value}]

Return = NewReturn = Config | SkipOrFail | term()

SkipOrFail = {fail,Reason} | {skip, Reason}

CTHState = NewCTHState = term()

Result = {NewReturn, NewCTHState}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called after init_per_group if it exists. It behaves the same way as post_init_per_suite, but for the init_per_group instead.

Module:pre_init_per_testcase(TestcaseName, Config, CTHState) -> Result

Types:

TestcaseName = atom()

Config = NewConfig = [{Key,Value}]

CTHState = NewCTHState = term()

Result = {NewConfig | SkipOrFail, NewCTHState}

SkipOrFail = {fail,Reason} | {skip, Reason}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called before init_per_testcase if it exists. It behaves the same way as pre_init_per_suite, but for the init_per_testcase function instead.

Note that it is not possible to add CTH's here right now, that feature might be added later, but it would right now break backwards compatability.

Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState) -> Result

Types:

TestcaseName = atom()

Config = [{Key,Value}]

Return = NewReturn = Config | SkipOrFail | term()

SkipOrFail = {fail,Reason} | {skip, Reason}

CTHState = NewCTHState = term()

Result = {NewReturn, NewCTHState}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called after end_per_testcase if it exists. It behaves the same way as post_init_per_suite, but for the end_per_testcase function instead.

Module:pre_end_per_group(GroupName, Config, CTHState) -> Result

Types:

GroupName = atom()

Config = NewConfig = [{Key,Value}]

CTHState = NewCTHState = term()

Result = {NewConfig | SkipOrFail, NewCTHState}

SkipOrFail = {fail,Reason} | {skip, Reason}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called before end_per_group if it exists. It behaves the same way as pre_init_per_suite, but for the end_per_group function instead.

Module:post_end_per_group(GroupName, Config, Return, CTHState) -> Result

Types:

GroupName = atom()

Config = [{Key,Value}]

Return = NewReturn = Config | SkipOrFail | term()

SkipOrFail = {fail,Reason} | {skip, Reason}

CTHState = NewCTHState = term()

Result = {NewReturn, NewCTHState}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called after end_per_group if it exists. It behaves the same way as post_init_per_suite, but for the end_per_group function instead.

Module:pre_end_per_suite(SuiteName, Config, CTHState) -> Result

Types:

SuiteName = atom()

Config = NewConfig = [{Key,Value}]

CTHState = NewCTHState = term()

Result = {NewConfig | SkipOrFail, NewCTHState}

SkipOrFail = {fail,Reason} | {skip, Reason}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called before end_per_suite if it exists. It behaves the same way as pre_init_per_suite, but for the end_per_suite function instead.

Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -> Result

Types:

SuiteName = atom()

Config = [{Key,Value}]

Return = NewReturn = Config | SkipOrFail | term()

SkipOrFail = {fail,Reason} | {skip, Reason}

CTHState = NewCTHState = term()

Result = {NewReturn, NewCTHState}

Key = atom()

Value = term()

Reason = term()

OPTIONAL

This function is called after end_per_suite if it exists. It behaves the same way as post_init_per_suite, but for the end_per_suite function instead.

Module:on_tc_fail(TestcaseName, Reason, CTHState) -> NewCTHState

Types:

TestcaseName = init_per_suite | end_per_suite | init_per_group | end_per_group | atom()

Reason = term()

CTHState = NewCTHState = term()

OPTIONAL

This function is called whenever a testcase fails. It is called after the post function has been called for the testcase which failed. i.e. if init_per_suite fails this function is called after post_init_per_suite, and if a testcase fails it is called after post_end_per_testcase.

The data which comes with the Reason follows the same format as the FailReason in the tc_done event. See Event Handling in the User's Guide for details.

Module:on_tc_skip(TestcaseName, Reason, CTHState) -> NewCTHState

Types:

TestcaseName = end_per_suite | init_per_group | end_per_group | atom()

Reason = {tc_auto_skip | tc_user_skip, term()}

CTHState = NewCTHState = term()

OPTIONAL

This function is called whenever a testcase is skipped. It is called after the post function has been called for the testcase which was skipped. i.e. if init_per_group is skipped this function is called after post_init_per_group , and if a testcase is skipped it is called after post_end_per_testcase .

The data which comes with the Reason follows the same format as tc_auto_skip and tc_user_skip events. See Event Handling in the User's Guide for details.

Module:terminate(CTHState)

Types:

CTHState = term()

OPTIONAL

This function is called at the end of a CTH's scope.

Module:id(Opts) -> Id

Types:

Opts = term()

Id = term()

OPTIONAL

The Id is used to uniquely identify a CTH instance, if two CTH's return the same Id the second CTH is ignored and subsequent calls to the CTH will only be made to the first instance. For more information see Installing a CTH in the User's Guide.

This function should NOT have any side effects as it might be called multiple times by Common Test.

If not implemented the CTH will act as if this function returned a call to make_ref/0.