FreeBSD Manual Pages
SHTK_UNITTEST(3) Library Functions Manual SHTK_UNITTEST(3) NAME unittest -- Utilities to implement test programs LIBRARY shtk_import unittest DESCRIPTION The unittest module provides a framework with which to implement test programs. A test program implemented with the unittest library is made up of a combination of standalone test cases and/or test cases within test fix- tures. The test program will exit with 0 (success) if and only if all tests pass, and 1 (failure) if any single test fails. Skips and ex- pected failures are considered passes. Program entry point Test programs implemented using unittest must use the shtk_unittest_main(3) function as their entry point. There are two mechanisms to achieve this. The first is to end the test program with the following definition of main: main() { shtk_unittest_main "${@}"; } The second is to tell shtk(1) at build time to use shtk_unittest_main(3) as the program's entry point: $ shtk build -m shtk_unittest_main module_test.sh In general, prefer the latter mechanism. Asserts vs. expects The unittest library provides a variety of helper check functions. One such example is the `equal' check, which allows the caller to compare two values for equality and fail the test case with the appropriate er- ror message if the values differ. Each check is offered in two flavors: an assert and an expect, which in the previous example means we have access to the two corresponding functions assert_equal and expect_equal. Assert-style checks are fa- tal: when the condition they test for is not met, the test case is im- mediately aborted; instead, expect-style checks are not fatal: they record the currently-running test case as failed but they let the test case continue execution. Assert-style checks should be used to vali- date conditions that must hold true in order to run the test (e.g. en- suring that the setup of the test case succeeds); expect-style checks should be used for all other cases. Scoping All functions provided by the unittest library are prefixed by `shtk_unittest_' as you would expect from the coding practices of shtk. However, such long prefix is inconvenient when writing test programs, as test programs are a very special case of scripts in which a domain- specific language makes sense. Therefore, for simplicity reasons, test cases have access to shortened names of the unittest functions without the `shtk_unittest_' prefix. In order words, a test case can simply call fail instead of having to call shtk_unittest_fail. Standalone test cases Standalone test cases are defined by top-level functions whose name ends with the `_test' suffix and are registered as test cases with the shtk_unittest_add_test(3) function. Such test cases are run indepen- dently of each other, with no common setup nor teardown code among them. Test fixtures Test fixtures are collections of related test cases that may share op- tional setup and teardown methods. Test fixtures are defined by top- level functions whose name ends with the `_fixture' suffix and are reg- istered as fixtures with the shtk_unittest_add_fixture(3) function. Within fixture fuctions, there may be a setup and a teardown method, and there should be one or more test cases registered with shtk_unittest_add_test(3). Each test within the fixture is executed independently of each other, but the setup and teardown methods (if any) are also executed with the test. Test environment The runtime engine (usually kyua(1)) is responsible for executing the test program under a controlled directory and with a sanitized environ- ment. However, due to the nature of shell-based test programs, unittest cre- ates one extra directory for each test case, and such directory becomes the work directory of the test case. These directories are all created within the directory in which the test program was executed. To ensure that any files created by the test case remain within its own subdirec- tory as much as possible, the HOME and TMPDIR variables are set to point to the individual test's work directory. One time setup and teardown Test programs may optionally define top-level one_time_setup and one_time_teardown. The code supplied in the one_time_setup function is executed exactly once at the beginning of the test program and all state set up by the function is shared across all tests. The code supplied in the one_time_teardown function is executed exactly once at the end of the test program and should be used to clean up any state prepared by one_time_setup. It is important to mention that these one-time setup and teardown rou- tines run in the parent directory of the executed test cases. In other words: if the setup routine creates any file to be shared across all tests, the tests will have access to such files by looking them up in their parent directory. Test results Any test case can complete with one of the following results: Expected failure (pass) The test case detected a failure, but such failure is expected. This condition is often use to denote test cases that are either incomplete or test cases that exercise a known bug. See shtk_unittest_set_expected_failure(3) for details. Failure The test case explicitly failed or it unexpectedly terminated. See shtk_unittest_delayed_fail(3) and shtk_unittest_fail(3) for details. Pass The test case completed successfully without reporting any er- rors. Skip (pass) The test case determined that it cannot run through completion and decided to abort early instead of reporting a false negative. See shtk_unittest_skip(3) for details. EXAMPLES The most basic test program that can be written is the following, which provides a simple test case for cp(1)'s -f flag: shtk_import unittest shtk_unittest_add_test cp_f_forces_override cp_f_forces_override_test() { echo "first" >first echo "second" >second chmod 555 second # First make sure that cp without -f fails. assert_command -s exit:1 -e ignore cp first second # Now run a second attempt with -f and verify that the command # exits successfully and it is silent. assert_command cp -f first second cmp -s first second || fail "source and destination do not match" } main() { shtk_unittest_main "${@}"; } A more complex test program can use fixtures to group related test cases and to provide common setup and teardown code for each of them: shtk_import unittest shtk_unittest_add_fixture cp setup() { # Create common files used by all test cases. Note that this runs # once per test case, so state is not shared among them. echo "first" >first echo "second" >second } teardown() { # Common cleanup to be executed at the end of the test case, no # matter if it passes or fails. rm -f first second } shtk_unittest_add_test override_existing_file override_existing_file_test() { assert_command cp first second cmp -s first second || fail "source and destination do not match" } shtk_unittest_add_test f_forces_override f_forces_override_test() { chmod 555 second assert_command -s exit:1 -e ignore cp first second assert_command cp -f first second cmp -s first second || fail "source and destination do not match" } } main() { shtk_unittest_main "${@}"; } Lastly, the most complex test program is depicted here, which includes a combination of fixtures and test cases with one-time setup and tear- down routines: shtk_import unittest one_time_setup() { ... initialization code shared by all tests in the program ... } one_time_teardown() { ... clean up code shared by all tests in the program ... } shtk_unittest_add_fixture clients clients_fixture() { setup() { ... initialization code shared by all tests in the fixture ... } teardown() { ... cleanup code shared by all tests in the fixture ... } shtk_unittest_add_test add add_test() { ... first test in the fixture ... } shtk_unittest_add_test modify modify_test() { ... second test in the fixture ... fail "And it fails" } } shtk_unittest_add_test initialization initialization_test() { ... standalone test not part of any fixture ... skip "But cannot run due to some unsatisfied condition" } # Either do this or, preferably, pass -mshtk_unittest_main to # "shtk build" when compiling the test program and don't define main. main() { shtk_unittest_main "${@}"; } SEE ALSO shtk(3), shtk_unittest_add_fixture(3), shtk_unittest_add_test(3), shtk_unittest_delayed_fail(3), shtk_unittest_fail(3), shtk_unittest_main(3), shtk_unittest_set_expected_failure(3), shtk_unittest_skip(3) Checks shtk_unittest_assert_command(3), shtk_unittest_assert_equal(3), shtk_unittest_assert_file(3), shtk_unittest_assert_not_equal(3), shtk_unittest_expect_command(3), shtk_unittest_expect_equal(3), shtk_unittest_expect_file(3), shtk_unittest_expect_not_equal(3) HISTORY unittest first appeared in shtk 1.6. FreeBSD Ports 14.quarterly November 16, 2014 SHTK_UNITTEST(3)
NAME | LIBRARY | DESCRIPTION | EXAMPLES | SEE ALSO | HISTORY
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=shtk_unittest&sektion=3&manpath=FreeBSD+Ports+14.3.quarterly>