QStudio
Query kdb+ Servers and Chart Results.
QUnit - Kdb Unit Testing Framework
Contents
Intro to Unit Testing
QUnit is a framework for implementing testing in kdb. Unit tests target specific areas of a q program e.g. a single method. Once a test has been written, we can be sure that area of code works and in future if we change that area of code we can be sure that it still works.
The framework enforces a specific structure for tests, this encourages best practices and helps other team members quickly understand the code. Tests should be declared in a separate file within the source code directory and within the file, the functions should be in a namespace. QUnit relies on certain naming conventions detailed below to identify tests and setup routines.
Special Methods - QUnit Naming of functions
QUnit uses special naming conventions to denote tests, all test functions begin with test in their name. The framework will run these tests (order is not guaranteed) and provide a table of successes and failures. An example is shown test for checking that adding works as expected is shown below:
The table below details all of the prefixes which have a special purpose in qunit.
| Name Prefix | Description |
|---|---|
| test* | Identifies that this method is a test. |
| setUp* | Code that should be ran before each test. |
| tearDown* | Code that should be ran before after test. |
| beforeNamespace* | Code that will be ran once, before any tests. |
| afterNamespace* | Code that will be ran once, after all tests have been ran. |
The prefixes other than test are for setting up state before the tests. For example beforeNamespace could be used to open a handle to a remote database that provides test data, rather than reconnecting within each test. All functions should accept no arguments, any results returned from tests will be shown in the result table. Any errors within these setup/teardown functions will cause no tests to run.
Assert Statements
Qunit provides methods for asserting that the output of a function was valid. If one of these assertions fail, their message will be shown and the test will fail. The following assertions are currently provided:
| Name | Description |
|---|---|
| assertEquals[actual; expected; msg] | Assert that actual and expected value are equal. |
| assertThat[actual; relation; expected; msg] | Assert that the relation between expected and actual value holds |
| assertTrue[actual; msg] | Assert that actual is true |
| assertError[func; arg; msg] | Assert that executing a given function causes an error to be thrown |
| assertKnown[actual; expectedFilename; msg] | Assert that the actual result matches the binary data stored in a kdb file. |
| fail[msg] | Make the test fail with given message. |
AssertThat is likely to be most useful as it allows performing any comparison between expected and actual result. The resultEquals tries to be smarter and compare table column names and rows if it expects a table, so as to provide more meaningful output. Each assertion accepts a msg parameter that is there to allow the developer to give information if the test fails to allow others to debug the problem.
Example Test .q file
As seen in q language style guidelines we recommended placing all related functions in one file/namespace. We recommend placing tests in a similarly named file and namespace. e.g. If you have functions for mathematical calculations in a file math.q, they would be placed in the namespace .math, .math.add[], .math.sub[] etc. The tests would then go in a file mathTest.q and the namespace .mathTest. As shown below:
Example math.q file
Example mathTest.q file
Running Tests
From Command Line
Make sure you have your actual code loaded, qunit and our tests then call .qunit.runTests on the test namespace. The framework will then automatically find all tests and log the output of each test and assertion. Finally returning a table with one row per test. Where the sucess or failure together with time and memory required. (Tests assertions can be set on max time/memory). Notice below that our testGetPrimesLessThanMinusOne function failed and that we can see where we expected an empty list, it returned 2.
QStudio Support
For a complete walk through of running example tests in QStudio see the QStudio/qUnit help page.
QStudio IDE has builtin support for qunit. Once your functions have been loaded onto your server you can press Ctrl + T to load qunit, load the tests and run them. Or you can also go "File Menu"->Query->"Unit Test Current Script". After a period the result panel should return a result atble with a row for each test and a pass or fail as shown in the screenshot above.
Test Results
The result of running tests is a table with one row for each test. The table will contain the following columns:
- status - Whether the test passed or failed.
- name - The fully qualified name of the test.
- result - The return value of the test
- actual - If the test failed, this will contain the actual value for the failing assertion.
- expected - If the test failed, this will contain the expected value for the failing assertion.
- msg - Reason why the test assertion failed
- time - The time in milliseconds that the test took to run.
- mem - The memory in bytes that the test took to run.
- maxTime - The maximium time in milliseconds that the test is allowed to run, else it would fail.
- maxMem - The maximium memory in bytes that the test is allowed to run, else it would fail.
HTML/Diff Test Reports
qUnit provides an HTML interface that allows visually diffing the result of any tests that failed. It looks like this:
FAQ
Can I have Timing and Memory Restricted Tests
In the example above we used the below code to set a maximum time/memory limit for the test.
qunitConfig:``!(); qunitConfig[`testGetFactorialSpeed]:`maxTime`maxMem!(100;20000000);Why another testing framework for kdb?
To reduce how much someone experienced in xunit had to learn we stuck as closely as possible to xunit ideas/syntax only using the special features of q where it really made sense. Qunit follows q language best practices allowing easy use within other users frameworks.