🧪 Why property-based testing?
At
Packmind, we’re convinced that methodologies such as
Test-Driven Development (TDD) can really improve the quality of our code. But developers need relevant inputs to ensure our code answers a business requirement. Behavior-Driven Development or Example Mapping sessions help to generate concrete examples of the behavior of a feature. This clearly brings value for developers and allows them to start writing tests and business code.
However, our tests usually cover a
restricted range of inputs that come from our examples. We assume they are representative enough for our tests. But what if we want to test edge cases? Or if we’re not sure that our test is representative?
This is where
property-based testing comes in. This post is a short intro to this concept.
❓ What is Property-Based Testing (PBT)?
Property-based testing aims to identify and test
invariants: predicates that should be true, whatever the input data are. An invariant is a business rule that can be written as a predicate.
A PBT framework will generate random input data and check if the invariant is valid. If one single execution fails, this means the code under test may have some defects in its implementation. But the PBT framework will give you the
input data, meaning you can reproduce the problem.
PBT is not something new, the first research studies on that topic are from 1994, from
Fink & Levitt.
🎓 Illustration with a test case
Our platform Packmind is designed for
best coding practices sharing. Users can
merge two practices if they have the same intention. Each practice can have zero, one, or multiple
categories. During a merge, categories of both practices should be merged into the
target practice.
Here is a simple implementation of this business rule (the
mergePractice method) and a unit test written in JS with Mocha and Chai. For simplicity, the code is written in the same file.
As you can see, this code does not handle any edge case. The implementation is clearly straightforward with a
concat operation. We consider only a typical case with two practices having a single category.
🚀 An implementation of Property-Based Testing with fast-check
Now comes the PBT with
fast-check, an open-source framework developed by
Nicolas Dubien for Javascript and Typescript.
From our example, we can identify
two invariants during the merge of two practices:
- There can’t be any duplications in the categories of the target practice.
- The categories of the target practice should not contain elements that do not appear in the source practices’ categories.
Here is the implementation of the two predicates:
And here is the test with fast-check :
In short:
fc.assert runs the property,
fc.property defines it, and
fc.array(fc.string()) generates a random array of string values, possible empty. We ask here to run 10,000 iterations of the test. Of course the documentation of the framework will give your more information!
After execution of our test (running with Mocha), we got the following error:
See the counterexample? You could argue that such a case should not normally happen, as categories should never be null or empty, but this is not the point here, it’s just an illustration of PBT :). We’ve got which inputs raise the error.
Thanks to that, we can slightly edit our business function:
Next run, we realized that our business case did not cover duplicated categories :
The
mergePractice method should now be updated to avoid duplications.
I think you’ve got it, right? PBT can complement your existing tests, and we showed an example of how this can help improve our codebase quality and the robustness of our tests.
There are many more exciting features, such as
Shrinking, which tries to simplify the understanding of a failing test by reducing the problem at its lowest level.
In a future post, we’ll discuss how mutation testing can also be relevant to improve our codebase.
