Property-Based Testing
A testing approach where you describe properties that must hold for all inputs, and a framework generates hundreds of random test cases to try to break them.
Also known as: Property Testing, PBT, Generative Testing, QuickCheck
Category: Software Development
Tags: software-development, software-engineering, testing, unit-testing, quality, reliability, functional-programming
Explanation
Most tests check examples. `sort([3, 1, 2])` should return `[1, 2, 3]`. You pick a few inputs, write the expected outputs, and hope you thought of the tricky cases. Usually, you didn't. The empty list, the list with duplicates, the list with a million elements, the one with negative numbers... The bugs live in the cases nobody wrote down.
Property-based testing flips the approach. Instead of examples, you describe properties: things that must be true for ANY valid input. The framework then generates many random inputs and checks the property against each one.
Koen Claessen and John Hughes introduced the idea with QuickCheck for Haskell, released in 1999. It has since been ported to most languages: Hypothesis for Python, fast-check for JavaScript and TypeScript, jqwik for Java, proptest for Rust and many more.
**An example**:
For a `sort` function, a few properties:
- The output has the same length as the input
- The output contains exactly the same elements as the input
- Each element is less than or equal to the next one
- Sorting an already sorted list changes nothing
The framework throws hundreds of random lists at these properties. If one fails, you get a concrete counterexample.
**Shrinking**:
This is where property-based testing really shines. When a failing input is found, the framework shrinks it: it searches for the SMALLEST input that still fails. Instead of a random 847-element list, you get `[0, -1]`. Debugging becomes much easier.
**Common property patterns**:
- **Round trip**: `decode(encode(x)) == x` (serialization, parsing, compression)
- **Invariants**: Some quantity never changes (length, total, set of elements)
- **Idempotence**: Doing it twice equals doing it once (`normalize(normalize(x)) == normalize(x)`)
- **Oracle**: Compare against a simpler, slower, trusted implementation
- **Never crashes**: The function returns a valid result for any input (see Total Functions)
- **Commutativity**: The order of operations doesn't matter (`a + b == b + a`)
**Where it fits**:
Property-based tests don't replace example-based unit tests. Examples document specific behavior and are easy to read. Properties explore the input space far more thoroughly than you could by hand. The combination is stronger than either alone.
**The connection to Design by Contract**:
Properties are contracts you check by experiment. Postconditions and invariants (see Design by Contract) make excellent properties: if a function claims `balance >= 0` after every operation, a property-based test can hammer it with random sequences of operations and try to prove it wrong.
Related Concepts
← Back to all concepts