leancheck alternatives and similar packages
Based on the "Testing" category
Do you think we are missing an alternative of leancheck or a related project?
LeanCheck is a simple enumerative property-based testing library. Properties
are defined as Haskell functions returning a boolean value which should be
True for all possible choices of argument values. LeanCheck applies
enumerated argument values to these properties in search for a counterexample.
Properties can be viewed as parameterized unit tests.
LeanCheck works by producing tiers of test values: a possibly infinite list of finite sublists of same-and-increasingly-sized values. This enumeration is similar to Feat's. However, the ranking and ordering of values are defined differently. The interface is also different.
Throughout this README lines that begin with the symbol
> indicate a line
entered into an interactive interpreter (
ghci). The result of evaluating the
expression is then printed on the following line.
LeanCheck implementation is easy to understand. LeanCheck's core is under 190 lines of code.
To install the latest LeanCheck version from Hackage, just run:
$ cabal update $ cabal install leancheck
Starting from Cabal v3.0, you need to pass
--lib as an argument to cabal
$ cabal install leancheck --lib
Checking if properties are True
To check if properties are True,
just use the function
:: Testable a => Int -> a -> Bool.
It takes two arguments:
the number of values to test
and a property (function returning Bool),
then, it returns a boolean indicating whether the property holds.
> import Test.LeanCheck > import Data.List > holds 100 $ \xs -> sort (sort xs) == sort (xs::[Int]) True > holds 100 $ \xs ->  `union` xs == (xs::[Int]) False
As a rule-of-thumb, you should run holds for 500, 1 000, or 10 000 tests. With more than that you may run out-of-memory depending on the types being tested.
Finding counter examples
To find counter examples to properties,
you can use the function
:: Testable a => Int -> a -> Maybe [String].
It takes two arguments:
the number of values to test
and a property (function returning Bool).
Then, it returns Nothing if no results are found or Just a list of Strings
representing the offending arguments to the property.
> import Test.LeanCheck > import Data.List > counterExample 100 $ \xs -> sort (sort xs) == sort (xs::[Int]) Nothing > counterExample 100 $ \xs ->  `union` xs == (xs::[Int]) Just ["[0,0]"] > counterExample 100 $ \xs ys -> xs `union` ys == ys `union` (xs::[Int]) Just ["","[0,0]"]
Checking properties like in SmallCheck/QuickCheck
> import Test.LeanCheck > import Data.List > check $ \xs -> sort (sort xs) == sort (xs::[Int]) +++ OK, passed 200 tests. > check $ \xs ys -> xs `union` ys == ys `union` (xs::[Int]) *** Failed! Falsifiable (after 4 tests):  [0,0]
check tests for a maximum of 200 tests.
To check for a maximum of
n tests, use
To get a boolean result wrapped in
There is no "quiet" option, just use
counterExample in that case.
Testing user-defined types
data MyType = MyConsA | MyConsB Int | MyConsC Int Char | MyConsD String instance Listable MyType where tiers = cons0 MyConsA \/ cons1 MyConsB \/ cons2 MyConsC \/ cons1 MyConsD
tiers function return a potentially infinite list of finite sub-lists
(tiers). Each successive tier has values of increasing size.
tiers :: Listable a => [[a]]
For convenience, the function
list returns a potentially infinite list
of values of the bound type:
list :: Listable a => [a]
So, for example:
> take 5 (list :: [(Int,Int)]) [(0,0),(0,1),(1,0),(0,-1),(1,1)]
list function can be used to debug your custom instances.
Listable class instances are more customizable than what is described here:
check source comments or haddock documentation for details.
Standard Listable Instances
LeanCheck comes out-of-the-box with
Listable instances for all types in the
Haskell 2010 Language Report with the intentional exception of a few types.
The leancheck-instances package aims to support types in the
Haskell Platform --
$ cabal install leancheck-instances.
Providers for Tasty, test-framework and Hspec
- LeanCheck provider for Tasty
$ cabal install tasty-leancheck;
- LeanCheck provider for test-framework
$ cabal install test-framework-leancheck;
- LeanCheck provider for Hspec
$ cabal install hspec-leancheck.
Due to the way it is implemented (using lists of lists), LeanCheck can be quite memory intensive if we set the maximum number of tests of a property to millions of values (YMMV).
For the default maximum number of tests (200) you should be safe on most cases. If you use 1 000 or 10 000 as the maximum number of tests for a property you're also generally safe. More than that, it is in a hit or miss basis.
For more details, see LeanCheck memory usage.
LeanCheck strives to be beginner/student friendly both in the interface and its implementation. For instance, to understand LeanCheck's core, one does not need to understand Monads as they aren't used at all there.
In the name of keeping the implementation easy to understand, a compromise were made in terms of performance (cf. LeanCheck memory usage).
LeanCheck is mostly Haskell 98 compliant and almost Haskell 2010 compliant.
With the exception of
Listable derivation modules (TH and Generics),
the only extension used by LeanCheck is CPP. This is to maintain
compatibility with different compilers. LeanCheck even compiles and runs on
Hugs98 from September 2006.
LeanCheck has 100% Haddock coverage with most functions having examples.
For a detailed documentation of each function, see LeanCheck's Haddock documentation.
For an introduction to property-based testing
and a step-by-step guide to LeanCheck, see the
tutorial on property-based testing with LeanCheck
doc/tutorial.md in the source repository).
LeanCheck is subject to a chapter in a PhD Thesis (2017).
LeanCheck has a list of frequently asked questions and answers.