This manual provides documentation for a Racket implementation of Quickcheck, a library that tests the specification of a program with randomly generated test inputs.

The original Quickcheck is a library for Haskell.

(require quickcheck ) package: quickcheck

1Quickstart🔗 i

1.1Installation🔗 i

If you have not installed the package yet, run the following command from the command-line:

raco pkg install quickcheck

Note that this package will require Racket v6.0 or newer to install.

Alternatively, you may use the GUI package manager from DrRacket to install the package.

1.2Running tests🔗 i

To use Quickcheck, you first write a specification for a program as a property in ordinary Racket code. For example, the following is a property that expresses the English specification “the string->number function produces numbers when given strings”:

> (define string->number-returns-number
(property ([strarbitrary-string ])
(number? (string->number str))))

Given such a property, we can run random tests using quickcheck :

> (quickcheck string->number-returns-number)

Falsifiable, after 0 tests:

str = "\u0002\u0003\u0002"

You may have already guessed that this property will be easily falsified, since most strings do not actually parse as numbers.

Next, let’s write a property that we expect will actually succeed. For example, the following encodes the English specification “given two lists of integers, the result of append ing them will have the same length as the sum of the original list lengths”:

> (define append-and-length-agree
(property ([lst-1(arbitrary-list arbitrary-integer )]
[lst-2(arbitrary-list arbitrary-integer )])
(= (+ (length lst-1)(length lst-2))
(length (append lst-1lst-2)))))

Testing the property reveals that it holds up:

> (quickcheck append-and-length-agree)

OK, passed 100 tests.

1.3Creating new generators🔗 i

Let’s say we have this code representing a 2D point.

> (struct point(xy)#:transparent)

> (define (distancep1p2)
(let ([dx(- (point-xp1)
(point-xp2))]
[dy(- (point-yp1)
(point-xp2))])
(sqrt (+ (* dxdx)
(* dydy)))))

We want to create a generator that will produce points. We can use bind-generators to do this:

> (define (choose-point[min-coord-9999][max-coord9999])
(bind-generators
([x(choose-integer min-coordmax-coord)]
[y(choose-integer min-coordmax-coord)])
(pointxy)))

Now we can use that generator to check that the distance from point A to point B should always be the same as the distance from point B to point A:

> (quickcheck
(property ([p1(choose-point)]
[p2(choose-point)])
(= (distancep1p2)
(distancep2p1))))

Falsifiable, after 0 tests:

p1 = #(struct:point -4526 6529) p2 = #(struct:point 8436 4434)

Oops, there is a bug in the distance function. If you fix it, all the checks should pass.

2Running checks🔗 i

procedure
( quickcheck prop)→void?
prop:testable?

Test the given prop, generating new test cases if prop specifies a generator.

Prints the result of the check, including a counterexample for the property if one was found.

Example:

> (quickcheck (property ((strarbitrary-string ))
(string=? str(list->string (string->list str)))))

OK, passed 100 tests.

procedure
( quickcheck-results prop)

→
integer? stamps (or/c boolean? result? )
prop:testable?

Like quickcheck , except it will return three values instead:

an integer representing the number of tests run,
a list of ...,
either #t if the tests were successful, #f if the arguments were exhausted, or an instance of the result structure type on a test failure.

Examples:

> (define-values (ntestsstampsok?)
(quickcheck-results (property ((strarbitrary-string ))
(string=? str(list->string (string->list str))))))
> ntests

100
> (firststamps)

'()
> ok?

#t

procedure
( checkconfigprop)→void?
config:config?
prop:testable?

Runs a check like quickcheck using the given config as a configuration.

Prints the result of the check, including a counterexample for the property if one was found.

procedure
( check-resultsconfigprop)→result?
config:config?
prop:testable?

Like check, except it will return three result values instead of printing the results. See the quickcheck-results function for an explanation of the return values.

3Configuration, results, and utilities🔗 i

struct
(struct config (max-testmax-failsizeprint-every)
#:extra-constructor-namemake-config )
max-test:number?
max-fail:number?
size:(-> integer? integer? )
print-every:(-> integer? (listof any/c )any )

A structure type that represents configurations for the test process. An instance of this type must be supplied when calling the check or check-results functions.

The max-test field represents the maximum number of succeeding tests that will be run. The max-fail field represents the maximum number of tests that can be run with no result before the checker terminates.

The size field should be a function of one argument that produces the test size given the current test number. The print-every field should be a function that takes the test number and the generated arguments and is called for its side effect.

The quickcheck and quickcheck-results functions use a default config where the test count is 100, the test size for test n is (+ 3(quotient n2)), the max fail count is ten times the test count, and nothing is printed. The default config can be adjusted to run different numbers of tests with with-small-test-count , with-medium-test-count , and with-large-test-count .

syntax
( with-small-test-count body...)

Within body... , the number of test cases used by the quickcheck functions is 100.

Example:

> (with-small-test-count
(quickcheck (property ((strarbitrary-string ))
(string=? str(list->string (string->list str))))))

OK, passed 100 tests.

syntax
( with-medium-test-count body...)

Within body... , the number of test cases used by the quickcheck functions is 1000.

Example:

> (with-medium-test-count
(quickcheck (property ((strarbitrary-string ))
(string=? str(list->string (string->list str))))))

OK, passed 1000 tests.

syntax
( with-large-test-count body...)

Within body... , the number of test cases used by the quickcheck functions is 10000.

Example:

> (with-large-test-count
(quickcheck (property ((strarbitrary-string ))
(string=? str(list->string (string->list str))))))

OK, passed 10000 tests.

syntax
( with-test-count test-count-exprbody...)

Within body... , the number of test cases used by teh quickcheck functions is test-count-expr.

Example:

> (with-test-count 42
(quickcheck (property ((strarbitrary-string ))
(string=? str(list->string (string->list str))))))

OK, passed 42 tests.

struct
(struct result (okstamparguments-list)
#:extra-constructor-namemake-result )
ok:(or/c null #t#f)
stamp:(listof string? )
arguments-list:(listof any/c )

Represents a single result from a test. The ok field is #t on success, #f on failure, and null if there is no test result.

The stamp field represents the labels that were relevant to this test execution. The arguments-list is a list of the values generated for checking this test case.

syntax
( property ([idgen/arb-expr]...)body0body...)

gen/arb-expr : (or/c arbitrary? generator? )

Constructs a testable property for functions like quickcheck .

The ids are bound to the result of the given gen/arb-exprs inside the body expressions. The body expressions are used as the bodies of a predicate function that will be run with newly generated values from the specified generators or arbitraries.

Example:

> (property ((strarbitrary-string ))
(string=? str(list->string (string->list str))))

#<property>

procedure
( property? val)→boolean?
val:any/c

Returns #t if val is a property for testing, otherwise returns #f.

procedure
( testable? val)→boolean?
val:any/c

Returns #t if val is a value that can be tested with functions like quickcheck . Returns #f otherwise.

Values that can be tested are the following:

boolean values (#t and #f),
instances of the result structure type,
instances of the property structure type,
or instances of the generator structure type.

4Integration with RackUnit🔗 i

By default quickcheck simply displays whether the property check results in success or, in case of failure, it prints the arguments that falsify the property along with some other metadata. This is not helpful for (semi-) automatic testing as it is done using RackUnit. Fortunately it is possible to combine the best of both worlds.

(require rackunit/quickcheck ) package: quickcheck

syntax
( check-property prop)

Like quickcheck but a RackUnit exception is raised if the property fails. The exception is raised using fail-check and it is augmented with additional data: the number of tests performed, the stamps, and the arguments that falsify the property.

For example, let us check the previous string->number-returns-number property:

> (check-property string->number-returns-number)

--------------------

FAILURE

name: check-property

location: eval:20:0

params: '(#<property>)

ntest:0

stamps: ()

arguments:"str = \"\\u0002\\u0003\\u0002\""

Falsifiable

--------------------

syntax
( check-property/config configprop)

Similar to check-property but taking a specific config object.

syntax
( check-property/quiet prop)

Like check-property but suppresses any output generated during the run.

syntax
( add-property-check-info ((namevalue)...))

Adds specific check-info data when checking a property, using either check-property or check-property/config . Its usage is like with-check-info but it does not define a new scope. Instead, it modifies an internal parameter that is cleared at every usage of check-property or check-property/config .

The purpose of this form is to improve error messages when a property fails. For instance, to compare whether two functions are observationally equivalent for arbitrary arguments, we can define the following form using add-property-check-info to state the expected and actual values inside the declaration of a property:

(define-syntax-rule (conforms-tofg([namegen]... ))
(property ([namegen]... )
(let ([expected(fname... )]
[actual(gname... )])
(add-property-check-info (['expectedexpected]
['actualactual]))
(equal? expectedactual))))

(define (fn)(+ n1))
(define (gn)(- n1))

> (check-property (conforms-tofg([narbitrary-integer ])))

--------------------

FAILURE

name: check-property

location: eval:24:0

expected: 1

actual: -1

ntest:0

stamps: ()

arguments:"n = 0"

Falsifiable

--------------------

Crucially, every time the property is tested with a different set of arguments add-property-check-info overrides the previous data.

5Generators🔗 i

struct
(struct generator (proc)
#:extra-constructor-namemake-generator )
proc:(-> integer? random-generator? any/c )

Represents a source of values for randomly testing a given property with functions like quickcheck .

The proc value should be a function that accepts an integer representing the size of the value, a random number generator, and returns a value for testing.

procedure
( choose-integer lowerupper)→generator?
lower:integer?
upper:integer?

Produces a generator that returns integers between lower and upper.

procedure
( choose-real lowerupper)→generator?
lower:real?
upper:real?

Produces a generator that returns real numbers between lower and upper.

value
choose-ascii-char :generator?

A generator that returns ASCII characters.

value
choose-ascii-letter :generator?

A generator that returns ASCII letter characters.

value
choose-printable-ascii-char :generator?

A generator that returns ASCII characters that have a printed representation.

procedure
( choose-char lowerupper)→generator?
lower:char?
upper:char?

Produces a generator that returns characters with integer values between lower and higher.

procedure
( choose-list elem-gensize)→generator?
elem-gen:generator?
size:integer?

Produces a generator that returns lists of the given size and with elements generated with elem-gen.

procedure
( choose-vector elem-gensize)→generator?
elem-gen:generator?
size:integer?

Produces a generator that returns vectors of the given size and with elements generated with elem-gen.

procedure
( choose-string char-gensize)→generator?
char-gen:generator?
size:integer?

Produces a generator that returns strings of the given size and with elements generated with elem-gen, which must generate characters.

procedure
( choose-symbol char-gensize)→generator?
char-gen:generator?
size:integer?

Produces a generator that returns symbols of the given size and with elements generated with elem-gen, which must generate characters.

procedure
( choose-one-of opts)→generator?
opts:(listof any/c )

Produces a generator that returns one of the values in the list opts.

procedure
( choose-mixed promises)→generator?
promises:(listof (promise/c generator? ))

Produces a generator that returns returns values from one of the generator promises, forcing the promise when it does so.

procedure
( choose-with-frequencies freqs)→generator?
freqs:(listof (cons/c integer? generator? ))

Produces a generator that returns a value from one of the given generators, weighing each generator by the matching integer in freqs.

procedure
( generator-unit val)→generator?
val:any/c

Produces a generator that always returns the given val.

syntax
( bind-generators ([idval-expr]...)body)

Produces a generator based on the results of other generators. Scoping of each id behaves as it does in let* .

If val-expr produces a generator? , the remaining code is threaded through generator-bind , and id will be bound to the value produced by the generator. Note that this delays evaluation until the generator is asked to produce a value.

(struct foo(ab)#:transparent)
;choose-foo creates a generator that produces a foo such that
; foo-a is an integer?
; foo-b is (or/c foo? #f)
(define (choose-foo[recurse-limit3])
(bind-generators
([rand(choose-integer 01)]
[recurse?(and (positive? recurse-limit)
(= 0rand))]
[a(choose-integer 09)]
[b(if recurse?
(choose-foo(sub1 recurse-limit))
#f)])
(fooab)))

procedure
( generator-bind genk)→generator?
gen:generator?
k:(-> any/c generator? )

Produce a generator from the result of calling k on the value generated from the generator gen.

procedure
( generator-sequence gens)→generator?
gens:(listof generator? )

Given the list of generators gens, produce a generator that produces values from them in sequence.

procedure
( sized f)→generator?
f:(-> integer? generator? )

Produces a generator from a function f that constructs a generator given an integer representing a value’s size.

value
arbitrary? :predicate/c

Recognizes arbitrary s.

procedure
( arbitrary gentrans)→arbitrary?
gen:generator?
trans:(-> any/c generator? generator? )

Returns a source of randomly generated values, except where the values are filtered by the function trans to produce a narrower set of new values.

value
arbitrary-boolean :arbitrary?

Generates a boolean value.

value
arbitrary-char :arbitrary?

Generates a character.

value
arbitrary-ascii-char :arbitrary?

Generates an ASCII character.

value
arbitrary-printable-ascii-char :arbitrary?

Generates a printable ASCII character.

value
arbitrary-integer :arbitrary?

Generates an integer.

value
arbitrary-natural :arbitrary?

Generates a non-negative integer.

value
arbitrary-rational :arbitrary?

Generates a rational number.

value
arbitrary-real :arbitrary?

Generates a real number.

procedure
( arbitrary-mixed pred+promises)→arbitrary?
pred+promises:(listof (cons/c (-> any/c any/c )(promise/c arbitrary? )))

Produces a arbitrary generator given a list matching up predicates to promises that produce arbitrary generators.

procedure
( arbitrary-one-of eql?vals)→arbitrary?
eql?:(any/c any/c -> any/c )
vals:(listof any/c )

Produces an arbitrary generator that generates values from the list of values vals. The values are filtered by the equality function eql?.

procedure
( arbitrary-pair fstrst)→arbitrary?
fst:arbitrary?
rst:arbitrary?

Produces an arbitrary generator that generates pairs of values drawn from the fst and rst generators respectively.

procedure
( arbitrary-list elem)→arbitrary?
elem:arbitrary?

Produces an arbitrary generator that generates lists in which the element values are drawn from elem.

procedure
( arbitrary-vector elem)→arbitrary?
elem:arbitrary?

Produces an arbitrary generator that generates vectors in which the element values are drawn from elem.

procedure
( arbitrary-tuple elem...)→arbitrary?
elem:arbitrary?

Produces an arbitrary generator that generates constant-length lists in which the element values are drawn from the generators elems in order.

procedure
( arbitrary-record constructor
accessors
elem...) → arbitrary?
constructor:procedure?
accessors:(listof procedure? )
elem:arbitrary?

Produces an arbitrary generator that generates values of some structure type given a constructor procedure and accessor procedures for that structure type. The values passed to the constructor are drawn from the generators elems.

value
arbitrary-string :arbitrary?

Generates a string.

value
arbitrary-ascii-string :arbitrary?

Generates an ASCII string.

value
arbitrary-printable-ascii-string :arbitrary?

Generates an ASCII string consisting of characters with printable representations.

value
arbitrary-symbol :arbitrary?

Generates a symbol.

procedure
( arbitrary-procedure resultarg...)→arbitrary?
result:arbitrary?
arg:arbitrary?

Generates a procedure that takes arguments drawn from the generators args and which produces a value drawn from the generator result.

6Operations on properties🔗 i

syntax
( ==> bool-exprprop)

Represents implication for testable properties.

If bool-expr is #t, equivalent to prop. Otherwise, produces a property that returns no result.

procedure
( label strtest)→generator?
str:string?
test:testable?

Labels the given test with a str label to help identify what portion of the property is failing.

syntax
( classify really?label-exprtestable-expr)

Labels the result of testable-expr with the value of label-expr if really? evaluates to #t. Otherwise just returns the result of testable-expr.

syntax
( trivial really?testable-expr)

Like classify , but always uses the label "trivial".

procedure
( collect lbltest)→generator?
lbl:any/c
test:testable?

Labels the given test with the written form of the value lbl. Similar to the label function.

7Random number generation🔗 i

procedure
( make-random-generator s1s2)→random-generator?
s1:number?
s2:number?

Constructs a random number generator, given two seeds s1 and s2.

procedure
( random-generator? val)→boolean?
val:any/c

Returns #t if val is a random number generator constructed by make-random-generator and returns #f in all other cases.

procedure
( random-generator-next rand) →
integer? random-generator?
rand:random-generator?

Produce a random integer and a random number generator.

procedure
( random-generator-split rand)

→
random-generator? random-generator?
rand:random-generator?

Splits the given random number generator and returns two random number generators.

procedure
( random-integer rglowhigh)→integer?
rg:random-generator?
low:integer?
high:integer?

Returns a random integer between the bounds low and high.

procedure
( random-real rglowhigh)→real?
rg:random-generator?
low:real?
high:real?

Returns a random real number between the bounds low and high.

top ← prev up next →