Unit testing of classes with functions as parameters in C++

Question

Let's say I have a function in a class with the following signature:

int fun(int x, int y,std::function<int(int, int)> funArg)

The output depends on the operations done in funArg.

My question is, how do you ensure that the code is tested in the right way? I can of course add a unit test with some random funArg, but as more people use the class, the more different functions they will pass as arguments. Do I have to make people add every new funArg to the unit test? Or is this approach wrong?

• Doc Brown: This question shows a misconception which we see on this site here every week, again and again: that the semantics of the code isn't important for design and tests, just the formal structure. Come on, you can do better: give us examples with more meaningful names than fun or funArg, tell us about the intended behaviour of this function, what it does and how complex it is. Don't assume this is not important for the tests.
• Caleth: Functions are values. Would you ask "Do I have to make people add every new x to the unit test?" You might find there are some constraints on valid inputs not modelled by the type system, but that's the same kind of constraint as "x must be even"

Answer 1

The testable part of fun is only the code around the call to funArg. So feed it with a trivial funArg parameter that doesn't have to do anything but return a value that causes fun to touch one of the possible code paths after calling funArg.

For example, if fun has special case handling for zero or negative results from calling funArg, those should be tested in individual unit tests.

The possible candidates for funArg in real life need their own unit tests, but those should not be mixed with testing of fun.

• Wballer3: Thank you for your answer. So if funArg is a lambda, how would you test it then in a unit-test? Or maybe one should write very simple lambdas, and if they contain a lot of logic, it is a sign that they should be extracted to a new class?
• bdsl: Let the test pass in a simple lambda that has one side effect (e.g. setting a property on the testcase class) and returns a value. You can then verify that your function under test executes the lamda only when it's supposed to, and verify that it does what it needs to do with the return value.
• Hans-Martin Mosner: @Wballer3 too big lambdas are a code smell, they should be extracted into separate methods or functions, and then they become testable.

Answer 2

As others have pointed out, the semantics of the function parameter matter. The tests for fun and funArg are two completely separate tests, and are different kinds of tests at that.

Testing funArg

Since funArg will ultimately be custom code provided by users of fun, you want to make sure that the abstract behavior of funArg is what fun expects - which means that you have to consider what that abstract behavior is, and how to express, test & document it. The significance of funArg is that it is an extension point for the logic in fun, and as such, it is defined by not just the signature, but also by its semantics with respect to fun.

Let me give you an example; suppose fun does some logic that involves user-defined comparisons of x and y. and that funArg is the familiar¹ comparison function int compare(a, b).

---

¹ It appears in various languages and libraries; the gist of it is:

int compare(a, b) returns: 
 Less than zero if a < b 
 Zero if a == b 
 Greater than zero if a > b

---

Suppose also that that fun expects funArg to establish a well defined order.

That constrains the behavior of compare; it has to be written a certain way for fun to work, as the code in fun makes certain assumptions.
E.g. it has to follow these rules (say):

• multiple calls to compare(a, b) have to return consistent results
• compare(a, a) must return 0
• if compare(a, b) == compare(b, a), then it must be that compare(a, b) == 0, and that a == b (by some compatible notion of equality).
• compare(a, b) and compare(b, a) cannot otherwise indicate the same ordering (e.g., they cannot both return negative, but different numbers; they have to have different signs).
• if compare(a, b) < 0 and compare(b, c) < 0, then compare(a, c) < 0 must also be true.
• etc., etc., depending on what you're trying to do

If this is what fun expects from funArg in order to do its job, than if a user supplies an implementation (a lambda) that violates some of these rules, then fun will not work (or worse, it will appear to work for a while, and then produce a weird bug). The supplied function would break the Liskov substitution principle with respect to fun. But that's on the caller, not on you.

You can write this set of bullet points as a set of tests that, by virtue of existing, (1) document this behavior, and (2) exercise the candidate lambda on some number of examples, thus providing some confidence that the implementation is correct.

I can of course add a unit test with some random funArg, but as more people use the class, the more different functions they will pass as arguments.

Your job here is not to test these functions for them; your job is to write the test for the abstract funArg, and make it accessible to people as part of the documentation of fun (which must also document what funArg is supposed to represent). That is, your job is to design the behavioral contract of funArg with respect to fun.

This is so that other people can plug in their own lambdas and run the test on their code, to check if it confirms to the specification of fun (a library function they have chosen to use). The test for funArg defines what kinds of extensions fun accepts.

You may write a funArg implementation for your own use, or a test-only mock implementation, but the test you're writing here is a high-level test for pluggable code, so it has to allow for different lambdas to be plugged in (TDD frameworks aren't great at this, but it can be done).

Another complication is that, depending on the application, the specific values you used for your test cases may not work well for user-provided implementations; it's not an easy problem, and it may be that the best you can do is publish the funArg tests as a reference that other people can use when writing their own test suite.

In any case, you may decide that you don't want to bother writing/publishing such a test - and that's fine; that's up to you (and your team), and depends on how critical it is to get the implementation right, how nuanced the expected behavior is, etc.

Testing fun

Unlike the other one, this is a test of code that is in your control. Here, in every test, you'd inject a different, test case–specific lambda. You're not trying to test funArg here at all. You're testing what fun will do given some completely predetermined behavior of funArg that you decided on in order to facilitate the needs of that particular test case.

E.g., suppose you want to test that, if the input is such that x < y, then fun should return y - x, continuing with the example above where funArg is a user-supplied comparison.

Your test would then supply concrete x and y values that make sense in the context of this premise, and for funArg it would supply a lambda that returns a value indicating that x is smaller than y. Remember, you're not testing the lambda here. For the purposes of this test, it can be hardcoded to return -1, without ever looking at x and y! It's a mock implementation, the sole purpose of which is to facilitate the test scenario.

So, within this group of tests, what other people who are going to use fun might provide as funArg is not of concern at all (beyond, you know, some sanity checks for things that may be beyond your control, like parameters resulting in a numerical overflow, or something that's a security concern, like input that needs to be sanitized, etc).