How to increase test granularity without modifying the public interface

Question

First of all, I'm aware that in python nothing is really private but let's assume that we can stick to using the leading underscore to indicate that something is private.

I have a module consisting of a bunch of private pure functions (marked with an underscore) and a single public one. We can think in the public one as a composition of the private ones. The module looks like the following:

.
.
.
def _h(x):
 ...
def _g(x):
 ...
def _f(x):
 ...
def foo(x):
 return ..._h(_g(_f(x)))

My unit test only calls the public one. I have two problems with this test:

1. It is too slow for a unit test (the process is CPU intensive)
2. Because it covers so much, when the test fails it takes extra-effort to determine what went wrong

I would prefer to unit test the individual private functions separately and leave the test of the composition (the public function) to an integration test. I have two questions in this regard:

1. Is this a sensible approach to tackle the issues mentioned above?
2. If yes, what is the right way to implement it? If not, are there any good alternatives?

For the second question, one option is to make the functions public. However, I see no reason to do this because no client uses these functions and it is unlikely that they are useful for anything but the public foo function. Exposing them just for the sake of testing seems not to be adequate (at least my colleagues consider it that way).

The other option is to ignore the underscore convention and test the private functions. I've read that this is usually considered a bad practice because the private functions are considered "implementation details" and can change anytime. Therefore, testing private functions results in brittle tests. However, the behavior of these private functions is rather stable. I would expect changes within these functions, but their input/output is unlikely to change over time. However, I've failed to find an effective convention/mechanism within python to communicate that this would be the case.

• Laiv: Havee you considered extracting the functions and make the principal component a composition? That way you can test everything separately.
• pob: Do you mean making the private functions public and letting the caller perform the composition? This indeed a possibility but the caller would have to perform the composition of various modules like this one, which would have a considerable impact on its readability.
• Laiv: No, that would cause temporal coupling. It's what @VoiceOfUnreason suggests when he says You design your private methods to delegate to a replaceable collaborator
• Steve Chamaillard: Also with that Laiv said, you can mock different calls and prove they’re properly being called and used. Because this function is simply orchestrating calls to other functions, that’s the logic that should be tested here.

Answer 1

If not, are there any good alternatives?

In my experience, not really. There are a number of different possible compromises; how suitable they are depends on local circumstance.

Martin Fowler described a distinction between Public vs Published methods; roughly, there are methods available in the interface that are not officially supported / "reserved for official use".

or

You accept that testing is a first class concern, and that the methods needed to test the system are part of the supported API. It's often the case that what the tests are highlighting are use cases that you haven't considered yet (ex: monitoring / observability / restart / retry)

or

You design your private method to delegate to a public method somewhere else, and test that public method according to its contract. That's great for testing the method, but not so much for ensuring that the original code does the right thing

or

You design your private methods to delegate to a replaceable collaborator - you end up with a bunch of tests for isolated parts, and tests that a collaborator executes its protocol correctly, and punt the "test that everything is compatible" problem to a higher level.

or

You test the private api

The good news is that "best practices" are not a fundamental law of nature. Figure out your real constraints, find practices that satisfy those constraints, then label those practice as the best practices for your context, and get on with providing business value.