Part 1 - A fluent unit testing framework in VBA: A fluent unit testing framework in VBA
Part 2 - Fluent VBA: One Year Later: Fluent VBA: One Year Later
Two (almost three) years have now passed since I initially wrote Fluent VBA. When I wrote "Fluent VBA: One Year Later", I added several updates to the project. When I had completed those updates, I felt that the project was near completion. I was very surprised when, after that post, I got all of these ideas for massive updates to the testing library. This post will illustrate some of the major changes I’ve implemented in the API since my last post.
Post notes: In the API itself, all of the classes are prefixed with I for interfaces and c for non-interface classes. So the fluent and fluentOf classes are actually cFluent and cFluentOf classes. In this post, I may or may not refer to these classes with the c prefix.
Custom iterables
One thing that I had long wanted to add to the testing library was a custom iterable. I just didn’t know where I would implement it in the library. I eventually got the idea to implement a Test class as a custom iterable. Previously, I had a TestResult class that contained various attributes for a given test result. Initially, TestResult was renamed to TestResults. And the custom iterable was added as TestResult. The custom iterable had several attributes, including a result property. So it made sense to rename it from TestResult to something else. So the TestResult and TestResults classes eventually became the Test and Tests classes respectively. I have several tests within the API. And many of these tests use custom iterables. You can see an example below:
Private Sub subby()
Dim f As IFluent
Dim test As ITest
Set f = New cFluent
f.TestValue = 10
f.Should.Be.GreaterThan 9
For Each test In f.Meta.Tests
Debug.Assert test.Result
Next test
Debug.Print "Finished!"
End Sub
The test class has lots of different properties that the user can use to get lots of different insights from the tests at a high level. I will discuss any relevant properties that the test class implements in other parts of this post.
InputToString method and recursive and iterative algorithms
As I noted in one of my previous posts, I had added an element to the testing library called a FluentPath. The fluent path is a string representation of the chain of objects that the user had followed to get to the testing method. So for a path like Fluent.Should.Be.Something, the fluent path would be "Should be something." The fluent path was updated to add additional attributes like TestingValue and the TestingInput. The TestingValue is the input that’s provided to the TestingValue in Fluent objects, or to the Of method in FluentOf objects. The TestingInput is the input that’s provided to the testing method (where applicable).
As I updated the fluent path to incorporate these methods, I noticed a problem: While I could add a value to the fluent path without issue, I could not do so for objects and arrays. I could add the type name for these things. But this was not really what I wanted for things like data structures. What I really wanted to add was a string representation of the data structure. I wanted the type of the data structure containing any elements that it contained, including other data structures. I decided to add functionality to the library to do this. I called the function that would do this ToString. And I later renamed this functionality to be called InputToString.
Since this function was going to support nested data structures, I had to write a recursive algorithm to do so. I had written recursive algorithms previously. InDataStructure implements both recursive and iterative algorithms to iterate through data structures (including nested ones) for example. I remember the recursive algorithm for InDataStructure taking me at least a few hours to write around 2 – 3 years ago. So I expected this algorithm to take a similar amount of time. After some trial and error, I had written an initial version of the algorithm in about 15 minutes. I was shocked to find out that the algorithm was mostly correct at that point. Some bug fixes at a later time pushed up the total time to perhaps 30 – 40 minutes. But that was still a smaller amount of time than I had initially anticipated. The result of my efforts was the function InputToStrRecur which implements this functionality.
This algorithm is capable of taking input like this:
Dim d As Scripting.Dictionary
Dim col As VBA.Collection
Set d = Nothing
Set col = New Collection
v = Array(New Collection, d, 10)
and producing an output like this:
"Variant(Collection(), Nothing, 10)"
And an input like this:
Set col = New Collection
Set col2 = New Collection
Set col3 = New Collection
col3.Add 1
col2.Add col3
col.Add col2
Produces an output like this:
"Collection(Collection(Collection(1)))"
In total I tested 27 different tests with lots of different object, data structure and value combinations. And all were showing as passing according to my tests. These string attributes can be accessed in the Test objects. They are stored in the StrTestValue and/or StrTestInput properties depending on whether they are used in the testing value and/or testing input respectively.
Attentive readers may have noticed that when I brought up InDataStructure earlier, that I mentioned that it implemented an iterative algorithm as well. So you may be wondering if there is an algorithm like InputToStrIter. I had tried to implement such an algorithm around the time I wrote inputToStrRecur but was ultimately unsuccessful. I spent a whole weekend (I kid you not, I estimate a few dozen hours) trying to implement it but the algorithm was too difficult. While the recursive algorithm handled nearly every input perfectly, every iterative attempt I tried kept failing under some edge case. I honestly believe that the iterative algorithm is the hardest algorithm I’ve ever tried to implement. And I say this as someone who’s written hundreds if not thousands of algorithms.
Months after failing to implement this iterative algorithm, I decided to give it another go. And the good news is that, this time I was successful. To give some reference, I am able to implement an easier algorithm in perhaps half an hour or less. Perhaps a more complex one may take a few hours. And a harder one may take a day or two. Getting the core algorithm for inputToStringIter took probably around four to five days. And work on the algorithm still took me around a week and a half to complete. So I still believe that this is the most difficult algorithm I’ve written.
Iterative algorithms for naturally recursive algorithms can be written for a few different reasons. In this instance, I did it for cross-checking reasons. With the iterative algorithm written, I can now use the iterative algorithm to crosscheck the outputs from the recursive algorithm and vice versa. Both of those 27 tests I mentioned earlier use both recursive and iterative algorithms to ensure the outputs from both algorithms are identical. And both algorithms have helped me catch bugs in the other algorithm when the outputs were not identical. These bugs would have been very difficult to spot without this cross-checking structure. So the tests work exactly as expected. And this proves that it was really worth all the investment in writing InputToStrIter.
Algorithm property updates
After I was able to add the inputToString algorithms, I now had two pairs of recursive and iterative algorithms. Since I now had more than one pair, it made sense to determine which algorithm to use using a property. So I added an Algorithm property to cTests that takes an flAlgorithm enumeration. Its value is set to flRecurisve implicitly by default. But can be set to flIterative or flRecurisve explicitly. These values determine whether an iterative or recursive algorithm is used for the methods that support it respectively. I refactored the data structure method to use this new property in mTests. You can see an example below:
Set tfRecur = MakeFluentOf ‘//creates an instance of a fluentOf object
Set tfIter = MakeFluentOf
tfRecur.Meta.Tests.Algorithm = flRecursive
tfIter.Meta.Tests.Algorithm = flIterative
arr = Array()
fluent.TestValue = testFluent.Of(10).Should.Be.InDataStructure(arr) 'with implicit recur
Debug.Assert fluent.ShouldNot.Be.EqualTo(True)
b = tfRecur.Of(10).Should.Be.InDataStructure(arr) = tfIter.Of(10).Should.Be.InDataStructure(arr)
fluent.TestValue = testFluent.Of(b).Should.Be.EqualTo(False) 'with explicit recur and iter
Debug.Assert fluent.ShouldNot.Be.EqualTo(True)
The data structure methods, which were created before this property, either required it or had the option to provided it optionally. Since this could now be determined by this property, those tests were refactored to no longer support the explicit property.
Null and Empty value updates
Null values
Fluent VBA is a unit testing library. Unit testing libraries typically return Booleans. And so, most unit testing libraries return either true or false. What should a unit testing library do in the case where a function does not evaluate to true, but does so not because the check evaluated to false, but because the parameters were of the wrong type, and so the test made no sense?
For example, Fluent VBA has a testing method called Something(). Something() is a testing method that expects objects as its testing value. So what should happen if the Something() method is called on a value like a String, Number, Boolean, etc.?
Someone might argue that we should return false in this case. But doing so you would not be able to distinguish between what you might call "real" false values and "technical" false values. And that could lead to difficult to spot bugs for the end-user.
Another option might be to raise an exception. This would typically be a good idea in code that could be compiled as a library. But that cannot happen with VBA code. The issue with raising an exception in VBA is that what happens when an exception is raised in custom VBA code is determined by the editor settings. So if you have break in class modules enabled, like I do, VBA would break within the testing module. That kind of experience would take the user out of the client API and into the internal API. And that would be a suboptimal experience. The user could alter their editor setting, but they may have to realter it based on the needs of their own VBA code. So neither of these approaches are ideal.
It turns out that there is a third approach: Using null values. Null is an obscure value that is not commonly found in VBA code. It can only be assigned to variables that are of the variant data type. And it turns out that null is used in cases like the one I brought up earlier. Excel has a few "has" functions in it’s API for example like HasFormula. HasFormula returns true if every cell within a range has formulas. It returns false if every cell in a range does not have formulas. But what if a range is a mix of formulas and non-formulas? What should HasFormula return in this case? While you may feel it should clearly return true or false in this instance, I do think it’s fair to say that it could be interpreted as going either way. And neither interpretation would be deemed as unreasonable. So what does have formula return in this third instance? It returns null. So I decided to update Fluent VBA to return these null values.
All of the testing method in fluent VBA now check that the inputs are of the expected data type. If they are not, then the functions return null. You can see an example below:
fluent.TestValue = testFluent.Of("Hello World!").Should.Be.Something
Debug.Assert fluent.Should.Be.EqualTo(Null) ‘//true
Adding nulls to fluent VBA required massive updates throughout the codebase. Many of the updates I’ve made to the library are completed within a day. And perhaps some of the longer updates take a weekend. The update to add nulls to Fluent VBA took two weekends. When I was writing some of the tests, I got an "invalid use of null error". I had gotten this error because I had accidentally set a variable to nothing when I did not mean to. Without the null updates, there’s a chance that I wouldn’t have caught that error. So issues like that show that the update is valuable and was worth my efforts.
Empty values
A similar update I made was for empty values. I won’t give as detailed of an update on Empty values since most of the points I made about null values also apply to empty values. And a lot of the updates I made for Empty values were able to reuse updates that I had previosuly made for null values. You can see an example below:
fluent.TestValue = testFluent.Of().Should.Be.EqualTo("Hello world")
Debug.Assert fluent.Should.Be.EqualTo(Empty) ‘//True
So with this update, functions return Empty when an initial testing value has not been provided.
Adding interfaces to almost all of the classes
The classes in Fluent VBA were initially designed to only implement interfaces if they needed them. If I needed to refactor a class, I would save a new file and make the updates in the class. If there was some issue with the implementation, I could open and reference the original file. While I didn’t do this personally, another option was to create a new class module, paste in the original code, and rename the old file. While these methods worked, they were suboptimal. The proper approach would have been for every class to implement a base interface. Doing this, I would be able to add any new classes I wished, implement the interface, and just update one assignment statement where the object is assigned in the class. So this approach was much more flexible and robust. As an example of the fluent objects, they went from this:
Private Sub originalFluentExamples()
Dim Fluent As cFluent
Dim FluentOf As cFluentOf
Set Fluent = New cFluent
Set FluentOf = New cFluentOf
'more testing code here
End Sub
To this:
Private Sub updatedFluentExamples()
Dim Fluent As IFluent
Dim FluentOf As IFluentOf
Set Fluent = New cFluent
Set FluentOf = New cFluentOf
'more testing code here
End Sub
And the other classes in the library went through similar updates. This seems like (and is) a small update from the perspective of the client API. But from a development perspective it makes the project much more robust.
Adding data structure methods
Fluent VBA had some testing methods to operate on data structures. The most prominent of these were the InDataStructure methods. While these methods are useful and powerful, I wanted to add other data structure methods that could provide different functionality. I wanted to add data structure that could determine whether:
- Two data structures are identical
- Two data structures contain the exact same elements
- Two data structures contain the same elements
- Two data structures contain the same unique elements
You can see a breakdown of the descriptions for the data structure methods that I added below:
IdenticalTo: Requires that the outer most data structures have the same type. And requires that both data structures have identical elements in the exact same order
ExactSameElementsAs: Does not require the outer most data structures to have the same type. Requires that both data structures have identical elements in the exact same order
SameElementsAs; Does not require the outer most data structures to have the same type or the elements to be in the same order. Requires that both data structures have identical elements.
SameUniqueElementsAs: Does not require the outer most data structures to have the same type, the elements to be in the same order, or the elements to be identical. Requires that the unique elements within the data structure are identical.
With these methods, the testing library has a solid foundation for working with data structures.
Adding dynamic data structure support
Fluent VBA supports four data structures out of the box. It supports the three main data structures in VBA which are the array, the collection, and the dictionary. In addition to these, it also supports the arraylist. As I’ve noted, I have several features in the library that operate on data structures. But if I limited these methods to only support pre-determined data structures, it would be limiting. So to work around this, I added dynamic data structure support. The user is able to add dynamic data structure types to the library and have them supported with the testing methods. To do this, the user needs to create an instance of the data structure that they want to add to the library. Once they’ve done that, they can use the AddDataStructure method in the Tests class to add the data structure. After they’ve added it, if the data structure is iterable, they should be able to use that data structure with the testing methods. You can see an example below that uses InDataStructure as well as the strTestInput property:
Dim q As Object
Set q = CreateObject("system.collections.Queue")
q.Enqueue ("Hello")
fluent.Meta.Tests.AddDataStructure q
fluent.TestValue = "Hello"
Debug.Assert fluent.Should.Be.InDataStructure(q)
Debug.Assert fluent.Meta.Tests(.Tests.Count).StrTestInput = "Queue(`Hello`)" ‘//True
Duplicate Test Event update
A feature that I wanted to add some time was a DuplicateTest event. This event is raised if a duplicate fluent path is encountered. You can see an implementation below:
Private Sub pTestDuplicate_DuplicateTest(ByVal test As ITest)
‘more code here
End Sub
This event works but has some issues. One issue is that for certain testing methods, duplicate fluent paths are currently being created when they should not be. As an example, the Contain method casts the inputs as string whether they are strings or not. So a tests like:
fluent.TestValue = 10
Debug.Assert fluent.Should.Contain(10)
fluent.TestValue = "10"
Debug.Assert fluent.Should.Contain("10")
Are treated as they same and produce the same fluent path even through they are technically distinct. A future update will probably distinguish between these tests so that duplicate fluent paths are not raised in this scenario.
To currently compensate for this scenario, the Tests class contains a SkipDupCheck Boolean property. If this property is set to true, then the event is not raised. You can see an example of using this property below:
fluent.TestValue = "10"
With fluent.Meta.Tests
.SkipDupCheck = True
Debug.Assert fluent.Should.Contain("10")
.SkipDupCheck = False
End With
TestInfo updates
One thing that I had wanted to do for some time is to get a breakdown of testing attributes for each of the different testing methods I had. The type of data I wanted to get for each of the testing methods includes:
- How many times a test was run
- How many times a test passed
- How many times a test failed
- And how many times a test returned an unexpected value
I decided to put this information in a class called cTestingInfo. Using this information, I’m able to determine attributes for each testing method specifically. So I can (and do) use this information to determine that each testing method was run once for example. This is very useful for debugging purposes.
The example below shows to get a count of how many times the EqualTo method was used in a given Fluent object:
Debug.Print fluent.Meta.Tests.TestingInfo.EqualTo.Count
ToString class updates
I decided to add a ToString class that can hold string related attributes which are not related to creating an expression. One of the functions I put in there is a WrapIfString function. This function can wrap string elements with a wrapper that can be defined by the user. This function is used in the fluent path on some of the strings. Using this wrapper allows the user to distinguish between string and non-string testing values and inputs that could not be distinguished previously. As an example, the following tests (and similar examples) would previously produce the same fluent path:
fluent.TestValue = True
Debug.Assert fluent.Should.Be.EqualTo(True)
fluent.TestValue = "True"
Debug.Assert fluent.Should.Be.EqualTo("True")
The fluent path would be like so:
"Testing value: True; Testing input: True; Expectation: Testing value should be equal to True; Result: True"
With the WrapIfString functionality, I can now get fluent paths like this:
"Testing value: `true`; Testing input: `true`; Expectation: Testing value should be equal to `true`; Result: True"
WrapIfString uses backticks by default. But the user is able to update this setting using the WrapChar property like so:
Dim fluent As IFluent
Set fluent = New cFluent
fluent.Meta.ToString.WrapChar = "'"
fluent.TestValue = "true"
fluent.Should.Be.EqualTo ("true")
Debug.Print fluent.Meta.Tests(1).FluentPath
This prints the following fluent path:
"Testing value: 'true'; Testing input: 'true'; Expectation: Testing value should be equal to 'true'; Result: True"
Test String updates
I was getting ready to finalize Fluent VBA and was thinking that I’d finished with the project until I thought of a gap in the testing: Fluent VBA had no tests to check nested strings. It seems like there are lots of different ways you could approach nested strings. The way I decided to approach them was by adding TestStrings class
The TestStrings class has what I call CleanString functionality. By default, it includes double quotes ("""") and spaces ( ). CleanString functionality is only used if the user sets the CleanTestValueStr property, the CleanTestInputStr property, or the CleanTestStrings property equal to true. If so, the TestValue, TestInput, or both are cleaned respectively. The "cleaning" replaces these values by default with an empty string. I also added an AddToCleanStringDict method that allows the user to add or replace the key-value pairs used to clean strings in the library.
The Test object has also been updated with various properties related to cleaning strings like CleanedTestValue, CleanedTestInput, HasCleanTestValue, HasCleanTestInput, HasCleanTestStrings. The FluentPath for the test object has also been updated to distinguish between the original testing string and the cleaned string.
You can see an example of using the CleanTestValStr property below:
fluent.Meta.Tests.TestStrings.CleanTestValStr = True
fluent.TestValue = """abc"""
Debug.Assert fluent.Should.Be.EqualTo("abc")
fluent.Meta.Tests.TestStrings.CleanTestValStr = False
The CleanString properties are best used in a limited scope like in the example above. One potential downside to using these properties is that the user may forget that they enabled this property. And they may have forgotten to disable it. A user may encounter similar lines of code to the ones in the example above. The above example passes even though it looks like it should not. And that doesn’t look like it makes sense.
For users looking to avoid this scenario, I offer a CleanString method. The CleanString method allows you to perform the same operations that CleanString properties perform implicitly, explicitly. You can see an example below:
fluent.TestValue = """bcd"""
fluent.TestValue = fluent.Meta.Tests.TestStrings.CleanString(fluent.TestValue)
Debug.Assert fluent.Should.Be.EqualTo(fluent.Meta.Tests.TestStrings.CleanString("""bcd"""))
Test file updates
All of the class modules within Fluent VBA are publicNotCreateable. This allows them to be used in external projects. And I even provided public functions for doing so. Another thing I wanted to do however was provide testing files for the users to do this. With these testing files, the user can simply reference the files in the distribution folder. From there, they can write their testing code in the testing files. Or they can also reference VBA code in their own projects as a reference as well. So in order to do this, I updated the PowerShell scripts within the scripts folder to dynamically create the files that do this. And these files are now included with the release.
Test updates
I added various tests in mTests. This was done to test against the new updates in the library as well as ensuring that the original tests were comprehensive. Doing this proved to be a good idea as it helped me discover and fix bugs within the library that I would not have caught otherwise. Fluent VBA now implements over 1,500 tests that it uses to test itself.
Twin Basic Updates
I decided to try to compile Fluent VBA in Twin Basic as a 64-bit ActiveX DLL. The bad news is that I was not able to run every test. But the good news is that I was able to run over 99% of them (1515/1523) and almost all of the functionality. So at some point in the future I may port this project to Twin Basic once everything is able to run. These tests were run in Excel, Word, PowerPoint, and Access
Future updates
While I’ve said this several times in the past, I do think that the library is now substantially complete. Its structure is unlikely to change significantly. And I do not anticipate making significant updates in the future but that’s always possible. I thought that this project was substantially completed before I added many of the major updates that it includes today.
I do have some additional updates I would like to add at some point in the future. There are lots of refactors I would like to add for example. And a few new features as well. But I do not expect to get to those updates until around the holidays in 2024.
Let me know if you have any questions, comments, or feedback. Thanks!