Unit Tests for Mathematica Packages

I’ve been writing some Mathematica/Wolfram Language packages recently. At some point I wanted to have some unit tests to make sure that my code is doing what I expect it to. Mathematica offers a feature called “Testing Notebooks” but so far I haven’t really figured out what is the best way to use them. Testing Notebooks are fancier versions of standard Mathematica notebooks where you can enter a function call and an expected result. Actually this sounds very good, however I have two issues with them.

1. I can’t get them to work properly with my self developped packages. I don’t load my packages all the time. So I need to add a Needs[] call to my notebooks whenever I want to use some of the functionality. I’m not sure where to place this Needs[] call. If I place it outside of the testing blocks, then my functions don’t get loaded always. If I place it inside, I get warnings about functions being loaded multiple times.
2. Having a GUI is nice, but I would like to have a more scriptable way to check the unit tests. Although I’m not planning to set up any large CI/CD pipeline, it would still be nice if I could check it from the command line or on the fly while I’m working in another notebook.

The approach that I came up with takes ideas from how julia does testing and how doctest works in python. For each package that I write, I add all my unit tests at the end of the package as a function called Evaluate<MyPackageName>Tests. Basically that function returns a nested list of VerificationTest calls, which is Mathematicas builtin way to evaluate tests. The returned TestReportObject can then be inspected in any notebook and I can write a simple wolframscript file that prints the test results to the command line.

Below is an example of a dummy package with a unit test at its end.

BeginPackage["MyPackageName`"]

Unprotect[
Foo,
EvaluateMyPackageNameTests
]

Foo::usage = "TBD";
SyntaxInformation[Foo]={"ArgumentsPattern"->{___}};

Begin["`Private`"]

Foo[___]:=Module[{}, Return[True]];

End[]

EvaluateUtilsTests[]:=Return[
TestReport[{
	TestReport[{
		VerificationTest[Foo[],    True, "TestID"->"Foo_NoArgument"],
		VerificationTest[Foo[1],   False, "TestID"->"Foo_IntegerArgument"],
		VerificationTest[Foo["x"], False, "TestID"->"Foo_StringArgument"]
	}]
}]]

Protect[
Foo
EvaluateMyPackageNameTests
];

EndPackage[]