Level up: jsonic revisited

Intro­ducing unit tests

In contracts, we saw how a contract can be a helpful devel­op­ment tool, because it lets us specify the accept­able data types for a func­tion’s input argu­ments and return value. Once the contract is in place, we’ll get a detailed error when­ever the func­tion misbe­haves.

But like any code, a contract is not self-executing. To verify that a func­tion and its contract work correctly, we need to call the func­tion with some sample cases and see what happens. These are known as unit tests.

Unit tests are valu­able because they remove guess­work:

We recall that func­tional program­ming encour­ages us to design func­tions so that their behavior depends only on their input argu­ments (instead of external state) and return a value (instead of relying on muta­tion or side effects). A side benefit of this approach is that it becomes easy to write unit tests, because our func­tions have no remote depen­den­cies or effects.

For this reason, Rack­e­teers often like to write unit tests for a func­tion before they start writing that func­tion, not after. It’s also common to put the unit tests near the func­tion defi­n­i­tion, so they can be worked in tandem.

Racket makes this style of coding easy with two features: the rackunit unit-testing library, and the test submodule.

Writing tests with rack­unit

The rackunit library helps us create unit tests called checks. To use it, we first add (require rackunit) to our program.

Then we write our checks. Each check is an asser­tion about the behavior of the func­tion. A check usually consists of a func­tion call and the expected result. If the check succeeds, it passes silently. If the check fails, an error is raised that tells us both the expected and actual result.

Below, we use the simplest test, check-equal?, with the our-div func­tion we wrote before. check-equal? takes two argu­ments and compares them with equal?:

When we run this sample, the second check raises an error:

We can write as many tests as we want. A thor­ough set of tests will cover obvious cases, edge cases, and error cases.

Because error cases don’t produce a return value, we instead check the error type. For instance, our-div will raise an error if we pass 0 as the denom argu­ment. Instead of check-equal?, we check an error with check-exn (exn is short for exception). The first argu­ment is the type of error we expect—in this case we’ll use the generic exn:fail? pred­i­cate.  + It’s better policy, as it was with contracts, to use the narrowest error pred­i­cate that applies. For more about Racket’s built-in error types, see errors and excep­tions. The second argu­ment is our test case, wrapped in a lambda, which prevents the error from being trig­gered imme­di­ately:

This time, all three checks will pass. For other types of checks, see unit testing.

Running tests in the test submodule

Putting unit tests directly next to our func­tion isn’t wrong. But it can be inef­fi­cient. Unit tests take time—espe­cially if we’re being thor­ough—and they don’t need to run every time a module is loaded.

To avoid this over­head, we could put our unit tests in a sepa­rate source file. But this can be a drag, because the func­tions and their corre­sponding unit tests then live in two places. It’s often more ergonomic to keep them together.

Thus, as a conve­nience, DrRacket specially coop­er­ates with an optional submodule called test. When we have the source file for a module open in DrRacket and click Run, DrRacket will run the body of the module as usual, but will also run the test submodule (if it exists). Other­wise—for instance, if the module is imported into another module with require—its test submodule will be ignored. This way, we can easily (and contin­u­ally) run our tests when we’re actively devel­oping the module.  + The test submodule also coop­er­ates with other Racket tools, like raco test.

Let’s move our tests into a test submodule. We’ve seen how to define a submodule with module. For test we’ll use a variant called module+. A submodule created with module+ has two bene­fits:

Because of these special behav­iors, a submodule declared with module+ neces­sarily runs after its enclosing module is eval­u­ated.

Starting with our example above, let’s move our unit tests into a test submodule:

When we run this code, DrRacket will first run the body of the module that contains the defi­n­i­tion of our-div. Then it will assemble the pieces of the test submodule—putting together the (require rackunit) piece at the top with the checks at the bottom—and run it. If we were to change one of the tests so it fails, we’d get the same kind of error message as before.

Suppose we add a new func­tion called our-mult. We could orga­nize the func­tion and its related tests like this:

Adding tests to the tokenizer

Now that we know how to use rackunit and the test submodule, we’ll add some unit tests for jsonic-token? and make-tokenizer.

We intro­duce the test submodule and import rackunit. Then we add the unit tests. For pred­i­cates like jsonic-token?, it’s conve­nient to use check-true and check-false. These are like check-equal?, but they auto­mat­i­cally compare the result to #t or #f. We write tests that cover each branch of the pred­i­cate, and one that fails:

Then we add tests for make-tokenizer. This func­tion is a little trickier, because we need to feed it an input-port?, and what we get back is a func­tion that emits tokens. For testing purposes, it would be simpler to pass in a string repre­senting a piece of source code, and get back a list of tokens.

Good news—the brag/support module provides a helper func­tion, apply-tokenizer-maker, that handles this minor house­keeping. We can use this func­tion with check-equal? to write our tests.

Now we just need some test expres­sions:

But more good news—we already wrote them when we first wrote make-tokenizer. Then, we tested make-tokenizer by inspecting the results we got by entering these expres­sions on the REPL. Now, we take the same expres­sions and results and move them inside unit tests.

At this point, the module seems to run the same way it always did. With unit tests, no news is good news. If we were to monkey with the code inside jsonic-token? or make-tokenizer—for those who haven’t tried it, go ahead—our unit tests would start raising errors.

With unit tests, we can contin­u­ally & auto­mat­i­cally verify the behavior of our func­tions. If we change these func­tions in a way that alters this behavior, the unit tests will flag the problem imme­di­ately. This makes it easier to improve the func­tions, because we have imme­diate testing feed­back.

Of course, not every update to a func­tion will be back­ward-compat­ible with our existing tests (we’ll see an example of that in source loca­tions). When that’s the case, we can update the tests too. Like contracts, tests aren’t meant to inhibit changes. Rather, by making the expected behavior of a func­tion explicit, they help us proceed in a more disci­plined and careful way.

Adding parser tests in a sepa­rate module

One limi­ta­tion of the test submodule is that we can only use it in modules that use a language that supports module+. Our "parser.rkt" module, for instance, is written in #lang brag, which doesn’t support module+:

In this case, we can still make unit tests for parse using rackunit. But we’ll have to move them into a sepa­rate file. Slightly less conve­nient, but a lot better than not testing at all.

As we did with make-tokenizer, we’ll take the informal tests we wrote in the last tuto­rial and convert them into real unit tests in a new file called "parser-test.rkt".

Recall that our parser takes a list of tokens and turns them into a parse tree. There­fore, we need to import our tokenizer so that we can process strings of source code into tokens. We’ll also import brag/support so we can use apply-tokenizer-maker func­tion again.

Then we can add our previous test cases, converted into check-equal? tests. To make things easier, in our tests we’ll use our parser’s parse-to-datum func­tion, which works the same as parse, but converts the result into a simple hier­ar­chical list.

The whole "parser-test.rkt" looks like this:

When we run this file, we should get no errors, because we already veri­fied that these test cases work.

Now, let’s delib­er­ately break "parser.rkt"—for instance, by adding a second CHAR-TOK to the second produc­tion rule:

This time, when we run "parser-test.rkt", one of our checks will fail:

Let’s reset "parser.rkt" to its orig­inal state and move on.

Unit-testing caveat

Those new to this idea of alter­nating between coding and testing some­times consider it a chore and a bore. “Why waste time writing tests that’ll have to be revised later? Why not just skip it?“ 

The choice is yours, of course. The problem is that any func­tion is likely to be used multiple places in a program. There­fore, the complexity of inter­ac­tions between func­tions grows faster than the number of func­tions.

Sure, writing tests takes a little more effort at the outset. But over the life of a program, that invest­ment is paid back in time and annoy­ance saved. Working on a program with a thor­ough suite of unit tests is a plea­sure. Working on a program without tests—now that’s a chore and a bore.