Finishing moves: jsonic

As we’ve already seen, it’s easy to install a direc­tory of Racket modules as a package with raco pkg, Racket’s package-manage­ment utility. So if our jsonic direc­tory looks like this:

From the command line, we can install it like so:

Or, equiv­a­lently:

One little gotcha—this variant won’t work:

In this case, because jsonic doesn’t look like a direc­tory name, raco pkg tries to find the jsonic package using Racket’s remote package server. (Which won’t work, because it doesn’t know about jsonic. But we’ll fix that problem shortly.) In this situ­a­tion, we can induce raco pkg to use the local direc­tory like this:

Intro­ducing info.rkt

Recall that once we install a direc­tory as a package, we can import its modules into another Racket program with (require jsonic/module-name). Or, if the package contains a language, use it in a source file as #lang jsonic. For proto­typing purposes, this suffices.

But if we want to control the instal­la­tion more precisely—e.g., auto­mat­i­cally build our new docu­men­ta­tion—we need to add an "info.rkt" file.

The "info.rkt" file contains meta­data that tells Racket how to handle the contents of the direc­tory. When Racket installs a package, it looks for an "info.rkt" in the top direc­tory, and each subdi­rec­tory it encoun­ters. If it doesn’t find one, instal­la­tion proceeds by default.

In general, once we’re ready to share our language, we’ll want to add an "info.rkt" to the top direc­tory of the package that refines how the package is compiled, installed, and tested.

Modules, pack­ages, collec­tions, and repos

A Racket source file is part of several orga­ni­za­tional schemes at once. In prac­tice, these can overlap. So let’s make sure the termi­nology is clear before we move forward:  + Hat tip to Vincent St-Amour for these expla­na­tions.

What can some­times make these terms confusing is that one name can be reused in several roles. For instance, consider our jsonic project:

In this case, our package is called jsonic, and lives in a repo called jsonic. When we install this with raco pkg, our modules become part of the jsonic collec­tion. Easy.

A more complex Racket project might look like this:

Here, the mega-repo is home to two inde­pen­dent pack­ages, foo-pkg and bar-pkg. The devel­oper would list these sepa­rately on the package server, so users could install them sepa­rately. foo-pkg provides modules that are part of the math collec­tion, while bar-pkg provides modules that are part of the math and games collec­tions. Mean­while, the "info.rkt" files in each direc­tory would make these inten­tions clear.

BTW, this is not a project orga­ni­za­tion we’d neces­sarily want to emulate.  + This capa­bility is typi­cally not used to smush unre­lated modules into common pack­ages, but rather to divide larger pack­ages into smaller ones. It simply illus­trates that repos, pack­ages, and collec­tions are inde­pen­dent ways of grouping modules. In simple projects, they’re often tele­scoped together.

Starting the info.rkt for jsonic

The "info.rkt" file uses a domain-specific language called #lang info, which looks like regular Racket but only supports a limited set of func­tions. The only purpose of #lang info is to make "info.rkt" files. So let’s start our file at the top level of the jsonic direc­tory:

An "info.rkt" is struc­tured as a series of define expres­sions that act as meta­data fields. Various Racket tools read these meta­data fields. If we want our package to coop­erate with these tools, we need at least one "info.rkt" in the top direc­tory of the package. We can also put "info.rkt" files inside any subdi­rec­tory that needs special handling. But often, the top-level file suffices.

Since we’re plan­ning to distribute our language, we want to start by spec­i­fying a collection name and a version number:

Per the expla­na­tion above, the collec­tion is the name­space where our package is avail­able. In this case, we want the collection value to be the same as the package name—"jsonic". Strictly speaking, when the collec­tion and package names match, this collection field is optional. But it’s a virtuous habit to set it explic­itly. That way, we remove the connec­tion between the two names. If we ever rename the package, or move these modules to be part of another package, they will still correctly be avail­able as part of the jsonic collec­tion.

Strictly speaking, the version field is also optional—without it, raco pkg treats jsonic as having version 0.0. But when we’re devel­oping a language, it’s another virtuous habit. Version numbers help those who rely on our language under­stand the rela­tion­ship between succes­sive releases. Also, if other devel­opers write pack­ages that depend on our language, the version number helps raco pkg figure out if a user needs a more recent version of our language.

Building docu­men­ta­tion

The "info.rkt" also spec­i­fies how to build the docu­men­ta­tion for our language. To do this, we define the scribblings field with a list of .scrbl docu­men­ta­tion files. In this case, we have only one:

Adjusting auto­matic testing

The raco test tool is used to run tests. When used with the -p flag, it runs all tests that exist in a package. Meaning, for each source file in the package, raco test runs the test submodule (if it exists), or the whole file.  + raco test is commonly used in shell scripts for services that auto­mat­i­cally build and test Racket pack­ages, for instance Travis CI.

We can see how this works by entering the following command at the terminal:

We’ll get some­thing like this:

"jsonic-test.rkt" prints its JSON result to the terminal. Nothing bad happens as a result.

But let’s suppose we prefer our test reports to be tidy. So we want to omit "jsonic-test.rkt" from our test suite.

We can do this by adding a test-omit-paths field to our "info.rkt", which is a list of source files that raco test should ignore:

After we make this change, we can run our tests again:

This time, "jsonic-test.rkt" is skipped:

Exactly right.

Spec­i­fying depen­den­cies

A depen­dency is any Racket package that’s neces­sary for our language to work. We use "info.rkt" to list both run-time depen­den­cies (= pack­ages needed every time we invoke a module from the package) and build depen­den­cies (= pack­ages only needed for testing or docu­men­ta­tion).  + Build depen­den­cies are spec­i­fied sepa­rately so they can be omitted from built pack­ages, allowing them to be smaller.

When a user tries to install a package, raco pkg checks the user’s existing pack­ages against the depen­den­cies for the new package. If any are missing, raco pkg gives the user the option to install them.

But we prob­ably forgot to keep track of all the depen­den­cies that we racked up during our project. Fortu­nately, that’s not a problem: we can use the raco setup command to tell us what they are. Let’s go to the terminal and issue this command:

raco setup is respon­sible for compiling every installed package. When we pass the --check-pkg-deps flag, raco setup inspects all our modules and deter­mines their depen­den­cies, and compares these to the depen­den­cies declared in our "info.rkt" (which right now is none). As it does so, we see a bunch of unde­clared depen­den­cies go flying up the screen. But at the end we get a summary like this:

This tells us all the pack­ages that need to be listed in "info.rkt" as ordi­nary depen­den­cies (the pack­ages in the top part of the list) and build depen­den­cies (just scribble-lib).

We can add these depen­den­cies to our "info.rkt" manu­ally. But more conve­niently, raco setup can take care of this house­keeping. If we’re satis­fied with the list of depen­den­cies, we can issue another terminal command to put them into our "info.rkt":

If we now reopen our "info.rkt", we see it’s been updated with the list of deps and build-deps that we just saw in the missing-depen­dency report:

How do we know this worked? We can go back to the terminal and use --check-pkg-deps again:

This time, it shows no errors, because all the depen­den­cies are now correctly declared in our "info.rkt". If we ever change jsonic to use different depen­den­cies, we can either manu­ally update these lists, or use raco setup with the --fix-pkg-deps flag to adjust them.

Unver­sioned vs. versioned depen­den­cies

The depen­dency list we just made contained only unver­sioned depen­den­cies. When we specify a depen­dency without a version number, what it means is “any version of this package is fine”. That works, as long as we’re only relying on parts of the package inter­face that have been consis­tent across every version.

But suppose that brag is updated to version 2.0 with a new xyzzy func­tion that’s essen­tial to our project. We can no longer specify brag as a unver­sioned depen­dency, because our language won’t work for users who only have version 1.0 of brag installed.

We can fix this problem by adding a version anno­ta­tion to brag:

In this case, when raco pkg installs our language, it checks if the user has brag version 2.0 or later installed. If brag isn’t installed at all, raco pkg auto­mat­i­cally installs the current version (as it would an unver­sioned depen­dency). But if a version of brag earlier than 2.0 is installed, raco pkg updates brag to the current version.

But in this case, we can leave our brag depen­dency unver­sioned.

Checking that every­thing works

Our finished "info.rkt" looks like this:

To make sure that it works, let’s unin­stall our package like so:

Then rein­stall as usual:

We should see some status messages as the package is installed. But no errors.

Let’s try running our tests again:

We should see no errors.

Now let’s test the docu­men­ta­tion with the raco docs command, which is just command-line short­hand for using the search box in the web inter­face:

Our web browser should show us a results screen like this:

That’s also correct—in the first search result, the docu­men­ta­tion system is correctly noticing the anchor to the jsonic language. For the second result, it’s noticing the word "jsonic" in the head­line. If we click either of these links, we’ll see our main docu­men­ta­tion page:

And now, if we go into DrRacket and run a test file:

It works as usual:

Great. Now that our jsonic package has been prepared with an "info.rkt" file, we’re ready to share it with the world using the package server.