Interposition points

During the expan­sion phase of program eval­u­a­tion, Racket wraps certain oper­a­tions in an extra macro that acts as a hook for providing default behavior. These macros are known as inter­po­si­tion points. We can recog­nize them by their names, which are prefixed with #%.

For instance, #%module-begin.  + The stacker tuto­rial intro­duces #%module-begin. When Racket expands a module expres­sion, it starts by wrap­ping the body of the expres­sion in #%module-begin. So this:

Becomes this:

Eval­u­a­tion continues from here, with #%module-begin taking the body-exprs ··· as input and returning a result.

If a needed inter­po­si­tion point isn’t avail­able in the language, then the oper­a­tion fails. This is why in our language tuto­rials, we always export a #%module-begin. Under the hood, every source file becomes a module expres­sion. So every language also needs a #%module-begin, or eval­u­a­tion will always fail.

In most language imple­men­ta­tions, it suffices to export the default inter­po­si­tion points, because they give us the vanilla behavior we usually want. But they can be modi­fied if we want to achieve certain special effects, as demon­strated below.

Common inter­po­si­tion points

The easiest way to under­stand inter­po­si­tion points is to see how they affect the eval­u­a­tion of a language. In partic­ular, it’s useful to be able to recog­nize the errors that arise when they’re missing.

Suppose we start making a language in a source file called "lang.rkt":

Don’t panic—it’s blank, except for the first line. In the same direc­tory, we create a source file that invokes this language:

Don’t panic—it’s blank too. When we run this test file, we get an error:

Though read-syntax is not an inter­po­si­tion point, it oper­ates by a similar prin­ciple. When Racket runs the language, it auto­mat­i­cally looks for the read-syntax func­tion, and passes the source code as input, so it can be turned into a module expres­sion.  + The stacker tuto­rial intro­duces read-syntax.

Thus, we can fix this error by adding a simple read-syntax func­tion:

This time, when we run our test file:

We get two new errors:

Strictly speaking, the first error is being reported by Racket itself (as it tries to run our tiny language), and the second by DrRacket (as it tries to create a REPL prompt). Notice that both errors arise from a missing inter­po­si­tion point: the first from #%module-begin, and the second from #%top-interaction.

We can fix these errors the same way we fixed the missing read-syntax: by exporting these iden­ti­fiers from our language. We can write our own version of each missing macro. Or we can also just export an existing one.

For #%module-begin, let’s do it that way:

When we run our test file again, we only get one error:

#%top-inter­ac­tion

#%top-interaction implic­itly wraps every expres­sion that’s entered at the REPL. There­fore, if we want the REPL to be avail­able in our language, we need to export #%top-interaction. As before, it suffices to reuse the #%top-interaction already avail­able:

This time, when we run our test file:

We don’t get an error.

Unlike #%module-begin, there’s rarely a useful reason to write a special #%top-interaction. If we want to customize the REPL for a language, we use other tools. (See the third BASIC tuto­rial for an example.) But #%top-interaction is still a prereq­ui­site for booting the REPL.

#%datum

Now let’s put a value in our test file:

When we run this file, we get a new error:

#%datum is the inter­po­si­tion point that wraps every instance of literal data in a program, like numbers and strings. So 42.0 becomes (#%datum . 42.0), and "str" becomes (#%datum . "str").

If we want to be able to use literal data in a program, we need to export #%datum:

With this change, our test file no longer raises an error, and instead prints our value as usual:

We can customize #%datum if we want to affect the eval­u­a­tion of literal values. For example, in the BASIC tuto­rial, we converted inexact inte­gers to exact inte­gers in the expander. We could move that house­keeping into #%datum:

Notice that we use the same basic tech­nique as when we customize #%module-begin. First, we write our new macro with a different name. Second, we pass our customized result to the default #%datum. Third, when we export our macro, we use rename-out to change its name to the expected #%datum.

This time, when we run our test file:

The literal value 42.0 gets converted to an exact integer:

#%app

Now suppose we want to accom­plish the epic feat of subtracting two inte­gers:

Unfor­tu­nately, it doesn’t work:

The “unbound iden­ti­fier” part of the error can be fixed by exporting - from our language (we also reset #%datum to its vanilla meaning):

With that change, we get a more descrip­tive error:

Just as #%datum is the inter­po­si­tion point that wraps every instance of literal data, #%app is the form that wraps every func­tion appli­ca­tion. So (- 42 10) becomes (#%app - 42 10). Of course, in this example, the inte­gers are still wrapped by #%datum, so the trans­for­ma­tion ends up looking like this:

We can customize #%app if we want to affect how func­tions are applied to their argu­ments. For instance, by default func­tion argu­ments are eval­u­ated from left to right. We can reverse this behavior by customizing #%app:

This version of #%app reverses the order of argu­ments, and then calls the default #%app. With this change, (- 42 10) instead behaves like (- 10 42):

Alter­na­tively, if we edit our language to use the default #%app:

We get the usual result:

#%top

To complete our tour of inter­po­si­tion points, let’s swap a vari­able into our math expres­sion:

This time, we get a new error:

Every iden­ti­fier in a program without a local binding (e.g., from let) gets wrapped with #%top, which signals “this iden­ti­fier uses the current top-level binding”. #%top doesn’t know or care whether that binding exists.

It’s true, however, that every unbound iden­ti­fier gets wrapped with #%top.  + This stands to reason: if the iden­ti­fier had some local binding, it wouldn’t be unbound. So we can use #%top to do some­thing about unbound iden­ti­fiers:

In this case, we rede­fine #%top to check if the iden­ti­fier is bound (with identifier-binding). If it’s bound, we pass it through. If not, we substi­tute the value 25.

When run our test file again, the unde­fined x is converted to 25:

Suppose we insert an explicit binding for x with define:

Because x is no longer unbound, the result changes:

But as with our other inter­po­si­tion points, if we just want the usual behavior of #%top, we can export the existing version:

Finally, if we revert to our orig­inal test case:

We get the usual error:

Inter­po­si­tion points in br/quick­lang

The br/quicklang dialect auto­mat­i­cally exports the default versions of #%top, #%app, #%datum, and #%top-interaction. Why? Because in prac­tice, these forms are rarely customized. But if you’re imple­menting a language starting from some­thing other than br/quicklang (say, racket/base) then you’ll need to handle this house­keeping your­self.