Thank you for your comment

Beau­tiful Racket / explainers

Hygiene

Hygiene is the orga­nizing policy of Racket’s macro system. Under­standing hygiene is the key to under­standing how macros work, and by exten­sion, how to write good macros of your own. Conversely, not under­standing hygiene can make Racket macros seem confusing and myste­rious. But they are not.

Hygiene is the answer to a simple ques­tion: a macro gener­ates code that gets deposited else­where. When that code is eval­u­ated, how should we deter­mine the bind­ings of the iden­ti­fiers inside?  + Because a Racket program can’t run unless every iden­ti­fier has a binding. See iden­ti­fiers.

We have two choices:

  1. Deter­mine the bind­ings according to what the iden­ti­fiers mean at the place where the macro was defined (aka the defi­n­i­tion site). This is Racket’s default policy, and part of what is implied by hygiene.

  2. Deter­mine the bind­ings according to what the iden­ti­fiers mean at the place where the macro was invoked (aka the calling site). Some­times this is the behavior we want, so Racket lets us “break hygiene” when we need to.

To deter­mine the binding of an iden­ti­fier, we look inside the lexical context attached to the code, which holds all the avail­able bind­ings. So hygiene can be seen as a way of managing lexical contexts. In the example below, there are two iden­ti­fiers named x. But thanks to hygiene, they live in sepa­rate lexical contexts, so they don’t collide:  + In Racket, begin groups expres­sions without creating a new lexical context (unlike let).

(define x 42)
(define-macro (mac)
  #'(begin
      (define x 84)
      (println x)))
(mac) ; 84
(println x) ; 42
(define x 42)
(define-macro (mac)
  #'(begin
      (define x 84)
      (println x)))
(mac) ; 84
(println x) ; 42
copy to clipboard

What problem does hygiene solve?

Without hygiene, every iden­ti­fier would be treated as if it lived in the lexical context of the calling site. While this has an appealing simplicity, it quickly runs aground on the shoals of prac­ti­cality.

Let’s convert the example above into an unhy­gienic macro, so it invades the lexical context of the calling site. It won’t work, because the two iden­ti­fiers named x now collide:

(define x 42)
(define-unhygienic-macro (mac)
  #'(begin
      (define x 84)
      (println x)))
(mac) ; error: duplicate definition
(define x 42)
(define-unhy­gienic-macro (mac)
  #'(begin
      (define x 84)
      (println x)))
(mac) ; error: dupli­cate defi­n­i­tion
copy to clipboard

“No problem—just change the macro to use an iden­ti­fier with a distinct name.” But how? The macro doesn’t know anything about where it will be called, or what iden­ti­fiers are already being used there. So there’s no way to guar­antee that any partic­ular iden­ti­fier will be distinct. Hygiene, by contrast, keeps lexical contexts sepa­rate, thereby elim­i­nating the possi­bility of inad­ver­tent name colli­sions (also known as iden­ti­fier capture).

The problem can run in the oppo­site direc­tion as well. In this example, our mac and u-mac macros implic­itly rely on println having its usual meaning. But it might not:

(define-macro (mac) #'(println "mac runs"))
(define-unhygienic-macro (u-mac) #'(println "u-mac runs"))
(mac)
(u-mac)
(module+ main
  (define (println thing) (error 'zombie-apocalypse-started))
  (mac)
  (u-mac))
(define-macro (mac) #'(println "mac runs"))
(define-unhy­gienic-macro (u-mac) #'(println "u-mac runs"))
(mac)
(u-mac)
(module+ main
  (define (println thing) (error 'zombie-apoc­a­lypse-started))
  (mac)
  (u-mac))
copy to clipboard
"mac runs"
"u-mac runs"
"mac runs"
error: zombie-apocalypse-started
"mac runs"
"u-mac runs"
"mac runs"
error: zombie-apoc­a­lypse-started
copy to clipboard

Within the main module, println has been rede­fined. At that point, the behavior of the hygienic macro remains consis­tent: thanks to hygiene, it retains the orig­inal binding of println. Whereas the unhy­gienic macro becomes unpre­dictable.

More broadly, hygiene can be seen as a way of making macros more compos­able, which is a general design goal of func­tional languages. Macros can’t know anything about the bind­ings used at the calling site. There­fore, the best way to ensure consis­tent behavior is to make macro-gener­ated code as self-contained as possible.

The golden rules

For the macro writer, there are four golden rules of hygiene:

  1. Code produced by a macro adopts the lexical context of the macro-defi­n­i­tion site. There­fore, this code can only rely on iden­ti­fiers that have bind­ings at the defi­n­i­tion site. Below, the mac macro can create code that refers to x, because x has a binding at the macro-defi­n­i­tion site (albeit outside the macro):

    (define x 42)
    (define-macro (mac)
      #'(println x))
    (mac) ; 42
    (define x 42)
(define-macro (mac)
  #'(println x))
(mac) ; 42
    copy to clipboard

  2. Within code produced by the macro, new bind­ings can shadow existing bind­ings at the defi­n­i­tion site (similar to how let works). For example, this updated mac macro defines its own x, which over­rides the x defined outside the macro, so the result is now 84:

    (define x 42)
    (define-macro (mac)
      #'(begin
          (define x 84)
          (println x)))
    (mac) ; 84
    (define x 42)
(define-macro (mac)
  #'(begin
      (define x 84)
      (println x)))
(mac) ; 84
    copy to clipboard

  3. Bind­ings intro­duced by a macro are only visible to other code produced by that macro. In the example below, one x is defined outside the mac macro; another is defined inside. The (println x) gener­ated by the macro refers to the x defined by the macro. The (println x) outside the macro refers to the x defined outside the macro. So we end up with two iden­ti­fiers called x with different values:

    (define x 42)
    (define-macro (mac)
      #'(begin
          (define x 84)
          (println x)))
    (mac) ; 84
    (println x) ; 42
    (define x 42)
(define-macro (mac)
  #'(begin
      (define x 84)
      (println x)))
(mac) ; 84
(println x) ; 42
    copy to clipboard

    The corol­lary to this rule is that bind­ings intro­duced by a macro are not visible outside the macro. Every early-stage Rack­e­teer tries to write a vari­a­tion of the define-x macro below, and is flum­moxed by the error:

    (define-macro (define-x)
      #'(define x 42))
    (define-x)
    (println x) ; error: unbound identifier
    (define-macro (define-x)
  #'(define x 42))
(define-x)
(println x) ; error: unbound iden­ti­fier
    copy to clipboard

    But it’s not surprising: the x defined inside the macro lives inside a sepa­rate lexical context, so the (println x) outside the macro can’t see it. If this still annoys you, consider an anal­o­gous example with let, which won’t surprise anyone.

    (let ([x 42])
      x)
    (println x) ; error: unbound identifier
    (let ([x 42])
  x)
(println x) ; error: unbound iden­ti­fier
    copy to clipboard

  4. Every iden­ti­fier retains its binding from its orig­inal lexical context.  + This is accom­plished by using syntax objects throughout the macro system. Here, we pass the outer x to our macro as an argu­ment and assign it to the pattern vari­able OUTER-X. When OUTER-X appears in our macro code, it still refers to the x defined outside the macro, while the other x refers to the one defined inside the macro:

    (define x 42)
    (define-macro (mac OUTER-X)
      #'(begin
          (define x 84)
          (println x)
          (println OUTER-X)))
    (mac x)
    (define x 42)
(define-macro (mac OUTER-X)
  #'(begin
      (define x 84)
      (println x)
      (println OUTER-X)))
(mac x)
    copy to clipboard
    84
    42
    84
42
    copy to clipboard

Breaking hygiene

Some­times a macro needs to intro­duce iden­ti­fiers into the lexical context of the calling site. These iden­ti­fiers are called unhy­gienic. Despite the shameful-sounding name, there’s nothing wrong with unhy­gienic iden­ti­fiers. On the contrary, some­times they’re the right tool for the job. But it’s wise to reserve them until needed.

Macros rely on hygiene by default. There­fore, breaking hygiene requires us to explic­itly inject our new iden­ti­fier into the target lexical context.

A common use for unhy­gienic iden­ti­fiers are macros that extend define by modi­fying the given iden­ti­fier name.  + For instance, struct unhy­gien­i­cally creates setter & getter iden­ti­fiers for its fields. See data struc­tures. As an example, let’s make define-$, which works like define for func­tions except that it appends $ to the name:

(define-macro (define-$ (ID ARG ...) BODY ...)
  (define id$-datum (format-datum '~a$ (syntax->datum #'ID)))
  (with-pattern ([ID$ (datum->syntax #'ID id$-datum)])
    #'(define (ID$ ARG ...)
      BODY ...)))
(define-$ (f x) (* x x))
(f$ 5) ; 25
(define-macro (define-$ (ID ARG ...) BODY ...)
  (define id$-datum (format-datum '~a$ (syntax->datum #'ID)))
  (with-pattern ([ID$ (datum->syntax #'ID id$-datum)])
    #'(define (ID$ ARG ...)
      BODY ...)))
(define-$ (f x) (* x x))
(f$ 5) ; 25
copy to clipboard

The macro starts with a syntax pattern that matches our ID, ARGs, and BODY expres­sions. We extract the datum from #'ID with syntax->datum. We use format-datum to make a new datum with a $ appended, called id$-datum.

The unhy­gienic iden­ti­fier is created next, with datum->syntax. The first argu­ment of datum->syntax is the lexical context for the iden­ti­fier being created. In this case, we use #'ID because it came from the calling site, and there­fore has the lexical context we want to borrow. The second argu­ment is our datum. We match this new iden­ti­fier to the pattern vari­able ID$, so we can use it in the syntax template below. Within the template, we use ID$ in the name posi­tion of a stan­dard define form.

The result is that we can then use the vari­able f$ as if we had defined it directly. (The iden­ti­fier f remains unbound.)

There’s no one right way to create an unhy­gienic iden­ti­fier. The mechanics always remain the same. For instance, if the syntax->datum / datum->syntax fandango gets tire­some, you can use prefix-id and suffix-id as helper func­tions to simplify creation of unhy­gienic iden­ti­fiers. They create a new iden­ti­fier derived from the name and lexical context of an existing iden­ti­fier:

(define-macro (define-$ (ID ARG ...) BODY ...)
  (with-pattern ([ID$ (suffix-id #'ID '$)])
    #'(define (ID$ ARG ...)
      BODY ...)))
(define-$ (f x) (* x x))
(f$ 5) ; 25
(define-macro (define-$ (ID ARG ...) BODY ...)
  (with-pattern ([ID$ (suffix-id #'ID '$)])
    #'(define (ID$ ARG ...)
      BODY ...)))
(define-$ (f x) (* x x))
(f$ 5) ; 25
copy to clipboard

Further

← prev next →