Thank you for your comment

Beau­tiful Racket / explainers

Syntax patterns

A syntax pattern is a tool for matching elements within a syntax object. Syntax patterns are used exten­sively in macros, espe­cially for sepa­rating the input into named pieces so they can be rearranged. In that way, a syntax pattern does for a syntax object what a regular expres­sion does for a string.

In #lang br, define-macro and define-macro-cases rely on syntax patterns to define calling patterns. In the macro m, the only matching input is the literal iden­ti­fier foo:

(define-macro (m foo) #'"match")
(m foo) ; "match"
(m bar) ; error: no matching case for pattern
(define-macro (m foo) #'"match")
(m foo) ; "match"
(m bar) ; error: no matching case for pattern
copy to clipboard

Whereas m2, defined with define-macro-cases, matches patterns of zero argu­ments, a literal foo iden­ti­fier, or a literal foo followed by anything:

(define-macro-cases m2
  [(m2) "first"]
  [(m2 foo) "second"]
  [(m2 foo ARG) #'ARG]
  [else "no match"])

(m2) ; "first"
(m2 foo) ; "second"
(m2 foo "bar") ; "bar"
(m2 bar) ; "no match"
(define-macro-cases m2
  [(m2) "first"]
  [(m2 foo) "second"]
  [(m2 foo ARG) #'ARG]
  [else "no match"])

(m2) ; "first"
(m2 foo) ; "second"
(m2 foo "bar") ; "bar"
(m2 bar) ; "no match"
copy to clipboard

Syntax patterns coop­erate closely with syntax templates. A syntax template is an expres­sion that creates a syntax object. In a syntax template, internal refer­ences to pattern vari­ables created by earlier syntax patterns are auto­mat­i­cally replaced with their under­lying matched value. Within a macro defi­n­i­tion or with-pattern expres­sion, any datum wrapped in a syntax expres­sion (or, equiv­a­lently, prefixed with #') is treated as a syntax template.

Syntax patterns are often used throughout a macro to destruc­ture syntax objects and syntax templates. For instance, this m3 macro contains three syntax patterns and three syntax templates:

(define-macro (m3 MID ... LAST)
  (with-pattern ([(ONE TWO THREE) (syntax LAST)]
                 [(ARG ...) #'(MID ...)])
    #'(list ARG ... THREE TWO ONE)))

(m3 25 42 ("foo" "bar" "zam")) ; '(25 42 "zam" "bar" "foo")
(define-macro (m3 MID ... LAST)
  (with-pattern ([(ONE TWO THREE) (syntax LAST)]
                 [(ARG ...) #'(MID ...)])
    #'(list ARG ... THREE TWO ONE)))

(m3 25 42 ("foo" "bar" "zam")) ; '(25 42 "zam" "bar" "foo")
copy to clipboard

Of course, for a syntax pattern to produce a match, the input syntax has to conform to the pattern. For instance, the syntax pattern (ONE TWO THREE) needs LAST to be a list of three elements. If it isn’t, an error arises:

(m3 25 42 ("foo" "bar"))
(m3 25 42 ("foo" "bar"))
copy to clipboard
with-pattern: unable to match pattern (ONE TWO THREE) in: ("foo" "bar")
with-pattern: unable to match pattern (ONE TWO THREE) in: ("foo" "bar")
copy to clipboard

Vocab­u­lary

A syntax pattern can have five possible ingre­di­ents:

  1. A literal, which only matches itself. Numbers, strings, and symbols are always literals. In define-macro, define-macro-cases, and with-pattern, iden­ti­fiers that are not in UPPERCASE are treated as literals.  + In stan­dard Racket, you need to list out literals sepa­rately (see, e.g., syntax-case). This is a chore that the br syntax func­tions handle auto­mat­i­cally.

    (define-macro-cases num
      [(num 42) "match"]
      [else "nope"])
    (num 42) ; "match"
    (num 24) ; "nope"
    (num "foo") ; "nope"

    (define-macro-cases str
      [(str "foo") "match"]
      [else "nope"])
    (str "foo") ; "match"
    (str foo) ; "nope"
    (str "bar") ; "nope"

    (define-macro-cases sym
      [(sym 'foo) "match"]
      [else "nope"])
    (sym 'foo) ; "match"
    (sym 'bar) ; "nope"
    (sym "foo") ; "nope"

    (define-macro-cases id
      [(id foo) "match"]
      [else "nope"])
    (id foo) ; "match"
    (id bar) ; "nope"
    (id "foo") ; "nope"
    (define-macro-cases num
  [(num 42) "match"]
  [else "nope"])
(num 42) ; "match"
(num 24) ; "nope"
(num "foo") ; "nope"

(define-macro-cases str
  [(str "foo") "match"]
  [else "nope"])
(str "foo") ; "match"
(str foo) ; "nope"
(str "bar") ; "nope"

(define-macro-cases sym
  [(sym 'foo) "match"]
  [else "nope"])
(sym 'foo) ; "match"
(sym 'bar) ; "nope"
(sym "foo") ; "nope"

(define-macro-cases id
  [(id foo) "match"]
  [else "nope"])
(id foo) ; "match"
(id bar) ; "nope"
(id "foo") ; "nope"
    copy to clipboard

    When matching literal iden­ti­fiers, a trap awaits the unwary. A literal iden­ti­fier in a syntax pattern is matched on the basis of its name but also its binding.  + Racket jocks might know this as equality in the sense of free-identifier=?. In the submodule below, mac looks like it will match the literal iden­ti­fier zeta. But outside the submodule, when we import zeta from math and then pass it as input to mac, we don’t get a match:

    (module mod br
      (define-macro-cases mac
        [(_ zeta) "match"]
        [else "nope"])
      (provide mac))
    (require (submod "." mod))

    (require math)
    (mac zeta) ; "nope"
    (module mod br
  (define-macro-cases mac
    [(_ zeta) "match"]
    [else "nope"])
  (provide mac))
(require (submod "." mod))

(require math)
(mac zeta) ; "nope"
    copy to clipboard

    Why not? Because the iden­ti­fier zeta has no bind­ing at the macro-defi­n­i­tion site.  + And this, in turn, is because of macro hygiene: mac lives in a sepa­rate lexical context, and can’t see the zeta binding at the calling site. Even though the names of the literal iden­ti­fiers are the same, their bind­ings are not. There­fore, the pat­tern doesn’t match, and the result is "nope".

    We can make the two zeta iden­ti­fiers match if we also make the same binding avail­able at the macro-defi­n­i­tion site, by importing math:

    (module mod br
      (require math)
      (define-macro-cases mac
        [(_ zeta) "match"]
        [else "nope"])
      (provide mac))
    (require (submod "." mod))

    (require math)
    (mac zeta) ; "match"
    (module mod br
  (require math)
  (define-macro-cases mac
    [(_ zeta) "match"]
    [else "nope"])
  (provide mac))
(require (submod "." mod))

(require math)
(mac zeta) ; "match"
    copy to clipboard

  2. A pattern vari­able (or wild­card) which can match anything (including a list of things) and assigns the matched item a name. Once a pattern vari­able is defined, all appear­ances of the pattern vari­able within a syntax object are replaced with the matched value.

    (define-macro (self ARG) #'ARG)
    (self "foo") ; "foo"
    (self (list 1 2 3)) ; '(1 2 3)

    (define-macro (add-three ARG) #'(+ ARG ARG ARG))
    (add-three 42) ; 126
    (define-macro (self ARG) #'ARG)
(self "foo") ; "foo"
(self (list 1 2 3)) ; '(1 2 3)

(define-macro (add-three ARG) #'(+ ARG ARG ARG))
(add-three 42) ; 126
    copy to clipboard

    The special wild­card _ also matches anything, but it can be used any number of times in a syntax pattern, and it cannot appear in a syntax template. It’s useful for signaling that an element of the syntax datum is being ignored.

    (define-macro (odds FIRST _ THIRD _ FIFTH)
      #'(list FIRST THIRD FIFTH))
    (odds 1 2 3 4 5) ; '(1 3 5)
    (define-macro (odds FIRST _ THIRD _ FIFTH)
  #'(list FIRST THIRD FIFTH))
(odds 1 2 3 4 5) ; '(1 3 5)
    copy to clipboard

  3. A sublist pattern, which will only match elements arranged with the same paren­the­siza­tion. If you know a certain element will be a list, a sublist pattern can be used to imme­di­ately match elements inside that list. Sublist patterns can be nested to any depth.

    (define-macro (m NUMS)
      (with-pattern ([(FIRST SECOND THIRD) #'NUMS])
        #'(list THIRD SECOND FIRST)))
    (m (1 2 3)) ; '(3 2 1)

    (define-macro (m2 (FIRST SECOND THIRD))
      #'(list THIRD SECOND FIRST))
    (m2 (1 2 3)) ; '(3 2 1)
    (define-macro (m NUMS)
  (with-pattern ([(FIRST SECOND THIRD) #'NUMS])
    #'(list THIRD SECOND FIRST)))
(m (1 2 3)) ; '(3 2 1)

(define-macro (m2 (FIRST SECOND THIRD))
  #'(list THIRD SECOND FIRST))
(m2 (1 2 3)) ; '(3 2 1)
    copy to clipboard

    By the way, a sublist pattern cannot create a sublist where none exists in the input:

    (define-macro (m2 (FIRST SECOND THIRD))
      #'(list THIRD SECOND FIRST))
    (m2 1 2 3) ; error: no match, because no sublist
    (define-macro (m2 (FIRST SECOND THIRD))
  #'(list THIRD SECOND FIRST))
(m2 1 2 3) ; error: no match, because no sublist
    copy to clipboard

  4. An ellipsis, which has to follow a pattern vari­able, and matches as many items as it can (similar to the “greedy” * oper­ator in regular expres­sions).

    (define-macro (ellip ARG ...)
      #'(list ARG ...))

    (ellip 1 2 3) ; '(1 2 3)
    (ellip "a" "b") ; '("a" "b")
    (ellip) ; '()
    (define-macro (ellip ARG ...)
  #'(list ARG ...))

(ellip 1 2 3) ; '(1 2 3)
(ellip "a" "b") ; '("a" "b")
(ellip) ; '()
    copy to clipboard

    Even though an ordi­nary pattern vari­able will match exactly one item, a pattern vari­able with an ellipsis can match zero items.

    If a pattern vari­able has an ellipsis, when the vari­able appears in a syntax object, the ellipsis must also appear (other­wise it’s an error):

    (define-macro (bad-ellip ARG ...)
      #'(list ARG))
    (bad-ellip 1 2 3) ; error: missing ellipsis
    (define-macro (bad-ellip ARG ...)
  #'(list ARG))
(bad-ellip 1 2 3) ; error: missing ellipsis
    copy to clipboard

    You can only have one ellipsis in each sublist of the pattern, including the top level:  + But see syntax/parse, an advanced macro-creation system that supports a richer pattern vocab­u­lary.

    ; OK: one ellipsis per sublist
    (define-macro (m (FOO ...) (BAR ...))
      #'(list FOO ... BAR ...))
    (m (1 2 3) (4 5 6)) ; '(1 2 3 4 5 6)

    ; not OK: two ellipses at top level
    (define-macro (m2 FOO ... BAR ...)
      #'(list FOO ... BAR ...))
    ; OK: one ellipsis per sublist
(define-macro (m (FOO ...) (BAR ...))
  #'(list FOO ... BAR ...))
(m (1 2 3) (4 5 6)) ; '(1 2 3 4 5 6)

; not OK: two ellipses at top level
(define-macro (m2 FOO ... BAR ...)
  #'(list FOO ... BAR ...))
    copy to clipboard

  5. A dot, which has to precede a pattern vari­able at the end of a list, and matches all remaining items in the list starting at the dot. The resulting pattern vari­able can either be used in a syntax object alone (in which case it’s treated as a list of items) or with another dot (in which case it’s spliced into the result). This expla­na­tion is more compli­cated than the example:

    (define-macro (m . ARGS) #'ARGS)
    (m + 1 2 3) ; means #'(+ 1 2 3) = 6

    (define-macro (m2 . ARGS) #'(list . ARGS))
    (m2 1 2 3) ; means #'(list 1 2 3) = '(1 2 3)
    (define-macro (m . ARGS) #'ARGS)
(m + 1 2 3) ; means #'(+ 1 2 3) = 6

(define-macro (m2 . ARGS) #'(list . ARGS))
(m2 1 2 3) ; means #'(list 1 2 3) = '(1 2 3)
    copy to clipboard

    “Isn’t a dot just a less flex­ible way of writing an ellipsis?” Pretty much. The above macros could be written with ellipses like so:

    (define-macro (m ARG ...) #'(ARG ...))
    (m + 1 2 3) ; means #'(+ 1 2 3) = 6

    (define-macro (m2 ARG ...) #'(list ARG ...))
    (m2 1 2 3) ; means #'(list 1 2 3) = '(1 2 3)
    (define-macro (m ARG ...) #'(ARG ...))
(m + 1 2 3) ; means #'(+ 1 2 3) = 6

(define-macro (m2 ARG ...) #'(list ARG ...))
(m2 1 2 3) ; means #'(list 1 2 3) = '(1 2 3)
    copy to clipboard

    You can prob­ably go your whole career as a macro writer without using the dot. But if you come across it in someone else’s source code, you’ll know what it means. (BTW, you can’t use a dot and ellipsis in the same level of a pattern.)

As a final exam, let’s combine all our vocab­u­lary elements into a single pattern. Hope­fully the result is not surprising:

(define-macro (m (WILD _ ELLIP ... literal) . DOT)
  #'(quote ((wild WILD) (ellip ELLIP ...) (dot . DOT))))

(m (1 2 3 4 literal) 5 6) ; '((wild 1) (ellip 3 4) (dot 5 6))
(define-macro (m (WILD _ ELLIP ... literal) . DOT)
  #'(quote ((wild WILD) (ellip ELLIP ...) (dot . DOT))))

(m (1 2 3 4 literal) 5 6) ; '((wild 1) (ellip 3 4) (dot 5 6))
copy to clipboard

More than one way to do it

As with regular expres­sions, you can often choose among multiple syntax patterns to match a single syntax object. Which you choose is a ques­tion of how strict you want to be about the match, and how you want to manip­u­late the pieces there­after.

For instance, these are all valid ways to match (and then reassemble) the list (+ 1 2):

(define-macro (m1 ARGS) #'ARGS)
(m1 (+ 1 2)) ; 3
(define-macro (m2 (ARG ...)) #'(ARG ...))
(m2 (+ 1 2)) ; 3
(define-macro (m3 (1ST 2ND 3RD)) #'(1ST 2ND 3RD))
(m3 (+ 1 2)) ; 3
(define-macro (m4 (1ST REST ...)) #'(1ST REST ...))
(m4 (+ 1 2)) ; 3
(define-macro (m5 (1ST . TAIL)) #'(1ST . TAIL))
(m5 (+ 1 2)) ; 3
(define-macro (m1 ARGS) #'ARGS)
(m1 (+ 1 2)) ; 3
(define-macro (m2 (ARG ...)) #'(ARG ...))
(m2 (+ 1 2)) ; 3
(define-macro (m3 (1ST 2ND 3RD)) #'(1ST 2ND 3RD))
(m3 (+ 1 2)) ; 3
(define-macro (m4 (1ST REST ...)) #'(1ST REST ...))
(m4 (+ 1 2)) ; 3
(define-macro (m5 (1ST . TAIL)) #'(1ST . TAIL))
(m5 (+ 1 2)) ; 3
copy to clipboard

Limi­ta­tions

You cannot make a syntax pattern that captures argu­ments two at a time (or three at a time, etc.).

You cannot take two ellip­sized pattern vari­ables and inter­leave their values within a syntax template.

You cannot make matches optional, in the sense of “match zero or one occur­rence of this item”. An ellipsis can approx­i­mate an optional argu­ment, because it will match zero occur­rences, but it will also match more than one:

(define-macro-cases m
  [(m REQ OPT ...) #'(list REQ OPT ...)]
  [else "nope"])

(m) ; "nope"
(m 1) ; '(1)
(m 1 2) ; '(1 2)
(m 1 2 3) ; '(1 2 3)
(define-macro-cases m
  [(m REQ OPT ...) #'(list REQ OPT ...)]
  [else "nope"])

(m) ; "nope"
(m 1) ; '(1)
(m 1 2) ; '(1 2)
(m 1 2 3) ; '(1 2 3)
copy to clipboard

If these short­com­ings bum you out, the syntax/parse library supports a richer vocab­u­lary of syntax patterns.

Further

← prev next →