Thank you for your comment

Beau­tiful Racket / tuto­rials

Extend a data format: jsonic

Ground rules

  1. Every jsonic program produces valid JSON. This isn’t a rule we’d impose on a general-purpose program­ming language. But for a DSL, it makes perfect sense. Since our DSL exists to produce JSON, we can treat this as a firm require­ment.

  2. Every valid JSON file is a valid jsonic program. This rule isn’t strictly neces­sary. But it’s consis­tent with the first rule: if we treat a JSON file as a jsonic program, the result should be itself.

  3. Racket expres­sions can be embedded anywhere within a jsonic program that a JSON value—i.e., a Boolean, null, number, string, array, or object—would appear.

  4. If the char­ac­ters // appear in a line, the rest of the line will be commented out. This rule also isn’t strictly neces­sary. But JSON doesn’t support comments. And every program­ming language—even a small one—really should.

Design

Let’s step through each of these ground rules and consider what it means for our language design.

  1. Every valid jsonic program produces valid JSON.
    This means we’ll need a way of checking the output of any program against the JSON spec­i­fi­ca­tion. If the output doesn’t meet the spec­i­fi­ca­tion, we’ll raise an error. For those worried that we’ll have to write a whole JSON checker within jsonic: we won’t.

  2. Every valid JSON file is a valid jsonic program.
    This means that our jsonic parser should process JSON files trans­par­ently. The rest will be handled by our JSON output checker mentioned above. Obvi­ously, if the input is valid JSON, it will still be valid JSON at output.

  3. Racket expres­sions can be embedded in place of any JSON value.
    Since Racket expres­sions have a different syntax than JSON, this means we’ll need a pair of delim­iters to set these expres­sions apart from the surrounding code. The simple delim­iter pairs like () and [] and {} are already used by Racket S-expres­sions, so we need to avoid those. Instead, let’s use @$ and $@. We could choose anything, but those delim­iters are easy to type, and won’t conflict with anything in JSON or Racket.

    Since we’re substi­tuting Racket expres­sions for JSON values, we’ll also want to enforce a corre­spon­dence between the two. JSON supports only six kinds of values—Booleans, nulls, numbers, strings, arrays, and objects. These natu­rally corre­spond to Racket Booleans, 'null symbols, numbers, strings, lists, and hash tables. So if our embedded Racket expres­sions don’t produce one of these JSON-compat­ible values, we’ll treat it as an error.

    Further­more, JSON is a text-based format. There­fore, our Racket expres­sions will ulti­mately need to be converted to JSON strings, so they can be combined with the surrounding JSON. But rather than making users deal with that house­keeping, we’ll have jsonic take care of these conver­sions auto­mat­i­cally.

  4. Line comments start with //.
    As we discov­ered in bf, a good place to imple­ment comments is in the tokenizer. We’ll do that again in jsonic.

Taken together, we should be able to take our sample JSON and treat it as a jsonic program that produces equiv­a­lent JSON (we can run this program in DrRacket—#lang jsonic-demo is a working sample imple­men­ta­tion):

#lang jsonic-demo
[
  null,
  42,
  true,
  ["array", "of", "strings"],
  {
    "key-1": null,
    "key-2": false,
    "key-3": {"subkey": 21}
  }
]
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
#lang jsonic-demo
[
  null,
  42,
  true,
  ["array", "of", "strings"],
  {
    "key-1": null,
    "key-2": false,
    "key-3": {"subkey": 21}
  }
]
copy to clipboard

We should also be able to replace these JSON values with equiv­a­lent Racket expres­sions:

#lang jsonic-demo
// a line comment
[
  @$ 'null $@,
  @$ (* 6 7) $@,
  @$ (= 2 (+ 1 1)) $@,
  @$ (list "array" "of" "strings") $@,
  @$ (hash 'key-1 'null
           'key-2 (even? 3)
           'key-3 (hash 'subkey 21)) $@
]
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
#lang jsonic-demo
// a line comment
[
  @$ 'null $@,
  @$ (* 6 7) $@,
  @$ (= 2 (+ 1 1)) $@,
  @$ (list "array" "of" "strings") $@,
  @$ (hash 'key-1 'null
           'key-2 (even? 3)
           'key-3 (hash 'subkey 21)) $@
]
copy to clipboard

When we run this code in DrRacket, the ordering of key-1, key-2, and key-3 in the two samples may be different. That’s okay, because a JSON object, like a Racket hash table, doesn’t preserve key order.

This kind of DSL is also known as a preprocessor, because it’s source code that produces some other kind of source code. Program­ming jocks might be familiar with the C preprocessor; web jocks might be familiar with SASS, which is a preprocessor for CSS. But unlike a typical preprocessor, which usually supports only a limited vocab­u­lary of oper­a­tions, jsonic can use every­thing in the Racket language (as long as it ulti­mately reduces to one of our six kinds of values).

Ready to roll

Beyond that, our language imple­men­ta­tion will proceed just as it did for stacker and bf. Our language will have two main parts: a reader (comprising a tokenizer and a parser) and an expander.

But this time, one change. At the end of bf, we learned about pack­aging our language. For jsonic, we’ll set up our project as a package right away, to make our language easier to work with as we build it.

← prev next →