Beau­tiful Racket / appendix



"Hello world"

1	"Hello world"



public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello world");
    }
}

1 2 3 4 5

public class HelloWorld {     public static void main(String[] args) {         System.out.println("Hello world");     } }

1. 

   Every­thing is an expres­sion. [Lisps] Most program­ming languages are a combi­na­tion of two distinct ingre­di­ents: expres­sions (things that are eval­u­ated to produce a value) and state­ments (things that denote an action). For instance, in Python, x = 42 is a state­ment, and x + 42 is an expres­sion.

   

   State­ments and expres­sions are distinct because while expres­sions can be natu­rally nested with each other, state­ments and expres­sions cannot. For instance, in Python, this is a valid expres­sion:

   42 + 101

   1

   42 + 101

   
   

   And so is this, substi­tuting another expres­sion for the right-hand value:

   42 + (100 + 100)

   1	42 + (100 + 100)

   
   

   But this is not valid:

   42 + if 1 < 0:
            100
        else:
            200

   1 2 3 4

   42 + if 1 < 0: 100 else: 200

   
   

   Why? Because in Python, a stan­dard if condi­tional is a state­ment, and can only be used in certain posi­tions.

   

   By making every­thing an expres­sion, however, Lisps remove this limi­ta­tion. Since expres­sions are nestable, anything in the language can be combined with nearly anything else. For instance, because an if condi­tional is an expres­sion, you can use it in place of a value:

   (+ 42 (if (< 1 0) 100 200))

   1	(+ 42 (if (< 1 0) 100 200))

   
   

   “But wait! Python has a ternary condi­tional expres­sion!” It doesn’t change the essen­tial point, but OK—you can indeed write this:

   42 + (100 if 1 < 0 else 200)

   1	42 + (100 if 1 < 0 else 200)

   
   

   But now suppose we want to use a condi­tional in place of the oper­ator, not the right-hand value. In a Lisp, because every­thing is an expres­sion—including the oper­ator itself—this is easy:

   ((if (< 1 0) + *) 42 100)

   1	((if (< 1 0) + *) 42 100)

   
   

   But if you try the same thing in Python, it will raise a syntax error:

   42 (+ if 1 < 0 else *) 100

   1	42 (+ if 1 < 0 else *) 100

   
   

   Why? Because Python oper­a­tors are not expres­sions.

   

   This is a synthetic example. The point is not that you’d neces­sarily want to do this, but that Lisps permit it. You don’t run into the syntactic guardrails that are lurking in other languages. As a programmer, this simpli­fies your work, because every­thing snaps together easily. It also expands your possi­bil­i­ties, because you can combine parts of the language in unusual ways if you feel like it.

   

   It’s similar to the basic idea behind Legos. Other building sets offer special­ized pieces that can only fit together certain ways. But by sharing uniform measure­ments, Lego bricks offer maximum possi­bil­i­ties for combi­na­tions. This ends up being more flex­ible & more fun.

   

   So it is with an expres­sion-based language. If you find this idea exciting, congrat­u­la­tions—you might be a Lisp programmer. (If you find this idea weird and scary, this is a good moment to bail out.)

2. 

   Every expres­sion is either a single value or a list. [Lisps] Single values are things like numbers and strings and hash tables. (In Lisps, they’re some­times called atoms.) That part is no big deal.

   

   The list part, however, is a big deal. In a language like Python, the list is one data type within the language. But in Lisps, the list is more like an orga­nizing prin­ciple for every­thing that happens.  + The name “Lisp” is an abbre­vi­a­tion for “list processing”. So yes, you can use the list as a data type. But a func­tion call is also a list. In fact, the source code for the func­tion is a list. Actu­ally, the rest of the program is too. Lists are every­where.  + The fancy CS term for this prop­erty is homoiconicity.

   

   The bene­fits of lists are similar to that of expres­sions. By bringing more of the language into a consis­tent form, more possi­bil­i­ties arise for how pieces can be combined and manip­u­lated.

   

   Seibel describes Lisp as a tool for getting “more done, faster”. Here, you can start to see why this is so. Lisp languages are immensely flex­ible and permis­sive in how their pieces can be connected. This means that the way you think about a program­ming problem can be quite close to the way you actu­ally program it. (This is also why Lisps have tradi­tion­ally excelled for proto­types and exploratory work.)

   

   To be fair, getting the most out of a Lisp means learning to think more in the Lisp idiom of lists and expres­sions. For that reason, I agree with Seibel—trying it your­self is the best way to be convinced of the bene­fits. As you get a feel for lists and expres­sions, it does pay increasing divi­dends throughout the language. You see how tiny lines of code can produce epic amounts of work. You also start to appre­ciate that even in a well-designed language like Python, you’re spending a lot of time shaping your ideas to fit its limi­ta­tions, like shaving an invis­ible yak.

3. 

   Func­tional program­ming. [Lisps] Yes, I know that other languages offer func­tional-program­ming features, and that Lisps aren’t consid­ered pure func­tional languages. But many program­mers haven’t been exposed to this idiom, and thus tend to under­rate its bene­fits. I know I was in that cate­gory.

   

   Func­tional program­ming doesn’t mean program­ming with func­tions. Every­body does that. Func­tional program­ming refers to a stricter style where func­tions receive certain data as input, process only that data, and return a result. In func­tional program­ming, func­tions avoid two habits common in other languages: muta­tion (= changing data in-place rather than returning a value) and relying on state (= extra context that’s not provided as input, for instance global vari­ables).

   

   “Wait—I love state and muta­tion. Why would you take them away?” Because they’re false friends. They contra­dict the essen­tial concept of a func­tion, which is to encap­su­late data and algo­rithms. When a func­tion relies on state or muta­tion, it’s oper­ating outside those bound­aries. There­fore, you either take on an increasing house­keeping burden to keep track of how func­tions affect each other, or watch your program sink into a swamp of myste­rious, compli­cated bugs.

   

   Program­ming in a func­tional style takes more effort at the outset. But it encour­ages you to struc­ture the program in a clean, compart­men­tal­ized way. This pays off imme­di­ately in programs that are easier to test and debug. It’s also more likely to lead to reusable compo­nents, since func­tions are truly inde­pen­dent.

   

   This bite-the-bullet aspect of func­tional program­ming is another reason why you can get “more done, faster” with a Lisp. The differ­ence between proto­type and produc­tion code often ends up being small, because you don’t take as many short­cuts at the start. The program grows and evolves more smoothly because it’s easy to change one part without causing ripple effects else­where.

4. 

   Macros. [Racket] Some Rack­e­teers quibble with this term, prefer­ring syntax trans­formers, because a Racket macro can be more sophis­ti­cated than the usual Common Lisp macro.

   

   A macro in Common Lisp is a func­tion that runs at compile time, accepting symbols as input and injecting them into a template to produce new code.

   

   Macros in Racket, on the other hand, rely on the concept of hygiene. They can handle Common Lisp-style macros, but also more elab­o­rate syntax rearrange­ments.

   

   But forget that—what’s in it for you? As a programmer, you end up getting two bites at the apple every time you run a file: Racket runs the macros (which alter the source code), and then the source code itself.

   

   Unlike some­thing like the C preprocessor, which is basi­cally a sepa­rate mini-language, Racket macros are them­selves Racket func­tions that give you access to every­thing in Racket. Like lists and expres­sions, macros add another layer of expres­sive possi­bil­i­ties.

5. 

   Create new program­ming languages. [Racket] When I first read that Racket could be used to create new languages, I had two thoughts—are they serious? and would I really want to do that? The answers were yes and oh hell yes.

   

   Between expres­sions, lists, and macros, Racket gives you a huge amount of semantic flex­i­bility. But on top of that, it also adds syntactic flex­i­bility, in that you can define a reader that converts surface syntax into stan­dard Racket S-expres­sions, and an expander that deter­mines the meaning of these S-expres­sions.  + Paul Graham’s program­ming language Arc, a dialect of Lisp, was built on top of Racket.

   

   You can use this facility to make special­ized dialects of Racket. Or imple­ment earlier languages. Or create entirely new languages with their own rules. You can use any of these languages within DrRacket to code new projects. (These special languages are some­times called domain-specific languages, or DSLs.) Scribble is a DSL based on Racket; Pollen is a set of DSLs based on Scribble.

   

   If you’re like most program­mers, you’ve never had a tool for making a new language, so you’ve not consid­ered it a real­istic approach to a problem. And you won’t need it all the time. But when you do, it is awesome, in both the new and old senses of that word.

   

   (The sequel to this piece—Why language-oriented program­ming? Why Racket?—is devoted to this topic.)

6. 

   Libraries & docu­men­ta­tion. [Racket] This might not look like a compet­i­tive differ­en­tiator—doesn’t every program­ming language have libraries & docu­men­ta­tion?

   

   Yes, but prob­ably not like this. As a conse­quence of being used in research settings for many years—Racket’s core devel­op­ment team is made of computer-science profes­sors—Racket’s libraries & docs are more like a trans­mis­sion from a highly evolved alien intel­li­gence.

   

   You get the essen­tials, of course: web server, JSON, XML, drawing, foreign-func­tion inter­face, and so on. Then you notice pack­ages you maybe didn’t expect: GUI appli­ca­tion frame­work, math plot­ting, package-distri­bu­tion system, unit tester. Beyond that, your face starts to melt a little bit: seman­tics engi­neering? Futures visu­al­izer?

   

   I won’t pretend to know what all this shit does. A lot of it is over my head. But I like that. Each week I use Racket, I end up exploring a new part of the library, and learning some­thing new. As opposed to other languages that seem to kill brain cells on contact (= pretty much anything named *Script, I find). 

   

   This learning is only possible because of Racket’s truly outstanding docu­men­ta­tion. It’s vast, thor­ough, precise, and approach­able. See for your­self.  + If you don’t like the typog­raphy and layout of the docs, blame me.

7. 

   DrRacket. [Racket] Yes, I know how to use a command line. But Racket includes a cross-plat­form graph­ical IDE called DrRacket that’s pretty great. DrRacket lets you edit, run, and debug Racket source files (or any other language based on Racket—see item #9 on this list.)

   

   No, it doesn’t have the Ginsu-level search-and-replace facil­i­ties of some­thing like Sublime Text. But it does have helpful editing features opti­mized for Racket code (for instance, you can right-click on a symbol name and rename it throughout the file, or jump from a func­tion to its docu­men­ta­tion).

   

   More­over, the command line within DrRacket doesn’t just show plain text—it can show stacked frac­tions, draw­ings, math plots, and other unex­pected guests. If your command line does all that, by all means keep using it.

8. 

   X-expres­sions. [Racket] This choice is some­what biased by my work with Racket, which mostly involves docu­ment processing and type­set­ting. But related topics arise in most web program­ming. An X-expres­sion is a special native data struc­ture that Lisps use to repre­sent HTML and other XML-ish data.

   

   Well, not “special” in a Lispy sense—keeping with the usual policy, an X-expres­sion is just another list—but special in the sense that other program­ming languages don’t have it. Usually your choice is to repre­sent HTML either as a string or as a full XML tree. A string is wrong because it doesn’t capture the struc­ture of the HTML, as defined by its tags and attrib­utes. An XML tree shows this struc­ture, but conceals the sequen­tial nature of the data elements, and is unwieldy to work with.

   

   An X-expres­sion ends up being an ideal hybrid between a string and a tree. More­over, because it’s just another list-based expres­sion in the language, you have a lot of options for processing it. Trans­lating an X-expres­sion to or from a text repre­sen­ta­tion using angle brackets is trivial and fast. (Details.)

   

   Given the close kinship between XML-ish data struc­tures and Lisp languages, I have no expla­na­tion why, during the Internet era, they’ve not been paired more often. They’re like peanut butter and jelly.

9. 

   Scribble. [Racket] Pollen wouldn’t have been possible without Scribble, so for me, this has been the stone-cold killer feature of Racket. But that won’t be true for everyone, so I’m moving it down the list.  + Scribble was orig­i­nally created to serve as Racket’s docu­men­ta­tion language (a job it does well).

   

   Scribble is a dialect of Racket that inverts the ordi­nary rela­tion­ship of plain text and code: rather than embed­ding text strings within source, a Scribble docu­ment consists of code expres­sions embedded within plain text.

   

   “So it’s like an HTML template language.” Yes, in the sense that a template language allows code to be embedded in text. But also no, because a template language is usually a pidgin version of a real program­ming language. Scribble, by contrast, lets you invoke any Racket code simply by adding a command char­acter to the front. In keeping with the theme already estab­lished, this approach is both simpler (because there’s almost nothing new to learn) and more powerful (because you can invoke anything in Racket).

   

   In its combi­na­tion of text and code, Scribble has more kinship with LaTeX. While it doesn’t have the type­set­ting facil­i­ties of LaTeX, the program­ming facil­i­ties are much better.

10. 

    Oppor­tu­ni­ties to partic­i­pate. [Racket] In theory, open-source soft­ware projects create the oppor­tu­nity for groups of devel­opers to join together and make better things in collab­o­ra­tion than they could sepa­rately.

    

    In prac­tice, I’ve found that they sort into a bimodal distri­bu­tion: over here, the under­doc­u­mented solo projects that sputter along fitfully (if at all); over there, the mature, popular projects that can be intim­i­dating for new contrib­u­tors.

    

    As an open-source project, Racket is posi­tioned at a happy medium. The core devel­op­ment team has been working together for years, and the commits remain fast & furious. But they’re friendly scien­tists, not Shire-dwelling egotists, and remain recep­tive to improve­ments across the whole system. If you have a better idea, they’ll listen; if you code it up to their stan­dards and make a pull request, they’ll take it.

    

    [2021 update: I no longer contribute to Racket due to abuse & bullying by the project lead­er­ship. Everyone in the broader Racket commu­nity, however, has always been helpful and kind.]

• 

  Racket’s killer app is language-oriented program­ming, which I discuss in the sequel to this piece—Why language-oriented program­ming? Why Racket?