This book teaches you how to design and implement programming languages, a powerful technique known as language-oriented programming. We’ll be using Racket, a language expressly built for making other languages.
My goal is to make this one of the most fun & interesting programming books you’ll ever read. That ought to be easy, because using Racket to make programming languages is the most fun & interesting thing I’ve done in more than 20 years of programming. (This book was itself made with a programming language called Pollen that I built with Racket.)
I assume you’ve done a little programming.
I don’t assume you’re an expert programmer.
I don’t assume you’ve used Racket.
I don’t assume you’ve made a programming language.
I don’t assume you’ve considered making a programming language.
I don’t even assume you’re sold on the whole idea yet. “Don’t we already have enough programming languages?” Short answer: no. (Though if you’re beyond persuasion, this book is probably not for you.)
I think of programming like writing or typography—a skill that can be valuable to anyone, not just computer scientists or professional software engineers. (If you’re already an expert, just skip ahead to the more challenging tutorials.)
But this book is as much about reasoning and abstract ideas as it is about technique or technology. For me, that’s what makes programming interesting and worthwhile.
Learning language-oriented programming will make you a better programmer. A programmer has exactly one indispensable tool: a programming language. Yet it’s possible to know a lot about using a language while the language itself remains opaque. Where did it come from? How does it work? This book will show you what’s inside the black box.
This knowledge will make you more proficient with languages generally, but also a more critical customer. It will increase your appreciation of certain languages, and—spoiler alert—diminish your affection for others.
At the high end, language-oriented programming will give you a deeper, radically different way of approaching programming problems. If a language is the one indispensable tool of any programmer, then it’s also the point of maximum leverage in a programming toolset. But historically, there’s been little opportunity for hacking our languages.
Far from being monolithic, languages can come in all different sizes. They can be tailored—even better than an ordinary program—to the exact requirements of a problem, or a group of programmers, or even just you.
Is a new language the right tool for every job? No. But when it is the right tool, it’s spectacular.
(Though it’s not mandatory reading, Why language-oriented programming? Why Racket? makes the case in more detail.)
It’s not. It’s just a different way of thinking about programming problems. And in fact, often makes them easier, because it brings more powerful mechanisms to bear.
Great! One of the most useful applications of Racket is for making domain-specific languages, which are “little languages” tailored to the needs of a certain task.
But the important ingredient in a domain-specific language is not the code, but the domain-specific knowledge that an expert brings to it. So even if you don’t program for a living, Racket can be a great tool for taking knowledge out of your brain and making it into a language—maybe just for your work, maybe for others too.
I doubt it.
Because there’s never been a dedicated platform for making languages quickly and cheaply. But now we have Racket.
It used to be that if you wanted to make a programming language, you’d need a C compiler and about five years free on your calendar. Even if your language could be designed quickly, you’d still have to build a minimal ecosystem around it to make it useful—compiler, debugger, libraries, documentation system, and so on. It’s a lot of work.
Racket, by contrast, is a programming environment designed in part as a laboratory for making other languages. Racket makes language experiments cheap & cheerful. You can focus on the parts that are important to you—for instance, the syntax and semantics of your new language. For the rest, you can rely on Racket’s existing tools and libraries. You can make a language in one hour.
Is Racket the last word in language implementation? Of course not. It is, in essence, a highly refined prototyping platform. Your language will not have the lightning reflexes of, say, hand-tailored C code.
But in return, Racket lets you make your language with a tiny fraction of the yak shaving that would otherwise be needed. For those of us without years to spare, that’s a welcome trade. And for most small- to mid-size languages, it’s a necessary trade—being able to make them cheaply is what allows them to be made at all.
I’m not hiding the ball—the book is called Beautiful Racket. Plainly, I’m a Racket partisan.
But I’m enthusiastic for good reasons:
Racket can do things that other languages can’t. We’ll start learning about those shortly (impatient skeptics can read about its hygienic macro system).
Racket is an open-source project (Apache/MIT), so it’s free to download, use, and deploy. It’s directed by a team of computer-science researchers who have been improving it for over 20 years, so it’s mature and stable.
Sure, it has a few gaps and warts. And we’ll cover those too. But if you can find a better tool for making programming languages—use that.
Of course. You can adapt the general ideas described here to any other implementation language. C? Python? JavaScript? Be my guest. This book, however, is still called Beautiful Racket. So you won’t find specific help here. We’re only going to use Racket to build new languages.
The tutorials take you through a series of programming-language projects of increasing difficulty. The tutorials are cumulative, so it’s fine to jump ahead. You’re always encouraged to fiddle with the code and improve, change, or break things.
The explainers teach you about core Racket features. The tutorials move briskly. If certain techniques are going over your head, you can detour into the related explainer. You can also refer to these later on, when you’re building your own languages.
Certain parts of this book require extra knowledge not covered within. For instance, the tutorial about the package server assumes a little knowledge of Git. But these parts are optional, and can be skipped without consequence.
This is necessarily an opinionated book. But I’ll never claim that my way is the best or only way to do things. (In fact, if I inspire someone to write An Even Better Book About Racket, I’ll be complimented.) My goal is to remove obstacles from your learning curve, not impose my style.
I sometimes point out that Racket has certain established idioms and virtuous habits. Whether you follow these is up to you. Sometimes it’s wise to do so.
And sometimes it’s fun not to. “You’re doing it wrong” is the lowest form of programming advice. You will not hear it from me.
The resources page lists other books, videos, and articles about making languages with Racket. They cover topics I do not, and offer different perspectives & techniques. These helped me when I was starting out with Racket.
Since I started using it in 2013, Racket has truly lit a fire in my brain, and transformed my work. By writing this book, I hope to pass along a little of that fire to others. I look forward to seeing what you do with it.
After you complete the setup, you can start the first tutorial.
—Matthew Butterick
PS. Hovering over the left margin of most paragraphs and code samples reveals a flag. Click this flag to reveal the URL for the paragraph and a comment form. If you find something that’s confusing or incorrect, let me know.