The making of Beautiful Racket

Tran­script of remarks given at Rack­etCon, 18 Sept 2016

I’m the author of the Pollen package on the Racket package server that I use to do a lot of crazy things for Racket. Including the Rack­etCon website.

I want to thank all the people who have been using Pollen in the last year. We have a tiny, ultra-tiny user commu­nity. But, like a lot of fun projects, it’s so neat to make some­thing and then see other people make things with it that you didn’t neces­sarily expect.

And I brought one. Because it fit into my brief­case. This is the book Flat­land. It’s not actu­ally about Racket’s first family. It’s a novel: “A Romance of Many Dimen­sions”. But a gentleman named Joel Dueck up in—sorry if I get this wrong, Joel, I think you’re up in Minnesota—used Pollen to make this. He made a web version and he made a PDF version. And then he sent the PDF version to Create­Space to make paper­backs. And he did this all from one set of Pollen sources. Pollen being this DSL that I created to do digital books.

And what’s fun about this for me is it’s such a great idea to see—what I was hoping for, to have one set of source files that you can do web and PDF with. I haven’t actu­ally done it yet! So thank you Joel. Pass that copy around if you like. Again, that’s all Racket-gener­ated product.

Right, so I’m here to talk about Beau­tiful Racket. Where are the slides? Well, you have laptops. So if you have a laptop or phone or you want to follow along at home, please go to beau­ti­ful­racket.com.

Beau­tiful Racket is a book I’m writing about making DSLs and languages with Racket. What is at beau­ti­ful­racket.com right now is a sort of initial excerpt, which shows all the major sections. You can start looking at it while I tell you a few more things about what I’ve been up to.

Because I actu­ally was speaking with my man Oliver Flatt last night. At the bar, where I speak to a lot of young people. And he was saying “I heard that you’ve pretty much spent the whole past year program­ming Racket.” And I said yes, that’s true. I decided, after last year’s Rack­etCon, that I was having such a great time with Racket, and learned so many things, and been so produc­tive—what would happen if I just set every­thing on my schedule aside and did nothing but Racket? And Beau­tiful Racket is the major project that I’ve been working on.

And he was very curious. And he said “So you haven’t been going to a job?” No. “You haven’t been going to school?” No. “You don’t have students?” No. I just do Racket.

And he said, “How do you pay for this?”

I thought, this is a fellow after my own heart. I said, “that’s the great thing, Oliver. I’m paying for it with Racket!”

Now he’s very inter­ested.

As he should be. I said yes, I’m totally a pro Racket programmer. Because the first thing I made with Racket—I mean, I made Pollen in order that I could make Prac­tical Typog­raphy, which you’ve all heard about. Which is this website that offers hints about typog­raphy. It offers hints about slide presen­ta­tions. I don’t know, if anybody here has to do a slide presen­ta­tion ... ? I don’t know, you might go on this free website.

And there are fonts for sale. And people buy the fonts and they send me money. So every day, the internet sends me money. And all I’m doing is working on Racket. So it’s a great setup: Racket project #1 is spurting all this money that then allows me to work on Racket project #2. So if you can make it work for your­self, great. I’m living the dream.

So, why Beau­tiful Racket? You know, this Rack­etCon’s actu­ally really exciting to see, because it’s nice to see so many talks about making DSLs with Racket. I feel like this is the highest concen­tra­tion yet. Because that’s really—I mean, for me, I came to Racket wanting to write a DSL. For what­ever reason, I had the genetic muta­tion already. And I actu­ally proto­typed my DSL in Python. I mean, that was a garbage way to do it. But the fact that I was out there, trying to make a DSL—you know the old adage: when the student is ready, the teacher appears. I was ready for Racket. So when I discov­ered it, I thought yes, these people have figured out every­thing I’m trying to do. Just much better.

So I’m trying to take what I learned on Pollen, and that expe­ri­ence, and put it into this book form. And the book really has two purposes, when I’m speaking of Beau­tiful Racket.

On the one hand, it’s meant to be a tuto­rial of both Racket as a language, and as a way of making languages. And to that regard, if you’re on the website, you can see that there’s going to be a set of tuto­rials that go from simpler to more complex, intro­ducing the key language-making features.

And then there’s also a set of what I call “explainers” that are short overviews of certain key topics in the Racket language. Because I still remember what it was like to be a total Racket newbie. I have all the ques­tions that I wrote down and strug­gled with. So it’s almost like the book that future me wants to go back and give to past me—if only I had contin­u­a­tions in real life.

A lot of the book links back to the docu­men­ta­tion. Our docu­men­ta­tion is wonderful. But it’s almost just to give more of an on-ramp for people who are new to the whole concept.

Because a ques­tion that I asked myself when I was starting this out is “what is a DSL?” Matthias [Felleisen], in his Racket Mani­festo, has given us a tech­nical answer. But I really appre­ci­ated what Emina [Torlak] had to say this morning, because I thought I was going to be the only one to say it. Now the keynote speaker agrees with me. That’s so much better.

She was talking about how a DSL is a way of taking domain-specific knowl­edge and pack­aging it inside a program­ming language. And I think that really encap­su­lates some­thing impor­tant, which is that a DSL has two ingre­di­ents:

And the thing I think is espe­cially inter­esting is that these people with the DSK, I’ll call it—the domain-specific knowl­edge—they’re not neces­sarily the profes­sional program­mers that you’re going to find on the streets of San Fran­cisco, brag­ging about the stock options they got in the new iOS app that—god knows what it does. The point is, that’s a different kind of programmer. Those are the brogram­mers. They’re about snap­ping necks and cashing checks. They’re about what­ever’s trendy. Domain-specific knowl­edge—they don’t have any domain-specific knowl­edge! They don’t know anything. I mean, they know about program­ming. But that’s not what we mean. We’re talking about being able to bring knowl­edge from another area, and bring it into a DSL. Which is such a powerful thing.

And it’s part of what has made Pollen successful. Hey—I am not on your level as a programmer. I am not. But I have a lot of domain-specific knowl­edge about typog­raphy, and design, and how writers and people who publish things want to work with their mate­rial. And that’s really where all the impor­tant ideas in Pollen come from. To see that I can take it, and as Emina said, package it into a language, turn around, and share it with people and docu­ment it. And you get things like [the Flat­land book] you see coming around. People really get it. And that, to me, is very satis­fying. And that’s really the expe­ri­ence I have: of sharing the knowl­edge, and seeing people say “oh yeah, the way this dude sees the world is a good way of doing things.”

So to go to the next ques­tion about who is Beau­tiful Racket for. I think the idea that Beau­tiful Racket can be for people who have that domain-specific knowl­edge. Right? Not neces­sarily profes­sional program­mers. I mean, if they want to come in and learn how to do DSLs, and bring it back to their iOS app—that’s great. But I really want to see—I would love to see more lawyers, and scien­tists, and accoun­tants, and psychi­a­trists, whoever—who are dealing with this domain-specific knowl­edge—come in to Beau­tiful Racket and start saying “oh yeah, I see how to do this and why.” Again, a big part of the book is going to be making the case for DSLs.

Now that said, because I’ve been spending a lot of time looking at the DSL appa­ratus. Racket has a lot of good features. Wonderful features, really. But then there are some little gaps along the way where if you’re a newbie, you can get your shoelace trapped. And I’m trying to sort of smooth over some of those things.

Like the fact that we talk about how Racket’s main feature is that it lets people make other languages. But that’s not really the lead message. It’s not the lead message on the website. Or in Realm of Racket. Or in How to Design Programs. So Beau­tiful Racket is really putting the language-making front and center. And putting the fun stuff too. Trying to get people into making languages quickly.

Another part of Racket: we have tuto­rials in the docu­men­ta­tion. They’re very clear. And they’re very good, in a certain sense. But they don’t help you make languages. The initial tuto­rial about drawing stuff. There’s a tuto­rial about doing a blog engine with the web server. Again—it’s a great tuto­rial. But Racket isn’t really going to compete in the web-server and blog-engine space, is it? It’s going to compete in domain-specific languages. So, again, just wanting to bring that forward for people who are new to Racket.

What else? Here’s one that’s been a kind of vexing issue for me. Which is that—and you’ve seen a little bit of this today, and Alexis [King] touched on this—is that DSLs based in S-expres­sions snap together really easily. But once you venture outside of S-expres­sions, and you start hacking on the reader, and having to deal with a parser—the road gets a little bit bumpy.

And I’ve had to spend some time going back, and try to find a coherent way of explaining this to people new to Racket. I don’t think, for instance, it’s OK to say “oh, and then you write a parser by hand. What­ever you feel like doing is fine.”

So I discov­ered that our friend Danny Yoo—who orig­i­nally made a fun tuto­rial for the bf language, which I repur­posed for Beau­tiful Racket. He also did a neat DSL called ragg. Which was his “Racket AST Gener­ator Gener­ator”. It’s a parser gener­ator that will take a BNF grammar and turn it into a parser. So I took his work and I forked it and I added some improve­ments to it. And that’s what I’m using in Beau­tiful Racket.

Again—there are upsides and down­sides to using a parser gener­ator. But I’m trying to show people the easiest path. It also lets you intro­duce the concept of a grammar to people who might not know about it.

Another place that I’ve had a little bit of trouble is with hygienic macros. Now, we love macros. We love hygiene in macros. It’s great. But some­times you need to do unhy­gienic iden­ti­fiers. And most of all, you some­times need to do them when you’re making DSLs. Right? Because you want to be able to type in iden­ti­fiers onto the REPL, or into a defi­n­i­tions window [in DrRacket], and have them behave as if they were defined there. Which is what an unhy­gienic iden­ti­fier is. So I think that the inter­face and the expla­na­tion of how to break hygiene smoothly and effi­ciently is a little bit clunky.

And I know that Matthias [Felleisen] is prob­ably saying “hey, use syntax para­me­ters.” They’re great. But you need to know the name of a syntax para­meter in advance. And again, if you want to do a language, like—one of the tuto­rials in Beau­tiful Racket is going to be a BASIC inter­peter. Because who doesn’t love BASIC. But you want to be able to do

And just have anything in the blank. If it’s a syntax para­meter, you have to know the name ahead of time. So that’s another example.

So that’s really it. The Beau­tiful Racket website—you can click on any para­graph—I’m kind of relying on Cunningham’s Law here. Have you heard of it? It says that on the internet, the fastest way to get the right answer is to post the wrong thing, and people will correct you. Click on any para­graph and there’ll be a comment form. So if you see some­thing you don’t like, I invite you. You don’t even have to put your email in. You can just type the nasty­gram. I don’t know who you are. But I’m saying there are plenty of things that could be improved.

I mean, obvi­ously I’m trying to find a balance here. There are certain aspects of Racket where I’m trying to stash some of the dirty details for later. Because again, the docu­men­ta­tion is wonderful. That is a chal­lenge too: to figure out how much to conceal at any point, and how much to show.

But this is where we are after eight or nine months. I think there’s another three or four months. There are more sections to come. There might be a couple special guests making an appear­ance.

But again, I thank everyone for this wonderful commu­nity. And really, this project is a tribute to every­thing that you have taught me. And it’s way that I can give it back. And I hope to bring more people into the fold so we keep hearing about all the great DSLs. So thank you.

Ques­tion from Matthias Felleisen: I want to talk back, because I think you’re wrong about one small point.

MB: Could be a big point. Fill out the form!

MF: Unfor­tu­nately, I would have to fill out all the forms. It’s perva­sive. And it’s the following. I think domain experts want to achieve some­thing with their knowl­edge. They just don’t want to write out the knowl­edge and then it’s there, and it’s inac­tive, passive, stuff on the web. They want to hook up to web services. And they want to distribute to many users of domain knowl­edge, [for instance] at a remote hospital. There’s a doctor some­where who puts down his domain knowl­edge [in one place]. But it’s really for people some­where else.

So you really need a real program­ming language under­neath that can do all the things that Racket can do. And Racket is as good as any other language. (If it’s not as good, you guys didn’t contribute enough libraries. It’s your fault.) We really need the other piece of the mani­festo too. It’s really a full-spec­trum language. You can do anything you want with the domain knowl­edge that was encoded in DSLs.

MB: Where do we disagree?

MF: You said, these domain experts don’t want web services.

MB: Oh, I’m sorry. I was unclear. What I was saying was, for instance, when I look at the Racket web-server tuto­rial, or the blog tuto­rial—that’s not the special, valu­able thing that Racket can do. I totally agree with you.

MF: We don’t disagree.

MB: All right. You know, I think back—a couple years ago on this stage, a gentleman named Michael Fogus gave the keynote. Michael Fogus, a wonderful guy, a friend of Racket.

But I completely disagreed with him when he was saying that maybe Racket could ride on the coat­tails of JavaScript. Or become a language on the server, on the back end. No! I think Racket should be Racket! Front and center, harnessing human knowl­edge, expert knowl­edge—that’s where it ought to be. It’s not Cinderella cleaning out the fire­place. It’s right in front of your face.

MF: So now I’m going to end with a chal­lenge. And that was the first picture in the mani­festo. And that is: if you’re a lawyer, or a doctor, with domain knowl­edge—they need to interact even­tu­ally. And we have not really figured out how DSLs talk to each other. We can kind of do it manu­ally—A is like X. Or descend into the common language of English. But that’s a manual process. Can we do better? For those of you who do research, for those of you who want to do research, for those of you with nothing else to do, and if you have spare cycles—that’s a problem.

MB: We need more books on Racket. That can be Beau­tiful Racket 2. I’ll license it. There you go.