In days of yore, a BASIC interpreter would include a tiny “standard library” of math operations like SIN, TAN, INT, and LOG. All other operations would need to be defined within the program as named functions.
We could reimplement these library functions in our own interpreter. But two considerations.
First, it would be really boring.
Second, given that Racket has authentically vast libraries of math functions, it would be nice to find a way to use them in our basic programs.
So let’s depart from tradition and introduce a new import statement. It’ll work like require and make Racket functions available within our program just like those created with def:
1 2 3 4 5 6 7 | #lang basic-demo-3 10 import [math/number-theory] 20 print [nth-prime](15) 30 print [prime?](24) 40 import [racket/base] 50 print [max](f(1), f(2), f(5), f(4)) 60 def f(x) = x + x |
1 2 3 | 53 ; the 15th prime 0 ; 24 is not prime 10 ; the max of 2, 4, 10, 8 |
We can see that the main tasks will be:
Converting import statements into require forms.
Placing these require forms at the top level of the module.
Making sure we get usable return values from Racket functions.
The first task isn’t too hard. But we need to step gingerly when we add Racket identifiers. BASIC supports infix notation, and Racket doesn’t. Therefore, certain source strings that are valid BASIC expressions could also be read as Racket identifiers or module names. If confusion ensues, programs will break:
1 2 3 4 | racket/list foo-bar/zim-zam random-seed x+y+z |
Our lexer and parser are going to need some extra help distinguishing these. So we’ll adopt a convention that only a name in [square-brackets] will be treated as a Racket identifier. Otherwise, it will be treated as an ordinary BASIC expression. That way, we can mix them in our source code without risk of ambiguity:
1 2 3 4 | [racket/list] foo-bar/zim-zam [random-seed] x+y+z |
Next, we need to move our require forms at the top level of the module. Why do we need to do this at all? require has special status because it introduces bindings, and resolving bindings is a prerequisite to expanding and evaluating the program (see evaluation). So Racket is picky about where require forms can appear. For instance, putting a require inside an expression won’t work:
1 2 | require: not at module level or top level in: (require math/number-theory) |
But moving the require ouside the let, to the top level, cures the problem:
“Can’t we just use a syntax lift, like we did in functions?” Unfortunately, no. Because require forms introduce bindings, these forms are resolved before deeper macro expansion begins. A syntax lift, by contrast, can only rearrange expressions during this deeper expansion. So lifts happen too late to help us with require.
Instead, we can adapt the technique we used to implement variables. In that case, we needed to find all the unique variable references in the program so we could convert them into top-level define expressions. In the parser, we marked all the variables with a special b-id rule. This rule was spliced, turning it into a syntax property that let us locate the right elements inside the parse tree.
This time, we’ll mark the import targets with a special parser rule called b-import-name. Then, in our #%module-begin, we’ll pull these names out of the parse tree and convert them to top-level require forms.
Before we finish, we’ll also need to smooth over the differences between Racket and BASIC data types. BASIC can only handle integers, decimals, and strings. Racket functions, by contrast, can return many other data types. To do this, we’ll update our b-func macro to reject any values that don’t have equivalents in BASIC.
Our current lexer rule for BASIC identifiers allows alphanumeric strings that start with a letter (and might also contain $). We’re not going to change that.
But Racket identifiers and module names aren’t as restricted, and frequently contain other characters—for instance, ?, =, -, and /. So if we want to use the Racket names in our source code, we need a new lexing rule. To keep them distinct from BASIC expressions, we’ll also require that these names appear between square brackets.
To help with this, we introduce a new lexer abbreviation, racket-id-kapu, which will denote the characters not allowed in a Racket identifier.
To derive this abbreviation, we look in the Racket documentation to see which characters are permitted for symbols and identifiers. It turns out that “any character can appear directly in an identifier, except for whitespace and the following special characters: ()[]{}",'`;#|\”. So we use :or to create a set of forbidden characters, and assign this to racket-id-kapu.
1 2 | (define-lex-abbrev racket-id-kapu (:or whitespace (char-set "()[]{}\",'`;#|\\"))) |
Then we use this abbreviation to make the lexer pattern for our Racket identifiers:
The lexer operator :~ means “any character but these”, so this pattern means “any sequence of characters not in racket-id-kapu, between square brackets”.
On the right side of the rule, we trim the brackets from the matched lexeme using trim-ends and convert it to a symbol before packaging it as a new RACKET-ID token type.
In addition to these changes, we also add our new import keyword to our list of reserved-terms. With these adjustments, our lexer looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | #lang br (require brag/support) (define-lex-abbrev digits (:+ (char-set "0123456789"))) (define-lex-abbrev reserved-terms (:or "print" "goto" "end" "+" ":" ";" "let" "=" "input" "-" "*" "/" "^" "mod" "(" ")" "if" "then" "else" "<" ">" "<>" "and" "or" "not" "gosub" "return" "for" "to" "step" "next" "def" "," "import")) (define-lex-abbrev racket-id-kapu (:or whitespace (char-set "()[]{}\",'`;#|\\"))) (define basic-lexer (lexer-srcloc ["\n" (token 'NEWLINE lexeme)] [whitespace (token lexeme #:skip? #t)] [(from/stop-before "rem" "\n") (token 'REM lexeme)] [(:seq "[" (:+ (:~ racket-id-kapu)) "]") (token 'RACKET-ID (string->symbol (trim-ends "[" lexeme "]")))] [reserved-terms (token lexeme lexeme)] [(:seq alphabetic (:* (:or alphabetic numeric "$"))) (token 'ID (string->symbol lexeme))] [digits (token 'INTEGER (string->number lexeme))] [(:or (:seq (:? digits) "." digits) (:seq digits ".")) (token 'DECIMAL (string->number lexeme))] [(:or (from/to "\"" "\"") (from/to "'" "'")) (token 'STRING (substring lexeme 1 (sub1 (string-length lexeme))))])) (provide basic-lexer) |
Our parser updates are simpler. We add a new b-import rule to handle the import statement, and add b-import to the right side of the b-statement rule.
We also mark the import statement argument with a spliced b-import-name rule. In the parse tree, this will be attached to the argument as a syntax property:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | #lang brag b-program : [b-line] (/NEWLINE [b-line])* b-line : b-line-num [b-statement] (/":" [b-statement])* [b-rem] @b-line-num : INTEGER b-rem : REM @b-statement : b-end | b-print | b-goto | b-let | b-input | b-if | b-gosub | b-return | b-for | b-next | b-def | b-import b-end : /"end" b-print : /"print" [b-printable] (/";" [b-printable])* @b-printable : STRING | b-expr b-goto : /"goto" b-expr b-let : [/"let"] b-id /"=" (STRING | b-expr) b-if : /"if" b-expr /"then" (b-statement | b-expr) [/"else" (b-statement | b-expr)] b-input : /"input" b-id @b-id : ID b-gosub : /"gosub" b-expr b-return : /"return" b-for : /"for" b-id /"=" b-expr /"to" b-expr [/"step" b-expr] b-next : /"next" b-id b-def : /"def" b-id /"(" b-id [/"," b-id]* /")" /"=" b-expr b-import : /"import" b-import-name @b-import-name : RACKET-ID | STRING b-expr : b-or-expr b-or-expr : [b-or-expr "or"] b-and-expr b-and-expr : [b-and-expr "and"] b-not-expr b-not-expr : ["not"] b-comp-expr b-comp-expr : [b-comp-expr ("="|"<"|">"|"<>")] b-sum b-sum : [b-sum ("+"|"-")] b-product b-product : [b-product ("*"|"/"|"mod")] b-neg b-neg : ["-"] b-expt b-expt : [b-expt ("^")] b-value @b-value : b-number | b-id | /"(" b-expr /")" | b-func b-func : (ID | RACKET-ID) /"(" b-expr [/"," b-expr]* /")" @b-number : INTEGER | DECIMAL |
The pattern for our b-import-name allows either a RACKET-ID or a STRING, because imported modules can be specified with either an identifier (e.g., math/number-theory) or a path name (e.g., "number-theory.rkt").
We also add RACKET-ID as a possibility in the function position of the b-func rule.
First, we need to update "expander.rkt":
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | #lang br/quicklang (require "struct.rkt" "run.rkt" "elements.rkt") (provide (rename-out [b-module-begin #%module-begin]) (all-from-out "elements.rkt")) (define-macro (b-module-begin (b-program LINE ...)) (with-pattern ([((b-line NUM STMT ...) ...) #'(LINE ...)] [(LINE-FUNC ...) (prefix-id "line-" #'(NUM ...))] [(VAR-ID ...) (find-property 'b-id #'(LINE ...))] [(IMPORT-NAME ...) (find-property 'b-import-name #'(LINE ...))]) #'(#%module-begin (require IMPORT-NAME ...) (define VAR-ID 0) ... LINE ... (define line-table (apply hasheqv (append (list NUM LINE-FUNC) ...))) (void (run line-table))))) (begin-for-syntax (require racket/list) (define (find-property which line-stxs) (remove-duplicates (for/list ([stx (in-list (stx-flatten line-stxs))] #:when (syntax-property stx which)) stx) #:key syntax->datum))) |
As promised at the outset, we extract our IMPORT-NAME arguments by searching through the parse tree for unique elements carrying the 'b-import-name syntax property.
To do this, we first update our find-unique-var-ids helper function to take an argument that determines which property to find. Since we’re generalizing the interface of this function, we’ll also give it a more general name: find-property. We then replace our calls to find-unique-var-ids with calls to find-property, passing it a property name and a list of #'(LINE ...) syntax objects. + In DrRacket, we could also right-click the name of our function and use Rename …. This will update the variable name throughout the source file. In this case, we’re also changing the arity of the function, which has to be updated manually.
Having located the IMPORT-NAME objects, all we need to do is put them into a require at the top of the module.
Once we have Racket functions available, we also need to smooth the differences between Racket values and BASIC values. So let’s also update our b-func macro in "expr.rkt":
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | #lang br (require "line.rkt") (provide b-expr b-sum b-product b-neg b-expt b-def b-func) (define (b-expr expr) (if (integer? expr) (inexact->exact expr) expr)) (define-macro-cases b-sum [(_ VAL) #'VAL] [(_ LEFT "+" RIGHT) #'(+ LEFT RIGHT)] [(_ LEFT "-" RIGHT) #'(- LEFT RIGHT)]) (define-macro-cases b-product [(_ VAL) #'VAL] [(_ LEFT "*" RIGHT) #'(* LEFT RIGHT)] [(_ LEFT "/" RIGHT) #'(/ LEFT RIGHT 1.0)] [(_ LEFT "mod" RIGHT) #'(modulo LEFT RIGHT)]) (define-macro-cases b-neg [(_ VAL) #'VAL] [(_ "-" VAL) #'(- VAL)]) (define-macro-cases b-expt [(_ VAL) #'VAL] [(_ LEFT "^" RIGHT) #'(expt LEFT RIGHT)]) (define-macro (b-def FUNC-ID VAR-ID ... EXPR) (syntax-local-lift-expression #'(set! FUNC-ID (λ (VAR-ID ...) EXPR)))) (define-macro (b-func FUNC-ID ARG ...) #'(if (procedure? FUNC-ID) (convert-result (FUNC-ID ARG ...)) (raise-line-error (format "expected ~a to be a function, got ~v" 'FUNC-ID FUNC-ID)))) (define (convert-result result) (cond [(number? result) (b-expr result)] [(string? result) result] [(boolean? result) (if result 1 0)] [else (raise-line-error (format "unknown data type: ~v" result))])) |
Here, we test the result of the function expression by passing it to the helper function convert-result. If it’s a number, we hand it off to b-expr. If it’s a string, we pass it through. If it’s a Boolean, we convert it to an integer. Otherwise, we raise an error. + We could’ve put the convert-result code inside the macro. But it’s good practice to make the output of a macro as small as possible. See macros.
Finally, though we’ve hoisted our actual imports into require forms at the top level of the module, we still need to do something with the b-import nodes that remain in the parse tree. So we just add a macro to "misc.rkt" that converts these nodes into (void):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | #lang br (require "struct.rkt" "expr.rkt") (provide b-rem b-print b-let b-input b-import) (define (b-rem val) (void)) (define (b-print . vals) (displayln (string-append* (map ~a vals)))) (define-macro (b-let ID VAL) #'(set! ID VAL)) (define-macro (b-input ID) #'(b-let ID (let* ([str (read-line)] [num (string->number (string-trim str))]) (or num str)))) (define-macro (b-import NAME) #'(void)) |
That should be all we need to make our original example work:
1 2 3 4 5 6 7 | #lang basic 10 import [math/number-theory] 20 print [nth-prime](15) 30 print [prime?](24) 40 import [racket/base] 50 print [max](f(1), f(2), f(5), f(4)) 60 def f(x) = x + x |
1 2 3 | 53 0 10 |
Let’s remove the first import statement and confirm that we get a sensible error:
1 2 3 4 5 6 | #lang basic 20 print [nth-prime](15) 30 print [prime?](24) 40 import [racket/base] 50 print [max](f(1), f(2), f(5), f(4)) 60 def f(x) = x + x |
1 | nth-prime: unbound identifier in module in: nth-prime |
Let’s also try calling a function that returns an unusable data type, in this case a list:
1 2 3 4 | #lang basic 40 import [racket/base] 50 print [list](f(1), f(2), f(5), f(4)) 60 def f(x) = x + x |
1 | error in line 50: unknown data type: '(2 4 10 8) |
Looks good.
Our focus here was math-related functions, so we only considered expressions as arguments to functions. It would be easy, however, to also allow strings as input, so we could also use Racket’s string-manipulation functions. We’d just need to change the right side of the b-func parser rule to accept STRING tokens as well.
Though it’s easy to convert a Boolean return value to an integer, it’s harder to go the other way. If our BASIC program invokes a Racket function with 0 as an argument, should it be treated as the integer 0 or the Boolean #f? A tricky question that we’ll leave for curious characters.