Lisp's Superpower: Saving Time Writing Code

A lot of people have heard about Lisp, but don't know what's so special about it. I decided to explore its potential by extending a json library using Lisp macros and saved a whole lot of time writing code. Here’s how.

I was writing a game-playing AI that needed information about the game from a json file, and I needed a way to read data from the json. The json library I was using turned json into lists and hash tables, but I wanted to map it to classes for two reasons:

1. To give the data meaning in the context of the game and
2. To give it structure and make it easy to manipulate.

Initially, I was doing this manually but quickly found that I was just writing the same code over and over. That’s when I realised that this was a perfect opportunity to use Lisp's macros. Lisp is one of the oldest and most revolutionary programming languages.

In other languages, there would be a sharp division between writing code and running it. Lisp's macros are useful because you can use all of Lisp’s runtime functionality as it’s generating and manipulating your code. With Lisp:

• You can easily convert from text to symbols and back to text. This makes it easy to generate the symbols you use in code.
• Lisp provides a templating syntax for generating lists. That means you can insert elements at arbitrary positions and also splice lists into other lists conveniently. This allows you to structure the list that represents your code in an easy-to-read way whereas function calls tend to make the structure you want less obvious.
• Code is just a list of symbols. You can loop over code, append it and rearrange it with Lisp’s list functions. This is useful when you want to manipulate and generate code in more complicated ways. The templating syntax usually doesn’t support this all too well.

In order to understand why this was useful for my game-playing AI, we first have to have a quick look at how I set up my programme in Lisp.

Implementation in Lisp

Mapping classes from Json

I first defined the player class from the json because it’s a nice and simple starting point:

(defclass player ()
   ((player-type :accessor player-type :initarg :player-type)
    (energy :accessor energy :initarg :energy)
    (health :accessor health :initarg :health)
    (hits-taken :accessor hits-taken :initarg :hits-taken)
    (score :accessor score :initarg :score)))

Then I converted the json to a hash table and wrote a function that pulled the values from the hash table to represent it as a class:

`(defun map-player-from-hash-table (hash-table)
    (make-instance 'player
                   :player-type (gethash "playerType" hash-table)
                   :energy (gethash "energy" hash-table)
                   :health (gethash "health" hash-table)
                   :hits-taken (gethash "hitsTaken" hash-table)
                   :score (gethash "score" hash-table)))

Since this template looks almost the same for all the classes, it can be generated with macros.

On a high-level, that ended up looking like this:

(defmacro define-hash-table-mapper (class-name slots)
  (let* ((hash-table-keys (make-keys slots))
         (init-args (mapcar #'make-keyword slots))
         (mapper-name (make-mapper-name class-name)))
    (alexandria:with-gensyms (hash)
      (let ((constructor-parameters
             (apply #'append
                    (loop for key in keys
                         for init-arg in init-args
                         for slot in slots
                         collect `(,init-arg (gethash ,key ,hash))))))
        `(defun ,mapper-name (,hash)
           (make-instance ',class-name ,@constructor-parameters))))))

Now, for someone who doesn’t know much about Lisp, this might be a bit dense. So let’s look at all these parts in a little more detail.

Generating hash keys and symbols

For convenience reasons, you want to make the macro as brief as possible. I used Lisp's easy conversion between text and symbols to turn symbols into the literal hash keys I wanted and generate the other symbols I needed. This works by passing in just a few symbols and generating extra symbols and strings based on a pattern that uses those symbols.

Here’s how I did that: I got the hash table keys by converting the slot symbols (properties in a class) to text with the symbol-name function and then camel casing the text to match how it appears in the json:

(defun camel-case (str)
  (let ((parts (cl-ppcre:split "-" str)))
    (format nil "~{~a~}" (cons (car parts) 
                               (mapcar #'string-capitalize (cdr parts))))))

(defun make-keys (slots)
  (mapcar (lambda (slot) 
            (camel-case (string-downcase (symbol-name (get-slot slot))))) 
          slots))

I generated two kinds of symbols:

• Keywords that start with a colon like :energy to specify the arguments to the class constructor and
• Ordinary symbols like the name of the function map-player-from-hash-table.

Ordinary symbols are created with (intern "YOUR-SYMBOL-NAME-HERE") and keywords are created with (intern "A-KEYWORD-SYMBOL" "KEYWORD").

Constructing a list of symbols

The next step was to create a template for what the function definition would look like. Lisp's list templating DSL let me easily construct a list of symbols with the code I wanted. The backtick operator tells Lisp that a templated list is about to begin. The comma operator tells Lisp to substitute the value of a variable at the next position in the list. The list splicing operator inserts a list into another list. Again, this is super valuable because it makes it a whole lot easier to structure your code in whatever way you choose.

(defun ,mapper-name (,hash)
   (make-instance ',class-name ,@constructor-params))

In other languages code isn’t a value. That means, it’s either impossible to generate code or it’s incredibly clumsy and done in a context that’s separate from ordinary programming in the language. A good example for this is the preprocessor in C.

Because code in Lisp is made up of lists, an ordinary value, I could flexibly manipulate it and rearrange it.

I collected symbols into lists and appended them together to get the body of the class constructor:

(apply #'append
  (loop for key in keys
        for init-arg in init-args
        for slot in slots
        collect `(,init-arg (gethash ,key ,hash))))

All of this is just the beginning. After tweaking the macro some more, I generated code that:

• Defines the class,
• Defines a function that maps json to a class and also
• Handles nested classes.

The final version can be found here and this is what it looks like to use it:

(define-poclo state ((game-details () game-details) 
                     (players () player)
                     (game-map () game-cell)) 
                     camel-case)

Lisp’s unique approach to code made this possible:

• I could easily convert from symbols to text and back again. This let me use text manipulation functions to generate symbols and hash keys based on the symbols I passed to the macro.

• The list templating syntax let me easily put symbols where they needed to be and splice pieces of code into other pieces of code.

• Because Lisp code is a list, it’s a value in the language and can be iterated over and manipulated like any other data structure. This let me map functions over the arguments passed to the macro and append the lists of symbols I needed to make the body of a function call.

Resources

If you’re interested in Lisp there are a lot of good books to take a look at. Many of them are freely available.

• On Lisp by Paul Graham (founder of Hacker News) is a classic book on Lisp focussing on Macros
• Let Over Lambda by Doug Hoyte is a very entertaining and interesting exploration of Lisp and Macros
• Paradigms of Artificial Intelligence Programming (PAIP) is a great book on solving interesting problems in Lisp by Peter Norvig (a director of research at Google)
• Practical Common Lisp by Peter Seibel is a good Lisp introduction and focuses on getting things done in Lisp

Henry is a Scala developer at Jemstep. He loves functional programming in Lisp and Haskell. When he's not coding, he loves to listen to classical music and also plays the piano.

If you enjoyed this article from Henry, check out his article on How he created his own AI Bot Game Using Lisp.