A LISPie day. Data, Code, Context and Hints

I came to two conclusions today, about LISP. One the one hand I crystallized some thoughts on some of the flaws in LISP. On the other hand, I finally got the point one of rallying cries of the LISP community, namely “In Lisp, your code is data, and your data is code”.

Ever since I first heard that phrase, and had a basic knowledge of LISP, I understood the basic idea behind it, but until today, I didn’t have the right context. As a result, I think I focused more on the first half, which equates to the concept of functional programming, or at least first class functions. The second part seemed far less interesting, and somewhat of a tautology. But reading yet another post from Steve Yegge, showed me a context that made the statement far more interesting. The section that really did it for me was Beyond Logs.

What is curious is that earlier in the day I had a discussion that really solidified some vague thoughts I had about why I, and probably many others, have been so turned off by LISP, despite its capability. You often hear LISP lovers comment, that LISP has no (or very little) syntax. And it’s true. But that isn’t always a good thing. In fact, I think it’s a fundamental flaw for the majority of what programmers traditionally consider code. It’s nice to be able to think of code as data, and data as code, but it’s nice to be able to tell the difference too, or more specifically to be able to tell what is “more” code, or “more” data.

A truth, which goes rather unnoticed by too many language, tool, framework and library designers, is that we spend much more time reading code than writing it. Reading may seem like a very simple and straightforward activity, because it comes so naturally after years of practice, especially when compared to writing English prose. But what makes writing English so difficult is that you have to structure it so that someone else can understand it later.

There are many tricks to English, and it’s not just stupid syntax. The complex syntax rules of English may seem to make writing more difficult, but many of them, when followed contribute to readability. Without those rules, writing would actually be more difficult because not only would you need to devise a similar structure, but you’d have to somehow communicate that to readers. Having a set of rules, no matter how arcane is preferable to no rules.

And that is where LISPs lack of strong syntax becomes a problem. Sure good LISP coders will create a well thought out sublanguage, and avoid using cryptic shortcuts and other reading impediments. With high level elements this works well, because it is the sublanguage which eventually describes the solution. But at other levels, this can be a serious impediment because it is hard to separate out the layers. The syntax of most other languages gives the readers more “hints”. Some of these hints may seem unnecessary when writing code (brackets, parentheses), but they help you spot different sections of code. In the C derived families, when you see parentheses, you’ll think function call. When you see braces, you think loop/conditional. Brackets make you think “array”, etc.

Those same structures exist in LISP, but they all look nearly identical. That’s great for writing code, you don’t have to think about which character to use (has any LISP user remapped the keyboard so they don’t need to use shift for a parenthesis?). For reading, however, the homogeneity is very poor. It would almost be like removing tense from all verbs. Usually you can decipher tense from context, but it’s almost always difficult if done wrong. Programming languages are more deterministic than natural languages so it’s always possible to decipher LISP code from context, but it’s not always going to be easy.

The most interesting part of all this is how it ties into the data is code concept. If you’re putting code into your data, you really should have a way to make it stand out. The complexity of code, is higher than data and deserves special attention, when inside a mixed environment. It’s best to separate data from core code, so that it can change independently. I see the value of one unified model for both code and data, and the ability to work in the blurry areas in between. But as a reader, I’d prefer there to be some syntactical differentiation regardless of what’s under the covers.

It’s possible a super-intelligent IDE might be able to fix these problems, but I haven’t put enough thought into it to know if that’s truly feasible with the existing LISP syntax, and from my current review of the state of the LISP IDEs, there is a very long way to go before that type of capability starts to show up.