Thursday, 19 September 2013

On editing, and Clojure

Back in the days when the world was young and we had proper software development tools, you'd write your top level function and evaluate it. And, of course, it would break, because you hadn't yet written your lower level functions. So you'd get a break inspector window pop up on the screen, and in that you could do one of several things.

Firstly and most obviously, you could supply a value as the return value of the function you hadn't yet written, and continue the computation. Secondly, you could inspect the stack, unwind a few levels of computation, and choose a different point to continue from, supplying a new return value there. But thirdly...

Thirdly, you could write your new lower level function right there on the stack, and continue the computation through your new function. And you could write it in a structure editor.

What is a structure editor?

OK, Lisp is a homoicinic language. Which is to say, a Lisp program is just a Lisp data structure. And the canonical form of the program is not a stream of byres in a file, it's the structure in memory. Now, OK, one can reconstruct the structure in memory by writing it out to a file as a stream of bytes and then reading that stream of bytes back in; and consequently, one could (as one must for more primitive languages) edit that stream of bytes in a text editor in order to modify the program, and then read in the modified stream of bytes to create a modified structure in memory, and consequently a modified program.

That is, of course, how the LMI and the Symbolics boys worked, how Richard Stallman worked. But that's a lot of hassle. It's much simpler to pop the data structure in memory up in a window (automatically formatted, since it's just data and not a stream of bytes so the indenting can't be wrong) and edit it right there. And the joy of that, of course, is just as you can't get white space wrong, you also can't get parenthesis nesting wrong, since all you can do is insert a new well formed symbolic expression, delete an existing well formed symbolic expression, or replace an existing well formed symbolic expression with a new well formed symbolic expression.

People don't understand Lisp syntax, and that's because they think of it as text. It isn't text, it's structure; and the structure is beautifully simple and perfectly regular. It consists of well formed symbolic expressions nested in other well formed symbolic expressions, and nothing else. There are no keywords. There are no operators. There are no control blocks. Nothing is magic, because everything is magic. The Lisp programmer is perfectly at liberty to define T as NIL, or NIL as something else. The world will probably stop working if he does, and not in a good way - but he can, because not even T and NIL are magic.

So after all that I'm going to talk about how wonderful it is to be able to do structure editing in Clojure, aren't I? No, sadly, I'm not, because (as yet) you can't. All you can do is the old-fashioned primitive thing of editing a text file, a stream of bytes, and then reloading it. The world of modern software tools is still years behind where we were thirty years ago.

But some of the modern editors for Clojure are nevertheless pointing the way forward.

Slime

There is an editor which these days is Generally Not Used Except by Middle Aged Computer Scientists; back in the day when eight megabytes was a lot of memory, we used to call it Eight Megabytes And Constant Swapping. Its original author called it 'editor macros', but since, in those days, on most computer file systems, file names were constrained to be short, it was more generally abbreviated to Emacs.

Emacs, of course, is written in Lisp and for Lisp. And consequently, there has long been a Superior Lisp Interaction Mode for Emacs, better known as Slime. Slime establishes a communication between the Emacs editing buffer and a separate Lisp process, and sends chosen symbolic expressions from the buffer to the Lisp process for evaluation. And the beauty of that, of course, is that it works not only with Emacs Lisp but, in principal, with any Lisp, from Portable Standard Lisp through Symbolics Lisp and Franz Lisp to the execrable Common Lisp (now) to Clojure.

The problem with this is that Emacs, although enormously powerful and in its way elegant, is now long in the tooth. It always was user hostile to an extreme degree, and has a very steep learning curve. Thirty years ago when it was designed, modern user interface design hadn't evolved. Now, not only are all the Emacs key commands incompatible with modern conventions, you can't even copy and paste simply between an Emacs buffer and any other window. Don't get me wrong, Emacs was my text editor of choice for fifteen years from 1988 until the early 2000s, and it still is my editor of choice for XML and SGML, since it parses DTDs properly and does helpful auto-completion on that basis (to be fair Oxygen XML does this equally well, but it's expensive), but otherwise in truth it's a pig to work with these days, and Slime, while elegant, doesn't marry up to modern tools.

Clojure Mode

Separately from Slime there's now a 'clojure mode' for Emacs, which (allegedly) works similarly to but better than Slime. The truth is I've no idea whether this is true, because it's built for the Emacs 24 package system, and while I spent an hour this morning trying to back-port it to the Emacs 23 which is supported by Debian, I eventually gave up. Life is too short. It would have to be utterly amazing to be better than the more modern Clojure editing tools.

Counterclockwise

Counterclockwise is a Clojure mode for my current editor of choice, Eclipse. Not only does it understand Leiningen projects, do syntax highlighting and intelligent auto-complete, structure analysis, linking, all the things one expects of a modern IDE, it also has an interactive Clojure process in which, just as with Slime, you can evaluate either the currently highlighted symbolic expression in a file, or the whole file. This is simple, powerful, easy to use, and, of course, integrates with all the other good Eclipse tools like Mylyn, and version control systems.

Counterclockwise is a joy to use. In fact, one might almost describe it as the perfect development environment for Clojure, were it not for one other competitor...

Light Table

Light Table is something new; something which, although it is still a text editor, has something of the power of the structure editors I still yearn for. It's live. Which means, you don't just send things from the buffer you're typing into to a separate Clojure process to evaluate them; they are continually evaluated as you type. It's remarkable, revolutionary, and, I think, potentially points to a future for programming editors.

Unfortunately, in its present state, it's alpha code, and it's fragile. While it works well for simple exploratory code, trying to load add a connection to a Leiningen project - something which Light Table allegedly supports, and must do if it is to be used for serious work, then, providing the project depends only on Clojure, everything works well. But if, as most projects do, the project depends on other projects, it frequently results in
Exception in thread "main" java.lang.RuntimeException: java.lang.NoSuchMethodError: clojure.lang.RT.mapUniqueKeys([Ljava/lang/Object;)Lclojure/lang/IPersistentMap;
 at clojure.lang.Util.runtimeException(Util.java:165)
 at clojure.lang.Compiler.eval(Compiler.java:6476)

That's really not helpful. Googling for it suggests that the problem is probably that Light Table is (on my machine) using Clojure 1.3, when it needs at least Clojure 1.5.1; however, while I have both versions on my machine, the path is set up to prefer Clojure 1.5.1, and the Lieningen project.clj file I'm using also specifies Clojure 1.51, so if it's using the wrong version I don't understand why it is.

In short, I'm really excited by Light Table, but I'm afraid it isn't yet ready for prime time.

Conclusion

Of course, there is not (yet) a structure editor for Clojure. It wouldn't at all be impossible to build one, and in fact that is the sort of tool-building project which is likely to pay off well in productivity in the long run; but it's also potentially tricky and I think I need to become more generally fluent in the language first. For the meantime, Counterclockwise is a tool which will serve me well.

No comments:

Creative Commons Licence
The fool on the hill by Simon Brooke is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License