How long does it take to change your language?

Sun once estimated that it takes on average 18 months to evaluate and approve proposals for changes to the Java language. That estimate doesn’t include building, testing, and releasing the change. Nor does it include the time and effort needed to update compilers, IDEs, and so on.

This means that if you want a useful change added to the Java language, you may have to wait years – if the change is made at all.

What if things were different?

What if you could change your programming language to suit your needs? What if you could change it without asking permission, and without waiting on any kind of community process or support? With Clojure, you can. Clojure is a Lisp, therefore has macros, and macros allow you to do these things.

So how do macros work? First let’s clarify what Clojure macros are not. Clojure macros are not like “macros” you might find elsewhere, such as Excel macros or C macros. We’re not talking about a tool that records keystrokes to play back later. Nor are we talking about simple text substitution.

Clojure macros allow your program to interact with the compiler, control order of evaluation and program flow, modify code, and write new code at compile time.

If this sounds like too much power, it’s because it is. But Lisp traditionally errors on the side of giving us too much power. As Clojure programmers, we benefit from this reckless allocation of power.

If you wish to modify the core Clojure language to better serve your own selfish needs, there’s a good chance you can do it with macros. Clojure’s macro facility is well defined, powerful, flexible and fun. And you don’t need anyone’s permission or help to use it.

We’re going to demonstrate some of this power with three small examples. Providing an actual tutorial on writing Clojure macros is beyond the scope of this article. What we want to achieve here is a healthy understanding of what the Clojure macro facility brings to the table, and why we as software developers should take it seriously.

Example 1: “unless”

Suppose you wanted to add an unless idiom to Java, so that you could write:

unless(false) {
  System.out.println("This will get printed out.");
}

Well, too bad. Java doesn’t support unless, and there’s no way for you to add it at the language level. But in Clojure we could write a simple macro:

(defmacro unless [condition & body]
  `(when (not ~condition) ~@body))

Now in our normal codebase we can do things like:

user> (unless (> 8 1) (println "The println is never even evaluated"))
nil
user> (unless (> 1 8) (println "This gets run!"))
This gets run!

So with a two-line snippet of code, we just added a fundamental flow control construct to our language. And we didn’t need Larry Ellison’s permission or James Gosling’s help to do it.

I double-dog dare you to petition Oracle to add unless to the Java language.

Example 2: “time”

Consider the common desire to get a simple micro benchmark for a chunk of code. Suppose we’re starting with this Java method…

public void myStuff(int n) {
  return betterStuff(
    ExternalLib.doStuff(n));
}

… and we want to wrap a micro benchmark around the call to doStuff(). We’re forced to do something like this:

public void myStuff(int n) {
  MyTimer t = new MyTimer();
  int stuff = ExternalLib.doStuff(n);
  t.stop();
  System.out.println("Duration: " + t.duration());
 
  return betterStuff(stuff);
}

Wow, that got real ugly real fast.

If you’re familiar with AOP you might be tempted to define an aspect that allows you to annotate arbitrary methods and get micro benchmarks. But AOP is a whole ‘nother language, has serious limitations, and must be explicitly bolted on to your project. By contrast, Clojure macros are written in Clojure, are a natural part of the language, and afford far more raw power and flexibility than AOP.

Let’s look at supporting micro benchmarks in Clojure. First let’s see how our code looks before:

(defn my-stuff [n]
  (better-stuff
    (ExternalLib/doStuff n)))

Now with explicit micro benchmarking:

(defn my-stuff [n]
  (let [start (System/currentTimeMillis))
        stuff (ExternalLib/do-stuff n)]
    (println "Duration:" (- System/currentTimeMillis start))
    (better-stuff stuff)))

Just like the Java example, that got real ugly real fast. But there’s clearly a pattern that can be abstracted out. What we’d really like to be able to write is this:

(defn my-stuff [n]
  (better-stuff
    (time (ExternalLib/do-stuff n))))

Notice that we’re not changing our initial syntax, nor are we introducing any extra syntax – we’re simply wrapping time around our target code.

But could that possibly work? It seems tricky because time would need full control over the order of evaluation. That is, time would need to start a timer, evaluate (do-stuff n), stop the timer and output the micro benchmark, then finally return the result from the call to (ExternalLib/do-stuff n).

In Java, this kind of shenanigans is simply not allowed. But in Clojure we can support exactly what we want, by writing a small macro:

(defmacro time [expr]
  `(let [start# (System/currentTimeMillis)
         ret# ~expr]
     (println (str "Duration: " (- (System/currentTimeMillis) start#)))
     ret#))

In some ways the time macro isn’t a big deal. We saved ourselves from some syntax ugliness – so what?

Well, if you take a step back you can see it goes deeper than that. We modified the Clojure language itself, including having our way with control flow. Without that ability, we wouldn’t be able to create such a nice time idiom. Instead, we’d be forced to change the syntax of all code we wanted to benchmark. But with Clojure’s macro facility we were able to support very concise syntax.

Example 3: “with-open”

Java 7 is going to be released with a new feature called try-with-resources. This is going to relieve us of some of the boilerplate try/finally Java code we’ve had to write all these years for making sure to close down resources when we’re done with them. This is a great little improvement to the Java language!

So how long did it take to get this feature added with Java 7? According to this page, the call for proposals began on February 27, 2009. At the time of writing (early 2011), the official release date for Java 7 is mid 2011. That’s well over 2 years turnaround time!

Actually, we should count ourselves lucky. What if this change hadn’t been approved by the JCP? We’d be totally out of luck. On the other hand, we can implement this idiom with a Clojure macro with very little fanfare. We might call it with-open. Once written, it would allow us to do things like this:

;; my-res will be automatically closed once do-something-with is done
(with-open [my-res (open-a-resource)]
  (do-something-with my-res))

More examples… are waiting for you in Clojure core

Clojure itself already comes with a wide variety of macros that do useful things. Since Clojure is an open source project, a great way to get more exposure to real world macros is to browse the part of Clojure that are implemented in Clojure, and look for the telltale defmacro definitions.

In fact, the macros discussed here are actually implemented in Clojure already. You can go code diving for time (a little fancier than shown here), unless (known as when-not and implemented a little differently), and with-open. And as you can imagine, that’s just the beginning.

Remember Spider-Man

With so much raw power and flexibility, macros come with their own unique risks. The best Clojure programmers are aware of this and exercise caution when creating macros. As Spider-Man might say: with great power comes great responsibility.

Macros should be viewed as a kind of “nuclear option”. We should use them sparingly and with great deliberation. The Clojure community has been known to joke that “the first rule of macro club is: don’t write macros”. What they mean is, we should avoid writing a macro when a simpler solution will do (such as writing a plain old function instead).

We should also place a high value on composability. Due to the nature of macros, especially their sheer power, it’s not hard to get carried away and lose sight of composability. But our code should be easy and natural to use with other code in a modular fashion.

An additional concern is code readability. Macros, if not written and documented well, can make it especially difficult for others to understand your code.

These concerns shouldn’t scare you away from the judicious use of macros. However, they should give you pause. In other words, be careful and remember your core values as you bring your nukes online.

“For The Win”

Macros are one reason many consider Lisp to be the most powerful language ever designed. While I won’t try to make that argument here, I will suggest that Clojure macros are very special.

If someone is considering picking up a new JVM language – or any language, for that matter – and they ask you something like…

Many languages support first class functions, have useful idioms like map and reduce, great libraries, plenty of syntactic sugar, etc. Is there anything unique to Clojure that makes it worth considering above the rest?

… you can tell them, “Dude, macros FTW.”