Something Same

Language, Expression and Design

Thursday

16

January 2014

Clojure, Dynamic Languages, Creativity and Simplicity

by Chris Zheng,

A while back, I started a little rant about 'proving' why dynamic languages are better on the mailing list here.


Me:

I'm a little bit miffed over this current craze of types and correctness of programs. It smells to me of the whole object craze of the last two decades. I agree that types (like objects) have their uses, especially in very well defined problems, but they have got me in trouble over and over again when I am working in an area where the goal is unclear and requirements are constantly changing.

My experience of programming in clojure has freed me from thinking about types and hierarchies and this article by Steve Yegge rings so true. However, everywhere I look, there are smug type-weenies telling me that my dynamically typed program is bad because it cannot be proven correct and not checked by the compiler. This question on SO really makes me angry. because no one is defending dynamic languages on there.


The discussion went on for a while, with many contributing their views. It was clear that even in the clojure community, there are polarizing views on the issue. For myself, there are some really good takeaways from this debate.:


Mikera on Being Pragmatic

I suspect you are going to have zero success 'proving' the superiority of dynamic languages (in an academic proof sense). For a start, people are even going to disagree about the precise definition of 'better'. What matters more: Runtime performance? Flexibility with respect to changing requirements? Ease of learning? Minimisation of typing? Ease of language implementation? Any such 'proof' based on your own definition of 'better' isn't likely to convince others who hold different views.

My suggestion: just don't worry about it, be pragmatic and use whatever you find helps you to build useful software. Ultimately, that is the main measure of success for a practical general purpose programming language.

In particular, I suggest ignoring anyone who launches a "barrage of abuse". Language trolls probably aren't going to give you a sensible conversation in any case.


Greg on issues with Javascript

js> Array(16).join("wat" - 1) + " Batman!"
"NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN Batman!"

Korny on Static Typing as Premature Optimization

This ties in nicely to my summary of how I feel about static typing: Static typing is a premature optimisation. Like most optimisations, it has genuine value, but if you apply it globally and too early, you end up causing more pain than you gain.

Sometimes type discussions lead to lead to early and convenient detection of bugs" - I'd agree with this; but in my experience, most of the bugs caught by type systems are relatively simple and straightforward, and not the sort of bugs that make it into production. I've almost never seen a serious bug caused by dynamic typing - and I've seen plenty of serious bugs that type systems would not have caught.

Static types also help with rapid IDE/compiler feedback of errors - but you often pay for this with slow compilation, especially when you need global type inferencing; and with complex (and often buggy) IDEs/compilers. I had huge pain getting the Scala IDEs to work reliably, last time I worked in Scala (admittedly this was a few years ago) - and they are still having lots of pain with performance, even though Scala doesn't have global type inference.

Statically typed code generally performs better - but if there's one major rule I've learned in 25 years in IT, it's that code performance is not your real problem, 99% of the time. Your real problem is more likely to be IO, or poor algorithm design, or inefficient scalability, or slow speed of development, or difficulty diagnosing bugs, or unreadable unmaintainable code. I had people telling me that C++ was too slow, I should stick to C. Then, that Java was too slow, I should stick to C++. Then, that Ruby/JavaScript was too slow, I should stick to Java. None of these people were right. These days, I'd generally optimise first for expressive code so it's fast to develop and maintain; then for powerful flexible languages that can do anything I need them to do, and last for raw performance.

I'm quite attracted by optional static typing, because it means I can rapidly code in a flexible dynamic language, and if I get to the point where I really need types, I can add them. But I suspect that most of the time, I'll never get to that point - or I'll use something like Prismatic Schema to define constraints at my external interfaces only, which is where I generally find I need them.


Rich Morin on Static Typing inhibiting Creativity

My take is that required types may force premature optimization and may inhibit the creative process. So, I like dynamic languages. However, optional types (preferably with type inference) give me the choice to add typing if, when, and how I think it will be worthwhile. So, I may still premature, but at least it's my mistake to make.


My Reflections

Type systems are pretty amazing and even magical. They provide a great first pass over the code and often will assist the programmer in learning to code. However, for those (like me) that have grown up programming with type systems they can be mental straightjackets that block amazing possibilities that can be done with code.

The more clojure I write, the more I believe that think that lisp (and clojure) was meant to be dynamic. Static types allow certain properties to be exploited. Dynamic types allow other properties to be exploited. It really should be up to the individual to CHOOSE what properties they wish to exploit. It is really a matter of taste.

In terms of dynamic vs static typing, I am very much in the dynamic camp. I will almost always choose freedom from types. Even if it means I could potentially shoot myself in the foot, I would still prefer having that choice rather than not having it at all.


Dynamic Typing Win (Case 1)

I want to give everyone a personal anecdote of how creative we can get with dynamic typing and clojure's code is data paradigm.

Ihe lein macro was written for my workflow library vinyasa. Using the repl version of lein allows commands normally run in shell to be run in the macro. It is extremely useful for me because I have a slow computer and having it within clojure allows me to bypass the jvm startup time.

Some examples of its use:

> (lein)             ;; Shows help screen

> (lein install)     ;; Install to local maven repo

> (lein uberjar)     ;; Create a jar-file

> (lein push)        ;; Deploy on clojars (I am using lein-clojars plugin) 

> (lein javac)       ;; Compile java classes (use vinyasa.reimport instead)

There are slight difference between the lein command in the repl vs the lein command in the shell:

  • the shell version terminates the jvm after every call.
  • the repl version crashes when (lein repl) is evoked.

In the first naive version of lein, I cut and copied the -main function (shown here) in leiningen.core.main, taking out all the calls to exit:

(ns vinyasa.lein
  (:require [leiningen.core.main :as lein]
            [leiningen.core.user :as user]
            [leiningen.core.project :as project]
            [clojure.java.io :as io])
  (:refer-clojure :exclude [lein]))

(defn lein-fn
  "Command-line entry point."
  [& raw-args]
  (try
    (user/init)
    (let [project (project/init-project
                   (if (.exists (io/file lein/*cwd* "project.clj"))
                     (project/read (str (io/file lein/*cwd* "project.clj")))
                     (-> (project/make {:eval-in :leiningen :prep-tasks []
                                        :source-paths ^:replace []
                                        :resource-paths ^:replace []
                                        :test-paths ^:replace []})
                         project/project-with-profiles
                         (project/init-profiles [:default]))))]
      (when (:min-lein-version project) (#'lein/verify-min-version project))
      (#'lein/configure-http)
      (#'lein/resolve-and-apply project raw-args))))

(defmacro lein [& args]
  `(lein-fn ~@(map str args)))

The results of such a function seem to work really well. All the tests passed locally and I posted the library up along with a blog post. However, before long, an issue caused by running a previous version of leiningen was raise:

I've tried to use the vinyasa and forgot that I had lein 2.2.0, so this caused following error:

Exception in thread "Thread-5" java.lang.RuntimeException: No such var: lein/*cwd*, compiling:(vinyasa/lein.clj:20:33)  
        at clojure.lang.Compiler.analyze(Compiler.java:6380)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3624)

I realised then that this was bad news because not only did version 2.2.0 break, but version 2.3.2 also broke (I was using 2.3.4 at the time). The fast and furious activity surrounding leiningen meant for sure that future versions of would break again and this was a really big problem for me. I started thinking that maybe this wasn't such a good idea anymore. After all, I really did not want to be support multiple versions of leiningen as well as all future upgrades for leiningen.

However, I had a flash of inspiration. How about using the source-fn function to fetch the source code of the current version of leiningen? This would also be the current version that is run. If I just took out all the calls to exit and recompiled it, then the repl version of lein would never require any upgrades anymore because it would always be using the version of lein that the user ran.

This was easy:

(read-string (source-fn 'leiningen.core.main/-main))

I first defined an alternate form, taking out all exit calls:

(def lein-main-form
  (postwalk
   (fn [f]
     (cond (and (list? f) (= 'exit (first f))) nil
           (list? f) (filter (comp not nil?) f)
           :else f))
   (read-string (source-fn 'leiningen.core.main/-main))))

Then after defining the form, I redefined -main in the leiningen.core.main namespace:

(in-ns 'leiningen.core.main)
(eval vinyasa.lein/lein-main-form)
(in-ns 'vinyasa.lein)

The macro was a very simple call to -main:

(defmacro lein [& args]
  `(leiningen.core.main/-main ~@(map str args)))

A whopping 29 lines of code now supports any version of leiningen as well as any future version (Unless they stop using the exit function to terminate the program). This code is stable and I don't think I need to change anything for a while.

I would not have been be able to do this so easily and so simplistically with static typing. Rich Morin is spot on. I am definitely alot more creative with Clojure.

comments powered by Disqus