Oliver Caldwell's blog
On languages, text editors and tools
Conversational software development
June 7, 2020
If you’re a seasoned Lisp (or similar) programmer a lot of this will just be rewording of what you already know. If all you’ve ever known is editing source files and restarting your program this should hopefully be new and inspiring.
Conversational software development is a term I’m half heartedly trying to coin to describe a way to develop your program with an incredibly short feedback loop. It aligns closely with hot module reloading for React based frontend applications but with a much tighter focus and an extra dimension that you may not have seen before.
Most languages require you to make changes to your code (possibly recompile) and then restart your program from the beginning with new instructions. This loop can be so long that you lose track of which change introduced a certain bad behaviour, linting tools will catch some things but others are only apparent after they’ve been executed.
We have no way to question our program, if we want to see a specific value at a point in time we have to insert multiple log statements and restart it each time until we get the information we need.
This becomes even more difficult if you’re working on something side effecting, you end up commenting out a chunk of code so it doesn’t execute each time which saves you from resetting a database or state file after every run.
Once you’ve got the information you need and you’ve applied your changes you have to undo any changes to the code you made to understand the problem in the first place. Raise your hand if you’ve ever accidentally committed and pushed log lines or commented out code you were using during development.
Yeah, me too.
Moving in the right direction
The next step up from this, which only applies to certain contexts such as frontend development, is hot module reloading. This means you don’t need to restart your entire application every time you make a change, you write your changes to a file and that file gets loaded in your evaluation environment (usually a browser).
However, hot module reloading needs to be implemented for every language (if it’s even possible), it also needs to be tweaked or re-written depending on the context. Hot reloading a React application requires hooking into React to re-render when the file is changed, in a server side node context this will be different. The reloading system tends to be framework specific.
With HMR we can get changes into our program, but it’s locked to the granularity of an entire file and it only goes one way. We can’t ask the file what a specific value is currently set to. We can’t evaluate a single function or method in isolation without evaluating everything surrounding it.
If we need to ask a question we still need to make changes to the source code to add logging and then either restart or poke the program in some other way to get that code to execute.
Indistinguishable from magic
The tightest feedback loop we can have with our program is the ability to execute an arbitrary string of code and immediately see the result. This is what people mean when they talk about REPL based development, it doesn’t have to happen inside a terminal prompt, but it’s a good start. The problem with REPL based development is that the name has been complected with terminal based prompt tools (such as when you run
python with no arguments in your terminal).
Conversational software development sets the concept free while still describing the core idea: We want to send a question or a statement to our program then listen and respond accordingly. It’s a two way street that can go down to a single character of granularity, such as "what’s currently in the variable
The questions and statements can even be generated by other programs, such as something that looks up documentation or completion results for a given prefix. We can build tooling into our editor that queries our running application through simple snippets of code, let your imagination run wild with the potential of that for a second.
Conjure is my own Neovim plugin for conversational software development with a variety of languages. At the time of writing it supports Clojure, Fennel (through Aniseed) and Janet out of the box. I plan on adding support for Racket among other Lisps as well as non-Lisp languages once some more features land in Neovim stable (tree-sitter).
I won’t be teaching you how to use Conjure specifically here, but you can learn about it through the interactive
:ConjureSchool once installed or by executing the demo script.
curl -fL conjure.fun/school | sh
Fixing an issue through a conversation
Let’s use Conjure and conversational software development to fix an issue with this simple Clojure Fizz Buzz program. I’ve called the repository fizzbugged since it’s slightly broken by design.
(ns fizzbugged.main (:require [clojure.string :as str])) (defn n->str [n] (let [by-5? (zero? (mod n 3)) by-3? (zero? (mod n 5))] (cond (and by-3? by-3?) "Fizz Buzz" by-3? "Fizz" by-5? "Buzz" :else (str n)))) (defn -main  (->> (range 1 101) (map n->str) (str/join ", ") (println)))
Because this is a contrived example to demonstrate a development style (not hard problems) you can probably already spot the issues with this program. It’s almost correct, but not quite, let’s get a REPL up and running so we can connect to it with Conjure and evaluate some code.
$ clojure -m nrepl.cmdline --middleware '["cider.nrepl/cider-middleware"]' nREPL server started on port 40285 on host localhost - nrepl://localhost:40285
Open Neovim with Conjure installed and it’ll connect automatically.
Now I’ll evaluate the contents of
-main by placing my cursor inside the
->> form and pressing
,ee (the comma prefix will depend on your
maplocalleader setting and Conjure configuration). The result will appear in the HUD (small window in the top right) but I’ll open the log up in a split so we can see more of it with
Here’s the expected and actual output of our program so far.
# Expected 1, 2, Fizz, 4, Buzz, Fizz, 7, 8, Fizz, Buzz, 11, Fizz, 13, 14, Fizz Buzz # Actual 1, 2, Buzz, 4, Fizz Buzz, Buzz, 7, 8, Buzz, Fizz Buzz, 11, Buzz, 13, 14, Fizz Buzz
So the "Fizz Buzz" combination (which should only appear for multiples of 15) is happening at odd places and it looks like "Fizz" and "Buzz" are inverted the rest of the time.
Let’s add a comment to the buffer with some test forms to check some of the specific return values of
I evaluated each form inside the comment with
,ee, we can see the results in the log buffer (which you can edit and interact with, by the way). Clearly my suspicions were correct, "Fizz" and "Buzz" are flipped and "Fizz Buzz" is applying to way more than multiples of 15.
I’m going to flip the
by-3? values around in the
let binding, evaluate the function with
,er (evaluates the top level or "root" form under the cursor) and then run my little ad-hoc test suite again.
And now I notice that the weird behaviour of "Fizz Buzz" occouring on things other than multiples of 15 is because I have
(and by-3? by-3?) when one of them should be
I’ll correct that, re-evaluate
n->str again and then execute the body of
-main one last time.
That looks good to me! Bear in mind that this is an extremely simple example so you might be thinking "I’d just run my program again after I made a change", which is perfectly acceptable in this instance.
Having a conversation with your program really shines when you’re talking directly with your staging server or poking at the state of intermediate steps in a complex pipeline that takes 10 minutes to run end to end.
While connected you can run each step individually, caching the results in vars, which allows you to run individual parts of the program on demand without having to change the code. When you go up against harder and more complex problems you’ll be thankful for having this tool close at hand.
Conjure is my labour of love, I’ve been writing, rewriting and refining it for years. I think I’m finally at a local maximum with the way it’s built and the problems I’m trying to solve.
I hope it proves extremely useful for you in your personal and work projects. If you aren’t interested in Conjure or the languages it supports I hope you can at least take the ideas of conversational software development with you.
Spread these ideas and build more tools that embody them, let’s shorten the feedback loop in editing source code to be effectively non-existent. I’d love to hear your thoughts on this topic, please do get in touch using the details in the footer.