Conjing

After being in Baltimore for the Clojure Conj last week, it's time to get back in the saddle and write about Toccata. It was great hanging out with Clojure community, seeing old friends and just generally geeking out. Thanks to everyone who came to the unsession on Toccata despite the late hour.

I got lots of positive feedback from folks when I mentioned what I was working on. The most common statement was "That would be great for embedded software.". Which is really gratifying, since that's what my career was before I started doing Clojure work. Toccata isn't specifically targeted to the embedded space and since it uses ref counting garbage collection, there are some places it's not suitable. But it seems there's quite a few Clojure programmers who want to do embedded stuff.

Tools

And now; tooling.

In programming, it's easy to assume that if you can just make the language "good enough", all problems go away and it's sunshine, rainbows, and ponies from then on. I don't believe that, so one of my goals with Toccata is to make it easy to write great tooling. I would like to see tools for writing, formatting, debugging, mapping, refactoring, and working with software written in Toccata. And yes, that includes static analysis, ala. types. One primary goal is to enable the creation of a tool to assist in reading Toccata code that you can point at a source file and it'll help you understand what's going on and why.

Given those goals, it's quite appropriate that the first merge to the toccata repo from someone else was by Michael Nygard (who has a new edition of his classic "Release It" coming out) that added support for CMake and also a Docker container with a self contained Toccata dev environment.

This is huge. I've always used lldb to debug my code and just recently was able to enable source level debugging in it. It's kind of a hack, but you can set break points in your Toccata source and the debugger will stop at those points. Then you can single step from that point and also examine variables. The Toccata symbol names are a little mangled and have a numeric identifier appended to the end, but you can probably figure it out.

Fire!

As I work on the Toccata compiler, I'm pretty focused on performance. I want the compiler to be fast so the edit/compile/run debug loop is takes as little time as possible. So tools for measuring the performance and identifying hotspots is prety important. One tool is the flamegraph. Unfortunately, the way the compiler works limits the utility of flame graphs. But thanks to the docker container, I got it working for Toccata so anyone should be able to generate them from their code.

One thing these graphs do show in stark detail is the effect compiler optimizations have on the Toccata compiler. Here's a flamegraph of Toccata compiling regression-tests/main1.toc from the repo. Obvious to see the massive call stack that builds up due to the structure of the recursive descent parser and the compiler.

However, this flamegraph is for a run where the compiler was compiled using the -O1 flag in Clang. That optimization level includes sibling call elimination and the difference in the flamegraph is dramatic. So much so, that I don't really trust it. At some point, I may dig into that further. OTOH, the difference in performance is dramatic.

FYI, if you intend to generate flamegraphs using the Docker container, you'll have to run it with the --privileged option so that perf will work properly.

Graphical Debugging

I would think it would be doable to enable gdbgui or something similar and get graphical debugging of Toccata code. It'll probably be awhile before I can do it, so let me know if get that working before I do.

Time Travel

But that's not the biggest thing.

Ever since I saw the rr project, I've wanted it for Toccata. But it doesn't work on OSX, which is what I mostly develop on. Now that there's a container, I thought it would be possible to get it working much like the flamegraphs. Sadly, this isn't the case and I have to drop that effort to make progress in other areas. I would love it if someone would take that up. Then we could have record/replay debugging at the Toccata source level. How cool would that be?