Ciaramella, a declarative audio programming language

Posted by Stefano at 27/05/2022, 14:04:12 UTC.

Domain-specific languages (DSLs) are ubiquitous in computer science and engineering today. You can find one or more successful examples in most engineering subfields, e.g., MATLAB for numerical computing, R for statistics, Mathematica and Maxima for symbolic computation, VHDL and Verilog for hardware description, SQL for querying relational databases, HTML and CSS for web hypertext and layout… and the list goes on and on.

When it comes to audio programming languages there’s no shortage of proposals either, yet in my experience none is taken seriously enough in the music tech industry to be used in actual products or even for prototyping (perhaps I might be partly wrong, but this is what I’ve witnessed so far). The academia also seems to largely ignore them, with the exception of the people directly involved in their development and a few enthusiasts. In pratice, it seems to me that prototypes are actually developed in Matlab and Python, while production code is mostly written in C/C++.

While I don’t necessarily believe that experts tend to choose their tools wisely, I think I can easily see why these tools are failing at being useful enough to get adopted for their intended purpose, and I also think we can do better. That is why we funded a doctoral position last year to research and develop Ciaramella, a new audio programming language that can actually be used.

The roots: BlockCompiler and Permafrost

It all started in 2009, when I went to Helsinki, Finland, for and Erasmus exchange at the Aalto University (back then it was still called TKK) under the supervision of Prof. Matti Karjalainen. He had developed an experimental audio programming language and compiler in Common Lisp called BlockCompiler which was probably the only such tool which could automatically schedule Wave Digital Filters (WDFs).

Unluckily the software was as quirky, clunky, unmaintainable, unmanageable, and user-hostile as you’d expect from a university project developed by a single person with a whole lot of other priorities in life — I always had the felling this thing was sort of a funny hobby to him. Therefore, since I had to write my thesis while being there, I proposed him that I could try using more modern tools.

After some research I opted for FAUST, as it was the most mature and capable language/environment out there, but after some discussion with the developers it was obvious to me that the language could not naturally represent highly-coupled systems such as WDFs.

At this point I had no choice but developing my own thing, and so a new audio programming language called Permafrost was born. Given the extremely limited timeframe, I could only create a proof-of-concept prototype. It had countless limitations and its syntax could be largely improved, yet it brilliantly solved that one problem by employing a declarative approach, somewhat like VHDL and Verilog, and by graph flattening and scheduling all operations as late as possible (don’t worry, we’ll get to that again in a second). After I defended my thesis no more serious development took place, even though I wished to put my hands on it again for over a decade.

New round, welcome Ciaramella

Ten years later (2019/2020) I have been developing an incredible amount of custom DSP algorithms, most of which were simulations of analog hardware. In order to make them perform efficiently, I’ve been using a modest amount of boilerplate code which I wished I could automatically generate somehow. Also, my work methodology became more homogeneous and standardised with time. As the pain was getting bigger with the increasing amount of requests, I decided it was time to scratch my own itch and have some young graduate student help me revive Permafrost with a focus on improving productivity.

This is how Paolo Marrone, the computer scientist in Orastron today, got involved. At the beginning he developed a first prototype of Ciaramella for his master thesis, which he defended in early 2021. Then we funded an internship and eventually his PhD position to push this project forward.

The Ciaramella language is, like Permafrost, fully declarative. This means that you don’t specify the order in which operations are executed one by one, as for example in C/C++ or JavaScript, but rather you describe how each output is computed starting from the inputs and then it’s up to the compiler to decide how to actually schedule the operations. Other than that, the syntax is C-like and the language is semantically based on the well-known patching and block interconnection paradigms that are already familiar to users of visual audio programming languages such as Pure Data and Max/MSP. It also strives to minimize the number of keywords and abstractions, and hence cognitive pressure on the user. Indeed, we believe this is one of the main problems with the adoption of current languages.

I mean... if you’ve ever done any DSP, you can probably guess what the following code does with no need for any further explaination on details of the syntax. And it shows already almost of all the syntax available at the moment.

# a comment

pi = 3.141592653589793 # a constant

# a block
y = onepole(x, mA1) {
	y = x + mA1 * (delay1(y) - x)
	@y = 0 # initial state
}

# a block using the previous block
y = onepole_double(x, mA1) {
	y = onepole(onepole(x, mA1), mA1)
}

Zampogna, a Ciaramella compiler written in JavaScript

Another fundamental aspect in getting adoption for a programming language is the tooling available. Now, we’re in extremely early stages and everything is very rough still, yet we decided to use JavaScript and support both Node.js and web browsers for a number of reasons:

  • we’re a tiny team and we don’t have time to waste on implementing stuff in lower-level languages such as C/C++ or address machine-code generation issues;
  • we’re already deeply familiar with JavaScript and basically everyone else also is to some degree;
  • we got web support (almost) for free and it required little effort to express an API to control the compiler (hence we have: native and web API, as well as CLI) — yes we have a web IDE/playground online;
  • writing a source-to-source compiler for a properly designed language is relatively straightforward in JavaScript using Jison for the frontend and dot.js for the code generation stage (this last one is a templating engine, which makes it really easy to add new targets);
  • the dynamic nature of JavaScript allows for easy experimentation and integration of other tools.

Zampogna — that is how we called the compiler — is internally structured in a series of individual and clearly separated stages, following current best practices in compiler design, and interally uses graphs to represent data flow, which should grant us great flexibility when implementing optimizations and complicated compilation algorithms in the future.

Combining scientific research and industrial applications

Now, this post doesn’t want to be an introduction to the technology. I rather refer you to the official website.

Since it is the topic of the doctoral studies of Paolo, he’ll be conducting proper scientific research and publishing papers regarding it. The first such paper, which describes the language and the compiler in detail, has been accepted and will be presented at the upcoming Sound and Music Computing (SMC) conference in Saint-Étienne, France, in a couple of weeks (Friday, June 10th, save the date).

If there is interest, we also wish to involve other researchers and to promote its usage in the academic community. Actually, I really wish it can help getting more researchers share their algorithms online for everybody to check and hack.

It is funny that Ciaramella has actually been already used in developing a good amount of code that is actually in production right now in some of the most acclaimed software and hardware products of today. Of course it needs a lot of love and work, as well as a caring user base, but if you ask me this is already a very decent start.

Comments