More Awesome Music DSP*

This is a curated list of my favourite music DSP and audio programming resources. * It was originally meant to be an official "Awesome list", but apparently you are not meant to write in the first person, so it is now a "more awesome" list.

Oli Larkin

Audio Plug-in/App Frameworks

• iPlug2 - Originally created by Cockos, iPlug is an awesome plug-in framework. For a long time, I was maintaining a fork of iPlug1, but over the last few years Alex Harker and I have reworked it into something new, with the imaginitive name: iPlug2. It's a massive improvement on the original version, adding support for GPU accelerated vector graphics, resizable high DPI UIs, iOS, WebViews, AUv3 and compiling to WebAudio/WASM, amongst many other things. iPlug2's syntax is super simple, for example, creating a parameter or a control in the UI is only a single line of C++ code.
• JUCE - JUCE is an undeniably awesome C++ application/plug-in framework with audio roots. It boasts a vast amount of functionality for the development of music software, including support for almost all plug-in formats and platforms. JUCE is used widely in the music technology industry and it has excellent documentation, code standards, features and support. The JUCE team organise the Audio Developer's Conference (ADC) which is one of the best conferences to attend if you like audio programming. What's more all the videos from all the ADCs so far are available on youtube.
• AudioKit - AudioKit uses the Swift programming language. It's a high level DSP and UI toolkit for audio things. If you want to make apps for Apple's devices this is a great option, although if you want to do more low level, custom DSP, you will still have to do some C/C++.
• ASPiK - This is a framework from Will Pirkle, used in his books, which is based on the VST3 SDK and it's AU/AAX wrappers, along with VST GUI and its runtime UI editor. It includes a project creator and a lot of extra "nuts and bolts" code for plug-in development.
• DPF - Distrho Plug-in Framework is a nice C++ plug-in framework by falkTX, supporting lots of Linux formats.
• Jamba - A new framework built on top of the VST3SDK & VSTGUI, with some useful extra widgets and good CMake support
• DPlug - A plug-in framework using the D programming language, that has had a huge amount of work over many years. I haven't tried it, but I've decided to include it here in solidarity with the authors. As a fellow open-source framework developer, it's nice if people know about your work.
• Steinberg VST3 SDK - The Steinberg VST3 SDK has been around for a long time, but recent versions start to add a lot of great features such as really good CMake support, Lots of documentation, and a nice template project. Since Steinberg include wrappers for VST2, AUv2, AUv3 and AAX it is somewhat like a plug-in framework.
• CPLUG - A smart new plug-in abstraction in C that targets VST3, AUv2, CLAP and Standalone formats on desktop. It's very new but a nice clean and minimal way to target multiple formats producing tiny binaries. Also includes hot reloading capabilities for the standalone targets.

Audio plug-in templates

If you want to create a robust cross-platform audio plug-in, there is a lot more involved than just programming your DSP and creating a user interface. There are many extra things to set up such as installers, testing, building in the cloud using github actions etc. Although these things are less fun than audio programming they will make your life much easier in the long term, but it becomes very tedious to set them up manually again and again. If you want to just be creative with a new project, it's smart to have a good template to start with, which is pre-configured to build and test your code in the cloud in a reproducible way. Here are some templates that people have made that you might want to use or look at if you are creating your own:

• iPlug2OOS - Continuing the theme of self-promotion, here is the latest and greatest iPlug2 template project, set up with CI/CD, installer scripts, github workspaces etc.
• pamplejuce - An excellent template for JUCE projects.
• wolfsound - audio-plugin-template - Jan Wilczek has made another excellent JUCE template
• JUCECmakeRepoPrototype - Eyal Amir also has one which looks good!
• Brad Howes AUv3 Template - A template for iOS/macOS AUv3 projects

UI Frameworks

• VSTGUI - VSTGUI is Steinberg's cross-platform UI framework for audio plug-ins. It is released under a BSD licence, it's an impressive piece of work and many plug-in developers use it for their products.
• Elements - A very promising UI framework for audio software using modern C++.
• PUGL - A cross platform GLFW-like solution that lets plug-in developers use OpenGL for their UIs

DSP Libraries

• Gamma - Gamma is a very awesome C++ DSP library by Lance Putnam. The beauty of Gamma is the conciseness of the implementation of certain techniques. How about this for a concise STFT example.
• Q - A very nice looking modern C++ DSP library with concise examples
• HIIR - HIIR is a seriously cool oversampling library by Laurent de Soras. Oversampling is something we often need in audio DSP, and this library handles it elegantly - providing a variety of classes for low latency IIR half band filtering (including SIMD optimizations). Originally this had a LGPL licence but now it's available under the WTFPL - my favourite licence.
• HOALibrary - A flexible DSP library for high order ambisonics (HOA) - a spatial audio platform that is becoming more and more relevant thanks to VR (GPL Licence).
• HISSTools Library - a nice BSD licensed library by Alex Harker, including FFT abstractions and multi-channel convolution, amongst other things
• Mach1 Spatial SDK - Vector based multichannel & spatial audio APIs for customizing multichannel or spatial audio pipelines & tools and agnostically playback any spatial, surround soundfields.
• Spatial Audio Framework - Another excellent library for ambisonics and other spatial audio related processing
• MadronaLib - Randy Jones' DSP library, which is awesome because it is all designed for SIMD processing.
• WDL - WDL is Cockos' library of reusable C++ code, that is used to make the DAW Reaper, amongst other things. It includes open source implementations of many useful things such as resampling and convolution, although there is next to no documentation and the code is difficult to understand. For more info about the various parts of WDL (which can be used independently), check the Cockos site
• DaisySP - A MIT licensed DSP library written for the Daisy platform, but also useable elsewhere. I like it because it includes some nice ports of Emilie Gillet's mutable instruments code.
• SignalSmithDSP - A really nice looking DSP library, along with a really nice blog/website with a collection of nice articles, e.g. How to write a reverb, see also a video presentation

Music Information Retrieval Libraries (C/C++)

• essentia - Essentia is a huge collection of MIR algorithms that you can use to extract high and low level features from audio. AGPL3 license. Somewhat painful to build cross-platform.
• audioflux - AudioFlux is a new-kid-on the block and some what similar but looks designed for deep learning, it looks very promising and has a liberal (MIT) license.

Domain Specific Languages (DSLs)

• FAUST - FAUST is a powerful functional DSL for audio DSP with many options for quickly compiling to different “architectures” including audio plug-ins and embedded devices. FAUST is not interpreted like programming languages such as Puredata/Max/Supercollider/javascript, it is "transpiled" to C++ and various other programming languages or bytecode formats such as Web Assembly (WASM). This functionality is suited to rapid prototyping but can also produce robust and performant binaries. Whilst the rapid prototyping possibilities of FAUST are appealing to me, what I like most about it is the extensive library of high quality DSP, particularly for physical modelling synthesis and the way in which you can compose algorithms using the concise syntax. FAUST includes a JIT compiler "libfaust" based on LLVM, which I integrated into a JUCE module. I've also integrated FAUST support into iPlug2 and used it to make a physical model of the Indian Tambura.
• Cmajor - Is an exciting new DSL created by Jules Storer and Cesare Ferrari, which also uses the LLVM compiler technology. It has a JIT compiler for a fast workflow and can spit out C++ and WASM and ready-to-go JUCE projects. Cmajor is an imperative language much more similar to C/C++/Javascript than FAUST. It has several advantages, for example multi-rate processing and in my opinion it is more suited for code that mixes event-handling with DSP, where as FAUST is good for discreet blocks of DSP processing. Since FAUST can output Cmajor code via faust2cmajor, you can use them together.

Filter design

• Filter Playground - A nice blog post and web audio tool demonstrating IIR filters
• MicroModeler Filter Design Tool - An excellent web based filter design tool
• DSPFilters - Vinno Falco made this excellent MIT licensed IIR filter library. Here is a fork to work with the latest JUCE. See also iir1, which seems a further developed and improved version.
• RBJ Cookbook - HTML version of Robert Bristow-Johnson's EQ cookbook
• Cytomic Technical Papers - Andy Simper's technical papers, which include code examples and theory for modern approaches to filter design
• Vadim Zavalishin - The Art of VA Filter Design - Very math-heavy text on those modern approachs

Reading

This is a small selection of websites, books and articles that have been helpful to me.

• Will Pirkle - Will Pirkle has written two books that will be invaluable the aspiring audio plug-in developer - "Designing Audio Effect Plug-Ins in C++" and "Designing Software Synthesizer Plug-Ins in C++". The books contain detailed information about important audio DSP, including modern virtual analogue techniques, and sample code for complete sythesizers that sound good. My only criticism is that a large part of the books relates to a bespoke plugin framework and providing information for multiple formats, AU, VST3 can be a bit overwhelming, and clutters the content. Nevertheless I highly recommend these books. For more info see here
• Dodge and Jerse - Computer Music: Synthesis, Composition and Performance, 2nd Edition - This is my absolute favourite book to recommend to students studying computer music/audio synthesis. I find the book does not date like some other computer music texts. The block diagrams are charming and the techniques are discussed very well.
• Udo Zölzer (Ed) - DAFX: Digital Audio Effects - This is a great book on audio DSP, written by a variety of domain experts, and it includes Matlab code examples.
• JOS - Julius Smith's site and his four books are an amazing resource. Matlab/Octave and FAUST code examples included.
• DAFX Conference Archive - All the papers from the DAFX conference are available online. Another great resource.
• EarLevel Engineering - Nigel Redmond's DSP blog contains some very nicely written explanations of a variety of topics.
• Michael Tyson blog - In depth article about Four common mistakes in audio development
• Ross Bencina's blog/site - A developer who writes a lot of interesting software and articles about lock free programming
• Sound on Sound Magazine Synth Secrets - A nice series on synthesis techniques.
• Jatin Chowdhury @ medium - This blog, along with Jatin's github is full of fascinating, cutting-edge stuff.
• RS-MET tutorials - Some nice pdfs with clear explanations of audio DSP topics.
• Melatonin Blog - Sudara is writing some really excellent articles on some of the painful tasks involved in audio plug-in development and also publishing lots of useful open source recipes on his github
• Katja's Home - a very cool site with writings about various DSP topics and examples in pd/c

Tools

These are the software tools that I find useful in my audio programming.

• ChatGPT+ Subscription - ChatGPT has become invaluable to me, allowing me to experiment and be creative with things that would have taken much more work before
• VSCode & Github Codespaces - VSCode is a brilliant text editor that has lots of interesting extensions and functionality. Lately I am using VSCode remote containers and Github Codespaces see video which I think is a game-changing technology.
• Cycling '74 Max - Max is a great environment to use for prototyping audio plug-ins. There are just so many options for integrating different technologies, I highly recommend it - even if nowadays most of the max patches I make only include a few objects.
• Desmos - This is an awesome online graphing calculator. Check out some interactive Casio CZ waveforms that I made saw square reso1
• Coliru - This is an online interactive C++ compiler, which can be a very nice and quick way to test out a particular feature of the language, without having to build a binary.
• Compiler Explorer - Compiler explorer is a great tool for checking the assembly code that different compilers will produce.
• FAUST Web IDE - This is an online FAUST IDE and compiler, that lets you test FAUST code with webaudio. You can then tell it to output a zip file with a binary for your preferred platform. This saves you all the trouble of installing FAUST and its dependencies on your local machine.
• Matplotlib-cpp - A C++ interface to the matplotlib python plotting tool. Allows you to visualize algorithms using the same code that you can use in production.
• xeus-cling - This is a really cool project allowing you to use JIT compiled C++ in jupyter notebooks - which, like Matplotlib-cpp above is great for visualizing algorithms using the same code that you can use in production, inside the popular jupyter notebook format. Here is an example of how you can use those things together, easily via a ready made docker image.
• StudioRack plugin manager - Open-source audio plugin manager tool and Github plugin templates to automatically build cross-platform audio plugins.
• dspplot - A python script that I find useful for plotting filter responses
• Docker desktop - Docker is incredibly useful for creating reproducible environments, e.g. with specific python libraries installed.
• Audio Programming Environment - APE - An audio plug-in that embeds a C++ JIT compiler which you can use to test your C++ DSP without leaving the DAW.
• OhMyZSH - Make macOS terminal/iTerm2 nice and colorful

Exemplary open source audio plug-in projects

• surge - Surge was an excellent product which has been open sourced and is now being maintained and continually improved by the surge-synth team, with a new JUCE UI.
• valentine - An awesome compressor and great, complete project example for a cross platform audio plug-in in 2023.
• vital - Vital is a very succesful wavetable synth with a slick interface done using the GPU. Amazing that a project like this can be open source!

Miscellaneous open source audio code/projects

• ni-media - A very nice audio file loading library from Native Instruments
• choc - From the creator of JUCE comes CHOC, an ISC licensed library of Classy Header-Only Classes. Including some useful audio related utilities
• BespokeSynth - A pretty amazing modular synth/composition environment with nice real-time visualisation, loads of modules, and lots of flexibility
• destroyfx - A brilliant collection of open source plug-ins that are very original have been around since the dawn of VST.
• puredata - Pd is a really nice cross platform dataflow programming environment. Also check out libpd, which you can use as an embeddable DSP runtime in your C++ audio plug-in etc. Camomille combines libpd with JUCE to let you make plug-ins based on Pd patches. plugdata is an excellent new effort to make PD run as a plug-in with a fresh new UI built with JUCE.
• supercollider - Likewise, but make sure you look at the scsynth part for DSP stuff.
• csound - CSound is still going strong after many years. It can also be used as an embeddable DSP library. There is a Web IDE, which is a great way to play with interesting scores/orchestras/UDOs that you find on the web and Cabbage lets you make plug-ins based on CSound orchestras.
• vcvrack - Whilst the other items in this list have been around a while and have somewhat arcane code-bases, this is pretty new and the API is very clean and simple. I think making a VCVRack module it is a great way to get into audio programming.
• tracktion engine - Source code for an entire DAW engine, using modern C++. An amazing resource for learning all sorts of things including how to structure and architect large audio projects. GPL/Commercial license.
• musicdsp.org - "A collection of algorithms, thoughts and snippets, gathered for the music dsp community". A long serving site that has recently been revamped. Beware: most of the code there was written a very long time ago, and optimization tricks etc, may not be relevant on modern machines. Also there are lots of code snippets programming languages other than C++ (delphi, java, C# etc). Could do with some curation.
• obxd - OBxd is a great sounding oberheim emulation plug-in, written with JUCE. DiscoDSP have updated it, and it has also been ported to work on the web as a WebAudioModule
• dexed - Dexed is a JUCE frontend to Raph Levien's "Music Synthesier For Android", which is an excellent DX7 emulation. Also ported to work on the web as a WebAudioModule
• AudioKit SynthOne - SynthOne is a great open source project built using Audiokit, that will be very interesting to anyone looking to build an iOS synthesiser.
• AirWindows Plugins - A large collection of GUI-less plug-ins open source but supported via patreon see airwindows site
• zita stuff - Fons Adriaensen's linux audio projects offer a lot of great tools for acoustic measurements, spatial audio etc - mostly as jack apps.
• Webkit's Web Audio API - This is the C++ code for WebKit's implementation of the Web Audio API, as used in Safari.
• Blink's Web Audio API - The same thing in Blink (Google chromium).
• Firefox's Web Audio API - And for Mozilla Firefox.

Machine Learning

• GuitarML - Very cool project building open source guitar fx/amp simulations using machine learning techniques.
• NeuralAmpModeler - A similar project, which is also very cool - plug-in built with iPlug2!
• ONNXRuntime - Microsoft's ONNX Runtime is a ML inference engine that is well documented, performant, and customizable. Here is an iPlug2 example that I prepared that integrates onnxruntime into an audio plugin, using ort-builder - another project that I made in order to build custom, slimmed down static libraries for the onnxruntime
• RTNeural - A library designed for realtime inference of Neural Network models with audio in mind. Very small when compared to linking a library like OnnxRuntime or libtensorflow.
• Torsion Audio's Neural Network Inference Template for Real-Time Cricital Audio Environments - A great JUCE template for a machine learning based plug-in, set up for libTorch, TFLite, or OnnxRuntime, including code to help you offload inference onto a separate thread.

Plug-in gui stuff

• KnobMan, WebKnobMan - "Safe for work" knob pics and an online tool to generate them.

Hardware/embedded

• the owl pedal/module - This is a programmable stomp box and eurorack module that I've been working with since the kick starter campaign to launch it. It's a really nice little unit, which you can program in C++, FAUST, Pd or with Max gen~. I find the limited interface with 4 controls makes me think quite carefully about what's important about my DSP algorithm. You can find some patches I made for it (using a mixture of c++ and FAUST) in the user library, and the original code here.
• bela - Bela is a wonderful little SoC + Audio Interface which is pretty revolutionary, allowing super low latency audio and sensor I/O all clocked together, in a tiny package. I wrote the FAUST support class for bela. You can also program it with C++ or libpd, and even supercollider.
• ELK Audio OS - An embedded linux OS & dev kit that runs VST2 & VST3 plugins with super low latency.
• mutable instruments - Emilie Gillet's work is inspirational. The code has been ported to VCVRack and to some nice iOS AUv3s
• pedalvite - An open source pedal board project from a developer who did a lot of the DSP in the best plugins of the last ~20 years.
• Daisy - Daisy is a brilliant programmable embedded music platform. A few different host devices are available into which you plug the "daisy seed" Of course you can also build your own device around the Daisy seed board if you don't suck at electronics, like me. It has a more powerful MCU than the OWL and I really like how the same board can be plugged into different hardware depending on your needs.

Youtubers/Podcasters

• Jan Wilczek - The WolfSound - Jan is an extremely driven, friendly and talented guy who has put a lot of work into making a great set of resources for people who want to learn audio programming. His content is a mixture of very good technical overviews of important topics and great interviews with people across the industry and involved in academia. I was honoured to be interviewed by Jan for episode 15 of the wolfsound podcast, which you can listen to here.
• TheAudioProgrammer - includes a lot of tutorials about JUCE with guest slots on things like web audio.
• TheChernoProject - although this channel is focused on game development, the C++ series is excellent. Highly recommended.
• Valerio Verlardo - The Sound of AI - Valerio has made a very impressive channel and community dedicated to AI and ML topics in Audio

Interesting Talks

• Teaching Yourself to Make Music Software: Steve Duda in Conversation | Loop - an inspiring interview with a great developer about a killer synth.
• Sean Costello (Valhalla DSP) on reverb design - great talk by another great plugin developer
• C++ Real-Time Audio Programming with Bela - a very clearly presented course.
• Vadim Zavalishin - The Art of VA Filter Design, ADC 2018
• How Moog Makes Synth Apps: Interview With Geert Bevin, Head Of Software At Moog - a great interview full of insight

Places

• musicdsp mailing list - The music DSP mailing list is pretty quiet these days, but it still worth signing up, despite the web 0-1 front page. Every now and again a music DSP legend posts something interesting.
• KvR Audio DSP and Plug-in Development Forum - This is probably the most active forum for audio DSP that is not aligned with a particular plug-in framework. There are some very smart people, some mavericks and some plain weirdos who hang out here.
• JUCE Forums - This is a collection of forums centred around the variety of things that JUCE does. Since there are such a huge amount of people using JUCE to make audio software, there is a lot of good info here.
• Cockos Forums - Another collection of forums centred around Cockos' tools including one for WDL/iPlug and one for Reaper JS.
• The audio developer conference (ADC) - Whilst the other places mentioned here are all virtual, this is a real conference where you can go and meet real people face-to-face who do audio programming - highly recommended!
• TheAudioProgrammer Discord server - A popular community for discussing audio programming.
• iPlug2 Forum - forum for iPlug2 discussions
• Steinberg VST3 Forum - A forum for the VST3 SDK/format

Oli Larkin 2018-2024 https://www.olilarkin.co.uk

License