The control-synthesis approach for making expressive and controllable neural music synthesizers

Deep neural networks have been successfully applied to audio synthesis. Such neural audio generation models can efficiently learn from data how to imitate a variety of instruments, such as piano and violin. However, effective control of these models is difficult. We introduce the “control-synthesis approach” to make neural audio synthesizers more controllable. This approach transforms user input into intermediate features to condition a neural audio synthesis model. We demonstrate this approach by implementing MIDI-controllable neural audio synthesizers and generating several examples for audition.