Latent Terrain: Adapting Neural Audio Autoencoders as Design Materials in NIME

Shuoyang Jasper Zheng; Keigo Yoshida; Nico García-Peguinho; Jiatong Liu; Dan Hearn; Anna Xambó Sedó; Nick Bryan-Kinns

Image credit: Shuoyang Jasper Zheng; Keigo Yoshida; Nico García-Peguinho; Jiatong Liu; Dan Hearn; Anna Xambó Sedó; Nick Bryan-Kinns

• oral
• Paper PDF link
• Presence: in person
• Duration: 16
• Type: long
• Session: Learning with the Machines

Abstract:

Neural audio autoencoders, a deep learning method for sound synthesis, are increasingly popular in AI-enhanced NIMEs. It is timely to explore as a community how this technological opportunity has opened a domain of design in NIME. This paper focuses on one compelling technique of using autoencoders for sound synthesis: navigating their latent space as a generative sound space. We introduce Latent Terrain, a Max/MSP tool package designed to tailor latent spaces into corpus-based sound spaces for NIMEs. We describe the rationale and development process of Latent Terrain to offer insights into the use of autoencoders in a material-oriented crafting space of musical interface design. We deliver an annotated portfolio resulting from a collaborative artistic exploration of Latent Terrain with four NIME makers, to showcase the design possibilities opened by autoencoders. We reflect on our practice-based account to discuss the challenges and opportunities of enabling neural audio autoencoders as design materials for AI-enhanced NIMEs.