Deep Drawing: Performance Surface Sound Source Localization

Lennon Seiders; Julie Zhu; John Granzow; Alex Zhang; Anusha Chinthamaduka

Image credit: Lennon Seiders; Julie Zhu; John Granzow; Alex Zhang; Anusha Chinthamaduka

• demo
-->
• Paper PDF link
• Presence: in person
• Type: short

Abstract:

Deep Drawing is an ongoing exploration of the sound of drawing through AI co-performance. As the sounds of a performer’s drawing gestures resonate through a wooden board, they are re-created, spatially localized, and visualized in real-time using a novel deep learning approach. The system foregrounds the often-overlooked, timbrally complex sounds of drawing and frames them as a shared interpretive space between human and machine. This paper presents an accessible hardware setup and a sound source localization (SSL) model that can be trained with minimal data, enabling expressive interaction without extensive calibration or large datasets. Because SSL for performance-sized surfaces is a relatively unexplored research topic, we introduce practical techniques for this setting, including high-pass filtering, data augmentation, and high-fidelity data capture. Deep Drawing contributes a replicable system that emphasizes performance, embodiment, and co-creative agency.