VoixTenue: Exploring Real-Time Gestural Control of Vocal Synthesis on a Mobile Phone

Adrien Scazzola; Xiao Xiao

VoixTenue: Exploring Real-Time Gestural Control of Vocal Synthesis on a Mobile Phone
Image credit: Adrien Scazzola; Xiao Xiao

Abstract:

We present VoixTenue, an interface exploring how a mobile phone’s on-board sensing can be used to control pitch, dynamics, and intonation in real time for expressive vocal synthesis. Voix- Tenue supports two main interaction modalities: one where users draw and replay fingertip-drawn intonation curves for real-time pitch control, and another where the phone’s orientation controls pitch and dynamics through inertial sensing. Using Pink Trom- bone as its synthesis engine, VoixTenue supports two modes for phonetic content: a vowel mode, in which users select a sustained vowel, and a phrase mode, in which a short English text can be entered, whose intonation is controlled by the user. We describe the system architecture and gestural mappings, and discuss po- tential use cases, including expressive performance and language learning.