nocturneAR: An Outdoor AR-Based Musical Interface in Dark Transitional Landscapes

Masami Hirabayashi; Kakuya Shiraishi; Scott Allen

nocturneAR: An Outdoor AR-Based Musical Interface in Dark Transitional Landscapes
Image credit: Masami Hirabayashi; Kakuya Shiraishi; Scott Allen

Abstract:

nocturneAR is a participatory outdoor music installation that explores liminal audio-visual experiences in darkness using Augmented Reality (AR). Although AR systems typically rely on sufficient illumination and thus fail in dark environments, we developed an AR-based musical interface for dimly lit outdoor spaces by utilizing visible light communication through LED flashing as AR markers. We propose an AR system in which light functions simultaneously as a navigational cue, interaction trigger, and musical interface. Participants walk through dark outdoor environments, such as night forests, guided by multiple luminous objects while holding a smartphone. Directing the smartphone toward a light source activates AR visuals and sound playback. As participants move between lights, sounds accumulate and transform, generating evolving harmonic structures that include chords, dissonance, and undulating textures. This system frames spatial navigation toward light as an embodied musical interaction, enabling participants to discover sound relationships through movement in an uncertain environment. By engaging with ambiguous visual cues and transitional sound states within darkness, the installation creates a liminal experience that blurs the boundaries between navigation, performance, and listening. We present the system design and describe an exploratory nighttime forest exhibition, discussing how darkness and visible light-based interaction can support novel forms of spatial musical experience in AR contexts.