iXeRemin: Designing a Polyphonic MR Instrument for Artistic Performance and Interaction

Gwangyu Lee

iXeRemin: Designing a Polyphonic MR Instrument for Artistic Performance and Interaction
Image credit: Gwangyu Lee

Abstract:

This paper introduces iXeRemin, a theremin-inspired mixed reality (MR) instrument for Apple Vision Pro that combines hand tracking, frequency modulation (FM) synthesis, chordal gesture mapping, fingertip labels, and Open Sound Control (OSC) output. The right hand controls pitch, note triggering, and vibrato, while the left hand controls amplitude, FM index, and harmonic mode. Chords are selected through left index-thumb distance, with seventh chords activated by left pinky-thumb distance. Real-time 3D labels display amplitude, note state, and pitch or chord information near the performer’s fingertips. Rather than claiming a direct continuation of traditional theremin technique, iXeRemin uses the theremin as a conceptual reference for touchless musical control and adapts it to a spatial computing context. The contribution of this short paper is the design and public release of a polyphonic MR instrument that connects gestural sound control with visual feedback and OSC-based interoperability.