e-baton: Recognising Conducting Gestures with Machine Learning

Marko Jeremic; Akito Van Troyer

Image credit: Marko Jeremic; Akito Van Troyer

• demo
-->
• Paper PDF link
• Presence: in person
• Type: short

Abstract:

We present e-baton, a wireless conducting controller that uses an inertial measurement unit (IMU) and a user-retrainable kNN classifier to simultaneously control human performers and electronic musical parameters. During iterative development, we found that accelerometer data alone was insufficient for recognising slower, fluid conducting gestures, and that extending the feature set to include jerk and rotational data was necessary to support the gesture vocabulary used in performance. We evaluate the system through a five-participant tempo-tracking study and a debut performance with an improvising keyboardist.