In the context of my Diploma thesis I developed new means of input for an existent system allowing users with any skill to conduct a virtual orchestra in realtime. The eWatch, a wearable computing and sensing platform, was selected as input device. It is worn like a regular wrist watch and transmits acceleration data via Bluetooth to the host computer.
By performing common conducting gestures, the user is able to indicate the measure. In addition, he or she can influence the orchestra’s tempo and volume by varying the gestures in speed and size. Hidden Markov models are used to perform the gesture recognition task. While realizing a reliable and continuous gesture recognition, the proposed system is able to successfully reject non-meaningful hand movements, applying a threshold model.
Related Publications
- Schmidt, D.
Acceleration-based gesture recognition for conducting with Hidden Markov Models
Diploma Thesis, 2007, Ludwig-Maximilians-Universität (unpublished)
PDF - Schmidt, D., Dannenberg, R. B., Smailagic, A., Siewiorek, D. P., and Brügge, B.
Learning an orchestra conductor’s technique using a wearable sensor platform
Poster at ISWC 2007
PDF - Brügge, B., Teschner, C., Lachenmaier, P., Fenzl, E., Schmidt, D., and Bierbaum, S.
Pinocchio: Conducting a virtual symphony orchestra
In Procceedings of ACE 2007, pp. 294-295
PDF
In the Press
- Wrist sensor lets you conduct a virtual orchestra. In New Scientist, October 18th, 2007.
Further Links
- Pinocchio (virtual symphony orchestra system)