Conducting a Virtual Orchestra

In the context of my Diploma thesis I developed new means of input for an existent system allowing users with any skill to conduct a virtual orchestra in realtime. The eWatch, a wearable computing and sensing platform, was selected as input device. It is worn like a regular wrist watch and transmits acceleration data via Bluetooth to the host computer.

EWatch3- and 4-beat conducting gestures

By performing common conducting gestures, the user is able to indicate the measure. In addition, he or she can influence the orchestra’s tempo and volume by varying the gestures in speed and size. Hidden Markov models are used to perform the gesture recognition task. While realizing a reliable and continuous gesture recognition, the proposed system is able to successfully reject non-meaningful hand movements, applying a threshold model.

Related Publications

  • Schmidt, D.
    Acceleration-based gesture recognition for conducting with Hidden Markov Models
    Diploma Thesis, 2007, Ludwig-Maximilians-Universität (unpublished)
  • Schmidt, D., Dannenberg, R. B., Smailagic, A., Siewiorek, D. P., and Brügge, B.
    Learning an orchestra conductor’s technique using a wearable sensor platform
    Poster at ISWC 2007
  • Brügge, B., Teschner, C., Lachenmaier, P., Fenzl, E., Schmidt, D., and Bierbaum, S.
    Pinocchio: Conducting a virtual symphony orchestra
    In Procceedings of ACE 2007, pp. 294-295

In the Press

Further Links