Air Violin: A Machine Learning Approach to Fingering Gesture Recognition

TitleAir Violin: A Machine Learning Approach to Fingering Gesture Recognition
Publication TypeConference Paper
Year of Publication2017
Conference NameMIE’17, November 13, 2017, Glasgow, UK
AuthorsDalmazzo, D. C., & Ramirez R.
Conference Start Date13/11/2017
PublisherProceedings of 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education (MIE’17). ACM, New York, NY, USA
Conference LocationGlasgow - Scotland
KeywordsGamification, Gestures, Hand Tracking, Hidden Markov model, Machine learning, Music education, Violin
AbstractWe train and evaluate two machine learning models for predicting fingering in violin performances using motion and EMG sensors integrated in the Myo device. Our aim is twofold: first, provide a fingering recognition model in the context of a gamification virtual violin application where we measure both right hand (i.e. bow) and left hand (i.e. ngering) gestures, and second, implement a tracking system for a computer assisted pedagogical tool for self-regulated learners in high-level music education. Our approach is based on the principle of mapping-by-demonstration in which the model is trained by the performer. We evaluated a model based on Decision Trees and compared it with a Hidden Markovian Model.
Final publication