Enriched Multimodal Representations of Music Performances: Online Access and Visualization

TitleEnriched Multimodal Representations of Music Performances: Online Access and Visualization
Publication TypeJournal Article
Year of Publication2017
AuthorsMaestre, E., Papiotis P., Marchini M., Llimona Q., Mayor O., Pérez A., & Wanderley M. M.
Journal TitleIEEE MultiMedia
Journal DateJan
Start Page24
Keywordsaudio, bowed string, Collaborateive work, Context awareness, data analysis, data exchange, Data visualization, Instruments, Internet/web technologies, Media, Microphones, motion capture, multimedia, multimodal data, Multiple signal classification, Music, music performance, Recording, remote visualization, string quartet, video, visualization, web access
AbstractIn this paper, we provide a first-person outlook on the technical challenges and developments involved in the recording, analysis, archiving, and cloud-based interchange of multimodal string quartet performance data as part of a collaborative research project on ensemble music making. In order to facilitate the sharing of our own collection of multimodal recordings and extracted descriptors and annotations, we developed a hosting platform and data archival protocol through which multimodal data (audio, video, motion capture, descriptor signals) can be stored, visualized, annotated, and selectively retrieved via a web interface and a dedicated API. By way of this paper we make a twofold contribution: (a) we open our collection of enriched multimodal datasets to the community, the Quartet Dataset; and (b) we introduce and enable access to our multimodal data exchange platform, the Repovizz system, through which users can upload recorded data, and navigate, playback, or edit existing datasets via a standard Internet browser.
preprint/postprint documenthttp://ieeexplore.ieee.org/document/7849104/
DOI of final publication10.1109/MMUL.2017.3