Note: This bibliographic page is archived and will no longer be updated. For an up-to-date list of publications from the Music Technology Group see the Publications list .

Immersive Orchestras: audio processing for orchestral music VR content

Title Immersive Orchestras: audio processing for orchestral music VR content
Publication Type Conference Paper
Year of Publication 2016
Conference Name VSGAMES 2016 - 8th International Conference on Virtual Worlds and Games for Serious Applications
Authors Janer, J. , Gómez E. , Martorell A. , Miron M. , & de Wit B.
Conference Start Date 07/09/2016
Publisher IEEE
Conference Location Barcelona
Abstract This paper combines Audio Signal Processing and Virtual Reality (VR) content to create novel immersive experi- ences for orchestral music audiences. In VR, the auralization of sound sources of recorded live content remains still a rather unexplored topic. We aim to build a multimodal experience, where visual and audio cues bring a sonic augmentation of the real scene. In the particular scenario of orchestral music content, our goal is to acoustically zoom on a particular instrument when the VR user stares at it. This work aims to improve the learning aspects of music listening, either for education or for personal enrichment. We use audio signal processing to separate different sound sources (instruments) in a acoustic scene (orchestral music recording). Given the signals captured by multiple microphones and the musical score of the piece, our system is able to isolate the different instruments. From the processed separated tracks, we use a binaural rendering technique to emphasize a give instrument. For these experiments we used original content from top European orchestras.
preprint/postprint document http://hdl.handle.net/10230/32953