Mobile Augmented Reality as a Control Mode for Real-time Music Systems

Mobile augmented reality has only been made possible in the last few years, owing to the increasing processing power, more responsive hardware sensors and improved cameras of modern mobile computers. As a result of this, the mobile app markets have started filling up with augmented reality applications, ranging from games to navigation assistants. Similarly, the increasing power of mobile devices combined with their multitude of interaction inputs has made them extremely suitable for music purposes – real-time synthesis, control and editing – and we are beginning to see more and more music creation apps on the market. However, as of yet there have been no attempts to mix these two ideas – augmented reality and music creation – in neither commercial products nor research. As such, an augmented reality real-time music system has been implemented inspired by the tangible table-top real-time music system, the Reactable with the aim of investigating the effects of an augmented-reality control mode on the enjoyment, intuitiveness of interaction, immersion and ease-of-use of a real-time music system. A series of subjective user tests were conducted and the results show that using an augmented reality control mode over a more traditional top-down screen-centric control mode improves the perceived enjoyment and engagement of the music experience, both of which are essential properties for such systems.