SALERO: Semantic AudiovisuaL Entertainment Reusable Objects
Started in January 2006, finished in December 2009.
SALERO was an European Project financed by the European Union under the Information Society Technologies Programme FP-027122
Objective
SALERO aims at making cross media-production for games, movies and broadcast faster, better and cheaper by combining computer graphics, language technology, semantic web technologies as well as content based search and retrieval.
Impact
SALERO will define and develop 'intelligent content' for media production, consisting of multimedia objects with context-aware behaviours for self-adaptive use and delivery across different platforms. 'Intelligent Content' should enable the creation and re-use of complex, compelling media by artists who need to know little of the technical aspects of how the tools that they use actually work.
Main Innovation
Based on research into methodologies for describing, creating and finding intelligent content, SALERO will develop toolsets to create, manage, edit, retrieve and deliver content objects, addressing characters, objects, sounds, language sets, and behaviours. The toolsets developed and the concept of intelligent content will be verified by experimental productions.
Our Task
The Music Technology Group of the Pompeu Fabra University has developed inside the SALERO project a set of Intelligent Authoring Tools for Audio Transformation. The Audio Transformation Tools are delivered as a set of VST plug-ins, an audio plug-in standard created by Steinberg to allow any third party developers to create VST plug-ins for use within VST host applications. The videos below show the functionality of these plug-ins as well as the powerful transformations that can be achieved using this real-time software.
* Voice Transformation *
Several singing and speech voice transformations are included in this real-time VST plug-in effect. The system uses spectral techniques for modifying the character of the voice. Transformations include: transposition, quantization, vibrato, roughness, breathiness, whisper, timbre mapping and other spectral transformations. This tool is targeted to recording or post-production studios that require voice processing. In the video we can see some of the available transformations applied in real-time to an original male voice.
* Tempo Transformation *
This VST plug-in combines automatic rhythm analysis and time-scaling in order to transform the rhythm of an audio mix. An input audio is first pre-analyzed in order to extract a musical rhythm description. After this process, the user can vary the tempo or add/subtract swing by using the GUI controls during the playback. In the video we can see how easily is, using this VST instrument plug-in, to modify in real-time the tempo of a song being played or adding a swing effect to it.
* Advanced Audio Equalizer *
This VST plug-in is aimed for the advanced equalization of commercial music productions. This is a powerful tool able to equalize the audio using energy histogram-based on different criteria. It is not limited to the typical energy distribution versus frequency, but it allows working with other useful energy distributions criteria such as panning, inter-channel phase difference, or energy variance. Some of its applications include: Remixing a piece by changing the volume of each instrument independently or removing instruments from the mix and then re-adding them processed by some effects or isolate one or several instruments to perform a musical analysis of a piece. In the video we can see how to mute or isolate guitar and voice from a commercial song very easily using the graphical user interface of the plug-in.
SALERO Partners list:
JOANNEUM RESEARCH, The Institute of Information Systems & Information Management (JRS, AT)
Activa Multimedia, IMAGINA Centre Audiovisual (AM, ES)
Blitz Games Studios Ltd (BLITZ, UK)
Pepper’s Ghost Productions Ltd (PGP, UK)
Universitat Pompeu Fabra, Fundació Barcelona Media (FBM-UPF, ES)
Universitat Ramon Llull, La Salle School of Engineering (URL, ES)
Dublin Institute of Technology (DIT, IE)
Taideteollinen Korkeakoulu (TAIK, FI)
The University of Glasgow (UG, UK)
Leopold-Franzens Universität Innsbruck, Digital Enterprise Research Insitute (LFUI, AT)
Thomson Systems Germany GmbH, Grass Valley Germany (GVG, DE)
DTS Digital Theater Systems, DLLNI Ltd (DTS, UK)
Universitat Pompeu Fabra, Music Technology Group (UPF-MTG. ES)
