Integrating Different Knowledge Sources for the Computational Modeling of Flamenco Music

There is a wealth of literature on music research that focuses on the understanding of music similarity from different viewpoints and on the computation of similarity distances to cluster different pieces according to composer, performer, genre or mood. This similarity measure is often based on comparing musical excerpts in audio format and measuring the distance of a set of content descriptors representative of different musical facets (e.g. the used instruments, rhythmic pattern or harmonic progression). Alternative approaches are based on comparing context information from the contrasted pieces (e.g. influences, temporal and geographical coincidences), which is usually extracted from the web or manually labelled. A combination of these two approaches (content and context) seems to be the most adequate solution, but there is still a limitation on current approaches. This might be due to the fact that there are still other information sources to consider, such as the listening conditions. State-of-the-art research has mainly focused on the analysis of music from the so-called “Western tradition,” given that most music retrieval systems are targeted toward this kind of music. Nevertheless, some studies are now considering if the available descriptors and similarity distances are suitable when dealing with music from other traditions. In this situation, the notion of similarity is also affected by the listener’s cultural background and his previous exposure to the considered musical structures.

We focus here in the study on flamenco music. Flamenco is a music tradition mostly originally from Andalusia, in southern Spain. The origin and evolution of flamenco styles and variants have been studied by different disciplines, mainly ethnomusicology, anthropology or literature. Prior studies have mainly focused on artists’ biographies, lyrics and social context, and there are few works on music analysis. There are some difficulties and motivations for developing computational models of similarity in flamenco music: being an oral tradition, there are no written scores; there exist few quality historical recordings and music collections are spread and not consistently documented; flamenco is not as present on the web as other musical styles; cultural institutions and music platforms are concerned with the preservation and spreading of flamenco music, given its commercial and cultural interest.

The goal of our project is to develop computational models for computer-assisted des- cription, similarity computation, comparative analysis and processing of flamenco music ( http://mtg.upf.edu/research/projects/cofla ). We want to integrate different knowledge

sources: content description (computational analysis of recordings and music similarity algo- rithms), context information (expert analyses) and user modeling (human judgements). As a case study, we deal with flamenco a capella singing. We focus on melodic similarity, and we evaluate state-of-the-art algorithms for automatic transcription and similarity computation. We work with a music collection of the most representative performances from 4 different a capella singing styles, mainly Debla, Martinete and Toná. We focus on analyzing the melodic exposition, and we approach both inter-style classification and intra-style similarity. Some of the challenges of this project are strongly connected to three of the topics discussed at Dagstuhl Seminar on Multimodal Music Processing: Multimodality, Evaluation and Ground Truth, and User Modeling.

• Multimodality . Flamenco research is highly multimodal, as we can combine different inputs when analyzing a certain performance: audio, video, context information (style and singer) and lyrics. These modalities are sometimes complementary and sometimes contradictory. In addition, we do not always have access to all of them simultaneously, so the main research to be done here is related to their integration in current algorithms. As mentioned before, this project intends to combine some of these modalities and their related knowledge sources by means of combined measures of similarity.
• Evaluation and Ground Truth . There are some difficulties in this project when evaluating algorithms for two different tasks: melodic transcription and similarity mea- surement. Regarding melodic transcription, we have adopted a two-stage evaluation procedure (annotations vs. corrections): first, we have collected manual melodic contour transcriptions from flamenco experts, where time information is not relevant and or- naments are removed; then, we have asked them to perform manual corrections and refinements of detailed transcriptions provided by the computational model. This allows us to gather and contrast both annotations of overall melodic contour and ornaments (melisma information). In the same way, we have gathered different sources of ground truth information for music similarity: list of relevant features, similarity ratings and validation of clusters/trees generated by computational models. We have observed that different ground-truth sources are complementary, and there is always a degree of sub- jectivity in each of them. Computational models should then integrate them and adopt procedures for interactive-validation and user-adapted annotation.
• User Modeling . Our project deals with two different user profiles: musicians with little knowledge of flamenco music and experts with high knowledge of flamenco music. We have observed a low correlation among their similarity ratings and the features they use to compare flamenco performances. Here, one particular challenge is to implement user-adapted similarity measures.

In conclusion, the integration of distinct knowledge sources as well as user adaptation are two key aspects in developing a computer-assisted model of flamenco music retrieval.