Note:
This bibliographic page is archived and will no longer be updated.
For an up-to-date list of publications from the Music Technology Group see the
Publications list
.
Evaluation in Music Information Retrieval
| Title | Evaluation in Music Information Retrieval |
| Publication Type | Journal Article |
| Year of Publication | 2013 |
| Authors | Urbano, J. , Schedl M. , & Serra X. |
| Journal Title | Journal of Intelligent Information Systems |
| Volume | 41 |
| Pages | 345-369 |
| Abstract | The field of Music Information Retrieval has always acknowledged the need for rigorous scientifi c evaluations, and several ef forts have set out to develop and provide the infrastructure, technology and methodologies needed to carry out these evaluations. The community has enormously gained from these evaluation forums, but we have reached a point where we are stuck with evaluation frameworks that do not allow us to improve as much and as well as we want. The community recently acknowledged this problem and showed interest in addressing it, though it is not clear what to do to improve the situation. We argue that a good place to start is again the Text IR field. Based on a formalization of the evaluation process, this paper presents a survey of past evaluation work in the context of Text IR, from the point of view of validity, reliability and eficiency of the experiments. We show the problems that our community currently has in terms of evaluation, point to several lines of research to improve it and make various proposals in that line. |
| Final publication | https://doi.org/10.1007/s10844-013-0249-4 |
