Note: This bibliographic page is archived and will no longer be updated. For an up-to-date list of publications from the Music Technology Group see the Publications list .

How do Gain and Discount Functions Affect the Correlation between DCG and User Satisfaction?

Title How do Gain and Discount Functions Affect the Correlation between DCG and User Satisfaction?
Publication Type Conference Paper
Year of Publication 2015
Conference Name 37th European Conference on Information Retrieval
Authors Urbano, J. , & Marrero M.
Pagination 197-202
Abstract We present an empirical analysis of the effect that the gain and discount functions have in the correlation between DCG and user satisfaction. Through a large user study we estimate the relationship between satisfaction and the effectiveness computed with a test collection. In particular, we estimate the probabilities that users find a system satisfactory given a DCG score, and that they agree with a difference in DCG as to which of two systems is more satisfactory. We study this relationship for 36 combinations of gain and discount, and find that a linear gain and a constant discount are best correlated with user satisfaction.