How do Gain and Discount Functions Affect the Correlation between DCG and User Satisfaction?

TitleHow do Gain and Discount Functions Affect the Correlation between DCG and User Satisfaction?
Publication TypeConference Paper
Year of Publication2015
Conference Name37th European Conference on Information Retrieval
AuthorsUrbano, J., & Marrero M.
Pagination197-202
AbstractWe present an empirical analysis of the effect that the gain and discount functions have in the correlation between DCG and user satisfaction. Through a large user study we estimate the relationship between satisfaction and the effectiveness computed with a test collection. In particular, we estimate the probabilities that users find a system satisfactory given a DCG score, and that they agree with a difference in DCG as to which of two systems is more satisfactory. We study this relationship for 36 combinations of gain and discount, and find that a linear gain and a constant discount are best correlated with user satisfaction.
intranet