Trinity College Dublin, Department of Computer Science
Hayes, Conor; Cunningham, Pádraig. 'An on-line evaluation framework for recommender systems'. - Dublin, Trinity College Dublin, Department of Computer Science, TCD-CS-2002-19, 2002, pp10
Computer Science Technical Report TCD-CS-2002-19
Several techniques are currently used to evaluate recommender
systems. These techniques involve off-line analysis using evaluation
methods from machine learning and information retrieval. We
argue that while off-line analysis is useful, user satisfaction with a recommendation
strategy can only be measured in an on-line context. We
propose a new evaluation framework which involves a paired test of two
recommender systems which simultaneously compete to give the best
recommendations to the same user at the same time. The user interface
and the interaction model for each system is the same. The framework
enables you to specify an API so that different recommendation strategies
may take part in such a competition. The API defines issues such as
access to data, the interaction model and the means of gathering positive
feedback from the user. In this way it is possible to obtain a relative
measure of user satisfaction with the two systems.
Please note: There is a known bug in some browsers that causes an
error when a user tries to view large pdf file within the browser window.
If you receive the message "The file is damaged and could not be
repaired", please try one of the solutions linked below based on the
browser you are using.
Items in TARA are protected by copyright, with all rights reserved, unless otherwise indicated.