Recommender Systems Handbook

pp 309-352

Evaluating Recommender Systems with User Experiments

  • Bart P. KnijnenburgAffiliated withClemson University Email author 
  • , Martijn C. WillemsenAffiliated withEindhoven University of Technology

* Final gross prices may vary according to local VAT.

Get Access


Proper evaluation of the user experience of recommender systems requires conducting user experiments. This chapter is a guideline for students and researchers aspiring to conduct user experiments with their recommender systems. It first covers the theory of user-centric evaluation of recommender systems, and gives an overview of recommender system aspects to evaluate. It then provides a detailed practical description of how to conduct user experiments, covering the following topics: formulating hypotheses, sampling participants, creating experimental manipulations, measuring subjective constructs with questionnaires, and statistically evaluating the results.