User Modeling and User-Adapted Interaction

, Volume 22, Issue 4–5, pp 441–504 | Cite as

Explaining the user experience of recommender systems

  • Bart P. Knijnenburg
  • Martijn C. Willemsen
  • Zeno Gantner
  • Hakan Soncu
  • Chris Newell
Open Access
Original Paper

Abstract

Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about. In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.

Keywords

Recommender systems Decision support systems User experience User-centric evaluation Decision-making Human-computer interaction User testing Preference elicitation Privacy 

References

  1. Ackerman, M., Cranor, L., Reagle, J.: Privacy in e-commerce: examining user scenarios and privacy preferences. In: Conference on Electronic Commerce, pp. 1–8. Denver, CO (1999)Google Scholar
  2. Adomavicius G., Tuzhilin A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17, 734–749 (2005)CrossRefGoogle Scholar
  3. Adomavicius G., Sankaranarayanan R., Sen S., Tuzhilin A.: Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Inf. Syst. 23, 103–145 (2005)CrossRefGoogle Scholar
  4. Anderson J.C., Gerbing D.W.: Structural equation modeling in practice: a review and recommended two-step approach. Psychol. Bull. 103, 411–423 (1988)CrossRefGoogle Scholar
  5. Bagozzi R., Yi Y.: On the evaluation of structural equation models. J. Acad. Market. Sci. 16, 74–94 (1988)CrossRefGoogle Scholar
  6. Baudisch, P., Terveen, L.: Interacting with recommender systems. In: SIGCHI Conference on Human Factors in Computing Systems, p. 164. Pittsburgh, PA (1999)Google Scholar
  7. Bechwati N., Xia L.: Do computers sweat? The impact of perceived effort of online decision aids on consumers’ satisfaction with the decision process. J. Consum. Psychol. 13, 139–148 (2003)Google Scholar
  8. Bentler P.M., Bonett D.G.: Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88, 588–606 (1980)CrossRefGoogle Scholar
  9. Berendt, B., Teltzrow, M.: Addressing users’ privacy concerns for improving personalization quality: towards an integration of user studies and algorithm evaluation. In: IJCAI 2003 Workshop on Intelligent Techniques for Web Personalization, LNAI, vol. 3169, pp. 69–88. Acapulco, Mexico (2005)Google Scholar
  10. Bharati P., Chaudhury A.: An empirical investigation of decision-making satisfaction in web-based decision support systems. Decis. Support Syst. 37, 187–197 (2004)CrossRefGoogle Scholar
  11. Bhatnagar A., Ghose S.: Online information search termination patterns across product categories and consumer demographics. J. Retail. 80, 221–228 (2004)CrossRefGoogle Scholar
  12. Bollen, D., Knijnenburg, B., Willemsen, M., Graus, M.: Understanding choice overload in recommender systems. In: Fourth ACM Conference on Recommender systems, pp. 63–70. Barcelona, Spain (2010)Google Scholar
  13. Bradley, K., Smyth, B.: Improving recommendation diversity. In: Twelfth Irish Conference on Artificial Intelligence and Cognitive Science, pp. 85–94. Maynooth, Ireland (2001)Google Scholar
  14. Brodie C., Karat C., Karat J.: Creating an E-commerce environment where consumers are willing to share personal information. In: Karat, C.-M., Blom, J.O., Karat, J. (eds.) Designing Personalized User Experiences in eCommerce, pp. 185–206. Kluwer, Dordrecht (2004)CrossRefGoogle Scholar
  15. Burke R.: Hybrid recommender systems: survey and experiments. User Model. User-Adap. Int. 12, 331–370 (2002)MATHCrossRefGoogle Scholar
  16. Cena, F., Vernero, F., Gena, C.: Towards a customization of rating scales in adaptive systems. In: 18th International Conference on User Modeling, Adaptation, and Personalization, vol. 6075, pp. 369–374. Big Island, HI, LNCS (2010)Google Scholar
  17. Chellappa R., Sin R.: Personalization versus privacy: an empirical examination of the online consumer’s dilemma. Inf. Technol. Manag. 6, 181–202 (2005)CrossRefGoogle Scholar
  18. Chen L., Pu P.: Interaction design guidelines on critiquing-based recommender systems. User Model. User-Adap. Inter. 19, 167–206 (2009)CrossRefGoogle Scholar
  19. Chen, L., Pu, P.: Critiquing-based recommenders: survey and emerging trends. User. Model. User-Adap. Inter. 22(1–2), 125–150 (2012)Google Scholar
  20. Chin D.: Empirical evaluation of user models and user-adapted systems. User Model. User-Adap. Inter. 11, 181–194 (2001)MATHCrossRefGoogle Scholar
  21. Cooke A., Sujan H., Sujan M., Weitz B.A.: Marketing the unfamiliar: the role of context and item-specific information in electronic agent recommendations. J. Market. Res. 39, 488–497 (2002)CrossRefGoogle Scholar
  22. Cosley, D., Lam, S., Albert, I., Konstan, J., Riedl, J.: Is seeing believing?: how recommender system interfaces affect users’ opinions. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 585–592. Ft. Lauderdale, FL (2003)Google Scholar
  23. Cramer H., Evers V., van Someren M., Ramlal S., Rutledge L., Stash N., Aroyo L., Wielinga B.: The effects of transparency on trust and acceptance in interaction with a content-based art recommender. User Model. User-Adap. Inter. 18, 455–496 (2008a)CrossRefGoogle Scholar
  24. Cramer, H., Evers, V., van Someren, M., Ramlal, S., Rutledge, L., Stash, N., Aroyo, L., Wielinga, B.: The effects of transparency on perceived and actual competence of a content-based recommender. In: CHI’08 Semantic Web User Interaction Workshop. Florence, Italy (2008b)Google Scholar
  25. Csikszentmihalyi M.: Beyond Boredom and Anxiety. Jossey-Bass Publishers, San Fransisco (1975)Google Scholar
  26. Davis F.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340 (1989)CrossRefGoogle Scholar
  27. Davis F., Bagozzi R., Warshaw P.: User acceptance of computer technology: a comparison of two theoretical models. Manag. Sci. 35, 982–1003 (1989)CrossRefGoogle Scholar
  28. Diehl K., Kornish L., Lynch J. Jr: Smart agents: when lower search costs for quality information increase price sensitivity. J. Consum. Res. 30, 56–71 (2003)CrossRefGoogle Scholar
  29. Felfernig A., Teppan E., Gula B.: Knowledge-based recommender technologies for marketing and sales. Int. J. Pattern Recog. Artif. Intell. 21, 333–354 (2007)CrossRefGoogle Scholar
  30. Felix, D., Niederberger, C., Steiger, P., Stolze, M.: Feature-oriented vs. needs-oriented product access for non-expert online shoppers. In: IFIP Conference on Towards the E-Society: E-commerce, E-business, and E-government, pp. 399–406. Zürich, Switzerland (2001)Google Scholar
  31. Fishbein M., Ajzen I.: Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Addison-Wesley, Reading (1975)Google Scholar
  32. Gena, C., Brogi, R., Cena, F., Vernero, F.: Impact of rating scales on user’s rating behavior. In: 19th International Conference on User Modeling, Adaptation, and Personalization, LNCS, vol. 6787, pp. 123–134. Girona, Spain (2011)Google Scholar
  33. Harper F., Li X., Chen Y., Konstan J.: An economic model of user rating in an online recommender system. In: 10th International Conference on User Modeling, LNCS, vol. 3538, pp. 307–316. Edinburgh, UK (2005)Google Scholar
  34. Hassenzahl M.: The thing and I: understanding the relationship between user and product. In: Blythe, M.A., Monk, A.F., Overbeeke, K., Wright, P.C. (eds.) Funology, pp. 31–42. Kluwer, Dordrecht (2005)CrossRefGoogle Scholar
  35. Hassenzahl, M.: User experience (UX): towards an experiential perspective on product quality. In: 20th International Conference of the Association Francophone d’Interaction Homme-Machine, pp. 11–15. Metz, France (2008)Google Scholar
  36. Häubl G., Trifts V.: Consumer decision making in online shopping environments: the effects of interactive decision aids. Market. Sci. 19, 4–21 (2000)CrossRefGoogle Scholar
  37. Häubl G., Dellaert B., Murray K., Trifts V.: Buyer behavior in personalized shopping environments. In: Karat, C.-M., Blom, J.O., Karat, J. (eds.) Designing Personalized User Experiences in eCommerce, pp. 207–229. Kluwer, Dordrecht (2004)CrossRefGoogle Scholar
  38. Hauser J., Urban G., Liberali G., Braun M.: Website morphing. Market. Sci. 28, 202–223 (2009)CrossRefGoogle Scholar
  39. Hayes, C., Massa, P., Avesani, P., Cunningham, P.: An on-line evaluation framework for recommender systems. In: AH’2002 Workshop on Recommendation and Personalization in E-Commerce, pp. 50–59. Málaga, Spain (2002)Google Scholar
  40. Herlocker, J., Konstan, J., Riedl, J.: Explaining Collaborative filtering recommendations. In: 2000 ACM Conference on Computer Supported Cooperative Work, pp. 241–250. Philadelphia, PA (2000)Google Scholar
  41. Herlocker J., Konstan J., Terveen L., Riedl J.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22, 5–53 (2004)CrossRefGoogle Scholar
  42. Ho S.Y., Tam K.Y.: An empirical examination of the effects of web personalization at different stages of decision making. Int. J. Human-Comput. Interact. 19, 95–112 (2005)CrossRefGoogle Scholar
  43. Hostler R., Yoon V., Guimaraes T.: Assessing the impact of internet agent on end users’ performance. Decis. Supp. Syst. 41, 313–323 (2005)CrossRefGoogle Scholar
  44. Hsu C., Lu H.: Why do people play on-line games? An extended TAM with social influences and flow experience. Inf. Manag. 41, 853–868 (2004)CrossRefGoogle Scholar
  45. Hu L.-T., Bentler P.: Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model: A Multidiscip. J. 6, 1–55 (1999)CrossRefGoogle Scholar
  46. Hu, R., Pu, P.: A comparative user study on rating vs. personality quiz based preference elicitation methods. In: 14th International Conference on Intelligent User Interfaces, pp. 367–371. Sanibel Island, FL (2009)Google Scholar
  47. Hu, R., Pu, P.: A study on user perception of personality-based recommender systems. In: 18th International Conference on User Modeling, Adaptation, and Personalization, LNCS, vol. 6075. pp. 291–302. Big Island, HI (2010)Google Scholar
  48. Hu, R., Pu., P.: Enhancing recommendation diversity with organization interfaces. In: 16th International Conference on Intelligent User Interfaces, pp. 347–350. Palo Alto, CA (2011)Google Scholar
  49. Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 263–272. Pisa, Italy (2008)Google Scholar
  50. Huang P., Lurie N.H., Mitra S.: Searching for experience on the web: an empirical examination of consumer behavior for search and experience goods. J. Market. 73(2), 55–69 (2009)CrossRefGoogle Scholar
  51. Iyengar S., Lepper M.: When choice is demotivating: can one desire too much of a good thing?. J. Pers. Soci. Psychol. 79, 995–1006 (2000)CrossRefGoogle Scholar
  52. Jones, N., Pu, P., Chen, L.: How users perceive and appraise personalized recommendations. In: 17th International Conference on User Modeling, Adaptation, and Personalization Conference, vol. 5535, pp. 461–466. Trento, Italy, LNCS (2009)Google Scholar
  53. Kamis, A., Davern, M.J.: Personalizing to product category knowledge: exploring the mediating effect of shopping tools on decision confidence. In: 37th Annual Hawaii International Conference on System Sciences. Big Island, HI (2004)Google Scholar
  54. Kaplan B., Duchon D.: Combining qualitative and quantitative methods in information systems research: a case study. Mis Q. 12, 571–586 (1988)CrossRefGoogle Scholar
  55. Kautz H., Selman B., Shah M.: Referral Web: combining social networks and collaborative filtering. Commun. ACM 40, 63–65 (1997)CrossRefGoogle Scholar
  56. Knijnenburg, B.P., Reijmer, N.J.M., Willemsen, M.C.: Each to his own: how different users call for different interaction methods in recommender systems. In: 5th ACM Conference on Recommender Systems, pp.141–148. Chicago, IL (2011)Google Scholar
  57. Knijnenburg, B.P., Willemsen, M.C.: Understanding the effect of adaptive preference elicitation methods on user satisfaction of a recommender system. In: Third ACM Conference on Recommender Systems, pp. 381–384. New York, NY (2009)Google Scholar
  58. Knijnenburg, B.P., Willemsen, M.C.: The effect of preference elicitation methods on the user experience of a recommender system. In: 28th International Conference on Human Factors in Computing Systems, pp. 3457–3462. Atlanta, GA (2010)Google Scholar
  59. Knijnenburg, B.P., Willemsen, M.C., Kobsa, A.: A pragmatic procedure to support the user-centric evaluation of recommender systems. In: 5th ACM Conference on Recommender Systems, pp. 321–324. Chicago, IL (2011a)Google Scholar
  60. Knijnenburg, B.P., Meesters, L., Marrow, P., Bouwhuis, D.: User-centric evaluation framework for multimedia recommender systems. In: First International Conference on User Centric Media, LNICST, vol. 40, pp. 366–369. Venice, Spain (2010a)Google Scholar
  61. Knijnenburg, B.P., Willemsen, M.C., Hirtbach, S.: Receiving recommendations and providing feedback: The user-experience of a recommender system. In: 11th International Conference on Electronic Commerce and Web Technologies, LNBIP, vol. 61, pp. 207–216. Bilbao, Spain (2010b)Google Scholar
  62. Kobsa, A. Teltzrow, M.: Contextualized communication of privacy practices and personalization benefits: impacts on users’ data sharing and purchase behavior. In: Workshop on Privacy Enhancing Technologies, LNCS, vol. 3424, pp. 329–343. Toronto, Canada (2005)Google Scholar
  63. Komiak S.Y.X., Benbasat I.: The effects of personalization and familiarity on trust and adoption of recommendation agents. Mis Q. 30, 941–960 (2006)Google Scholar
  64. Konstan J.A., Riedl J.: Recommender Systems: From Algorithms to User Experience. User Model. User-Adap. Inter. 22(1–2), 101–123 (2012)Google Scholar
  65. Koren Y., Bell R., Volinsky C.: Matrix factorization techniques for recommender systems. IEEE Comput. 42, 30–37 (2009)CrossRefGoogle Scholar
  66. Koren Y.: Factor in the neighbors: Scalable and accurate collaborative filtering. Trans. Knowl. Discov. Data 4, 1–24 (2010)CrossRefGoogle Scholar
  67. Koufaris M.: Applying the technology acceptance model and flow theory to online consumer behavior. Inf. Syst. Res. 13, 205–223 (2003)CrossRefGoogle Scholar
  68. Kramer T.: The effect of measurement task transparency on preference construction and evaluations of personalized recommendations. J. Market. Res. 44, 224–233 (2007)CrossRefGoogle Scholar
  69. Krishnan, V., Narayanashetty, P., Nathan, M., Davies, R., Konstan, J.: Who predicts better?: Results from an online study comparing humans and an online recommender system. In: 2008 ACM Conference on Recommender Systems, pp. 211–218. Lausanne, Switzerland (2008)Google Scholar
  70. Law, E., Roto, V., Hassenzahl, M., Vermeeren, A., Kort, J.: Understanding, scoping and defining user experience: a survey approach. In: 27th International Conference on Human Factors in Computing Systems, pp. 719–728. Boston, MA (2009)Google Scholar
  71. Lynch J.G. Jr, Ariely D.: Wine online: search cost and competition on price, quality, and distribution. Market. Sci. 19, 83–103 (2000)CrossRefGoogle Scholar
  72. Marrow, P., Hanbidge, R., Rendle, S., Wartena, C., Freudenthaler, C.: MyMedia: producing an extensible framework for recommendation. In: Networked Electronic Media Summit 2009. Saint-Malo, France (2009)Google Scholar
  73. McNamara N., Kirakowski J.: Functionality, usability, and user experience: three areas of concern. ACM Interact. 13, 26–28 (2006)CrossRefGoogle Scholar
  74. McNee, S., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S., Rashid, A., Konstan, J., Riedl, J.: On the recommending of citations for research papers. In: 2002 ACM Conference on Computer Supported Cooperative Work, pp. 116–125. New Orleans, LA (2002)Google Scholar
  75. McNee, S., Riedl, J., Konstan, J.: Being accurate is not enough: how accuracy metrics have hurt recommender systems. In: 24th International Conference Human factors in computing systems, pp. 1097–1101. Montréal, Canada (2006a)Google Scholar
  76. McNee, S., Riedl, J., Konstan, J.: Making recommendations better: an analytic model for human-recommender interaction. In: 24th International Conference Human factors in computing systems, pp. 1103–1108. Montréal, Canada (2006b)Google Scholar
  77. Meesters, L., Marrow, P., Knijnenburg, B.P., Bouwhuis, D., Glancy, M.: MyMedia Deliverable 1.5 End-user Recommendation Evaluation Metrics (2008) http://www.mymediaproject.org/Publications/WP1/MyMedia_D1.5.pdf
  78. Murray K., Häubl G.: Interactive consumer decision aids. In: Wierenga, B. (ed.) Handbook of Marketing Decision Models, pp. 55–77. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  79. Murray K., Häubl G.: Personalization without interrogation: towards more effective interactions between consumers and feature-based recommendation agents. J. Interact. Market. 23, 138–146 (2009)CrossRefGoogle Scholar
  80. Muthen B.: A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika 49, 115–132 (1984)CrossRefGoogle Scholar
  81. Nelson P.: Information and consumer behavior. J. Polit. Econ. 78, 311–329 (1970)CrossRefGoogle Scholar
  82. Netemeyer R., Bentler P.: Structural equations modeling and statements regarding causality. J. Consum. Psychol. 10, 83–85 (2001)CrossRefGoogle Scholar
  83. Ochi P., Rao S., Takayama L., Nass C.: Predictors of user perceptions of web recommender systems: how the basis for generating experience and search product recommendations affects user responses. Int. J. Hum.-Comput. Stud. 68, 472–482 (2010)CrossRefGoogle Scholar
  84. Ozok A.A., Fan Q., Norcio A.F.: Design guidelines for e?ective recommender system interfaces based on a usability criteria conceptual model: results from a college student population. Behav. Inf. Technol. 29, 57–83 (2010)CrossRefGoogle Scholar
  85. Paramythis A., Weibelzahl S., Masthoff J.: Layered evaluation of interactive adaptive systems: framework and formative methods. User Model. User-Adap. Inter. 20, 383–453 (2010)CrossRefGoogle Scholar
  86. Pathak B., Garfinkel R., Gopal R.D., Venkatesan R., Yin F.: Empirical analysis of the impact of recommender systems on sales. J. Manag. Inf. Syst. 27, 159–188 (2010)CrossRefGoogle Scholar
  87. Pedersen P.: Behavioral effects of using software agents for product and merchant brokering: an experimental study of consumer decision-making. Int. J. Electron. Commer. 5, 125–141 (2000)Google Scholar
  88. Pommeranz, A., Broekens, J., Wiggers, P., Brinkman, W.-P., Jonker, C. M.: Designing interfaces for explicit preference elicitation: a user-centered investigation of preference representation and elicitation process. User Model. User-Adap. Inter. 22 (2012). doi:10.1007/s11257-011-9116-6
  89. Preece J., Rogers Y., Sharp H.: Interaction Design: Beyond Human-Computer Interaction. Wiley, New York (2002)Google Scholar
  90. Pu, P., Chen, L.: Trust building with explanation interfaces. In: 11th International Conference on Intelligent User Interfaces, pp. 93–100. Sydney, Australia (2006)Google Scholar
  91. Pu P., Chen L.: Trust-inspiring explanation interfaces for recommender systems. Knowl.-Based Syst. 20, 542–556 (2007)MathSciNetCrossRefGoogle Scholar
  92. Pu, P., Chen, L.: A user-centric evaluation framework of recommender systems. In: ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces, pp. 14–21. Barcelona, Spain (2010)Google Scholar
  93. Pu, P., Chen, L., Hu, R.: Evaluating recommender systems from the user’s perspective: survey of the state of the art. User Model. User-Adap. Inter. 22 (2012). doi:10.1007/s11257-011-9115-7
  94. Pu P., Chen L., Kumar P.: Evaluating product search and recommender systems for E-commerce environments. Electron. Commer. Res. 8, 1–27 (2008)MATHCrossRefGoogle Scholar
  95. Rendle, S., Schmidt-Thieme, L.: Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In: 2008 ACM Conference on Recommender systems, pp. 251–258. Lausanne, Switzerland (2008)Google Scholar
  96. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp. 452–461. Montreal, Canada (2009)Google Scholar
  97. Resnick P., Varian H.: Recommender systems. Commun. ACM 40, 56–58 (1997)CrossRefGoogle Scholar
  98. Scheibehenne B., Greifeneder R., Todd P.: Can there ever be too many options? A meta-analytic review of choice overload. J. Consum. Res. 37, 409–425 (2010)CrossRefGoogle Scholar
  99. Schwartz B.: The Paradox of Choice: Why More Is Less. HarperCollins, New York (2004)Google Scholar
  100. Senecal S., Nantel J.: The influence of online product recommendations on consumers’ online choices. J. Retail. 80, 159–169 (2004)CrossRefGoogle Scholar
  101. Sheeran P.: Intention-behavior relations: a conceptual and empirical review. Eur. Rev. Soc Psychol. 12, 1–36 (2002)CrossRefGoogle Scholar
  102. Simonson I., Tversky A.: Choice in context: tradeoff contrast and extremeness aversion. J. Market. Res. 29, 281–295 (1992)CrossRefGoogle Scholar
  103. Spiekermann, S., Grossklags, J., Berendt, B.: E-privacy in 2nd generation E-commerce: privacy preferences versus actual behavior. In: 3rd ACM Conference on Electronic Commerce, pp. 38–47. Tampa, FL (2001)Google Scholar
  104. Stolze, M., Nart, F.: Well-integrated needs-oriented recommender components regarded as helpful. In: 22nd International Conference on Human Factors in Computing Systems, p. 1571. Vienna, Austria (2004)Google Scholar
  105. Tam K.Y., Ho S.Y.: Web personalization as a Persuasion strategy: an elaboration likelihood model perspective. Inf. Syst. Res. 16, 271–291 (2005)CrossRefGoogle Scholar
  106. Teltzrow M., Kobsa A.: Impacts of user privacy preferences on personalized systems. Hum.-Comput. Interact. Ser. 5, 315–332 (2004)CrossRefGoogle Scholar
  107. Tintarev, N., Masthoff, J.: Evaluating the Effectiveness of Explanations for Recommender Systems. User Model. User-Adap. Inter. 22 (2012). doi:10.1007/s11257-011-9117-5
  108. Torres, R., McNee, S., Abel, M., Konstan, J., Riedl, J.: Enhancing digital libraries with TechLens+. In: 4th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 228–236. Tucson, AZ (2004)Google Scholar
  109. Van Velsen L., VanDer Geest T., Klaassen R., Steehouder M.: User-centered evaluation of adaptive and adaptable systems: a literature review. Knowl. Eng. Rev. 23, 261–281 (2008)CrossRefGoogle Scholar
  110. Venkatesh V., Morris M., Davis G., Davis F.: User acceptance of information technology: toward a unified view. Mis Q. 27, 425–478 (2003)Google Scholar
  111. Viappiani P., Faltings B., Pu P.: Preference-based search using example-critiquing with suggestions. J. Artif. Intell. Res. 27, 465–503 (2006)MATHGoogle Scholar
  112. Viappiani P., Pu P., Faltings B.: Preference-based search with adaptive recommendations. AI Commun. 21, 155–175 (2008)MathSciNetMATHGoogle Scholar
  113. Vijayasarathy L.R., Jones J.M.: Do internet shopping aids make a difference? an empirical investigation. Electron. Markets 11, 75–83 (2001)CrossRefGoogle Scholar
  114. Wang W., Benbasat I.: Recommendation agents for electronic commerce: effects of explanation facilities on trusting beliefs. J. Manag. Inf. Syst. 23, 217–246 (2007)CrossRefGoogle Scholar
  115. Willemsen, M.C., Knijnenburg, B.P., Graus, M.P., Velter-Bremmers, L.C.M., Fu, K.: Using latent features diversification to reduce choice difficulty in recommendation lists. In: RecSys’11 Workshop on Human Decision Making in Recommender Systems, CEUR-WS, vol. 811, pp. 14–20. Chicago, IL (2011)Google Scholar
  116. Xiao B., Benbasat I.: ECommerce product recommendation agents: use, characteristics, and impact. Mis Q. 31, 137–209 (2007)Google Scholar
  117. Yu J., Ha I., Choi M., Rho J.: Extending the TAM for a t-commerce. Inf. Manag. 42, 965–976 (2005)CrossRefGoogle Scholar
  118. Ziegler, C., McNee, S., Konstan, J., Lausen, G.: Improving recommendation lists through topic diversification. In: 14th international World Wide Web Conference, pp. 22–32. Chiba, Japan (2005)Google Scholar
  119. Zins, A., Bauernfeind, U.: Explaining online purchase planning experiences with recommender websites. In: International Conference on Information and Communication Technologies in Tourism, pp. 137–148. Innsbruck, Austria (2005)Google Scholar

Copyright information

© The Author(s) 2012

Authors and Affiliations

  • Bart P. Knijnenburg
    • 1
    • 2
  • Martijn C. Willemsen
    • 2
  • Zeno Gantner
    • 3
  • Hakan Soncu
    • 4
  • Chris Newell
    • 5
  1. 1.Department of Informatics, Donald Bren School of Information and Computer SciencesUniversity of CaliforniaIrvineUSA
  2. 2.Human-Technology Interaction Group, School of Innovation SciencesEindhoven University of Technology (TU/e)EindhovenThe Netherlands
  3. 3.Information Systems and Machine Learning Lab (ISMLL)University of HildesheimHildesheimGermany
  4. 4.European Microsoft Innovation Center GmbHAachenGermany
  5. 5.BBC Research & Development, Centre HouseLondonUK

Personalised recommendations