Skip to main content
SpringerLink
Log in
Book cover

Recommender Systems Handbook pp 479–510Cite as

Designing and Evaluating Explanations for Recommender Systems

  • Chapter
  • First Online:

Abstract

This chapter gives an overview of the area of explanations in recommender systems. We approach the literature from the angle of evaluation: that is, we are interested in what makes an explanation “good”, and suggest guidelines as how to best evaluate this. We identify seven benefits that explanations may contribute to a recommender system, and relate them to criteria used in evaluations of explanations in existing systems, and how these relate to evaluations with live recommender systems. We also discuss how explanations can be affected by how recommendations are presented, and the role the interaction with the recommender system plays w.r.t. explanations. Finally, we describe a number of explanation styles, and how they may be related to the underlying algorithms. Examples of explanations in existing systems are mentioned throughout.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   179.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Pandora (2006). http://www.pandora.com

  2. Movielens dataset (2009). http://www.grouplens.org/node/73

  3. Netflix dataset (2009). http://www.netflixprize.com/

  4. Newsmap (2009). http://www.marumushi.com/apps/newsmap/index.cfm

  5. Nutking (2010). http://nutking.ectrldev.com/nutking/jsp/language. do?action=english

  6. Adrissono, L., Goy, A., Petrone, G., Segnan, M., Torasso, P.: Intrigue: Personalized recommendation of tourist attractions for desktop and handheld devices. Applied Artificial Intelligence 17, 687–714 (2003)

    Article  Google Scholar 

  7. Ahn, J.W., Brusilovsky, P., Grady, J., He, D., Syn, S.Y.: Open user profiles for adaptive news systems: help or harm? In: WWW ’07: Proceedings of the 16th international conference on World Wide Web, pp. 11–20. ACM Press, New York, NY, USA (2007).

    Chapter  Google Scholar 

  8. Andersen, S.K., Olesen, K.G., Jensen, F.V.: HUGIN—a shell for building Bayesian belief universes for expert systems. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1990)

    Google Scholar 

  9. Bederson, B., Shneiderman, B., Wattenberg, M.: Ordered and quantum treemaps: Making effective use of 2d space to display hierarchies. ACM Transactions on Graphics 21(4), 833–854. (2002)

    Article  Google Scholar 

  10. Bennett, S.W., Scott., A.C.: The Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project, chap. 19 - Specialized Explanations for Dosage Selection, pp. 363–370. Addison-Wesley Publishing Company (1985)

    Google Scholar 

  11. Bilgic, M., Mooney, R.J.: Explaining recommendations: Satisfaction vs. promotion. In: Proceedings of the Wokshop Beyond Personalization, in conjunction with the International Conference on Intelligent User Interfaces, pp. 13–18 (2005)

    Google Scholar 

  12. Billsus, D., Pazzani, M.J.: A personal news agent that talks, learns, and explains. In: Proceedings of the Third International Conference on Autonomous Agents, pp. 268–275 (1999)

    Google Scholar 

  13. Burke, R.: Hybrid recommender systems: Survey and experiments. User Modeling and User- Adapted Interaction 12(4), 331–370 (2002)

    Article  MATH  Google Scholar 

  14. Burke, R.D., Hammond, K.J., Young, B.C.: Knowledge-based navigation of complex information spaces. In: AAAI/IAAI, Vol. 1, pp. 462–468 (1996)

    Google Scholar 

  15. Carenini, G., Mittal, V., Moore, J.: Generating patient-specific interactive natural language explanations. Proc Annu Symp Comput Appl Med Care pp. 5–9 (1994)

    Google Scholar 

  16. Chen, L., Pu, P.: Trust building in recommender agents. In: WPRSIUI in conjunction with Intelligent User Interfaces, pp. 93–100 (2002)

    Google Scholar 

  17. Chen, L., Pu, P.: Hybrid critiquing-based recommender systems. In: Intelligent User Interfaces, pp. 22–31 (2007)

    Google Scholar 

  18. Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J.: Is seeing believing?: how recommender system interfaces affect users’ opinions. In: CHI, Recommender systems and social computing, vol. 1, pp. 585–592 (2003).

    Google Scholar 

  19. Cramer, H., Evers, V., Someren, M.V., Ramlal, S., Rutledge, L., Stash, N., Aroyo, L., Wielinga, B.: The effects of transparency on perceived and actual competence of a contentbased recommender. In: Semantic Web User Interaction Workshop, CHI (2008)

    Google Scholar 

  20. Cramer, H.S.M., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., Wielinga, B.J.: The effects of transparency on trust in and acceptance of a content-based art recommender. User Model. User-Adapt. Interact 18(5), 455–496 (2008).

    Article  Google Scholar 

  21. Czarkowski, M.: A scrutable adaptive hypertext. Ph.D. thesis, University of Sydney (2006)

    Google Scholar 

  22. Doyle, D., Tsymbal, A., Cunningham, P.: A review of explanation and explanation in casebased reasoning. Tech. rep., Department of Computer Science, Trinity College, Dublin (2003)

    Google Scholar 

  23. Felfernig, A., Gula, B.: Consumer behavior in the interaction with knowledge-based recommender applications. In: ECAI 2006Workshop on Recommender Systems, pp. 37–41 (2006)

    Google Scholar 

  24. Fogg, B., Marshall, J., Kameda, T., Solomon, J., Rangnekar, A., Boyd, J., Brown, B.: Web credibility research: A method for online experiments and early study results. In: CHI 2001, pp. 295–296 (2001)

    Google Scholar 

  25. Fogg, B.J., Soohoo, C., Danielson, D.R., Marable, L., Stanford, J., Tauber, E.R.: How do users evaluate the credibility of web sites?: a study with over 2,500 participants. In: Proceedings of DUX’03: Designing for User Experiences, no. 15 in Focusing on user-to-product relationships, pp. 1–15 (2003). URL http://doi.acm.org/10.1145/997078.997097

  26. Ginty, L.M., Smyth, B.: Comparison-based recommendation. Lecture Notes in Computer Science 2416, 731–737 (2002).

    Google Scholar 

  27. Hance, E., Buchanan, B.: Rule-based expert systems: the MYCIN experiments of the Stanford Heuristic Programming Project. Addison-Wesley (1984)

    Google Scholar 

  28. Häubl, G., Trifts, V.: Consumer decision making in online shopping environments: The effects of interactive decision aids. Marketing Science 19, 4–21 (2000)

    Article  Google Scholar 

  29. Herlocker, J.L., Konstan, J.A., Riedl, J.: Explaining collaborative filtering recommendations. In: ACM conference on Computer supported cooperative work, pp. 241–250 (2000)

    Google Scholar 

  30. Hingston, M.: User friendly recommender systems. Master’s thesis, Sydney University (2006)

    Google Scholar 

  31. Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: ICDM (2008)

    Google Scholar 

  32. Hunt, J.E., Price, C.J.: Explaining qualitative diagnosis. Engineering Applications of Artificial Intelligence 1(3), Pages 161–169 (1988)

    Article  Google Scholar 

  33. Khan, O.Z., Poupart, P., Black, J.P.: Minimal sufficient explanations for mdps. In: Workshop on Explanation-Aware Computing associated with IJCAI (2009)

    Google Scholar 

  34. Krulwich, B.: The infofinder agent: Learning user interests through heuristic phrase extraction. IEEE Intelligent Systems 12, 22–27 (1997)

    Google Scholar 

  35. Lacave, C., Diéz, F.J.: A review of explanation methods for bayesian networks. The Knowledge Engineering Review 17:2, 107–127 (2002)

    Google Scholar 

  36. Lacave, C., Diéz, F.J.: A review of explanation methods for heuristic expert systems. The Knowledge Engineering Review 17:2, 107–127 (2004)

    Google Scholar 

  37. Lewis, C., Rieman, J.: Task-centered user interface design: a practical introduction. University of Colorado (1994)

    Google Scholar 

  38. Lopez-Suarez, A., Kamel, M.: Dykor: a method for generating the content of explanations in knowledge systems. Knowledge-based Systems 7(3), 177–188 (1994)

    Article  Google Scholar 

  39. McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: Thinking positively - explanatory feedback for conversational recommender systems. In: Proceedings of the European Conference on Case-Based Reasoning (ECCBR-04) Explanation Workshop,, pp. 115–124 (2004)

    Google Scholar 

  40. McNee, S., Lam S.K.and Guetzlaff, C., Konstan J.A.and Riedl, J.: Confidence displays and training in recommender systems. In: INTERACT IFIP TC13 International Conference on Human-Computer Interaction, pp. 176–183 (2003)

    Google Scholar 

  41. McNee, S.M., Lam, S.K., Konstan, J.A., Riedl, J.: Interfaces for eliciting new user preferences in recommender systems. User Modeling pp. pp. 178–187 (2003)

    Google Scholar 

  42. McNee, S.M., Riedl, J., Konstan, J.A.: Being accurate is not enough: How accuracy metrics have hurt recommender systems. In: Extended Abstracts of the 2006 ACM Conference on Human Factors in Computing Systems (CHI 2006) (2006)

    Google Scholar 

  43. McSherry, D.: Explanation in recommender systems. Artificial Intelligence Review 24(2), 179 – 197 (2005)

    Article  MATH  Google Scholar 

  44. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: ACM CHI’90, pp. 249–256 (1990)

    Google Scholar 

  45. Ohanian, R.: Construction and validation of a scale to measure celebrity endorsers’ perceived expertise, trustworthiness, and attractiveness. Journal of Advertising 19:3, 39–52 (1990)

    Google Scholar 

  46. O’Sullivan, D., Smyth, B., Wilson, D.C., McDonald, K., Smeaton, A.: Improving the quality of the personalized electronic program guide. User Modeling and User-Adapted Interaction 14, pp. 5–36 (2004)

    Article  Google Scholar 

  47. Paramythis, A., Totter, A., Stephanidis, C.: A modular approach to the evaluation of adaptive user interfaces. In: S.Weibelzahl, D.N. Chin, G.Weber (eds.) Evaluation of Adaptive Systems in conjunction with UM’01, pp. 9–24 (2001)

    Google Scholar 

  48. Pazzani, M.J.: A framework for collaborative, content-based and demographic filtering. Artificial Intelligence Review 13, 393–408 (1999)

    Article  Google Scholar 

  49. Pu, P., Chen, L.: Trust building with explanation interfaces. In: IUI’06, Recommendations I, pp. 93–100 (2006).

    Google Scholar 

  50. Pu, P., Chen, L.: Trust-inspiring explanation interfaces for recommender systems. Knowledge-based Systems 20, 542–556 (2007)

    Article  MathSciNet  Google Scholar 

  51. Rafter, R., Smyth, B.: Conversational collaborative recommendation - an experimental analysis. Artif. Intell. Rev 24(3-4), 301–318 (2005). URL http://dx.doi.org/10.1007/s10462-005-9004-8

  52. Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Dynamic critiquing. In: P. Funk, P.A. González-Calero (eds.) ECCBR, Lecture Notes in Computer Science, vol. 3155, pp. 763–777. Springer (2004)

    Google Scholar 

  53. Roth-Berghofer, T., Schulz, S., Leake, D.B., Bahls, D.:Workshop on explanation-aware computing. In: ECAI (2008)

    Google Scholar 

  54. Roth-Berghofer, T., Tintarev, N., Leake, D.B.: Workshop on explanation-aware computing. In: IJCAI (2009)

    Google Scholar 

  55. Sinha, R., Swearingen, K.: The role of transparency in recommender systems. In: Conference on Human Factors in Computing Systems, pp. 830–831 (2002)

    Google Scholar 

  56. Sørmo, F., Cassens, J., Aamodt, A.: Explanation in case-based reasoning perspectives and goals. Artificial Intelligence Review 24(2), 109 – 143 (2005)

    Article  Google Scholar 

  57. Swearingen, K., Sinha, R.: Interaction design for recommender systems. In: Designing Interactive Systems, pp. 25–28 (2002).

    Google Scholar 

  58. Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Justified recommendations based on content and rating data. In:WebKDDWorkshop onWeb Mining andWeb Usage Analysis (2008)

    Google Scholar 

  59. Thompson, C.A., G¨oker, M.H., Langley, P.: A personalized system for conversational recommendations. J. Artif. Intell. Res. (JAIR) 21, 393–428 (2004).

    Google Scholar 

  60. Tintarev, N., Masthoff, J.: Over- and underestimation in different product domains. In:Workshop on Recommender Systems associated with ECAI (2008)

    Google Scholar 

  61. Tintarev, N., Masthoff, J.: Personalizing movie explanations using commercial meta-data. In: Adaptive Hypermedia (2008)

    Google Scholar 

  62. Vig, J., Sen, S., Riedl, J.: Tagsplanations: Explaining recommendations using tags. In: Intelligent User Interfaces (2009)

    Google Scholar 

  63. Wang, W., Benbasat, I.: Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. Journal of Managment Information Systems 23, 217–246 (2007)

    Article  Google Scholar 

  64. Wärnestål, P.: Modeling a dialogue strategy for personalized movie recommendations. In: Beyond Personalization Workshop, pp. 77–82 (2005)

    Google Scholar 

  65. Wärnestål, P.: User evaluation of a conversational recommender system. In: Proceedings of the 4th Workshop on Knowledge and Reasoning in Practical Dialogue Systems, pp. 32–39 (2005)

    Google Scholar 

  66. Wick, M.R., Thompson, W.B.: Reconstructive expert system explanation. Artif. Intell. 54(1- 2), 33–70 (1992).

    Article  Google Scholar 

  67. Ye, L., Johnson, P., Ye, L.R., Johnson, P.E.: The impact of explanation facilities on user acceptance of expert systems advice. MIS Quarterly 19(2), 157–172 (1995).

    Article  Google Scholar 

  68. Yee, K.P., Swearingen, K., Li, K., Hearst, M.: Faceted metadata for image search and browsing. In: ACM Conference on Computer-Human Interaction (2003)

    Google Scholar 

  69. Zaslow, J.: Oh no! My TiVo thinks I’m gay (2002).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Aberdeen, Aberdeen, U.K

    Nava Tintarev & Judith Masthoff

Authors
  1. Nava Tintarev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Judith Masthoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nava Tintarev .

Editor information

Editors and Affiliations

  1. , Faculty of Computer Science, Free University of Bozen-Bolzano, Piazza Domenicani 3, Bolzano, 39100, Italy

    Francesco Ricci

  2. , Dept. Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, 84105, Israel

    Lior Rokach

  3. Dept. Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Bracha Shapira

  4. School of Communication,, Information & Library Studies, Rutgers University, Huntington Street 4, New Brunswick, 08901-1071, New Jersey, USA

    Paul B. Kantor

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Tintarev, N., Masthoff, J. (2011). Designing and Evaluating Explanations for Recommender Systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P. (eds) Recommender Systems Handbook. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-85820-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-85820-3_15

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-85819-7

  • Online ISBN: 978-0-387-85820-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation