The effects of transparency on trust in and acceptance of a content-based art recommender

  • Henriette Cramer
  • Vanessa Evers
  • Satyan Ramlal
  • Maarten van Someren
  • Lloyd Rutledge
  • Natalia Stash
  • Lora Aroyo
  • Bob Wielinga
Open Access
Original Paper


The increasing availability of (digital) cultural heritage artefacts offers great potential for increased access to art content, but also necessitates tools to help users deal with such abundance of information. User-adaptive art recommender systems aim to present their users with art content tailored to their interests. These systems try to adapt to the user based on feedback from the user on which artworks he or she finds interesting. Users need to be able to depend on the system to competently adapt to their feedback and find the artworks that are most interesting to them. This paper investigates the influence of transparency on user trust in and acceptance of content-based recommender systems. A between-subject experiment (N = 60) evaluated interaction with three versions of a content-based art recommender in the cultural heritage domain. This recommender system provides users with artworks that are of interest to them, based on their ratings of other artworks. Version 1 was not transparent, version 2 explained to the user why a recommendation had been made and version 3 showed a rating of how certain the system was that a recommendation would be of interest to the user. Results show that explaining to the user why a recommendation was made increased acceptance of the recommendations. Trust in the system itself was not improved by transparency. Showing how certain the system was of a recommendation did not influence trust and acceptance. A number of guidelines for design of recommender systems in the cultural heritage domain have been derived from the study’s results.


User-adaptivity Human-computer interaction Recommender systems Transparency Trust Acceptance Cultural heritage 



We would like to thank all participants in this study and colleagues and anonymous reviewers for their helpful comments. This research is funded by the Interactive Collaborative Information Systems (ICIS) project nr: BSIK03024, by the Dutch Ministry of Economical Affairs under contract to the Human-Computer Studies Laboratory of the University of Amsterdam. The CHIP system is developed by the CHIP (Cultural Heritage Information Personalization— project, part of the CATCH (Continuous Access To Cultural Heritage) program funded by the NWO (Netherlands Organisation for Scientific Research). The Rijksmuseum Amsterdam gave permission for use of its artwork images.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.


  1. Alpert S.R., Karat J., Karat C., Brodie C. and Vergo J.G. (2003). User attitudes regarding a user-adaptive e-Commerce web site. User. Model. User-adapt. Interact. 13(4): 373–396 CrossRefGoogle Scholar
  2. Aroyo L., Wang Y., Brussee R., Gorgels P., Rutledge L. and Stash N. (2007). Personalised Museum Experience: The Rijksmuseum Use Case. The International Museums and the Web Conference, San Francisco, USA Google Scholar
  3. Bartneck C. (2001). How convincing is Mr. Data’s smile: affective expressions of machines. User Model. User-adapt. Interact. 11(4): 279–295 zbMATHCrossRefGoogle Scholar
  4. Benyon D. (1993). Adaptive systems: a solution to usability problems. User Model. User-adapt. Interact. 3(1): 65–87 CrossRefGoogle Scholar
  5. Bilgic, M., Mooney, R.J.: Explaining Recommendations: Satisfaction vs. Promotion. Beyond Personalization Workshop, The International Conference on Intelligent User Interfaces, pp. 470–474. San Diego, California, USA (2005)Google Scholar
  6. Bowen, J.P., Filippini-Fantoni, S.: Personalization and the Web from a Museum Perspective. The International Museums and the Web Conference, pp. 63–78 (2004)Google Scholar
  7. Briggs, P., Simpson, B., De Angeli, A.: Trust and personalisation: a reciprocal relationship? In: Karat, C.-M., Blom, J., Karat, J. (eds.) Designing Personalised User Experiences for e-Commerce, pp. 39–55. Kluwer (2004)Google Scholar
  8. Brusilovsky P. (1996). Methods and techniques of adaptive hypermedia. User Model. User-adapt. Interact. 6(2–3): 87–129 CrossRefGoogle Scholar
  9. Burke R. (2002). Hybrid recommender systems: survey and experiments. User Model. User-adapt. Interact. 12(4): 331–370 zbMATHCrossRefGoogle Scholar
  10. Carmagnola, R., Cena, F., Comsole, L., Cortassa, O., Gena, C., Goy, A., Torre, I.: Tag-based User Models for Social Multi-Device Adaptive Guides. This issue (2008)Google Scholar
  11. Carmichael, D., Kay, J., Kummerfeld, B., Niu, W.: Why did you show/tell/hide that? The need for scrutability in ubiquitous personalisation. ECHISE Workshop Exploiting Context Histories in Smart Environments at UbiComp, Irvine, CA, USA (2006)Google Scholar
  12. Castelfranchi C. and Falcone R. (2000). Trust and control: a dialectic link. Appl. Artif. Intell. J. 14(8): 799–823 CrossRefGoogle Scholar
  13. Cheverst K., Byun H.E., Fitton D., Sas C., Kray C. and Villar N. (2005). Exploring issues of user model transparency and proactive behaviour in an office environment control system. User Model. User-adapt. Interact. 15(3–4): 235–273 CrossRefGoogle Scholar
  14. Cortellessa, G., Giuliani, M.V., Scopelliti, M., Cesta, A.: Key issues in interactive roblem solving: an empirical investigation on users attitude. Interact, pp. 657–670. Rome, Italy (2005)Google Scholar
  15. Cramer, H.S.M., Evers, V., Van Someren, M., Wielinga, B., Besselink, S., Rutledge, L., Stash, N., Aroyo, L.: User Interaction with User-Adaptive Information Filters, pp. 324–333. HCI International, Beijing, China (2007)Google Scholar
  16. Cramer, H.S.M., Wielinga, B.J., Evers, V., Rutledge, L., Stash, N.: The Effects of Transparency on Perceived and Actual Competence of a Content-Based Recommender. Semantic Web User Interaction workshop at CHI, Florence, Italy (2008)Google Scholar
  17. Damiano, R., Gena., C., Lombardo, V., Nunnari, F., Pizzo, A.: A stroll with carletto. Adaptation in Drama-Based Tours with Virtual Characters. This issue (2008)Google Scholar
  18. Davis F.D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quart. 13(2): 318–340 Google Scholar
  19. Dzindolet M., Peterson S.A., Pomranky R.A., Pierce L.G. and Beck H.P. (2003). The role of trust in automation reliance. Int. J. Human-Comput. Stud. 58(6): 697–718 CrossRefGoogle Scholar
  20. Fog g, B.J.: Prominence-interpretation theory: Explaining how people assess credibility online. Conference on Human Factors in Computing Systems CHI, PP. 722–723 (2003).Google Scholar
  21. Fogg, B.J., Tseng, H.: The Elements of Computer Credibility. Conference on Human Factors in Computing Systems CHI, pp. 80–87 (1999)Google Scholar
  22. Gefen D., Karahanna E. and Straub D.W. (2003). Trust and TAM in online shopping: an integrated model. MIS Quart. 27(1): 51–90 Google Scholar
  23. Goren-Bar D., Graziola I., Pianesi F. and Zancanaro M. (2006). The influence of personality factors on visitor attitudes towards adaptivity dimensions for mobile museum guides. User Model. User-adapt. Interact. 16(1): 31–62 CrossRefGoogle Scholar
  24. Gregor S. and Benbasat I. (1999). Explanations from intelligent systems: theoretical foundations and implications for ractice. MIS Quart. 23(4): 497–530 CrossRefGoogle Scholar
  25. Hanani U., Shapira B. and Shoval P. (2001). Information filtering: overview of issues, research and systems. User Model. User-adapt. Interact. 11(3): 203–259 zbMATHCrossRefGoogle Scholar
  26. Herlocker, J.L., Konstan, J.A., Riedl, J.T.: Explaining Collaborative Filtering Recommendations. Conference on Computer Supported Cooperative Work CSCW, pp. 241–250. Philadelphia, Pennsylvania, USA, (2000)Google Scholar
  27. Herlocker J.L., Konstan J.A., Terveen L.G. and Riedl J.T. (2004). Evaluating collaborative filtering recommender systems. ACM Trans. Inform. Syst. 22(1): 5–53 CrossRefGoogle Scholar
  28. Höök, K.: Evaluating the Utility and Usability of an Adaptive Hypermedia System. International Conference on Intelligent User Interfaces IUI, pp. 179–186. ACM, Orlando, Florida, USA (1997)Google Scholar
  29. Höök K. (2000). Steps to take before intelligent interfaces become real. Interact. Comput. 12(4): 409–426 CrossRefGoogle Scholar
  30. Höök K., Karlgren J., Waern A., Dahlbck N., Jansson C.G., Karlgren K. and Lemaire B. (1996). A glass box approach to adaptive hypermedia. User Model. User-adapt. Interact. 6(2–3): 157–184 CrossRefGoogle Scholar
  31. Hyvönen E., Mäkelä E., Salminen M., Valo A., Viljanen K., Saarela S., Junnila M. and Kettula S. (2005). MuseumFinland—Finnish Museums on the semantic web. J. Web Semantics 3(2): 224–241 Google Scholar
  32. Jameson A. (2003). Adaptive interfaces and agents. In: Jacko, J. and Sears, A. (eds) The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp 305–330. Erlbaum, Mahwah, NJGoogle Scholar
  33. Jameson, A., Schwarzkopf, E.: Pros and Cons of Controllability: An Empirical Study. Adaptive Hypermedia and Adaptive Web-based Systems, pp. 193–202. Springer (2002)Google Scholar
  34. Jian J.Y., Bisantz A.M. and Drury C.G. (2000). Foundations for an empirically determined scale of trust in automated systems. Int. J. Cogn Ergonom. 4(1): 53–71 CrossRefGoogle Scholar
  35. Jiang X., Khasawneh M.T., Master R., Bowling S.R., Gramopadhye A.K., Melloy B.J. and Grimes L. (2004). Measurement of human trust in a hybrid inspection system based on signal detection theory measures. Int. J. Ind. Ergonom. 34(5): 407–419 CrossRefGoogle Scholar
  36. Jøsang, A., Lo Presti, S.: Analysing the relationship between risk and trust. International Conference on Trust Management, Oxford, UK (2004)Google Scholar
  37. Kurasaki K.S. (2000). Intercoder reliability for validating conclusions drawn from open-ended interview data. Field Methods 12(3): 179–194 Google Scholar
  38. Kay, J.: Scrutable adaptation: Because we can and must. Adaptive Hypermedia and Adaptive Web-Based Systems AH, pp. 11–19. Dublin, Ireland (2006)Google Scholar
  39. Kim, S., Alani, H., Hall, W., Lewis, P., Millard, D.E., Shadbolt, N.G., Weal, M.J.: Artequakt: Generating Tailored Biographies with Automatically Annotated Fragments from the Web. Semantic Authoring, Annotation and Knowledge Markup Workshop at ECAI, pp. 1–6. Riva del Garda, Italy (2006)Google Scholar
  40. Klopping I.M. and McKinney E. (2004). Extending the technology acceptance model and the task-technology fit model to consumer e-commerce. Inform. Technol. Learn. Perform. J. 22(1): 35–48 Google Scholar
  41. Langheinrich, M.: Privacy by Design—Principles for Privacy-Aware Ubiquitous Systems. International Symposium on Ubiquitous Computing Ubicomp, pp. 273–291. Atlanta, GA (2001)Google Scholar
  42. Lee J.D. and Moray N. (1994). Trust, self-confidence and operator’s adaptation to automation. Int. J. Hum.-Comput. Stud. 40: 153–184 CrossRefGoogle Scholar
  43. Lee J.D. and See K.A. (2004). Trust in automation: designing for appropriate reliance. Hum. Factors 42(1): 50–80 CrossRefGoogle Scholar
  44. Ma Q. and Liu L. (2004). The technology acceptance model: a meta-analysis of empirical findings. J. Organ. End User Comput. 16(1): 59–72 Google Scholar
  45. McAllister D.J. (1995). Affect and cognition-based trust as foundations for interpersonal cooperation in organizations. Acad. Manage. J. 38(1): 24–59 CrossRefMathSciNetGoogle Scholar
  46. McGuinness D.L. and Pinheiro da Silva P. (2004). Explaining answers from the semantic web: the inference web approach. Web Semantics. Sci. Serv. Agent World Wide Web 1(4): 397–413 CrossRefGoogle Scholar
  47. McNee, S.M., Lam, S.K., Guetzlaff, C., Konstan, J.A., Riedl, J.: Confidence Metrics and Displays in Recommender Systems. Interact, pp. 176–183. Zurich, Switzerland (2003)Google Scholar
  48. McSherry D. (2005). Explanation in recommender systems. Artif. Intell. Rev. 24: 179–197 zbMATHCrossRefGoogle Scholar
  49. Muir B.M. (1994). Trust in automation: part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37: 1905–1922 CrossRefGoogle Scholar
  50. Muir B.M. and Moray N. (1996). Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3): 429–460 CrossRefGoogle Scholar
  51. Ndubisi N.O., Gupta O.K. and Ndubisi G.C. (2005). The Moguls’ model of computing: integrating the moderating impact of users’ persona into the technology acceptance model. J. Glob. Inform. Technol. Manage. 8(1): 27–47 Google Scholar
  52. Nückles M., Winter A., Wittwer J., Herbert M. and Hübner S. (2006). How do experts adapt their explanations to a layperson’s knowledge in asynchronous communication? An experimental study. User Model. User-adapt. Interact. 16(2): 87–127 CrossRefGoogle Scholar
  53. Oppermann R. and Specht M. (1999). A nomadic information system for adaptive exhibition guidance. Arch. Museum Inform. 13(2): 127–138 CrossRefGoogle Scholar
  54. Parasuraman R. and Miller C. (2004). Trust and etiquette in high-criticality automated systems. Commun. ACM 47(4): 51–55 CrossRefGoogle Scholar
  55. Pavlou P.A. (2003). Consumer acceptance of electronic commerce: integrating trust and risk with the technology acceptance model. Int. J. Electron. Comm. 7(3): 101–134 Google Scholar
  56. Proctor, N., Tellis, C.: The state of the art in museum handhelds in 2003. Museums and the Web. Charlotte, NC, USA (2003)Google Scholar
  57. Picard R.W. (1997). Affective Computing. MIT Press, Cambridge, MA, USA Google Scholar
  58. Pu P. and Chen L. (2007). Trust-inspiring explanation interfaces for recommender systems. Knowl.-based Syst. J. 20: 542–556 CrossRefMathSciNetGoogle Scholar
  59. Rajaonah B., Anceaux F. and Vienne F. (2006). Trust and the use of adaptive cruise control: a study of a cut-in situation. Cogn. Technol. Work 8(2): 146–155 CrossRefGoogle Scholar
  60. Rajaonah B., Anceaux F. and Vienne F. (2006). Study of driver trust during cooperation with adaptive cruise control. Le Travail Humain 69: 99–127 CrossRefGoogle Scholar
  61. Rajaonah B., Tricot N., Anceaux F. and Millot P. (2008). Role of intervening variables in driver-ACC cooperation. Int. J. Hum. Comput. Stud. 66(3): 185–197 CrossRefGoogle Scholar
  62. Reeves B. and Nass C. (1996). The media equation: how people treat computers, television and new media like real people and places. Cambridge University Press, Cambridge, UK Google Scholar
  63. Rocchi, C., Stock, O., Zancanaro, M., Kruppa, M., Krueger, A.: The Museum Visit: Generating Seamless Personalized Presentations on Multiple Devices. International Conference on Intelligent User Interfaces, pp. 316–318. Madeira, Portugal (2004)Google Scholar
  64. Schreiber, G., Amin, A., van Assem, M., de Boer, V., Hardman, L., Hildebrand, M., Hollink, L., Huang, Z., van Kersen, J., de Niet, M., Omelayenko, B., van Ossenbruggen, J., Siebes, R., Taekema, J., Wielemaker, J., Wielinga, B.J.: Multimedian e-culture demonstrator. International Semantic Web Conference, pp. 951–958. Athens, USA (2006)Google Scholar
  65. Shimazu H. (2002). ExpertClerk: a conversational case-based reasoning tool for developing salesclerk agents in e-commerce webshops. Artif. Intell. Rev. 18: 223–244 CrossRefGoogle Scholar
  66. Shneiderman B. and Maes P. (2007). Direct manipulation vs. interface agents. Interactions 4(6): 42–61 CrossRefGoogle Scholar
  67. Simmel, G.: In: Wolff, K.H. (ed.) The Sociology of George Simmel. Free Press, New York (1964)Google Scholar
  68. Sinclair, P., Lewis, P., Martinez, K., Addis, M., Prideaux, D.: Semantic Web Integration of Cultural Heritage Sources. 15th International Conference on World Wide Web, pp. 1047–1048. New York, USA (2006)Google Scholar
  69. Sinha, R., Swearingen, K.: The role of transparency in recommender systems. Conference on Human Factors in Computing Systems, pp. 830–831. ACM Press (2002)Google Scholar
  70. Stock O., Zancanaro M., Busetta P., Callaway C., Krüger A., Kruppa M., Kuflik T., Not E. and Rocchi C. (2007). Adaptive, intelligent presentation of information for the museum visitor in PEACH. User-Model. User-adapt. Interact. 17(3): 257–304 CrossRefGoogle Scholar
  71. Tintarev, N., Masthoff, J.: Effective Explanations of Recommendations: User-Centered Design. ACM Conference on Recommender systems, pp. 153–156 (2007)Google Scholar
  72. Trant J. (2006). Exploring the potential for social tagging and folksonomy in art museums: proof of concept. New Rev. Hypermedia Multimedia 12(1): 83–105 CrossRefGoogle Scholar
  73. van der Heijden H. (2004). User acceptance of hedonic information systems. MIS Quart. 28: 695–704 Google Scholar
  74. Van Setten, M.: Supporting People in finding information: hybrid recommender systems and goal-based structuring. Telematica Instituut, The Netherlands (2005)Google Scholar
  75. Venkatesh V., Morris M.G., Davis G.B. and Davis F.D. (2003). User acceptance of information technology: toward a unified view. MIS Quart. 27(3): 425–478 Google Scholar
  76. Victor, P., Cornelis, C., De Cock, M., Pinheiro da Silva, P.: Gradual trust and distrust in recommender systems. To appear in Fuzzy Set. Syst. (2008)Google Scholar
  77. Waern A. (2004). User involvement in automatic filtering: an experimental study. User Model. User-adapt. Interact. 14(2–3): 201–237 CrossRefGoogle Scholar
  78. Wang Y. (2005). The Presentation of Media-rich Collections of Culture Heritage in the Age of Digital Reproduction. Vrije Universiteit, Amsterdam, The Netherlands Google Scholar
  79. Wang W. and Benbasat I. (2005). Trust in and adoption of online recommendation agents. J. Assoc. Inform. Syst. 6(3): 72–101 Google Scholar
  80. Wang W. and Benbasat I. (2007). Recommendation agents for electronic commerce: effects of explanation facilities on trusting beliefs. J. Manage. Inform. Syst. 23(4): 217–246 CrossRefGoogle Scholar
  81. Wang, Y., Aroyo, L., Stash, N., Rutledge, L.: Interactive User Modeling for Personalised Access to Museum Collections: The Rijksmuseum Case Study. International User Modeling Conference UM, pp. 385–389. Corfu, Greece (2007)Google Scholar
  82. Wu I.L. and Chen J.L. (2005). An extension of trust and TAM model with TPB in the initial adoption of on-line tax: an empirical study. Int. J. Hum.-Comput. Stud. 62(6): 784–808 CrossRefGoogle Scholar
  83. Wubs H. and Huysmans F. (2006). Snuffelen en Graven. Sociaal en Cultureel Planbureau, The Netherlands Google Scholar
  84. Xiao, S., Benbasat, I.: The formation of trust and distrust in recommendation agents in repeated interactions: a process-tracing analysis. The 5th international conference on Electronic commerce, pp. 287–293 (2003)Google Scholar
  85. Zimmermann, A., Lorenz, A.: LISTEN: a User-Adaptive Audio-Augmented Museum Guide. This issue (2008)Google Scholar

Copyright information

© The Author(s) 2008

Authors and Affiliations

  • Henriette Cramer
    • 1
  • Vanessa Evers
    • 1
  • Satyan Ramlal
    • 1
  • Maarten van Someren
    • 1
  • Lloyd Rutledge
    • 2
    • 3
  • Natalia Stash
    • 4
    • 5
  • Lora Aroyo
    • 4
    • 5
  • Bob Wielinga
    • 6
  1. 1.Human Computer Studies LabUniversity of AmsterdamAmsterdamThe Netherlands
  2. 2.Telematica InstituteEnschedeThe Netherlands
  3. 3.CWIAmsterdamThe Netherlands
  4. 4.Eindhoven University of TechnologyEindhovenThe Netherlands
  5. 5.VU University AmsterdamAmsterdamThe Netherlands
  6. 6.Human Computer Studies LabUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations