Skip to main content
SpringerLink
Log in

Usability Guidelines for Product Recommenders Based on Example Critiquing Research

  • Chapter
  • First Online:
Recommender Systems Handbook

Abstract

Over the past decade, our group has developed a suite of decision tools based on example critiquing to help users find their preferred products in e-commerce environments. In this chapter, we survey important usability research work relative to example critiquing and summarize the major results by deriving a set of usability guidelines. Our survey is focused on three key interaction activities between the user and the system: the initial preference elicitation process, the preference revision process, and the presentation of the systems recommendation results. To provide a basis for the derivation of the guidelines, we developed a multi-objective framework of three interacting criteria: accuracy, confidence, and effort (ACE). We use this framework to analyze our past work and provide a specific context for each guideline: when the system should maximize its ability to increase users’ decision accuracy, when to increase user confidence, and when to minimize the interaction effort for the users. Due to the general nature of this multi-criteria model, the set of guidelines that we propose can be used to ease the usability engineering process of other recommender systems, especially those used in e-commerce environments. The ACE framework presented here is also the first in the field to evaluate the performance of preference-based recommenders from a user-centric point of view.

Designers can use these guidelines for the implementation of an effective and successful product recommender.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 179.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adomavicius, G., Tuzhilin, A., Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data 542 Pearl Pu, Boi Faltings, Li Chen, Jiyong Zhang and Paolo Viappiani Engineering 17 (6) (2005) 734-749.

    Google Scholar 

  2. Agrawal, R., Imielinski, T., Swami, A., Mining association rules between sets of items in large databases. Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, ACM Press, 1993, 207216.

    Google Scholar 

  3. Belkin, N.J., Croft, W.B., Information filtering and information retrieval: two sides of the same coin? Communications of the ACM 35 (12) (1992) 29-38.

    Google Scholar 

  4. Bernoulli, D., Exposition of a new theory on the measurement of risk (original 1738). Econometrica 22 (1) (1954) 23-36.

    Article  MathSciNet  Google Scholar 

  5. Blythe, J., Visual exploration and incremental utility elicitation. Proceedings of the 18th National Conference on Artificial Intelligence, AAAI press, 2002, 526-532.

    Google Scholar 

  6. Bradley, K., Smyth, B., Improving recommendation diversity. Proceedings of the 12th Irish Conference on Artificial Intelligence and Cognitive Science, 2001, 85-94.

    Google Scholar 

  7. Burke, R., Hybrid recommender systems: survey and experiments. User Modeling and User- Adapted Interaction 12 (4) (2002) 331-370.

    Google Scholar 

  8. Burke, R., Hammond, K., Cooper, E., Knowledge-based navigation of complex information spaces. Proceedings of the 13th National Conference on Artificial Intelligence, AAAI press, 1996, 462-468.

    Google Scholar 

  9. Burke, R., Hammond, K., Young, B., The FindMe approach to assisted browsing. IEEE Expert: Intelligent Systems and Their Applications 12 (4) (1997) 32-40.

    Google Scholar 

  10. Chen, L., Pu, P., Trust building in recommender agents. Proceedings of the Workshop on Web Personalization, Recommender Systems and Intelligent User Interfaces at the 2nd International Conference on E-Business and Telecommunication Networks, 2005, 135-145.

    Google Scholar 

  11. Chen, L., Pu, P., Evaluating critiquing-based recommender agents. Proceedings of TwentyfirstNational Conference on Artificial Intelligence (AAAI-06), 2006, 157-162.

    Google Scholar 

  12. Cover, T.M., Hart, P.E., Nearest neighbor pattern classification. IEEE Transactions on Information Theory, IT-13, 1967, 21-27.

    Google Scholar 

  13. Faltings, B., Pu, P., Torrens, M., Viappiani, P., Designing example-critiquing interaction. Proceedings of the 9th International Conference on Intelligent User Interfaces (IUI’04), ACM Press, 2004, 22-29.

    Google Scholar 

  14. Faltings, B., Torrens, M., Pu, P., Solution generation with qualitative models of preferences. Computational Intelligence 20 (2) (2004), 246-263.

    Google Scholar 

  15. Freuder, E.C., Wallace, R.J., Partial constraint satisfaction. Artificial Intelligence 58 (1-3) (1992) 21-70.

    Google Scholar 

  16. Goker, M., Thompson, C., The adaptive place advisor: a conversational recommendation system. Proceedings of the 8th German Workshop on Case Based Reasoning, 2000.

    Google Scholar 

  17. Goldberg, D., Nichols, D., Oki, B.M., Terry, D., Using collaborative filtering to weave an information tapestry. Communications of the ACM (35) 12, Special issue on information filtering (1992) 61-70.

    Google Scholar 

  18. Good, N., Schafer, J.B., Konstan, J.K., Borchers, A., Sarwar, B.M., Herlocker, J.L., Riedl, J., Combining collaborative filtering with personal agents for better recommendations. Proceedings of the 16th National Conference on Artificial Intelligence (AAAI’99), AAAI press, 1999, 439-446.

    Google Scholar 

  19. Ha, V.A., Haddawy, P., Problem-focused incremental elicitation of multi-attribute utility models. In Shenoy, P. (ed.), Proceedings of the 13th Conference on Uncertainty in Artificial Intelligence (UAI’97), 1997, 215-222.

    Google Scholar 

  20. Haubl, G., Trifts, V., Consumer decision making in online shopping environments: the effects of interactive decision aids. Marketing Science 19 (1) (2000) 4-21.

    Google Scholar 

  21. Herstein, I.N., Milnor, J., An axiomatic approach to measurable utility. Econometrica 21 (2)(1953) 291-297.

    Article  MATH  MathSciNet  Google Scholar 

  22. Hill, W., Stead, L., Rosenstein, M., Furnas, G., Recommending and evaluating choices in a virtual community of use. Proceedings of the CHI ’95 Conference on Human Factors in Computing Systems, 1995, 194-201.

    Google Scholar 

  23. Keeney, R.L., Value-Focused Thinking: A Path to Creative Decision Making, Harvard University Press (1992).

    Google Scholar 

  24. Keeney, R.L., Raiffa, H., Decisions with Multiple Objectives: Preferences and Value Tradeoffs, New York: Wiley (1976).

    Google Scholar 

  25. L, J., Kolodner. Case-Based Reasoning. San Mateo, CA: Morgan Kaufmann (1993).

    Google Scholar 

  26. Krulwich, B., Lifestyle finder: intelligent user profiling using large-scale demographic data. Artificial Intelligence Magazine 18 (2) (1997) 37-45.

    Google Scholar 

  27. Lang, K., Newsweeder: learning to filter news. Proceedings of the 12th International Conference on Machine Learning, 1995, 331-339.

    Google Scholar 

  28. Linden, G., Hanks, S., Lesh, N., Interactive assessment of user preference models: the automated travel assistant. Proceedings of the 6th International Conference on User Modeling (UM’97), New York: Springer Wien New York, 1997, 67-78.

    Google Scholar 

  29. Marschak, J., Rational Behavior, Uncertain Prospects, and Measurable Utility. Econometrica 18 (2) (1950) 111-141.

    Article  MathSciNet  Google Scholar 

  30. McCarthy K., McGinty L., Smyth, B., Reilly, J., A live-user evaluation of incremental dynamic critiquing. Proceedings of the 6th International Conference on Case-Based Reasoning (ICCBR’05), 2005, 339-352.

    Google Scholar 

  31. McCarthy K., Reilly, J., L. McGinty, Smyth, B., Thinking positively explanatory feedback for conversational recommender systems. Proceedings of the Workshop on Explanation in CBR at the 7th European Conference on Case-Based Reasoning (ECCB’04), 2004, 115-124.

    Google Scholar 

  32. McCarthy K., Reilly, J., L. McGinty, Smyth, B., Experiments in dynamic critiquing. Proceedings of the 10th International Conference on Intelligent User Interfaces (IUI’05), New York: ACM Press, 2005, 175-182.

    Google Scholar 

  33. McGinty L., Smyth, B., On the role of diversity in conversational recommender systems. Proceedings of the 5th International Conference on Case-Based Reasoning (ICCBR’03), 2003, 276-290.

    Google Scholar 

  34. McNee S.M., Lam, S.K., Konstan, J., Riedl, J., Interfaces for eliciting new user preferences in recommender systems. Proceedings of User Modeling Conference, Springer, 2003, 178–187.

    Google Scholar 

  35. McNee S.M., Riedl, J., Konstan, J., Being accurate is not enough: how accuracy metrics have hurt recommender systems. In CHI ’06 Extended Abstracts on Human Factors in Computing Systems (CHI ’06), ACM, New York, NY, 1097-1101.

    Google Scholar 

  36. McSherry, D., Diversity-conscious retrieval. In, Craw, S., Preece, A. (eds.), Proceedings of the 6th European Conference on Advances in Case-Based Reasoning, London: Springer-Verlag, 2002, 219-233.

    Google Scholar 

  37. McSherry, D., Similarity and compromise. Proceedings of the 5th International Conference on Case-Based Reasoning (ICCBR’03), Springer-Verlag, 2003, 291-305.

    Google Scholar 

  38. McSherry, D., Explanation in recommender systems. Workshop Proceedings of the 7th European Conference on Case-Based Reasoning (ECCBR’04), 2004, 125-134.

    Google Scholar 

  39. Mongin, P., Expected Utility Theory. Handbook of Economic Methodology, Edward Elgar, 1998, 342-350.

    Google Scholar 

  40. Payne, J.W., Bettman, J.R., Johnson, E.J., The Adaptive Decision Maker, Cambridge University Press (1993).

    Google Scholar 

  41. Payne, J.W., Bettman, J.R., Schkade, D.A., Measuring constructed preferences: towards a building code. Journal of Risk and Uncertainty 19 (1999) 243-270.

    Google Scholar 

  42. Price, B., Messinger, P.R., Optimal recommendation sets: covering uncertainty over user preferences. Proceedings of the 20th National Conference on Artificial Intelligence (AAAI’05), 2005, 541-548.

    Google Scholar 

  43. Pu, P., Faltings, B., Enriching buyers’ experiences: the SmartClient approach. Proceedings of the SIGCHI conference on Human factors in computing systems (CHI’00), New York: ACM Press, 2000, 289-296.

    Google Scholar 

  44. Pu, P., Faltings, B., Torrens, M., User-involved preference elicitation. Working Notes of the Workshop on Configuration. Eighteenth International Joint Conference on Artificial Intelligence (IJCAI’03), 2003, 56-63.

    Google Scholar 

  45. Pu, P., Faltings, B., Decision tradeoff using example-critiquing and constraint programming. Constraints: an International Journal 9 (4) (2004) 289-310. 544 Pearl Pu, Boi Faltings, Li Chen, Jiyong Zhang and Paolo Viappiani

    Article  Google Scholar 

  46. Pu, P., Faltings, B., Torrens, M., Effective interaction principles for online product search environments. Proceedings of the IEEE/WIC/ACM International Joint Conference on Intelligent Agent Technology and Web Intelligence, 2004, 724-727.

    Google Scholar 

  47. Pu, P., Kumar, P., Evaluating example-based search tools. Proceedings of the 5th ACM Conference on Electronic Commerce (EC’04), ACM Press, 2004, 208-217.

    Google Scholar 

  48. Pu, P., Chen, L., Integrating tradeoff support in product search tools for e-commerce sites. Proceeding of the 6th ACM Conference on Electronic Commerce (EC’05), ACM Press, 2005, 269-278.

    Google Scholar 

  49. Pu, P., Chen, L., Trust building with explanation interfaces. Proceedings of the 11th International Conference on Intelligent User Interface (IUI’06), 2006, 93-100.

    Google Scholar 

  50. Pu, P., Viappiani, P., Faltings, B., Stimulating decision accuracy using suggestions. SIGCHI conference on Human factors in computing systems (CHI’06), 2006, 121-130.

    Google Scholar 

  51. Pu P., Chen L. and Kumar P., Evaluating Product Search and Recommender Systems for ECommerce Environments. Electronic Commerce Research Journal 8(1-2), June 2008, 1-27.

    Google Scholar 

  52. Pu P., Chen L., User-Involved Preference Elicitation for Product Search and Recommender Systems. AI Magazine 29(4), Winter 2008, 93-103.

    Google Scholar 

  53. Reilly, J., K. McCarthy, L. McGinty, Smyth, B., Dynamic critiquing. Proceedings of the 7th European Conference on Case-Based Reasoning (ECCBR’04), Springer, 2004, 763-777.

    Google Scholar 

  54. Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J., GroupLens: an open architecture for collaborative filtering of Netnews. CSCW ’94: Conference on Computer Supported Cooperative Work, ACM, 1994, 175-186.

    Google Scholar 

  55. Rich, E., User modeling via stereotypes. Cognitive Science 3 (1979) 329-354.

    Google Scholar 

  56. Rokach, L., Maimon, O., Averbuch, M., Information Retrieval System for Medical Narrative Reports, Lecture Notes in Artificial intelligence 3055, page 217-228 Springer-Verlag, 2004.

    Google Scholar 

  57. Schafer, J.B., Konstan, J.A., Riedl, J., Recommender systems in e-commerce. Proceedings of the ACM Conference on Electronic Commerce, ACM, 1999, 158-166.

    Google Scholar 

  58. Shardanand, U., Maes, P., Social information filtering: algorithms for automating ”Word of Mouth”. Proceedings of the Conference on Human Factors in Computing Systems (CHI ’95), 1995, 210-217.

    Google Scholar 

  59. Schoemaker, P., The Expected Utility Model: Its Variants, Purposes, Evidence and Limitations. Journal of Economic Literature 20 (2) (1982), 529-563.

    Google Scholar 

  60. Shimazu, H., ExpertClerk: navigating shoppers’ buying process with the combination of asking and proposing. Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI’01), 2001, 1443-1448

    Google Scholar 

  61. Smyth, B., P. McClave. Similarity vs. diversity. Proceedings of the 4th International Conference on Case-Based Reasoning (ICCBR’01), Springer-Verlag, 2001, 347-361.

    Google Scholar 

  62. Spiekermann, S., Parachiv, C., Motivating human-agent interaction: transferring insights from behavioral marketing to interface design. Journal of Electronic Commerce Research 2 (3) (2002) 255-285.

    Article  MATH  Google Scholar 

  63. Stolze, M., Soft navigation in electronic product catalogs. International Journal on Digital Libraries 3 (1) (2000) 60-66.

    Article  Google Scholar 

  64. Torrens, M., Faltings, B., Pu, P., SmartClients: constraint satisfaction as a paradigm forscaleable intelligent information systems. International Journal of Constraints 7 (1) (2002) 49-69.

    Article  MATH  Google Scholar 

  65. Tversky, A., Contrasting rational and psychological principles in choice. Wise Choices: Decisions, Games, and Negotiations, Boston, MA: Harvard Business School Press (1996) 5-21.

    Google Scholar 

  66. Tversky, A., Simonson, I., Context-dependent preferences. Management Science 39 (10) (1993) 1179-1189.

    Google Scholar 

  67. Viappiani, P., Faltings, B., V. Schickel-Zuber, Pu, P., Stimulating preference expression using suggestions. Mixed-Initiative Problem-Solving Assistants, AAAI Fall Symposium Series, AAAI press, 2005, 128-133.

    Google Scholar 

  68. Viappiani, P., Faltings, B., Pu, P., Evaluating preference-based search tools: a tale of two approaches. Proceedings of the Twenty-first National Conference on Artificial Intelligence (AAAI-06), 2006, 205-210. 16 Usability Guidelines for Product Recommenders 545

    Google Scholar 

  69. Viappiani, P., B. Faltings and Pu, P., Preference-based Search using Example-Critiquing with Suggestions. Journal of Artificial Intelligence Research (JAIR), 27, 2006, 465-503.

    Google Scholar 

  70. Paolo Viappiani, Pearl Pu, and Boi Faltings. Preference-based Search with Adaptive Recommendations. AI Communications 21>(2-3), 2008, 155-175.

    MATH  MathSciNet  Google Scholar 

  71. J. von Neumann, Morgenstern, O., The Theory of Games and Economic Behavior, Princeton University Press (1944).

    Google Scholar 

  72. Williams, M.D., Tou, F.N., RABBIT: an interface for database access. Proceedings of the ACM ’82 Conference, ACM Press, 1982, 83-87.

    Google Scholar 

  73. Zhang, J., Pu, P., Performance evaluation of consumer decision support systems. International Journal of E-Business Research 2 (2006) Idea Group Publishing.

    Google Scholar 

  74. Ziegler, C.N., S.M. McNee, Konstan, J.A., Lausen, G., Improving recommendation lists through topic diversification. Proceedings of the 14th International World Wide Web Conference (WWW’05), 2005, 22-32.

    Google Scholar 

  75. Zukerman, I., Albrecht, D.W., Predictive statistical models for user modeling. User Modeling and User-Adapted Interaction 11> (1-2) (2001) 5-18.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Human Computer Interaction Group, School of Computer and Communication Sciences, Swiss Federal Institute of Technology in Lausanne (EPFL), CH-1015, Lausanne, Switzerland

    Pearl Pu, Li Chen & Jiyong Zhang

  2. Artificial Intelligence Laboratory, School of Computer and Communication Sciences, Swiss Federal Institute of Technology in Lausanne (EPFL), CH-1015, Lausanne, Switzerland

    Boi Faltings

  3. Department of Computer Science, University of Toronto, 6 King’s College Road, M5S3G4, Toronto, ON, Canada

    Paolo Viappiani

Authors
  1. Pearl Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Boi Faltings
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiyong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paolo Viappiani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pearl Pu .

Editor information

Editors and Affiliations

  1. , Faculty of Computer Science, Free University of Bozen-Bolzano, Piazza Domenicani 3, Bolzano, 39100, Italy

    Francesco Ricci

  2. , Dept. Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, 84105, Israel

    Lior Rokach

  3. Dept. Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Bracha Shapira

  4. School of Communication,, Information & Library Studies, Rutgers University, Huntington Street 4, New Brunswick, 08901-1071, New Jersey, USA

    Paul B. Kantor

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Pu, P., Faltings, B., Chen, L., Zhang, J., Viappiani, P. (2011). Usability Guidelines for Product Recommenders Based on Example Critiquing Research. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P. (eds) Recommender Systems Handbook. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-85820-3_16

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-85820-3_16

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-85819-7

  • Online ISBN: 978-0-387-85820-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation