Abstract
Two problems may arise when an intelligent (recommender) system elicits users’ preferences. First, there may be a mismatch between the quantitative preference representations in most preference models and the users’ mental preference models. Giving exact numbers, e.g., such as “I like 30 days of vacation 2.5 times better than 28 days” is difficult for people. Second, the elicitation process can greatly influence the acquired model (e.g., people may prefer different options based on whether a choice is represented as a loss or gain). We explored these issues in three studies. In the first experiment we presented users with different preference elicitation methods and found that cognitively less demanding methods were perceived low in effort and high in liking. However, for methods enabling users to be more expressive, the perceived effort was not an indicator of how much the methods were liked. We thus hypothesized that users are willing to spend more effort if the feedback mechanism enables them to be more expressive. We examined this hypothesis in two follow-up studies. In the second experiment, we explored the trade-off between giving detailed preference feedback and effort. We found that familiarity with and opinion about an item are important factors mediating this trade-off. Additionally, affective feedback was preferred over a finer grained one-dimensional rating scale for giving additional detail. In the third study, we explored the influence of the interface on the elicitation process in a participatory set-up. People considered it helpful to be able to explore the link between their interests, preferences and the desirability of outcomes. We also confirmed that people do not want to spend additional effort in cases where it seemed unnecessary. Based on the findings, we propose four design guidelines to foster interface design of preference elicitation from a user view.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Adomavicius G., Tuzhilin A.: Towards the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6), 734–749 (2005)
Aloysius J.A., Davis F.D., Wilson D.D., Taylor A.R., Kottemann J.E.: User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques. Eur J Oper Res 169, 273–285 (2006)
Ames, M., Naaman, M.: Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the SIGCHI conference on Human factors in computing systems, Gaithersburg (2007)
Ardissono L., Felfernig A., Friedrich G., Goy A., Jannach D., Petrone G., Schäfer R., Zanker M.: A framework for the development of personalized, distributed web-based configuration systems. AI Mag 24, 93–108 (2003)
Barneveld, J., Setten, M.: Designing usable interfaces for tv recommender systems. In: Personalized digital television, human-computer interaction series, vol 6, pp. 259–285. Springer, Amsterdam (2004)
Bellucini E., Zeleznikow J.: Developing negotiation decision support systems that support mediators: a case study of the familywinner system. J Artif Intell Law 13(2), 233–271 (2006)
Bettman J.R., Luce M.F., Payne J.W.: Constructive consumer choice processes. J Consum Res 25(3), 187–217 (1998)
Broekens J., Brinkman W.P.: Affectbutton: towards a standard for dynamic affective user feedback. In: Mühl, C., Heylen, D., Nijholt, A. (eds) Proceedings of Affective Computing and Intelligent Interaction (ACII), pp. 1–8. IEEE Computer Society Press, Amsterdam (2009)
Broekens, J., Pommeranz, A., Wiggers, P., Jonker, C.M.: Factors influencing user motivation for giving preference feedback. In: Fifth Multidisciplinary Workshop on Advances in Preference Handling in Conjunction with ECAI 2010, Ulrich Junker, Jérôme Lang and Patrice Perny, pp. 19–24. Lisbon, Portugal, (2010a)
Broekens, J., Pronker, A., Neuteboom, M.: Real time labeling of affect in music using the affectbutton. In: Proceedings of the 3rd international workshop on Affective interaction in natural environments, Ginevra Castellano, Kostas Karpouzis, Jean-Claude Martin, Louis-Philippe Morency, Laurel Riek and Christopher Peters, pp. 21–26. ACM, New York, AFFINE ’10 (2010b)
Burke R.: Knowledge-based recommender systems. In: Kent, A. (ed) Encyclopedia of Library and Information Systems, vol 69, Marcel Dekker, New York (2000)
Burke R.: Hybrid recommender systems: Survey and experiments. User Model User-Adapt Interact 12(4), 331–370 (2002)
Burke, R.D., Hammond, K.J., Young, B.C.: Knowledge-based navigation of complex information spaces. In: 13th National Conference on Artificial Intelligence, pp. 462–468. AAAI Press, Portland, (1996)
Carenini, G., Loyd, J.: Valuecharts: analyzing linear models expressing preferences and evaluations. In: AVI ’04: Proceedings of the working conference on Advanced visual interfaces, pp. 150–157. ACM, New York (2004)
Carenini, G., Poole, D.: Constructed preferences and value-focused thinking: Implications for ai research on preference elicitation. In: AAAI-02 Workshop on Preferences in AI and CP: symbolic approaches, pp. 1–10. AAAI, Edmonton, Canada (2002)
Carenini, G., Smith, J., Poole, D.: Towards more conversational and collaborative recommender systems. In: 8th international conference on intelligent user interfaces, pp. 12–18. ACM, Miami (2003)
Chen, L., Pu, P.: Survey of preference elicitation methods. Tech. rep., Swiss Federal Institute of Technology In Lausanne (EPFL), Lausanne (2004)
Chen L., Pu P.: Interaction design guidelines on critiquing-based recommender systems. User Model User-Adapt Interact J (UMUAI) 19(3), 167–206 (2009)
Curhan J.R., Neale M.A., Ross L.: Dynamic valuation: preference changes in the context of face-to-face negotiation. J Exp Soc Psychol 40(2), 142–151 (2004)
Faltings, B., Pu, P., Torrens, M., Viappiani, P.: Designing example-critiquing interaction. In: IUI ’04: Proceedings of the ninth international conference on intelligent user interfaces, pp. 22–29. ACM, New York (2004)
Fano, A., Kurth, S.W.: Personal choice point: helping users visualize what it means to buy a bmw. In: IUI ’03: Proceedings of the eighth international conference on intelligent user interfaces, pp. 46–52. ACM, New York (2003)
Fischer G.W., Carmon Z., Ariely D., Zauberman G.: Goal-based construction of preferences: task goals and the prominence effect. Manag Sci 45(8), 1057–1075 (1999)
Gregorc A.: The Mind Styles Model: Theory, Principles, and Practice. AFG, Columbia (2006)
Herlocker J.L., Konstan J.A., Terveen L.G., Riedl J.T.: Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst 22(1), 5–53 (2004)
Hunt D., Haynes R., Hanna S., Smith K.: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. J Am Med Assoc 280(15), 1339–1346 (1998)
Johnson E., Steffel M., Goldstein D.: Making better decisions: from measuring to constructing preferences. Health Psychol 24(8), 17–22 (2005)
Kay J.: Stereotypes, student models and scrutability. In: Gauthier, G., Frasson, C., VanLehn, K. (eds) Intelligent Tutoring Systems, Lecture Notes in Computer Science, vol 1839, pp. 19–30. Springer, Berlin (2000)
Keeney R.: Value-Focused Thinking: A Path to Creative Decision Making. Harvard University Press, Cambridge (1992)
Keeney R., Raiffa H.: Decisions with Multiple Objectives: Preferences and Value Trade-Offs. Cambridge University Press, Cambridge (1993)
Knijnenburg, B., Willemsen, M., Ganter, Z., Soncu, H., Newell, C.: Explaining the user experience of recommender systems. User Model User-Adapt Interact 22. (2012). doi:10.1007/s11257-011-9118-4
Kramer T.: The effect of measurement task transparency on preference construction and evaluations of personalized recommendations. J Market Res 44(2), 224–233 (2007)
Linden, G., Hanks, S., Lesh, N.: Interactive assessment of user preference models: The automated travel assistant. In: User Modeling: Proceedings of the Sixth International Conference, pp. 67–78. Vienna, Austria (1997)
Liu, F.: Changing for the better: preference dynamics and agent diversity. PhD thesis, University of Amsterdam, Amsterdam (2008)
McCarthy, K.J., Reilly, K., McGinty, L., Smyth, B.: Experiments in dynamic critiquing. In: Proceedings of the 10th International Conference on Intelligent User Interfaces, pp. 175–182. ACM, San Diego (2005)
McFadden D.: Rationality for economists?. J Risk Uncert 19(1), 73–105 (1999)
Mehrabian, A.: Basic dimensions for a general psychological theory: Implications for personality, social, environmental, and developmental studies. Cambridge (1980)
Miller, B.N., Albert, I., Lam, S.K., Konstan, J.A., Riedl, J.: Movielens unplugged: experiences with an occasionally connected recommender system. In: Proceedings of the eighth international conference on intelligent user interfaces, pp. 263–266. ACM, New York, IUI ’03 (2003)
Mooney, R., Roy, L.: Content-based book recommending using learning for text categorizations. In: Proceedings of the Fifth ACM Conference on Digital Libraries, pp. 195–204. ACM, San Antonio (2000)
Ono C., Kurokawa M., Motomura Y., Asoh H.: A context-aware movie preference model using a bayesian network for recommendation and promotion. In: Conati, C., McCoy, K., Paliouras, G. (eds) User Modeling 2007, Lecture Notes in Computer Science, vol 4511, pp. 247–257. Springer, Berlin (2007)
Payne J.W., Bettman J.R., Schkade D.A.: Measuring constructed preferences: towards a building code. J Risk Uncert 19(1–3), 243–270 (1999)
Pazzani M.J., Billsus D.: Content-based recommendation systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds) The adaptive web: methods and strategies of web personalization, pp. 325–341. Springer-Verlag, Berlin (2007)
Peintner B., Viappiani P., Yorke-Smith N.: Preferences in interactive systems: technical challenges and case studies. AI Mag 29(4), 13–24 (2008)
Pommeranz A., Broekens J., Visser W., Brinkman W.P., Wiggers P., Jonker C.: Multi-angle view on preference elicitation for negotiation support systems. In: Brinkman, W.P., Hindriks, K. (eds) Proceedings of First International Working Conference on Human Factors and Computational Models in Negotiation (HuCom08), pp. 19–26. Delft University of Technology, Mediamatica, Delft (2008)
Pommeranz, A., Wiggers, P., Jonker, C.: User-centered design of preference elicitation interfaces for decision support. In: Leitner G, Hitz M, Holzinger A (eds) USAB2010: HCI in Work & Learning, Life & Leisure, Lecture Notes in Computer Science, vol Vol. 6389, pp. 14–33. Springer, Klagenfurt (2010)
Pu P., Chen L.: Trust-inspiring explanation interfaces for recommender systems. Knowledge-Based Systems 20(6), 542–556 (2007)
Pu P., Chen L.: User-involved preference elicitation for product search and recommender systems. AI Mag 29(4), 93–103 (2008)
Pu, P., Faltings, B., Torrens, M.: User-involved preference elicitation. In: Workshop notes of the Workshop on Configuration, the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI’03), pp. 56–63. Menlo Park (2003)
Pu, P., Chen, L., Hu, R.: Evaluating recommender systems from the user’s perspective: Survey of the state of the art. User Modeling and User-Adap Inter 22. (2012). doi:10.1007/s11257-011-9115-7
Resnick P., Iacovou N., Sushak M., Bergstrom P., Riedl J.: GroupLens: an open architecture for collaborative filtering of netnews. In: Smith, J.B., Smith, F.D., Malone, T.W. (eds) ACM Conference on Computer Supported Collaborative Work Conference, Association of Computing Machinery, pp. 175–186. ACM Press, Chapel Hill (1994)
Shearin S., Lieberman H.: Intelligent profiling by example. In: Sidner, C., Moore, J. (eds) IUI ’01: Proceedings of the Sixth International Conference on Intelligent User Interfaces, pp. 145–151. ACM, New York (2001)
Shiv B., Fedorikhin A.: Heart and mind in conflict: the interplay of affect and cognition in consumer decision making. J Consum Res 26(3), 278–292 (1999)
Simon D., Krawczyk D.C., Holyoak K.J.: Construction of preferences by constraint satisfaction. Psychol Sci 15(5), 331–336 (2004)
Sinha R., Swearingen K.: The role of transparency in recommender systems. In: Wixon, D. (ed) CHI ’02 Extended Abstracts on Human Factors in Computing Systems, pp. 830–831. ACM, New York (2002)
Smyth B., McGinty L.: The power of suggestion. In: Gottlob, G., Walsh, T. (eds) IJCAI-03, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 127–132. Morgan Kauffman, Acapulco (2003)
Stolze, M., Ströbel, M.: Dealing with learning in ecommerce product navigation and decision support: the teaching salesman problem. In: Piller, F.T., Reichwald, R., Tseng, M. (eds) In: Proceedings of the Second Interdisciplinary World Congress on Mass Customization and Personalization, TUM, Munchen, Working Paper Series of the Department for General and Industrial Management, TUM Business School, Technische Universität München (2003)
Viappiani P., Faltings B., Zuber V.S., Pu P.: Stimulating preference expression using suggestions. In: Aha, D.W., Tecuci, G. (eds) Proceedings of the AAAI Fall Symposium on Mixed-Initiative Problem Solving Assistants (FS-05-07), pp. 128–133. AAAI, Washington, Arlington (2005)
Weber E.U., Johnson E.J.: Constructing Preferences from Memory. Cambridge University Press, Cambridge (2006)
Acknowledgements
We would like to thank the participants of all three studies. This research is supported by the Dutch Technology Foundation stw, the Applied Science Division of nwo and the Technology Program of the Ministry of Economic Affairs. It is part of the Pocket Negotiator project with grant number vici-project 08075.
Open Access
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Pommeranz, A., Broekens, J., Wiggers, P. et al. Designing interfaces for explicit preference elicitation: a user-centered investigation of preference representation and elicitation process. User Model User-Adap Inter 22, 357–397 (2012). https://doi.org/10.1007/s11257-011-9116-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11257-011-9116-6