A Live-User Evaluation of Incremental Dynamic Critiquing
Feature critiquing has emerged as an important feedback strategy for conversational recommender systems as it offers a useful balance between user effort and recommendation efficiency. Dynamic critiquing has recently been presented as an extension to conventional (single-feature) critiquing that supports the simultaneous critiquing of multiple features. To date, dynamic critiquing has been evaluated through a variety of artificial user trials to demonstrate its potential advantages, when it comes to improving recommendation efficiency and quality. However these advantages have never been verified through any large-scale user trial. The contribution of this paper is that we present the results of such an evaluation, which confirms the advantages of dynamic critiquing in a realistic online, e-commerce setting. Furthermore we investigate the impact of implicitly maintaining session specific user models to influence the selection of compound critiques. These models are incrementally constructed as the user critiques example recommendations from cycle to cycle. Our live-user evaluation also enabled us to analyse how real users interact with the compound critiques that are produced in this way. The results demonstrate that our incremental critiquing approach has the capability of generating more relevant critique options, and that users frequently recognise the benefits associated with using these as feedback options, leading to significantly shorter recommendation sessions.
Unable to display preview. Download preview PDF.
- 1.Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., Inkeri Verkamo, A.: Fast Discovery of Association Rules in Large Databases. In: Advances in Knowledge Discovery and Data Mining, pp. 307–328 (1996)Google Scholar
- 2.Burke, R., Hammond, K., Young, B.: Knowledge-based Navigation of Complex Information Spaces. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 462–468. AAAI Press/MIT Press, Portland (1996)Google Scholar
- 5.Linden, G., Hanks, S., Lesh, N.: Interactive Assessment of User Preference Models: The Automated Travel Assistant. In: Paris, C., Jameson, A., Tasso, C. (eds.) User Modeling: Proceedings of the Sixth International Conference, pp. 67–78. Springer, Wien (1997)Google Scholar
- 6.McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: On the Dynamic Generation of Compound Critiques in Conversational Recommender Systems. In: De Bra, P. (ed.) Proceedings of the Third International Conference on Adaptive Hypermedia and Web-Based Systems (AH 2004), pp. 176–184. Springer, Eindhoven (2004)CrossRefGoogle Scholar
- 7.McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: On the Evaluation of Dynamic Critiquing: A Large-Scale User Study. Submitted to the Twentieth National Conference on Artificial Intelligence (AAAI 2005), Pittsburgh, Pennsylvania, USA (2005)Google Scholar
- 8.McSherry, D.: Minimizing Dialog Length in Interactive Case-Based Reasoning. In: Nebel, B. (ed.) Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 993–998. Morgan Kaufmann, Seattle (2001)Google Scholar
- 10.Nguyen, Q.N., Ricci, F., Cavada, D.: User Preferences Initialization and Integration in Critique-Based Mobile Recommender Systems. In: Proceedings of Artificial Intelligence in Mobile Systems 2004, in conjunction with UbiComp 2004, pp. 71–78. Iniversitat des Saarlandes Press, Nottingham (2004)Google Scholar
- 13.Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Incremental Critiquing. In: Bramer, M., Coenen, F., Allen, T. (eds.) Research and Development in Intelligent Systems XXI. Proceedings of AI 2004, pp. 101–114. Springer, Cambridge (2004)Google Scholar