Efficient and non-parametric reasoning over user preferences

  • Carmel DomshlakEmail author
  • Thorsten Joachims
Original Paper


We consider the problem of modeling and reasoning about statements of ordinal preferences expressed by a user, such as monadic statement like “X is good,” dyadic statements like “X is better than Y,” etc. Such qualitative statements may be explicitly expressed by the user, or may be inferred from observable user behavior. This paper presents a novel technique for efficient reasoning about sets of such preference statements in a semantically rigorous manner. Specifically, we propose a novel approach for generating an ordinal utility function from a set of qualitative preference statements, drawing upon techniques from knowledge representation and machine learning. We provide theoretical evidence that the new method provides an efficient and expressive tool for reasoning about ordinal user preferences. Empirical results further confirm that the new method is effective on real-world data, making it promising for a wide spectrum of applications that require modeling and reasoning about user preferences.


Preference elicitation Ordinal utility function Reasoning over preferences Support vector machines Kernels 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bacchus, F., Grove, A.: Graphical models for preference and utility. In: UAI-1995, pp. 3–10. Montreal (1995)Google Scholar
  2. Bertsekas, D., Nedic, A., Ozdaglar, A.: Convex Analysis and Optimization. Athena Scientific. Nashua, NH (2003)Google Scholar
  3. Blythe, J.: Visual exploration and incremental utility elicitation. In: AAAI-02, pp. 526–532. Edmonton (2002)Google Scholar
  4. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A traininig algorithm for optimal margin classifiers. In: Haussler, D. (ed.) Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152. Pittsburg, PA (1992)Google Scholar
  5. Boutilier, C.: Toward a logic for qualitative decision theory. In: Proceedings of the Third Conference on Knowledge Representation (KR–94), pp. 75–86. Bonn (1994)Google Scholar
  6. Boutilier, C., Bacchus, F., Brafman, R.I.: UCP-networks: A directed graphical representation of conditional utilities. In: UAI-2001, pp. 56–64. Seatlle, WA (2001)Google Scholar
  7. Boutilier C., Brafman R., Domshlak C., Hoos H., Poole D. (2004). CP-nets: A tool for representing and reasoning about conditional ceteris paribus preference statements. J. Artif. Intel. Res. 21: 135–191 zbMATHMathSciNetGoogle Scholar
  8. Boutilier, C., Patrascu, R., Poupart, P., Schuurmans, D.: Regret-based utility elicitation in constraint-based decision problems. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence. Edinburgh, Scotland (2005)Google Scholar
  9. Brafman, R., Domshlak, C., Kogan, T.: Compact value-function representations for qualitative preferences. In: UAI-04, Banff, Canada (2004)Google Scholar
  10. Burke R.D., Hammond K.J., Young B.C. (1997). The FindMe approach to assisted browsing. IEEE Expert 12(4): 32–40 CrossRefGoogle Scholar
  11. Chai J., Horvath V., Nicolov N., Stys M., Kambhatla N., Zadrozny W., Melville P. (2002). Natural language assistant: A dialog system for online product recommendation. AI Mag. 23(2): 63–76 Google Scholar
  12. Chisholm R.M., Sosa E. (1966). On the logic of ‘Intrinsically Better’. Am. Philos. Q. 3: 244–249 Google Scholar
  13. Chomicki J. (2003). Preference formulas in relational queries. ACM Trans. Database Syst. 28(4): 427–466 CrossRefGoogle Scholar
  14. Cohen W., Shapire R., Singer Y. (1999). Learning to order things. J. Artif. Intell. Res. 10: 243–270 zbMATHGoogle Scholar
  15. Cortes C., Vapnik V. (1995). Support–vector networks. Mach. Learn. J. 20: 273–297 zbMATHGoogle Scholar
  16. Domshlak, C., Joachims, T.: Unstructuring user preferences: Efficient non-parametric utility revelation. In: Conference on Uncertainty in Artificial Intelligence (UAI), Edinburgh, Scotland (2005)Google Scholar
  17. Doyle J. (2004). Prospects for preferences. Comput. Intell. 20(2): 111–136 CrossRefMathSciNetGoogle Scholar
  18. Doyle J., Thomason R.H. (1999). Background to qualitative decision theory. AI Mag. 20(2): 55–68 Google Scholar
  19. Doyle, J., Wellman, M.: Representing preferences as ceteris paribus comparatives. In: Proceedings of the AAAI Spring Symposium on Decision-Theoretic Planning, pp. 69–75. Stanford, CA (1994)Google Scholar
  20. Fishburn P.C. (1982). The Foundations of Expected Utility. Reidel, Dordrecht zbMATHGoogle Scholar
  21. Freund Y., Iyer R., Schapire R.E., Singer Y. (2003). An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res. 4: 933–969 CrossRefMathSciNetGoogle Scholar
  22. Goldsmith, J., Lang, J., Truszczynski, M., Wilson, N.: The computational complexity of dominance and consistency in CP-nets. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 144–149. Edinburgh, Scotland (2005)Google Scholar
  23. Green P.E., Krieger A.M., Wind Y. (2001). Thirty years of conjoint analysis: Reflections and prospects. Interfaces 31(3): 56–73 CrossRefGoogle Scholar
  24. Ha, V., Haddawy, P.: A hybrid approach to reasoning with partially elicited preference models. In: Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence, pp. 263–270. Morgan Kaufmann, Stockholm, Sweden (1999)Google Scholar
  25. Haddawy P., Ha V., Restificar A., Geisler B., Miyamoto J. (2003). Preference elicitation via theory refinement. J. Mach. Learn. Res. 4: 317–337 CrossRefMathSciNetGoogle Scholar
  26. Hansson S.O. (2001a). Preference logic. In: Gabbay, D.M., Guenthner, F. (eds) Handbook of Philosophical Logic, IInd edn, vol. 4, pp 319–394. Kluwer, Dordrecht Google Scholar
  27. Hansson S.O. (2001b). The Structure of Values and Norms. Cambridge University Press, Cambridge zbMATHGoogle Scholar
  28. Herbrich, R., Graepel, T., Obermayer, K.: Large margin rank boundaries for ordinal regression. In: Advances in Large Margin Classifiers, pp. 115–132. MIT Press, Cambridge, MA (2000)Google Scholar
  29. Herfert, M., La Mura, P.: Estimation of consumer preferences via ordinal decision-theoretic entropy. Leipzig Graduate School of Management, Working Paper HHL-Arbeitspapier Nr. 64 (2004)Google Scholar
  30. Joachims, T., Granka, L., Pang, B., Hembrooke, H., Gay, G.: Accurately interpreting clickthrough data as implicit feedback. In: ACM Conference on Research and Development in Information Retrieval (SIGIR), pp. 154–161. Salvador, Brazil (2005)Google Scholar
  31. Keeney R.L., Raiffa H. (1976). Decision with Multiple Objectives. Wiley, NY Google Scholar
  32. Kimeldorf G., Wahba G. (1971). Some results on tchebycheffian spline functions. J. Math. Anal. Appl. 33: 82–95 zbMATHCrossRefMathSciNetGoogle Scholar
  33. Krantz D.H., Luce R.D., Suppes P., Tversky A. (1971). Foundations of Measurement. Academic, NY zbMATHGoogle Scholar
  34. La Mura, P.: Foundations of Multi-Agent Systems. Ph.D. thesis, Graduate School of Business, Stanford (1999)Google Scholar
  35. La Mura, P.: Decision-theoretic entropy. In: Proceedings of the Ninth Conference on Theoretical Aspects of Rationality and Knowledge, pp. 35–44. Bloomington, IN (2003)Google Scholar
  36. La Mura, P., Shoham, Y.: Expected utility networks. In: UAI-1999, pp. 367–373. Stockholm, Sweden (1999)Google Scholar
  37. Lang J. (2004). Logical preference representation and combinatorial vote. Ann. Math. Artif. Intell. 42(1–3): 37–71 zbMATHCrossRefMathSciNetGoogle Scholar
  38. Linden, G., Hanks, S., Lesh, N.: Interactive assessment of user preference models: The automated travel assistant. In: Proceedings of the Sixth International Conference on User Modeling, pp. 67–78. Chia Laguna, Sardinia, Italy (1997)Google Scholar
  39. McGeachie M., Doyle J. (2004). Utility functions for ceteris paribus preferences. Comput. Intell. 20(2): 158–217 CrossRefMathSciNetGoogle Scholar
  40. McJones, P.: Eachmovie collaborative filtering data set. DEC Systems Research Center (1997)Google Scholar
  41. Packard D.J. (1975). A preference logic minimally complete for expected utility maximization. J. Philos. Logic. 4(2): 223–235 CrossRefMathSciNetGoogle Scholar
  42. Pu P., Faltings B. (2004). Decision tradeoff using example critiquing and constraint programming. Constr. Int. J. 9(4): 289–310 CrossRefGoogle Scholar
  43. Pu, P., Faltings, B., Torrens, M.: User-involved preference elicitation. In: Working notes of the Workshop on Configuration (IJCAI-2003). Acapulco, Mexico (2003)Google Scholar
  44. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA (1993)Google Scholar
  45. Riedl, J., Konstan, J., Lam, S., Herlocker, J.: Movielens collaborative filtering data set. (2006)Google Scholar
  46. Shearin, S., Lieberman, H.: Intelligent profiling by example. In: Proceedings of the Sixth International Conference on Intelligent User Interfaces. Santa Fe, NM, USA (2001)Google Scholar
  47. Shoham, Y.: Conditional utility, utility independence, and utility networks. In: Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence, pp. 429–436. Morgan Kaufmann Publishers, San Francisco, CA (1997a)Google Scholar
  48. Shoham, Y.: A symmetric view of probabilities and utilities. In: IJCAI-1997, pp. 1324–1329. Nagoya, Japan (1997b)Google Scholar
  49. Vapnik, V.: Statistical Learning Theory. Wiley (1998)Google Scholar
  50. von Wright, G.H.: The logic of preference reconsidered. Theory and Decisions 3, 140–167. Reprinted in [von Wright, 1984] (1972)Google Scholar
  51. Wright G.H. (1984). Philosophical logic: Philosophical Papers, vol. 2. Cornell University Press, Ithaca, NY Google Scholar
  52. Wahba, G.: Spline Models for Observational Data, vol. 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM (1990)Google Scholar
  53. Wilson, N.: Extending CP-nets with stronger conditional preference statements. In: Proceedings of the Nineteenth National Conference on Artificial Intelligence. San Jose, CL (2004)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.Faculty of Industrial Engineering and ManagementTechnion - Israel Institute of TechnologyHaifaIsrael
  2. 2.Department of Computer ScienceCornell UniversityIthacaUSA

Personalised recommendations