Artificial Intelligence Review

, Volume 25, Issue 3, pp 231–246 | Cite as

Supervised ordering by regression combined with Thurstone’s model

  • Toshihiro KamishimaEmail author
  • Shotaro Akaho


In this paper, we advocate a learning task that deals with the orders of objects, which we call the Supervised Ordering task. The term order means a sequence of objects sorted according to a specific property, such as preference, size, cost. The aim of this task is to acquire the rule that is used for estimating an appropriate order of a given unordered object set. The rule is acquired from sample orders consisting of objects represented by attribute vectors. Developing solution methods for accomplishing this task would be useful, for example, in carrying out a questionnaire survey to predict one’s preferences. We develop a solution method based on a regression technique imposing a Thurstone’s model and evaluate the performance and characteristics of these methods based on the experimental results of tests using both artificial data and real data.


Order Rank Supervised learning Regression analysis Thurstone’s law of comparative judgment Sensory survey 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Agresti A (1996) Categorical data analysis. WileyGoogle Scholar
  2. Anderson JA and Philips PR (1981). Regression, discrimination and measurement models for ordered categorical variables. J R Stat Soc (C) 30(1): 22–31 zbMATHMathSciNetGoogle Scholar
  3. Arnold BC, Balakrishnan N, Nagaraja HN (1992) A first course in order statistics. WileyGoogle Scholar
  4. Arrow KJ (1963) Social choice and individual values, 2nd edn. Yale University PressGoogle Scholar
  5. Babington Smith B (1950) Discussion on professor ross’s paper. J R Stat Soc (B) 12:53–56, (A. S. C. Ross, Philological Probability Problems, pp 19–41)Google Scholar
  6. Cohen WW, Schapire RE and Singer Y (1999). Learning to order things. J Artif Intel Res 10: 243–270 zbMATHMathSciNetGoogle Scholar
  7. de Condorcet M (1995) On constitution and the functions of provincial assemblies (1788). In: McLean I, Urken AB (eds) Classics of social choice, The University of Michigan Press, chap 7, pp 113–144Google Scholar
  8. Fligner MA and Verducci JS (1990). Posterior probabilities for a consensus ordering. Psychometrika 55(1): 53–63 CrossRefMathSciNetGoogle Scholar
  9. Hayashi C (1952). On the prediction of phenomena from qualitative data and the quantification of qualitative data from the mathematico-statistical point of view. Ann Inst Stat Math 3: 69–98 zbMATHCrossRefGoogle Scholar
  10. Herbrich R, Graepel T, Bollmann-Sdorra P, Obermayer K (1998) Learning preference relations for information retrieval. In: ICML-98 Workshop: text categorization and machine learning, pp 80–84Google Scholar
  11. Joachims T (2002) Optimizing search engines using clickthrough data. In: Proceedings of the 8th international conference on knowledge discovery and data mining, pp 133–142Google Scholar
  12. Kamishima T (2003) Nantonac collaborative filtering: Recommendation based on order responses. In: Proceedings of the 9th International conference on knowledge discovery and data mining, pp 583–588Google Scholar
  13. Kamishima T, Akaho S (2004) Filling-in missing objects in orders. In: Proceedings of the IEEE International conference on data miningGoogle Scholar
  14. Kamishima T, Fujiki J (2003) Clustering orders. In: Proceedings of the 6th International conference on discovery science, pp 194–207, [LNAI 2843]Google Scholar
  15. Kendall M, Gibbons JD (1990) Rank correlation methods, 5th edn. Oxford University PressGoogle Scholar
  16. Lebanon G, Lafferty J (2002) Crankng: combining rankings using conditional probability models on permutations. In: Proceedings of the 19th International conference on machine learning, pp 363–370Google Scholar
  17. Mallows CL (1957). Non-null ranking models. I Biometrika 44: 114–130 zbMATHMathSciNetGoogle Scholar
  18. Mannila H, Meek C (2000) Global partial orders from sequential data. In: Proceedings of the 6th International conference on knowledge discovery and data mining, pp 161–168Google Scholar
  19. Marden JI (1995) Analyzing and modeling rank data, monographs on statistics and applied probability, vol 64. Chapman & HallGoogle Scholar
  20. McCullagh P (1980). Regression models for ordinal data. J R Stat Soc (B) 42(2): 109–142 zbMATHMathSciNetGoogle Scholar
  21. Mitchell TM (1997) Machine learning. The McGraw-HillGoogle Scholar
  22. Mosteller F (1951). Remarks on the method of paired comparisons: I—the least squares solution assuming equal standard deviations and equal correlations. Psychometrika 16(1): 3–9 CrossRefGoogle Scholar
  23. Nakamori Y (2000) Kansei data Kaiseki. Morikita Shuppan Co., Ltd., (in Japanese)Google Scholar
  24. Osgood CE, Suci GJ, Tannenbaum PH (1957) The Measurement of meaning. University of Illinois PressGoogle Scholar
  25. Plackett RL (1975). The analysis of permutations. J R Stat Soc (C) 24(2): 193–202 MathSciNetGoogle Scholar
  26. Sai Y, Yao YY, Zhong N (2001) Data analysis and mining in ordered information tables. In: Proceedings of the IEEE International conference on data mining, pp 497–504Google Scholar
  27. Thurstone LL (1927). A law of comparative judgment. Psychol Rev 34: 273–286 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.National Institute of Advanced Industrial Science and Technology (AIST)Tsukuba, IbarakiJapan

Personalised recommendations