Ordinal regression with continuous pairwise preferences

  • Maria DobrskaEmail author
  • Hui Wang
  • William Blackburn
Original Article


We investigate the problem of data-driven ordinal regression—the problem of learning to rank order new data items based on information inherent in existing data items. Ordinal regression shares common features with multi-category classification and metric regression. However, it requires new, tailor-made methodologies to reduce prediction error. The approach has application in various domains, including information retrieval, collaborative filtering and social sciences. We propose a new distribution-independent methodology for ordinal regression based on pairwise preferences employing information about strength of dependency between two data instances, which we refer to as continuous preferences. Our hypothesis is that additional information about strength of preference as well as its direction can improve algorithmic performance. We also provide a novel technique for deriving ordinal regression labels from pairwise information. Experimental results on real-world ordinal and metric regression data sets confirm usefulness of the methodology compared with other state-of-the-art approaches.


Ordinal regression Pairwise preference Preference training Continuous preference 


  1. 1.
    Burges C, Shaked T, Renshaw E, Lazier A, Deeds M, Hamilton N, Hullender G (2005) Learning to rank using gradient descent. In: ICML’05: Proceedings of the 22nd international conference on machine learning. ACM, New York, NY, USA, pp 89–96Google Scholar
  2. 2.
    Chu W, Keerthi SS (2007) Support vector ordinal regression. Neural Comput 19(3):792–815MathSciNetCrossRefGoogle Scholar
  3. 3.
    Cohen WW, Schapire RE, Singer Y (1998) Learning to order things. In: NIPS’97: Proceedings of the 1997 conference on advances in neural information processing systems 10. MIT Press, Cambridge, MA, USA, pp 451–457Google Scholar
  4. 4.
    Crammer K, Singer Y (2001) Pranking with ranking. In: NIPS, pp 641–647Google Scholar
  5. 5.
    Frank E, Hall M (2001) A simple approach to ordinal classification. In: EMCL’01: Proceedings of the 12th European conference on machine learning. Springer, London, UK, pp 145–156Google Scholar
  6. 6.
    Freund Y, Iyer R, Schapire RE, Singer Y (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4:933–969MathSciNetGoogle Scholar
  7. 7.
    Herbrich R, Graepel T, Obermayer K (1999) Support vector learning for ordinal regression. In: In international conference on artificial neural networks, pp 97–102Google Scholar
  8. 8.
    Herbrich R, Graepel T, Obermayer K (2000) Large margin rank boundaries for ordinal regression. MIT Press, Cambridge, MA, pp 115–132.
  9. 9.
    Huhn J, Hullermeier E (2009) Is an ordinal class structure useful in classifier learning?. Int J Data Min Model Manag 1(1):45–67CrossRefGoogle Scholar
  10. 10.
    Jiao T, Peng J, Terlaky T (2009) A confidence voting process for ranking problems based on support vector machines. Ann Oper Res 166(1):23–38. Google Scholar
  11. 11.
    Rennie JD, Srebro N (2005) Loss functions for preference levels: regression with discrete ordered labels. In: Proceedings of the IJCAI multidisciplinary workshop on advances in preference handling, pp 180–186Google Scholar
  12. 12.
    Shashua A, Levin A (2002) Ranking with large margin principle: two approaches. In: NIPS, pp 937–944Google Scholar
  13. 13.
    Shevade SK, Chu W (2006) Minimum enclosing spheres formulations for support vector ordinal regression. In: ICDM’06: Proceedings of the sixth international conference on data mining. IEEE Computer Society, Washington, DC, USA, pp 1054–1058Google Scholar
  14. 14.
    Torra V, Domingo-Ferrer J, Mateo-Sanz JM, Ng M (2006) Regression for ordinal variables without underlying continuous variables. Inf Sci 176(4):465–474MathSciNetCrossRefGoogle Scholar
  15. 15.
    Waegeman W, Baets BD, Boullart L (2008) Roc analysis in ordinal regression learning. Pattern Recogn Lett 29(1):1–9CrossRefGoogle Scholar
  16. 16.
    Witten IH, Frank E (2005) Data Mining: practical machine learning tools and techniques. Morgan Kaufmann, San FransiscoGoogle Scholar
  17. 17.
    Zhao B, Wang F, Zhang C (2009) Block-quantized support vector ordinal regression. IEEE Trans Neural Netw 20(5):882–90CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.School of Computing and MathematicsUniversity of UlsterNewtownabbeyUnited Kingdom

Personalised recommendations