Progress in Artificial Intelligence

, Volume 5, Issue 3, pp 171–179 | Cite as

Current prospects on ordinal and monotonic classification

  • Pedro Antonio GutiérrezEmail author
  • Salvador García
Regular Paper


Ordinal classification covers those classification tasks where the different labels show an ordering relation, which is related to the nature of the target variable. In addition, if a set of monotonicity constraints between independent and dependent variables has to be satisfied, then the problem is known as monotonic classification. Both issues are of great practical importance in machine learning. Ordinal classification has been widely studied in specialized literature, but monotonic classification has received relatively low attention. In this paper, we define and relate both tasks in a common framework, providing proper descriptions, characteristics, and a categorization of existing approaches in the state-of-the-art. Moreover, research challenges and open issues are discussed, with focus on frequent experimental behaviours and pitfalls, commonly used evaluation measures and the encouragement in devoting substantial research efforts in specific learning paradigms.


Machine learning Ordinal classification Ordinal regression Monotonic classification Evaluation measures 



This work is supported by the National Research Projects TIN2014-57251-P and TIN2014-54583-C2-1-R of the Spanish Ministry of Economy and Competitiveness (MINECO), by FEDER Funds and by the P11-TIC-7508 project of the Junta de Andalucía, Spain.


  1. 1.
    Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Multiple-Valued Logic Soft Comput. 17(2–3), 255–287 (2010)Google Scholar
  2. 2.
    Asuncion, A., Newman, D.: UCI Machine Learning Repository (2007).
  3. 3.
    Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA’09), pp. 283–287 (2009)Google Scholar
  4. 4.
    Ben-David, A.: Automatic generation of symbolic multiattribute ordinal knowledge-based dsss: methodology and applications. Decision Sci. 23, 1357–1372 (1992)CrossRefGoogle Scholar
  5. 5.
    Ben-David, A.: Monotonicity maintenance in information-theoretic machine learning algorithms. Mach. Learn. 19(1), 29–43 (1995)Google Scholar
  6. 6.
    Ben-David, A., Sterling, L., Pao, Y.H.: Learning, classification of monotonic ordinal concepts. Comput. Intell. 5, 45–49 (1989)CrossRefGoogle Scholar
  7. 7.
    Ben-David, A., Sterling, L., Tran, T.: Adding monotonicity to learning algorithms may impair their accuracy. Expert Syst. Appl. 36(3), 6627–6634 (2009)CrossRefGoogle Scholar
  8. 8.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)Google Scholar
  9. 9.
    Blaszczynski, J., Slowinski, R., Stefanowski, J.: Ordinal classification with monotonicity constraints by variable consistency bagging. In: Proceedings of 7th International Conference on Rough Sets and Current Trends in Computing, RSCTC 2010, Warsaw, Poland, June 28–30, 2010, pp. 392–401 (2010)Google Scholar
  10. 10.
    Cao-Van, K., De Baets, B.: Growing decision trees in an ordinal setting. Int. J. Intell. Syst. 18(7), 733–750 (2003)CrossRefzbMATHGoogle Scholar
  11. 11.
    Cardoso, J.S., da Costa, J.F.P.: Learning to classify ordinal data: the data replication method. J. Mach. Learn. Res. 8, 1393–1429 (2007)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Cardoso, J.S., Sousa, R.G.: Measuring the performance of ordinal classification. Int. J. Pattern Recognit. Artif. Intell. 25(8), 1173–1195 (2011)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)zbMATHGoogle Scholar
  14. 14.
    Chen, C.C., Li, S.T.: Credit rating with a monotonicity-constrained support vector machine model. Expert Syst. Appl. 41(16), 7235–7247 (2014)CrossRefGoogle Scholar
  15. 15.
    Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. J. Mach. Learn. Res. 6, 1019–1041 (2005)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Chu, W., Keerthi, S.S.: Support vector ordinal regression. Neural Comput. 19(3), 792–815 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)CrossRefGoogle Scholar
  18. 18.
    Cruz-Ramírez, M., Hervás-Martínez, C., Sánchez-Monedero, J., Gutiérrez, P.A.: Metrics to guide a multi-objective evolutionary algorithm for ordinal classification. Neurocomputing 135, 21–31 (2014). doi: 10.1016/j.neucom.2013.05.058 CrossRefGoogle Scholar
  19. 19.
    Daniels, H., Velikova, M.: Derivation of monotone decision models from noisy data. IEEE Trans. Syst. Man Cybern. Part C 36, 705–710 (2006)CrossRefGoogle Scholar
  20. 20.
    Daniels, H., Velikova, M.: Monotone and partially monotone neural networks. IEEE Trans. Neural Netw. 21(6), 906–917 (2010)CrossRefGoogle Scholar
  21. 21.
    Dembczyński, K., Kotłowski, W., Słowiński, R.: Learning rule ensembles for ordinal classification with monotonicity constraints. Fundamenta Informaticae 94(2), 163–178 (2009)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Duivesteijn, W., Feelders, A.: Nearest neighbour classification with monotonicity constraints. ECML/PKDD 1, 301–316 (2008)Google Scholar
  23. 23.
    Fawcett, T.: An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861–874 (2006)CrossRefGoogle Scholar
  24. 24.
    Feelders, A.: Monotone relabeling in ordinal classification. In: IEEE International Conference on Data Mining (ICDM), pp. 803–808 (2010)Google Scholar
  25. 25.
    Feelders, A.J., Pardoel, M.: Pruning for monotone classification trees. In: IDA, Lecture Notes in Computer Science, vol. 2810, pp. 1–12. Springer, New York (2003)Google Scholar
  26. 26.
    Fernández-Navarro, F., Riccardi, A., Carloni, S.: Ordinal neural networks without iterative tuning. IEEE Trans. Neural Netw. Learn. Syst. 25(11), 2075–2085 (2014)CrossRefGoogle Scholar
  27. 27.
    Fullerton, A.S., Xu, J.: The proportional odds with partial proportionality constraints model for ordinal response variables. Soc. Sci. Res. 41(1), 182–198 (2012). doi: 10.1016/j.ssresearch.2011.09.003 CrossRefGoogle Scholar
  28. 28.
    Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 42(4), 463–484 (2012)CrossRefGoogle Scholar
  29. 29.
    Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recognit. 44(8), 1761–1776 (2011). doi: 10.1016/j.patcog.2011.01.017 CrossRefGoogle Scholar
  30. 30.
    García, J., AlBar, A.M., Aljohani, N.R., Cano, J.R., García, S.: Hyperrectangles selection for monotonic classification by using evolutionary algorithms. Int. J. Comput. Intell. Syst. 9(1), 184–202 (2016)CrossRefGoogle Scholar
  31. 31.
    García, J., Fardoun, H.M., Alghazzawi, D.M., Cano, J.R., García, S.: MoNGEL: monotonic nested generalized exemplar learning. Pattern Anal. Appl. (2016) (In press). doi: 10.1007/s10044-015-0506-y
  32. 32.
    García, S., Luengo, J., Herrera, F.: Data Preprocessing in Data Mining. Springer, New York (2015)CrossRefGoogle Scholar
  33. 33.
    González, S., Herrera, F., García, S.: Monotonic random forest with an ensemble pruning mechanism based on the degree of monotonicity. New Gener. Comput. 33(4), 367–388 (2015)CrossRefGoogle Scholar
  34. 34.
    Gutiérrez, P.A., Pérez-Ortiz, M., Sánchez-Monedero, J., Fernandez-Navarro, F., Hervás-Martínez, C.: Ordinal regression methods: survey and experimental study. IEEE Trans. Knowl. Data Eng. 28(1), 127–146 (2016). doi: 10.1109/TKDE.2015.2457911 CrossRefGoogle Scholar
  35. 35.
    Harrington, E.F.: Online ranking/collaborative filtering using the perceptron algorithm. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML2003) (2003)Google Scholar
  36. 36.
    Hu, Q., Che, X.Z.L., Zhang, D., Guo, M., Yu, D.: Rank entropy-based decision trees for monotonic classification. IEEE Trans. Knowl. Data Eng. 24(11), 2052–2064 (2012)CrossRefGoogle Scholar
  37. 37.
    Hühn, J.C., Hüllermeier, E.: Is an ordinal class structure useful in classifier learning? Int. J. Data Mining Model. Manag. 1(1), 45–67 (2008)CrossRefzbMATHGoogle Scholar
  38. 38.
    van de Kamp, R., Feelders, A., Barile, N.: Isotonic classification trees. In: Advances in Intelligent Data Analysis VIII, Proceedings of 8th International Symposium on Intelligent Data Analysis, IDA 2009, Lyon, France, August 31–September 2, 2009, pp. 405–416 (2009)Google Scholar
  39. 39.
    Kotlowski, W., Slowinski, R.: Rule learning with monotonicity constraints. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14–18, 2009, pp. 537–544 (2009)Google Scholar
  40. 40.
    Kotlowski, W., Slowiński, R.: On nonparametric ordinal classification with monotonicity constraints. IEEE Trans. Knowl. Eng. 25(11), 2576–2589 (2013)CrossRefGoogle Scholar
  41. 41.
    Lee, J.W.T., Yeung, D.S., Wang, X.: Monotonic decision tree for ordinal classification. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 2623–2628 (2003)Google Scholar
  42. 42.
    Lievens, S., De Baets, B., Cao-Van, K.: A probabilistic framework for the design of instance-based supervised ranking algorithms in an ordinal setting. Ann. Oper. Res. 163(1), 115–142 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Lin, H.T., Li, L.: Reduction from cost-sensitive ordinal ranking to weighted binary classification. Neural Comput. 24(5), 1329–1367 (2012)CrossRefzbMATHGoogle Scholar
  44. 44.
    Makino, K., Suda, T., Ono, H., Ibaraki, T.: Data analysis by positive decision trees. IEICE Trans. Inf. Syst. E82(D(1)), 76–88 (1999)Google Scholar
  45. 45.
    Marsala, C., Petturiti, D.: Rank discrimination measures for enforcing monotonicity in decision tree induction. Inf. Sci. 291, 143–171 (2015)MathSciNetCrossRefGoogle Scholar
  46. 46.
    McCullagh, P.: Regression models for ordinal data. J. R. Stat. Soc. Ser. B (Methodological) 42(2), 109–142 (1980)MathSciNetzbMATHGoogle Scholar
  47. 47.
    Milstein, I., Ben-David, A., Potharst, R.: Generating noisy monotone ordinal datasets. Artif. Intell. Res. 3(1), 30–37 (2014)Google Scholar
  48. 48.
    Nathalie Japkowicz, M.S.: Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, Cambridge (2011)Google Scholar
  49. 49.
    Novak, P.K., Lavrac, N., Webb, G.I.: Supervised descriptive rule discovery: a unifying survey of contrast set, emerging pattern and subgroup mining. J. Mach. Learn. Res. 10, 377–403 (2009)Google Scholar
  50. 50.
    PASCAL: Pascal (Pattern Analysis, Statistical Modelling and Computational Learning) Machine Learning Benchmarks Repository (2011).
  51. 51.
    Perez-Ortiz, M., Gutierrez, P.A., Hervas-Martinez, C., Yao, X.: Graph-based approaches for over-sampling in the context of ordinal regression. IEEE Trans. Knowl. Data Eng. 27(5), 1233–1245 (2015)CrossRefGoogle Scholar
  52. 52.
    Potharst, R., Ben-David, A., van Wezel, M.C.: Two algorithms for generating structured and unstructured monotone ordinal datasets. Eng. Appl. Artif. Intell. 22(4–5), 491–496 (2009)CrossRefGoogle Scholar
  53. 53.
    Potharst, R., Feelders, A.J.: Classification trees for problems with monotonicity constraints. SIGKDD Explor. 4(1), 1–10 (2002)CrossRefGoogle Scholar
  54. 54.
    Qian, Y., Xu, H., Liang, J., Liu, B., Wang, J.: Fusing monotonic decision trees. IEEE Trans. Knowl. Data Eng. 27(10), 2717–2728 (2015)CrossRefGoogle Scholar
  55. 55.
    Rademaker, M., De Baets, B., De Meyer, H.: Optimal monotone relabelling of partially non-monotone ordinal data. Optim. Methods Softw. 27(1), 17–31 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Sousa, R.G., Cardoso, J.S.: Ensemble of decision trees with global constraints for ordinal classification. In: 11th International Conference on Intelligent Systems Design and Applications, ISDA 2011, Córdoba, Spain, November 22–24, 2011, pp. 1164–1169 (2011)Google Scholar
  57. 57.
    Sun, B.Y., Li, J., Wu, D.D., Zhang, X.M., Li, W.B.: Kernel discriminant learning for ordinal regression. IEEE Trans. Knowl. Data Eng. 22(6), 906–910 (2010)CrossRefGoogle Scholar
  58. 58.
    Torra, V., Domingo-Ferrer, J., Mateo-Sanz, J.M., Ng, M.: Regression for ordinal variables without underlying continuous variables. Inf. Sci. 176(4), 465–474 (2006)MathSciNetCrossRefGoogle Scholar
  59. 59.
    Triguero, I., García, S., Herrera, F.: Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study. Knowl. Inf. Syst. 42(2), 245–284 (2015)CrossRefGoogle Scholar
  60. 60.
    Van Gestel, T., Baesens, B., Van Dijcke, P., Garcia, J., Suykens, J., Vanthienen, J.: A process model to develop an internal rating system: Sovereign credit ratings. Decision Support Syst. 42(2), 1131–1151 (2006)CrossRefGoogle Scholar
  61. 61.
    Waegeman, W., De Baets, B., Boullart, L.: Roc analysis in ordinal regression learning. Pattern Recognit. Lett. 29(1), 1–9 (2008)CrossRefzbMATHGoogle Scholar
  62. 62.
    Williams, R.: Generalized ordered logit/partial proportional odds models for ordinal dependent variables. Stata J. 6(1), 58–82 (2006)Google Scholar
  63. 63.
    Zhu, H., Zhai, J., Wang, S., Wang, X.: Monotonic decision tree for interval valued data. In: 13th International Conference on Machine Learning and Cybernetics, pp. 231–240 (2014)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of Computer Science and Numerical AnalysisUniversity of CórdobaCórdobaSpain
  2. 2.Department of Computer Science and Artificial IntelligenceUniversity of GranadaGranadaSpain

Personalised recommendations