Skip to main content
SpringerLink
Log in

A Generalized Logistic Link Function for Cumulative Link Models in Ordinal Regression

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Ordinal regression is a kind of regression analysis used for predicting an ordered response variable. In these problems, the patterns are labelled by a set of ranks with an ordering among the different categories. The most common type of ordinal regression model is the cumulative link model. The cumulative link model relates an unobserved continuous latent variable with a monotone link function. Logit and probit functions are examples of link functions used in cumulative link models. In this paper, a novel generalized link function based on a generalization of the logistic distribution is proposed. The generalized link function proposed is able to reproduce other different link functions by changing two real parameters: \(\alpha \) and \(\lambda \). The generalized link function has been included in a cumulative link model where the latent function is determined by a standard neural network in order to test the performance of the proposal. For this model, a reformulation of the tunable thresholds and distribution parameters was applied to convert the constrained optimization problem into an unconstrained optimization problem. Experimental results demonstrate that our proposed approach can achieve competitive generalization performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The datasets and results are available at http://www.gatsby.ucl.ac.uk/~chuwei/ordinalregression.html.

  2. JCLEC url: http://jclec.sourceforge.net.

  3. JCLEC url: http://jclec.sourceforge.net.

References

  1. Agresti A (1984) Analysis of ordinal categorical data. Wiley Series in Probability and Statistics, New York

    MATH  Google Scholar 

  2. Ahuja J, Nash SW (1967) The generalized gompertz-verhulst family of distributions. Sankhya 29(Ser.A.):141–156

  3. Aitchison J, Silvey SD (1957) The generalization of probit analysis to the case of multiple responses. Biometrika 44:131–140

    Article  MathSciNet  MATH  Google Scholar 

  4. Anderson JA (1984) Regression and ordered categorical variables. J R Stat Soc Ser B (Methodological) 46(1):1–30

    MathSciNet  MATH  Google Scholar 

  5. Aranda-Ordaz F (1983) An extension of the proportional-hazards model for grouped data. Biometrics 39(1):109–117

    Article  MathSciNet  MATH  Google Scholar 

  6. Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html

  7. Baccianella S, Esuli A, Sebastiani F (2009) Evaluation measures for ordinal regression. In: Proceedings of the 2009 9th international conference on intelligent systems design and applications, ISDA ’09, pp 283–287

  8. Cardoso J, Pinto da Costa J (2007) Learning to classify ordinal data: the data replication method. J Mach Learn Res 8:1393–1429

    MathSciNet  MATH  Google Scholar 

  9. Caruana R, Baluja S, Mitchell T (1996) Using the future to sort out the present: rankprop and multitask learning for medical risk evaluation. Adv Neural Inf Process Syst 8:959–965

    Google Scholar 

  10. Cheng J, Wang Z, Pollastri G (2008) A neural network approach to ordinal regression. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN2008, IEEE world congress on computational intelligence). IEEE Press, pp 1279–1284

  11. Chu W, Ghahramani Z (2005) Gaussian processes for ordinal regression. J Mach Learn Res 6:1019–1041

    MathSciNet  MATH  Google Scholar 

  12. Chu W, Keerthi SS (2005) New approaches to support vector ordinal regression. In: ICML 05: proceedings of the 22nd international conference on machine learning, pp 145–152

  13. Chu W, Keerthi SS (2007) Support vector ordinal regression. Neural Comput 19:792–815

    Article  MathSciNet  MATH  Google Scholar 

  14. Crammer K, Singer Y (2005) Online ranking by projecting. Neural Comput 17:145–175

    Article  MATH  Google Scholar 

  15. da Costa P, Alonso H, Cardoso JS (2008) The unimodal model for the classification of ordinal data. Neural Netw 21(1):78–91

    Article  MATH  Google Scholar 

  16. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  17. Deng WY, Zheng QH, Lian S, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74(1–3):447–456

    Article  Google Scholar 

  18. Fahrmeir L, Tutz G (2001) Multivariate statistical modelling based on generalized linear models, 2nd edn. Springer, New York

    Book  MATH  Google Scholar 

  19. Fernández-Navarro F, Campoy P, De la Paz M, Hervás-Martínez C, Yao X (2013) Addressing the EU sovereign ratings using an ordinal regression approach. IEEE Trans Cybern 43(6):2228–2240

    Article  Google Scholar 

  20. Fernández-Navarro F, Hervás-Martínez C, Gutierrez PA (2013) Generalised gaussian radial basis function neural networks. Soft Comput 17(3):519–533

    Article  Google Scholar 

  21. Fernandez-Navarro F, Riccardi A, Carloni S (2014) Ordinal neural networks without iterative tuning. IEEE Trans Neural Netw Learn Syst 25(11):2075–2085

    Article  Google Scholar 

  22. Friedman M (1940) A comparison of alternative tests of significance for the problem of \(m\) rankings. Ann Math Stat 11(1):86–92

    Article  MathSciNet  MATH  Google Scholar 

  23. Genter FC, Farewell VT (1985) Goodness-of-link testing in ordinal regression models. Can J Stat 13(1):37–44

    Article  MATH  Google Scholar 

  24. Gupta RD, Kundu D (1999) Theory & methods: generalized exponential distributions. Aust N Z J Stat 41(2):173–188

    Article  MathSciNet  MATH  Google Scholar 

  25. Gutiérrez PA, Pérez-Ortiz M, Sánchez-Monedero J, Fernández-Navarro F, Hervás-Martínez C (2016) Ordinal regression methods: survey and experimental study. IEEE Trans Knowl Data Eng 28(1):127–146

    Article  Google Scholar 

  26. Hedeker D, Berbaum M, Mermelstein R (2006) Location-scale models for multilevel ordinal data: between and within subjects variance modeling. J Probab Stat Sci 1(4):1–20

    MathSciNet  Google Scholar 

  27. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 42(2):513–529

    Article  Google Scholar 

  28. Igel C, Hüsken M (2003) Empirical evaluation of the improved rprop learning algorithms. Neurocomputing 50(6):105–123

    Article  MATH  Google Scholar 

  29. Ishwaran H, Gatsonis CA (2000) A general class of hierarchical ordinal regression models with applications to correlated ROC analysis. Can J Stat 28(4):731–750

    Article  MathSciNet  MATH  Google Scholar 

  30. Kendall M (1948) Rank correlation methods. Griffin, London

    MATH  Google Scholar 

  31. Kosmidis I (2014) Improved estimation in cumulative link models. J R Stat Soc Ser B (Stat Methodol) 76(1):169–196

    Article  MathSciNet  Google Scholar 

  32. Li L, Lin HT (2007) Ordinal regression by extended binary classification. Adv Neural Inf Process Syst 19:865–872

    Google Scholar 

  33. Lim TS, Loh WY, Shih YS (2000) A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Mach Learn 40(3):203–228

    Article  MATH  Google Scholar 

  34. Martínez-Estudillo FJ, Hervás-Martínez C, Peña PG, Martínez-Estudillo AC, Ventura-Soto S (2006) Evolutionary product-unit neural networks for classification. In: Intelligent data engineering and automated learning—IDEAL 2006. Springer, New York, pp 1320–1328

  35. Mathieson MJ (1996) Ordinal models for neural networks. In: Refenes APN, Abu-Mostafa Y, Weigend A (eds) Proceedings of the 3rd international conference on neural networks in the capital markets, neural networks in financial engineering. World Scientific, Singapore, pp 523–536

  36. McCullagh P (1980) Regression models for ordinal data. J R Stat Soc Ser B 4:109–142

    MathSciNet  MATH  Google Scholar 

  37. Pratt JWT (1981) Concavity of the log likelihood. J Am Stat Assoc 76(373):103–106

    Article  MathSciNet  MATH  Google Scholar 

  38. Prentice RL (1974) A log gamma model and its maximum likelihood estimation. Biometrika 61(3):539–544

    Article  MathSciNet  MATH  Google Scholar 

  39. Riccardi A, Fernandez-Navarro F, Carloni S (2014) Cost-sensitive adaboost algorithm for ordinal regression based on extreme learning machine. IEEE Trans Cybern 44(10):1898–1909

    Article  Google Scholar 

  40. Seah CW, Tsang IW, Ong YS (2012) Transductive ordinal regression. IEEE Trans Neural Netw Learn Syst 23(7):1074–1086

    Article  Google Scholar 

  41. Sun BY, Wang HL, Li WB, Wang HJ, Li J, Du ZQ (2014) Constructing and combining orthogonal projection vectors for ordinal regression. Neural Process Lett 41(1):139–155

    Article  Google Scholar 

  42. Sun P, Yao X (2010) Sparse approximation through boosting for learning large scale kernel machines. IEEE Trans Neural Netw 21(6):883–894

    Article  Google Scholar 

  43. Thomas P, Suhner MC (2015) A new multilayer perceptron pruning algorithm for classification and regression applications. Neural Process Lett 42(2):437–458

    Article  Google Scholar 

  44. Verwaeren J, Waegeman W, De Baets B (2012) Learning partial ordinal class memberships with kernel-based proportional odds models. Comput Stat Data Anal 56(4):928–942

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The author would like to thank Professor Jaime Cardoso who kindly provided us with the source code of the oNN algorithm. This work has been partially subsidised by the TIN2014-54583-C2-1-R project of the Spanish Ministry of Economy and Competitiveness (MINECO), FEDER funds and the P2011-TIC-7508 project of the “Junta de Andalucia” (Spain).

Author information

Authors and Affiliations

  1. Department of Quantitative Methods, Universidad Loyola Andalucia, Seville, Spain

    F. Fernández-Navarro

Authors
  1. F. Fernández-Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Fernández-Navarro.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernández-Navarro, F. A Generalized Logistic Link Function for Cumulative Link Models in Ordinal Regression. Neural Process Lett 46, 251–269 (2017). https://doi.org/10.1007/s11063-017-9589-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-017-9589-3

Keywords

Search

Navigation