Abstract
The evaluation of results in Cluster Analysis frequently appears in the literature, and a variety of evaluation measures have been proposed. On the contrary, in supervised classification, particularly in the discrete case, the subject of results’ evaluation is relatively scarce in this field of the literature. This is the motto underlying this study. The evaluation of the performance of any model of supervised classification is, generally, based on the number of cases correctly or incorrectly predicted by the model. However, these measures can lead to a misleading evaluation when the data is not balanced. More recently, other types of measures have been studied as association or agreement coefficients, the Huberty index, Mutual information, and even ROC curves. Exploratory studies were conducted in this study to understand the relationship between each measure and data characteristics, namely, sample size, balance, and separability of classes. To this end, simulated data and a Beta regression model in the performance of the models were used.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bacelar-Nicolau, H.: The affinity coefficient in cluster analysis. Methods Oper. Res. 53, 507–512 (1985)
Cribari-Neto, F., Zeileis, A.: Beta regression in R. J. Stat. Softw. 34, 1–24 (2010)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20, 37–46 (1960)
Ferreira, A.S., Cardoso, M.G.: Evaluating discriminant analysis results. In: Lita da Silva, J., Caeiro, F., Natário, I., Braumann C. (eds.) Advances in Regression, Survival Analysis, Extreme Values, Markov Processes and and Other Statistical Applications. Studies in Theoretical and Applied Statistics, pp. 155–162. Springer, Heidelberg (2013)
Goldstein, M., Dillon, W.R.: Discrete Discriminant Analysis. Wiley, New York (1978)
Ho, T.K., Basu, M.: Complexity measures of supervised classification problems. IEEE Trans. Pattern Anal. Mach. Intell. 24, 289–300 (2002)
Huberty, C.J., Olejnik, S.: Applied MANOVA and Discriminant Analysis. Wiley-Interscience. Wiley, New Jersey (2006)
Paik, H.: The effect of prior probability on skill in two-group discriminant analysis. Qual Quant. 32, 201–211 (1998)
Santafe, G., Inza, I., Lozano, J.A.: Dealing with the evaluation of supervised classification algorithms. Artif. Intell. Rev. 44, 467–508 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ferreira, A.S., Marques, A. (2021). Performance Measures in Discrete Supervised Classification. In: Chadjipadelis, T., Lausen, B., Markos, A., Lee, T.R., Montanari, A., Nugent, R. (eds) Data Analysis and Rationality in a Complex World. IFCS 2019. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Cham. https://doi.org/10.1007/978-3-030-60104-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-60104-1_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60103-4
Online ISBN: 978-3-030-60104-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)