Abstract
In the paper we consider the problem of the statistical evaluation and comparison of different classification algorithms. For this purpose we apply the methodology of statistical tests for testing independence in the case the multinomial distribution. We propose to use two-sample tests for the comparison of different classification algorithms. In the paper we consider only the case of the supervised classification when an external ‘expert’ evaluates the correctness of classification. The results of the proposed statistical tests are interpreted using possibilistic methodology based on indices of dominance introduced by [7].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Agresti, A.: Categorical Data Analysis, 2nd edn. J. Wiley, Hoboken (2006)
Berthold, M., Hand, D.J. (eds.): Intelligent Data Analysis. An Introduction, 2nd edn. Springer, Berlin (2007)
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. CRC Press, Boca Raton (1984)
Charytanowicz, M., Niewczas, J., Kulczycki, P., Kowalski, P.A., Łukasik, S., Żak, S.: Complete Gradient Clustering Algorithm for Features Analysis of X-Ray Images. In: Piętka, E., Kawa, J. (eds.) Information Technologies in Biomedicine. AISC, vol. 69, pp. 15–24. Springer, Heidelberg (2010)
Desu, M.M., Raghavarao, D.: Nonparametric Statistical Methods for Complete and Censored Data. Chapman & Hall, Boca Raton (2004)
Dubois, D., Prade, H.: Ranking Fuzzy Numbers in the Setting of Possibility Theory. Information Science 30, 183–224 (1983)
Gil, M.A., Hryniewicz, O.: Statistics with Imprecise Data. In: Meyers, R.A. (ed.) Encyclopedia of Complexity and Systems Science, pp. 8679–8690. Springer, Heidelberg (2009)
Hryniewicz, O.: Possibilistic Interpretation of the Results of Statistical Tests. In: Proceedings of Eight International Conference on Information Processing and Management of Uncertainty in Knowledge-based Systems, IPMU 2000, Madrid, pp. 215–219 (2000)
Hryniewicz, O.: Possibilistic decisions and fuzzy statistical tests. Fuzzy Sets and Systems 157, 2665–2673 (2006)
Hryniewicz, O.: Possibilistic methodology for the evaluation of classification algorithms. In: Proceedings of the 6th International Conference on Software and data Technology, ICSOFT 2011, Seville (July 2011)
Krzanowski, W.J.: Principles of Multivariate Analysis: A User’s Perspective. Oxford University Press, New York (1988)
Kulczycki, P., Kowalski, P.A.: Bayes classification of imprecise information of interval type. Control and Cybernetics 40, 101–123 (2011)
Mehta, C.R., Patel, N.R.: Network algorithm for performing Fisher’s exact test in r × c contingency tables. Journ. Amer. Stat. Assoc. 78, 427–434 (1983)
Mehta, C.R., Patel, N.R.: ALGORITHM 643: FEXACT: a FORTRAN subroutine for Fisher’s exact test on unordered r × c contingency tables. ACM Transactions on Mathematical Software (TOMS) 12, 154–161 (1986)
Nisbet, R., Elder, J., Miner, G.: Statistical Analysis and Data Mining. Applications. Elsevier Inc., Amsterdam (2009)
Yarnold, J.K.: The Minimum Expectation in X2 Goodness of fit test and the Accuracy of Approximations for the Null Distribution. Journ. Amer. Stat. Assoc. 70, 864–886 (1970)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hryniewicz, O. (2013). Statistical and Possibilistic Methodology for the Evaluation of Classification Algorithms. In: Escalona, M.J., Cordeiro, J., Shishkov, B. (eds) Software and Data Technologies. ICSOFT 2011. Communications in Computer and Information Science, vol 303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36177-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-36177-7_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36176-0
Online ISBN: 978-3-642-36177-7
eBook Packages: Computer ScienceComputer Science (R0)