Abstract
Five classification algorithms namely J48, Naive Bayes, Multilayer Perceptron, IBK and Bayes Net are evaluated using Mc Nemar’s test over datasets including both nominal and numeric attributes. It was found that Multilayer Perceptron performed better than the two other classification methods for both nominal and numerical datasets. Furthermore, it was observed that the results of our evaluation concur with Kappa statistic and Root Mean Squared Error, two well-known metrics used for evaluating machine learning algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
T. G. Dietterich, “Approximate statistical tests for comparing supervised classification learning algorithms”, Neural Computation, vol. 10, pp. 1895–1923, 1998.
R. R. Bouckaert and E. Frank, “Evaluating the replicability of significance tests for comparing learning algorithms”, in Proceedings 8th Pacific-Asia Conference, pp. 3–12, 2004.
J. Demsar, “Statistical comparisons of classifiers over multiple data sets”, Journal of Machine Learning Research, pp. 1–6, 2006.
D. A. Berry, “Logarithmic transformations in ANOVA”, Biometrics, vol. 43, no. 2, pp. 439–456, 1987.
M. Friedman, “The use of ranks to avoid the assumption of normality implicit in the analysis of variance”, Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937.
M. Friedman, “A comparison of alternative tests of significance for the problem of m rankings”, The Annals of Mathematical Statistics, vol. 1, no. 1, pp. 86–92, 1940.
P. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Pearson, 2006.
D. Aha and D. Kibler, “Instance-based learning algorithms”, Machine Learning, vol. 6, pp. 37–66, 1991.
N. Friedman, D. Geiger, and Goldszmidt, “Bayesian network classifiers”, Machine Learning, vol. 29, pp. 131–163, 1997.
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, “The WEKA data mining software: An update”, SIGKDD Explorations, vol. 11, 2009.
“UCI Machine Learning Repository”. http://archive.ics.uci.edu/ml/, 2012
Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages”, Psychometrika, no. 12, pp. 153–157, 1947.
A. F. Clark and C. Clark, “Performance Characterization in Computer Vision: A Tutorial”.
D. Liddell, “Practical tests of 2 2 contingency tables”, Journal of the Royal Statistical Society, vol. 25, no. 4, pp. 295–304, 1976.
I. H. Witten, E. Frank, and M. A. Hall, Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2011.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer India
About this paper
Cite this paper
Bostanci, B., Bostanci, E. (2013). An Evaluation of Classification Algorithms Using Mc Nemar’s Test. In: Bansal, J., Singh, P., Deep, K., Pant, M., Nagar, A. (eds) Proceedings of Seventh International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA 2012). Advances in Intelligent Systems and Computing, vol 201. Springer, India. https://doi.org/10.1007/978-81-322-1038-2_2
Download citation
DOI: https://doi.org/10.1007/978-81-322-1038-2_2
Published:
Publisher Name: Springer, India
Print ISBN: 978-81-322-1037-5
Online ISBN: 978-81-322-1038-2
eBook Packages: EngineeringEngineering (R0)