Skip to main content

An Evaluation of Classification Algorithms Using Mc Nemar’s Test

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 201)

Abstract

Five classification algorithms namely J48, Naive Bayes, Multilayer Perceptron, IBK and Bayes Net are evaluated using Mc Nemar’s test over datasets including both nominal and numeric attributes. It was found that Multilayer Perceptron performed better than the two other classification methods for both nominal and numerical datasets. Furthermore, it was observed that the results of our evaluation concur with Kappa statistic and Root Mean Squared Error, two well-known metrics used for evaluating machine learning algorithms.

Keywords

  • Classifier evaluation
  • Classification algorithms
  • Mc Nemar’s test

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-81-322-1038-2_2
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-81-322-1038-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   279.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • T. G. Dietterich, “Approximate statistical tests for comparing supervised classification learning algorithms”, Neural Computation, vol. 10, pp. 1895–1923, 1998.

    Google Scholar 

  • R. R. Bouckaert and E. Frank, “Evaluating the replicability of significance tests for comparing learning algorithms”, in Proceedings 8th Pacific-Asia Conference, pp. 3–12, 2004.

    Google Scholar 

  • J. Demsar, “Statistical comparisons of classifiers over multiple data sets”, Journal of Machine Learning Research, pp. 1–6, 2006.

    Google Scholar 

  • D. A. Berry, “Logarithmic transformations in ANOVA”, Biometrics, vol. 43, no. 2, pp. 439–456, 1987.

    Google Scholar 

  • M. Friedman, “The use of ranks to avoid the assumption of normality implicit in the analysis of variance”, Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937.

    Google Scholar 

  • M. Friedman, “A comparison of alternative tests of significance for the problem of m rankings”, The Annals of Mathematical Statistics, vol. 1, no. 1, pp. 86–92, 1940.

    Google Scholar 

  • P. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Pearson, 2006.

    Google Scholar 

  • D. Aha and D. Kibler, “Instance-based learning algorithms”, Machine Learning, vol. 6, pp. 37–66, 1991.

    Google Scholar 

  • N. Friedman, D. Geiger, and Goldszmidt, “Bayesian network classifiers”, Machine Learning, vol. 29, pp. 131–163, 1997.

    Google Scholar 

  • M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, “The WEKA data mining software: An update”, SIGKDD Explorations, vol. 11, 2009.

    Google Scholar 

  • “UCI Machine Learning Repository”. http://archive.ics.uci.edu/ml/, 2012

  • Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages”, Psychometrika, no. 12, pp. 153–157, 1947.

    Google Scholar 

  • A. F. Clark and C. Clark, “Performance Characterization in Computer Vision: A Tutorial”.

    Google Scholar 

  • D. Liddell, “Practical tests of 2 2 contingency tables”, Journal of the Royal Statistical Society, vol. 25, no. 4, pp. 295–304, 1976.

    Google Scholar 

  • I. H. Witten, E. Frank, and M. A. Hall, Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2011.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Betul Bostanci .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer India

About this paper

Cite this paper

Bostanci, B., Bostanci, E. (2013). An Evaluation of Classification Algorithms Using Mc Nemar’s Test. In: Bansal, J., Singh, P., Deep, K., Pant, M., Nagar, A. (eds) Proceedings of Seventh International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA 2012). Advances in Intelligent Systems and Computing, vol 201. Springer, India. https://doi.org/10.1007/978-81-322-1038-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-81-322-1038-2_2

  • Published:

  • Publisher Name: Springer, India

  • Print ISBN: 978-81-322-1037-5

  • Online ISBN: 978-81-322-1038-2

  • eBook Packages: EngineeringEngineering (R0)