Skip to main content
Book cover

Proceedings of Seventh International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA 2012) pp 15–26Cite as

An Evaluation of Classification Algorithms Using Mc Nemar’s Test

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 201)

Abstract

Five classification algorithms namely J48, Naive Bayes, Multilayer Perceptron, IBK and Bayes Net are evaluated using Mc Nemar’s test over datasets including both nominal and numeric attributes. It was found that Multilayer Perceptron performed better than the two other classification methods for both nominal and numerical datasets. Furthermore, it was observed that the results of our evaluation concur with Kappa statistic and Root Mean Squared Error, two well-known metrics used for evaluating machine learning algorithms.

Keywords

  • Classifier evaluation
  • Classification algorithms
  • Mc Nemar’s test

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-81-322-1038-2_2
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-81-322-1038-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   279.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • T. G. Dietterich, “Approximate statistical tests for comparing supervised classification learning algorithms”, Neural Computation, vol. 10, pp. 1895–1923, 1998.

    Google Scholar 

  • R. R. Bouckaert and E. Frank, “Evaluating the replicability of significance tests for comparing learning algorithms”, in Proceedings 8th Pacific-Asia Conference, pp. 3–12, 2004.

    Google Scholar 

  • J. Demsar, “Statistical comparisons of classifiers over multiple data sets”, Journal of Machine Learning Research, pp. 1–6, 2006.

    Google Scholar 

  • D. A. Berry, “Logarithmic transformations in ANOVA”, Biometrics, vol. 43, no. 2, pp. 439–456, 1987.

    Google Scholar 

  • M. Friedman, “The use of ranks to avoid the assumption of normality implicit in the analysis of variance”, Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937.

    Google Scholar 

  • M. Friedman, “A comparison of alternative tests of significance for the problem of m rankings”, The Annals of Mathematical Statistics, vol. 1, no. 1, pp. 86–92, 1940.

    Google Scholar 

  • P. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Pearson, 2006.

    Google Scholar 

  • D. Aha and D. Kibler, “Instance-based learning algorithms”, Machine Learning, vol. 6, pp. 37–66, 1991.

    Google Scholar 

  • N. Friedman, D. Geiger, and Goldszmidt, “Bayesian network classifiers”, Machine Learning, vol. 29, pp. 131–163, 1997.

    Google Scholar 

  • M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, “The WEKA data mining software: An update”, SIGKDD Explorations, vol. 11, 2009.

    Google Scholar 

  • “UCI Machine Learning Repository”. http://archive.ics.uci.edu/ml/, 2012

  • Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages”, Psychometrika, no. 12, pp. 153–157, 1947.

    Google Scholar 

  • A. F. Clark and C. Clark, “Performance Characterization in Computer Vision: A Tutorial”.

    Google Scholar 

  • D. Liddell, “Practical tests of 2 2 contingency tables”, Journal of the Royal Statistical Society, vol. 25, no. 4, pp. 295–304, 1976.

    Google Scholar 

  • I. H. Witten, E. Frank, and M. A. Hall, Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2011.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK

    Betul Bostanci & Erkan Bostanci

Authors
  1. Betul Bostanci
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erkan Bostanci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Betul Bostanci .

Editor information

Editors and Affiliations

  1. South Asian University, Chankya Puri, New Delhi, 110021, India

    Jagdish Chand Bansal

  2. ABV - IIITM, Gwalior, Gwalior, 474015, Madhya Pradesh, India

    Pramod Kumar Singh

  3. , Department of Mathematics, Indian Institute of Technology Roorkee, Roorkee, 247667, India

    Kusum Deep

  4. , Department of Paper Technology, Indian Institute of Technology Roorkee, Saharanpur Campus, Roorkee, India

    Millie Pant

  5. , Department of Computer Science, Liverpool Hope University, Office: FML 412, Liverpool, Liverpool, L16 9JD, United Kingdom

    Atulya K. Nagar

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer India

About this paper

Cite this paper

Bostanci, B., Bostanci, E. (2013). An Evaluation of Classification Algorithms Using Mc Nemar’s Test. In: Bansal, J., Singh, P., Deep, K., Pant, M., Nagar, A. (eds) Proceedings of Seventh International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA 2012). Advances in Intelligent Systems and Computing, vol 201. Springer, India. https://doi.org/10.1007/978-81-322-1038-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-81-322-1038-2_2

  • Published:

  • Publisher Name: Springer, India

  • Print ISBN: 978-81-322-1037-5

  • Online ISBN: 978-81-322-1038-2

  • eBook Packages: EngineeringEngineering (R0)