Skip to main content
SpringerLink
Log in

On the statistical analysis of the parameters’ trend in a machine learning algorithm

  • Regular Paper
  • Published:
Progress in Artificial Intelligence Aims and scope Submit manuscript

Abstract

Statistical validation of results for supporting the conclusions achieved in an experimental study is more and more demanded in research results. Although statistics are usually used in the analysis of results for comparing the performance of several algorithms, they could be used in other tasks, such as proper selection of parameters’s value or study of the trend of a parameter. In this short paper, we describe a non-parametric test, the Page test, which can be used for predicting the order of experimental conditions. We include an illustrative example for using it on classification problems taking the well-known \(k\)-nearest neighbour algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Altinçay, J.: Decision trees using model ensemble-based nodes. Pattern Recognit. 40, 3540–3551 (2007)

    Article  MATH  Google Scholar 

  2. Conover, W.J.: Practical Nonparametric Statistic, 3rd edn. Wiley, New York (1999)

  3. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MATH  MathSciNet  Google Scholar 

  4. Derrac, J., García, S., Molina, D., Herrera, F.: A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1, 3–18 (2011)

    Article  Google Scholar 

  5. García, S., Herrera, F.: An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J. Mach. Learn. Res. 9, 2677–2694 (2008)

    MATH  Google Scholar 

  6. García, S., Molina, D., Lozano, M., Herrera, F.: A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J. Heuristics 15, 617–644 (2009)

    Article  MATH  Google Scholar 

  7. García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental Analysis of Power. Inf. Sci. 180, 2044–2064 (2010)

    Article  Google Scholar 

  8. Gibbons, J., Chakraborti, S.: Nonparametric Statistical Inference, 5th edn. Chapman & Hall, Boca Raton (2010)

  9. Manocha, S., Girolami, M.A.: An empirical analysis of the probabilistic K-nearest neighbour classifier. Pattern Recognit. Lett. 28, 1818–1824 (2007)

    Article  Google Scholar 

  10. Page, E.B.: Ordered hypotheses for multiple treatments: a significance test for linear ranks. J. Am. Stat. Assoc. 58, 216–230 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  11. Sheskin, D.: Handbook of Parametric and Nonparametric Statistical Procedures, 5th edn. Chapman & Hall/CRC, Boca Raton (2011)

  12. Sun, S., Zhang, C., Zhang, D.: An experimental evaluation of ensemble methods for EEG signal classification. Pattern Recognit. Lett. 28, 2157–2163 (2007)

    Article  Google Scholar 

  13. Zar, J.H.: Biostatistical Analysis, 5th edn. Prentice Hall, New Jersey (2009)

Download references

Acknowledgments

This work was partially supported by the Spanish Ministry of Science and Technology under project TIN2011-28488 and the Andalusian Research Plans P11-TIC-7765, P10-TIC-6858.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Jaén, Campus las Lagunillas S/N, 23071 , Jaén, Spain

    Salvador García

  2. School of Computer Science and Informatics, Cardiff University, 5 The Parade, CF24 3AA , Cardiff, UK

    Joaquín Derrac

  3. Department of Computer Science and AI, Research Center on Information and Communications Technology (CITIC-UGR), University of Granada, Granada, Spain

    Sergio Ramírez-Gallego & Francisco Herrera

Authors
  1. Salvador García
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joaquín Derrac
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergio Ramírez-Gallego
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francisco Herrera
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salvador García.

Appendix

Appendix

See Table 2.

Table 2 Critical \(L\) values at a level of significance \(\alpha = 0.05\)
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

García, S., Derrac, J., Ramírez-Gallego, S. et al. On the statistical analysis of the parameters’ trend in a machine learning algorithm. Prog Artif Intell 3, 51–53 (2014). https://doi.org/10.1007/s13748-014-0043-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13748-014-0043-8

Keywords

Search

Navigation