Advertisement

Progress in Artificial Intelligence

, Volume 3, Issue 1, pp 51–53 | Cite as

On the statistical analysis of the parameters’ trend in a machine learning algorithm

  • Salvador García
  • Joaquín Derrac
  • Sergio Ramírez-Gallego
  • Francisco Herrera
Regular Paper

Abstract

Statistical validation of results for supporting the conclusions achieved in an experimental study is more and more demanded in research results. Although statistics are usually used in the analysis of results for comparing the performance of several algorithms, they could be used in other tasks, such as proper selection of parameters’s value or study of the trend of a parameter. In this short paper, we describe a non-parametric test, the Page test, which can be used for predicting the order of experimental conditions. We include an illustrative example for using it on classification problems taking the well-known \(k\)-nearest neighbour algorithm.

Keywords

Statistical analysis Non-parametric statistics Parameter trend Classification 

Notes

Acknowledgments

This work was partially supported by the Spanish Ministry of Science and Technology under project TIN2011-28488 and the Andalusian Research Plans P11-TIC-7765, P10-TIC-6858.

References

  1. 1.
    Altinçay, J.: Decision trees using model ensemble-based nodes. Pattern Recognit. 40, 3540–3551 (2007)CrossRefzbMATHGoogle Scholar
  2. 2.
    Conover, W.J.: Practical Nonparametric Statistic, 3rd edn. Wiley, New York (1999)Google Scholar
  3. 3.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Derrac, J., García, S., Molina, D., Herrera, F.: A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1, 3–18 (2011)CrossRefGoogle Scholar
  5. 5.
    García, S., Herrera, F.: An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J. Mach. Learn. Res. 9, 2677–2694 (2008)zbMATHGoogle Scholar
  6. 6.
    García, S., Molina, D., Lozano, M., Herrera, F.: A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J. Heuristics 15, 617–644 (2009)CrossRefzbMATHGoogle Scholar
  7. 7.
    García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental Analysis of Power. Inf. Sci. 180, 2044–2064 (2010)CrossRefGoogle Scholar
  8. 8.
    Gibbons, J., Chakraborti, S.: Nonparametric Statistical Inference, 5th edn. Chapman & Hall, Boca Raton (2010)Google Scholar
  9. 9.
    Manocha, S., Girolami, M.A.: An empirical analysis of the probabilistic K-nearest neighbour classifier. Pattern Recognit. Lett. 28, 1818–1824 (2007)CrossRefGoogle Scholar
  10. 10.
    Page, E.B.: Ordered hypotheses for multiple treatments: a significance test for linear ranks. J. Am. Stat. Assoc. 58, 216–230 (1963)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Sheskin, D.: Handbook of Parametric and Nonparametric Statistical Procedures, 5th edn. Chapman & Hall/CRC, Boca Raton (2011)Google Scholar
  12. 12.
    Sun, S., Zhang, C., Zhang, D.: An experimental evaluation of ensemble methods for EEG signal classification. Pattern Recognit. Lett. 28, 2157–2163 (2007)CrossRefGoogle Scholar
  13. 13.
    Zar, J.H.: Biostatistical Analysis, 5th edn. Prentice Hall, New Jersey (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Salvador García
    • 1
  • Joaquín Derrac
    • 2
  • Sergio Ramírez-Gallego
    • 3
  • Francisco Herrera
    • 3
  1. 1.Department of Computer ScienceUniversity of JaénJaénSpain
  2. 2.School of Computer Science and InformaticsCardiff UniversityCardiffUK
  3. 3.Department of Computer Science and AI, Research Center on Information and Communications Technology (CITIC-UGR)University of GranadaGranadaSpain

Personalised recommendations