A Comparison of Validation Methods for Learning Vector Quantization and for Support Vector Machines on Two Biomedical Data Sets

  • David Sommer
  • Martin Golz
Conference paper
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)

Abstract

We compare two comprehensive classification algorithms, support vector machines (SVM) and several variants of learning vector quantization (LVQ), with respect to different validation methods. The generalization ability is estimated by “multiple-hold-out” (MHO) and by “leave-one-out” (LOO) cross v method. The ξα-method, a further estimation method, which is only applicable for SVM and is computationally more efficient, is also used.

Calculations on two different biomedical data sets generated of experimental data measured in our own laboratory are presented. The first data set contains 748 feature vectors extracted of posturographic signals which were obtained in investigations of balance control in upright standing of 48 young adults. Two different classes are labelled as “without alcoholic impairment” and “with alcoholic impairment”. This classification task aims the detection of small unknown changes in a relative complex signal with high inter-individual variability.

The second data set contains 6432 feature vectors extracted of electroencephalographic and electroocculographic signals recorded during overnight driving simulations of 22 young adults. Short intrusions of sleep during driving, so-called microsleep events, were observed. They form examples of the first class. The second class contains examples of fatigue states, whereas driving is still possible. If microsleep events happen in typical states of brain activity, the recorded signals should contain typical alterations, and therefore discrimination from signals of the second class, which do not refer to such states, should be possible.

Optimal kernel parameters of SVM are found by searching minimal test errors with all three validation methods. Results obtained on both different biomedical data sets show different optimal kernel parameters depending on the validation method. It is shown, that the ξα-method seems to be biased and therefore LOO or MHO method should be preferred.

A comparison of eight different variants of LVQ and six other classification methods using MHO validation yields that SVM performs best for the second and more complex data set and SVM, GRLVQ and OLVQ1 show nearly the same performance for the first data set.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. CAO, L.J. and TAY, F.E.H. (2003): Support Vector Machine With Adaptive Parameters in Financial Time Series Forecasting. IEEE Transactions on Neural Networks, 14, 1506–1518.CrossRefGoogle Scholar
  2. CORTES, C. and VAPNIK, V. (1995): Support Vector Networks. Machine Learning, 20, 273–297.Google Scholar
  3. DEVROYE, L.; GYORFI, L.; LUGOSI, G. (1996): A probabilistic theory of pattern recognition. Springer; New York.Google Scholar
  4. FRITZKE, B. (1994): Growing Cell Structures — A Self-Organizing Network for Unsupervised and Supervised Learning. Neural Networks, 7, 1441–1460.CrossRefGoogle Scholar
  5. GOLZ, M.; SOMMER, D.; LEMBCKE, T.; KURELLA, B. (1998): Classification of the pre-stimulus-EEG of k-complexes using competitive learning networks. EUFIT’ 98. Aachen, 1767–1771.Google Scholar
  6. GOLZ, M.; SOMMER, D.; WALTHER, L.; EURICH, C. (2004): Discriminance Analysis of Postural Sway Trajectories with Neural Networks SCI2004, VII. Orlando, USA, 151–155.Google Scholar
  7. HAMMER, B. and VILLMANN, T. (2002): Generalized relevance learning vector quantization. Neural Networks, 15, 1059–1068.CrossRefGoogle Scholar
  8. JOACHIMS, T. (2002): Learning to Classify Text Using Support Vector Machines. Kluwer; Boston.Google Scholar
  9. KEARNS, M. (1996): A Bound on the Error of Cross Validation Using the Approximation and Estimation Rates, with Consequences for the Training-Test Split. Advances in Neural Information Processing Systems, 8, 183–189.Google Scholar
  10. KOHONEN, T. (2001): Self-Organizing Maps (third edition). Springer, New York.Google Scholar
  11. MÜLLER, K.-R.; S. MIKA; RÄTSCH, G.; TSUDA, K.; SCHÖLKOPF, B. (2001): An Introduction to Kernel-Based Learning Algorithms. IEEE Transactions on Neural Networks, 12(2), 181–201.Google Scholar
  12. OSUNA, E.; FREUND, R.; GIROSI, F.; (1997): Training Support Vector Machines: an Application to Face Detection. Proceedings of CVPR’ 97. Puerto Rico.Google Scholar
  13. SATO, A. (1999): An Analysis of Initial State Dependence in Generalized LVQ. In: D. Willshaw et al. (Eds.): (ICANN’ 99). IEEE Press;, 928–933.Google Scholar
  14. SOMMER, D. and GOLZ, M. (2003): Short-Time Prognosis of Microsleep Events by Artificial Neural Networks. Proc. Medizin und Mobilität. Berlin, 149–151.Google Scholar
  15. SONG, H. and LEE, S. (1996): LVQ Combined with Simulated Annealing for Optimal Design of Large-set Reference Models. Neural Networks, 9, 329–336.CrossRefGoogle Scholar
  16. VAN GESTEL, T.; SUYKENS, J.; BAESENS, B.; VIAENE, S.; VANTHIENEN, J.; DEDENE, G.; DE MOOR, B.; VANDEWALLE, J. (2002): Benchmarking least squares support vector machine classifiers. Machine Learning.Google Scholar
  17. VAPNIK, V. (1995): The Nature of Statistical Learning Theory. Springer, New York.Google Scholar

Copyright information

© Springer Berlin · Heidelberg 2006

Authors and Affiliations

  • David Sommer
    • 1
  • Martin Golz
    • 1
  1. 1.Department of Computer ScienceUniversity of Applied Sciences SchmalkaldenGermany

Personalised recommendations