Statistical re-sampling techniques have been used extensively and successfully in the machine learning approaches for generation of classifier and predictor ensembles. It has been frequently shown that combining so called unstable predictors has a stabilizing effect on and improves the performance of the prediction system generated in this way. In this paper we use the re-sampling techniques in the context of Principal Component Analysis (PCA). We show that the proposed PCA ensembles exhibit a much more robust behaviour in the presence of outliers which can seriously affect the performance of an individual PCA algorithm. The performance and characteristics of the proposed approaches are illustrated on a number of experimental studies where an individual PCA is compared to the introduced PCA ensemble.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Pearson, K.: On lines and planes of closest fit to systems of points in space. Philosophical Magazine 2, 559–572 (1901)Google Scholar
  2. Hotelling, H.: Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24, 417–441, 498–520 (1933)CrossRefGoogle Scholar
  3. Cook, R.D.: Detection of influential observations in linear regression. Technometrics 19, 15–18 (1977)zbMATHCrossRefMathSciNetGoogle Scholar
  4. Dixon, W.J.: Analysis of extreme values. Ann. Math. Stat. 21, 488–506 (1950)CrossRefGoogle Scholar
  5. Oja, E.: Neural networks, principal components and subspaces. International Journal of Neural Systems 1(1), 61–68 (1989)CrossRefMathSciNetGoogle Scholar
  6. Sanger, D.: Contribution analysis: A technique for assigning responsibilities to hidden units in connectionist networks. Connection Science 1, 115–138 (1989)CrossRefGoogle Scholar
  7. Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  8. Schapire, R.E., Freud, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. The Annals of Statistics 26(5), 1651–1686 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  9. Gabrys, B.: Combining neuro-fuzzy classifiers for improved generalisation and reliability. In: Proceedings the Int. Joint Conference on Neural Networks (IJCNN 2002) a part of the WCCI 2002 Congress, Honolulu, USA, pp. 2410–2415 (2002)Google Scholar
  10. Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley Interscience, Chichester (2004)zbMATHCrossRefGoogle Scholar
  11. Ruta, D., Gabrys, B.: Classifier Selection for Majority Voting. Special issue of the journal of information fusion on Diversity in Multiple Classifier Systems 6(1), 63–81 (2005)Google Scholar
  12. Gabrys, B.: Learning Hybrid Neuro-Fuzzy Classifier Models From Data: To Combine or not to Combine? Fuzzy Sets and Systems 147, 39–56 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  13. Ruta, D., Gabrys, B.: A Theoretical Analysis of the Limits of Majority Voting Errors for Multiple Classifier Systems. Pattern Analysis and Applications 5, 333–350 (2002)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Bogdan Gabrys
    • 1
  • Bruno Baruque
    • 2
  • Emilio Corchado
    • 2
  1. 1.Computational Intelligence Research GroupBournemouth UniversityUnited Kingdom
  2. 2.Department of Civil EngeneeringUniversity of BurgosSpain

Personalised recommendations