Genetic algorithms and particle swarm optimization for exploratory projection pursuit

  • Alain Berro
  • Souad Larabi Marie-Sainte
  • Anne Ruiz-GazenEmail author


Exploratory Projection Pursuit (EPP) methods have been developed thirty years ago in the context of exploratory analysis of large data sets. These methods consist in looking for low-dimensional projections that reveal some interesting structure existing in the data set but not visible in high dimension. Each projection is associated with a real valued index which optima correspond to valuable projections. Several EPP indices have been proposed in the statistics literature but the main problem lies in their optimization. In the present paper, we propose to apply Genetic Algorithms (GA) and recent Particle Swarm Optimization (PSO) algorithm to the optimization of several projection pursuit indices. We explain how the EPP methods can be implemented in order to become an efficient and powerful tool for the statistician. We illustrate our proposal on several simulated and real data sets.


Clustering Exploratory projection pursuit Genetic algorithm Particle swarm optimization 

Mathematics Subject Classifications (2010)

62-07 62-09 62H99 68-04 68T20 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Achard, V., Landrevie, A., Fort, J.-C.: Anomalies detection in hyperspectral imagery using projection pursuit algorithm, image and signal processing for remote sensing X. In: Bruzzone, L. (ed.) Proceedings of the SPIE, vol. 5573, pp. 193–202 (2004)Google Scholar
  2. 2.
    Art, D., Gnanadesikan, R., Kettenring, J.R.: Data-based metrics for cluster analysis. Util. Math. 21A, 75–99 (1982)MathSciNetGoogle Scholar
  3. 3.
    Caussinus, H., Ruiz-Gazen, A.: Metrics for finding typical structures by means of principal component analysis. In: Escoufier, Y. et al. (eds.) Data Science and its Applications, pp. 177–192. Academic (1995)Google Scholar
  4. 4.
    Caussinus, H., Ruiz-Gazen, A.: Classification and generalized principal component analysis. In: Brito et al. (eds.) Selected Contributions in Data Analysis and Classification, pp. 539–548. Springer (2007)Google Scholar
  5. 5.
    Caussinus, H., Ruiz-Gazen, A.: Exploratory projection pursuit. In: Govaert, G. (ed.) Data Analysis (Digital Signal and Image Processing series). ISTE (2009)Google Scholar
  6. 6.
    Chiang, S., Chang, C.: Unsupervised target detection in hyperspectral images using projection pursuit. IEEE Trans. Geosci. Remote Sens. 39(7), 1380–1391 (2001)CrossRefGoogle Scholar
  7. 7.
    Clerc, M.: Particle swarm optimization. International Scientific and Technical Encyclopaedia. Wiley, Hoboken (2006)Google Scholar
  8. 8.
    Cook, D., Buja, A., Cabrera, J.: Projection pursuit indices based on orthogonal function expansions. J. Comput. Graph. Stat. 2(3), 225–250 (1993)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Cook, D., Caragea, D., Honavar, H.: Visualization in classification problems. In: Antoch, J. (ed.) Proceeding in Computational Statistics (COMPSTAT 2004), pp. 799–806. Springer, Berlin (2004)Google Scholar
  10. 10.
    Cook, D., Swayne, D.F.: Interactive and Dynamic Graphics for Data Analysis. Springer, New York (2007)zbMATHCrossRefGoogle Scholar
  11. 11.
    Crawford, S.L.: Genetic optimization for exploratory projection pursuit. In: Computer Science and Statistics: Proc. 23rd Symp. Interface (1992)Google Scholar
  12. 12.
    Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer (2003)Google Scholar
  13. 13.
    Fogel, L.J., Owens, A.J., Walsh, M.J.: Artificial Intelligence Through Simulated Evolution. Wiley, New York (1966)zbMATHGoogle Scholar
  14. 14.
    Friedman, J.H.: Exploratory projection pursuit. J. Am. Stat. Assoc. 82(1), 249–266 (1987)zbMATHCrossRefGoogle Scholar
  15. 15.
    Friedman, J.H., Tukey, J.W.: A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. C-23, 881–889 (1974)CrossRefGoogle Scholar
  16. 16.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning, 412 p. Addison Wesley Longman (1989)Google Scholar
  17. 17.
    Glover, D.M., Hopke, P.K.: Exploration of multivariate chemical data by projection pursuit. Chemometr. Intell. Lab. Syst. 16, 45–59 (1992)CrossRefGoogle Scholar
  18. 18.
    Guo, Q., Wu, W., Massart, D., Boucon, S., de Jong, S.: Sequential projection pursuit using genetic algorithms for data mining of analytical data. Anal. Chem. 72(13), 2846–2855 (2000)CrossRefGoogle Scholar
  19. 19.
    Hansen, N.: The CMA Evolution Strategy. Website∼hansen/cmaesintro.html (1996)
  20. 20.
    Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Harbor (1975)Google Scholar
  21. 21.
    Huber, P.J.: Projection pursuit. Ann. Stat. 13(2), 435–475 (1985)zbMATHCrossRefGoogle Scholar
  22. 22.
    Jones, M.C., Sibson, R.: What is projection pursuit? (with discussion). J. R. Stat. Soc. A 150, 1–36 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Yuhui Shi (1995)Google Scholar
  24. 24.
    Klinke, S.: Data Structures in Computational Statistics. Physica-Verlag (1997)Google Scholar
  25. 25.
    Krink, T., Vesterstrom, J., Riget, J.: Particle swarm optimization with spatial particle extension. In: Proceedings of the Fourth Congress on Evolutionary Computation (CEC-2002), vol. 2, pp. 1474–1479. IEE, Piscataway (2002)CrossRefGoogle Scholar
  26. 26.
    Kruskal, J.B.: Toward a practical method which helps uncover the structure of a set of multivariate observations by finding the linear transformation which optimizes a new index of condensation. Statistical Computing, pp. 427–440. Academic, New York (1969)Google Scholar
  27. 27.
    Lee, E., Klinke, S., Cook, D., Thomas, L.: Projection pursuit for exploratory supervised classification. J. Comput. Graph. Stat. 14(4), 831–846 (2005)CrossRefGoogle Scholar
  28. 28.
    Li, X.: A multimodal particle swarm optimizer based on fitness euclidean-distance ratio. In: Thierens, D. (ed.) Proceeding of Genetic and Evolutionary Computation Conference 2007 (GECCO’07), pp. 78–85. ACM (2007)Google Scholar
  29. 29.
    Lovbjerg, M., Krink, T.: Extending particle swarms with self-organized criticality. In: Proceedings of the Fourth Congress on Evolutionary Computation (CEC-2002), vol. 2, pp. 1588–1593. IEE, Piscataway (2002)CrossRefGoogle Scholar
  30. 30.
    Lubischew, A.A.: On the use of discriminant functions in Taxonomy. Biometrics 18, 455–477 (1962)zbMATHCrossRefGoogle Scholar
  31. 31.
    Malpica, J.A., Rejas, J.G., Alonso, M.C.: A projection pursuit algorithms for anomaly detection in hyperspectral imagery. Pattern Recogn. 41, 3313–3327 (2008)zbMATHCrossRefGoogle Scholar
  32. 32.
    Martinez, W., Martinez, A.: Computational Statistics Handbook with Matlab. CRC (Taylor and Francis Group) (2001)Google Scholar
  33. 33.
    Nason, G.P.: Design and choice of projection indices. Ph.D. dissertation, University of Bath (1992)Google Scholar
  34. 34.
    Nason, G.P.: Three-dimensional projection pursuit. J. R. Stat. Soc. C 44, 411–430 (1995)zbMATHMathSciNetGoogle Scholar
  35. 35.
    Peña, D., Prieto, F.: Cluster identification using projections. J. Am. Stat. Assoc. 96(456), 1433–1445 (2001)zbMATHCrossRefGoogle Scholar
  36. 36.
    Posse, C.: Tools for two-dimensional exploratory projection pursuit. J. Comput. Graph. Stat. 4(2), 83–100 (1995)CrossRefGoogle Scholar
  37. 37.
    Rechenberg, R.I.: Evolutionsstrategie: Optimierung Technischer Systeme Nach Prinzipien Der Biologischen Evolution. Frommann-Holzboog, Stuttgart (1973)Google Scholar
  38. 38.
    Riani, M., Atkinson, A.C., Cerioli, A.: Finding an unknown number of multivariate outliers. J. R. Stat. Soc. B 71(2), 447–466 (2009)CrossRefMathSciNetGoogle Scholar
  39. 39.
    Schwefel, H.P.: Numerical Optimization of Computer Models. Wiley, New York (1981)zbMATHGoogle Scholar
  40. 40.
    Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, New York (1986)zbMATHGoogle Scholar
  41. 41.
    Storn, R., Price, K.: Differential evolution a simple and efficient heuristic for global optimization over continuous space. J. Glob. Optim. 11(4), 341–359 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Sun, J.: Some practical aspects of exploratory projection pursuit. SIAM J. Sci. Comput. 14(1), 68–80 (1993)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Alain Berro
    • 1
  • Souad Larabi Marie-Sainte
    • 1
  • Anne Ruiz-Gazen
    • 2
    Email author
  1. 1.IRITUniversity Toulouse 1ToulouseFrance
  2. 2.Toulouse School of Economics (Gremaq and IMT)University Toulouse 1ToulouseFrance

Personalised recommendations