Evolving Kernel PCA Pipelines with Evolution Strategies

  • Oliver Kramer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10505)


This paper introduces an evolutionary tuning approach for a pipeline of preprocessing methods and kernel principal component analysis (PCA) employing evolution strategies (ES). A simple (1+1)-ES adapts the imputation method, various preprocessing steps like normalization and standardization, and optimizes the parameters of kernel PCA. A small experimental study on a benchmark data set with missing values demonstrates that the evolutionary kernel PCA pipeline can be tuned with relatively few optimization steps, which makes evolutionary tuning applicable to scenarios with very large data sets.


Dimension reduction Evolutionary machine learning Machine learning pipelines Kernel PCA 


  1. 1.
    Das, S., Abraham, A., Konar, A.: Automatic clustering using an improved differential evolution algorithm. IEEE Trans. Syst. Man Cybern. Part A 38(1), 218–237 (2008)CrossRefGoogle Scholar
  2. 2.
    Friedrichs, F., Igel, C.: Evolutionary tuning of multiple SVM parameters. Neurocomputing 64, 107–117 (2005)CrossRefGoogle Scholar
  3. 3.
    Jolliffe, I.: Principal Component Analysis. Springer Series in Statistics. Springer, New York (1986)CrossRefzbMATHGoogle Scholar
  4. 4.
    Karami, A., Johansson, R.: Choosing DBSCAN parameters automatically using differential evolution. Int. J. Comput. Appl. 91(7), 1–11 (2014)Google Scholar
  5. 5.
    Kramer, O.: Hybrid manifold clustering with evolutionary tuning. In: Mora, A.M., Squillero, G. (eds.) EvoApplications 2015. LNCS, vol. 9028, pp. 481–490. Springer, Cham (2015). doi: 10.1007/978-3-319-16549-3_39 Google Scholar
  6. 6.
    Kwok, J.T., Mak, B., Ho, S.K.: Eigenvoice speaker adaptation via composite kernel PCA. In: Neural Information Processing Systems (NIPS), pp. 1401–1408 (2003)Google Scholar
  7. 7.
    Lückehe, D., Kramer, O.: Alternating optimization of unsupervised regression with evolutionary embeddings. In: Mora, A.M., Squillero, G. (eds.) EvoApplications 2015. LNCS, vol. 9028, pp. 471–480. Springer, Cham (2015). doi: 10.1007/978-3-319-16549-3_38 Google Scholar
  8. 8.
    Olson, R.S., Bartley, N., Urbanowicz, R.J., Moore, J.H.: Evaluation of a tree-based pipeline optimization tool for automating data science. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 485–492 (2016)Google Scholar
  9. 9.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Schölkopf, B., Smola, A., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRefGoogle Scholar
  11. 11.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: Neural Information Processing Systems (NIPS), pp. 2960–2968 (2012)Google Scholar
  12. 12.
    Sun, R., Tsung, F., Qu, L.: Integrating KPCA with an improved evolutionary algorithm for knowledge discovery in fault diagnosis. In: Leung, K.S., Chan, L.-W., Meng, H. (eds.) IDEAL 2000. LNCS, vol. 1983, pp. 174–179. Springer, Heidelberg (2000). doi: 10.1007/3-540-44491-2_26 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Computational Intelligence Group, Department of Computer ScienceUniversity of OldenburgOldenburgGermany

Personalised recommendations