Dimensionality Reduction for Evolving RBF Networks with Particle Swarms

  • Junying Chen
  • Zheng Qin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3971)


Dimensionality reduction including both feature selection and feature extraction techniques are useful for improving the performance of neural networks. In this paper, particle swarm optimization (PSO) algorithm was proposed for simultaneous feature extraction and feature selection. First PSO was used to simultaneous feature extraction and selection in conjunction with knearest- neighbor (k-NN) for individual fitness evaluation. With the derived feature set, PSO was then used to evolve RBF networks dynamically. Experimental results on four datasets show that RBF networks evolved with the derived feature set by our proposed algorithm have more simple architecture and stronger generalization ability with the similar classification performance when compared with the networks evolved with the full feature set.


Particle Swarm Optimization Feature Selection Dimensionality Reduction Particle Swarm Optimization Algorithm Feature Subset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fu, X., Wang, L.: Data Dimensionality Reduction with Application to Simplifying RBF Network Structure and Improving Classification Performance. IEEE Transactions on Systems, Man and Cybernetics 33(3), 399–409 (2003)CrossRefGoogle Scholar
  2. 2.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, Boston (1990)MATHGoogle Scholar
  3. 3.
    Pfleger, K., John, G., Kohavi, R.: Irrelevant Features and the Subset Selection Problem. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 121–129 (1994)Google Scholar
  4. 4.
    Pudil, P., Ferri, F.J., Novovicova, J., Kittler, J.: Floating Search Methods for Feature Selection with Nonmonotonic Criterion Functions. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 2, pp. 279–283 (1994)Google Scholar
  5. 5.
    Somol, P., Pudil, P., Kittler, J.: Fast Branch and Bound Algorithms for Optimal Feature Selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(7), 900–912 (2004)CrossRefGoogle Scholar
  6. 6.
    Siedlecki, W., Sklansky, J.: A Note on Genetic Algorithms for Large Scale Feature Selection. Pattern Recognition Letters 10(5), 335–347 (1989)MATHCrossRefGoogle Scholar
  7. 7.
    Scherf, M., Brauer, W.: Feature Selection by Means of A Feature Weighting Approach. Technical Report FKI-221-97, Institute fur Informatik, Technische Universitt Mnchen, Munich (1997)Google Scholar
  8. 8.
    Yang, J., Honavar, V.: Feature Subset Selection Using A Genetic Algorithm. In: Liu, H., Motoda, H. (eds.) Feature Extraction, Construction and Selection: A Data Mining Perspective, pp. 117–136 (1998)Google Scholar
  9. 9.
    Raymer, M.L., Punch, W.F., Goodman, E.D., Sanschagrin, P.C., Kuhn, L.A.: Simultane-ous Feature Extraction and Selection Using A Masking Genetic Algorithm. In: Proceedings of ICGA 1997, pp. 561–567. San Francisco, CA (1997)Google Scholar
  10. 10.
    Firpi, H.A., Goodman, E.: Swarmed Feature Selection. In: Applied Imagery Pattern Recognition Workshop, pp. 112–118 (2004)Google Scholar
  11. 11.
    Liu, Y., Qin, Z., Xu, Z.L., He, X.S.: Feature Selection with Particle Swarms. In: International Symposium on Computational and Information Science, Shanghai, China, pp. 425–430 (2004)Google Scholar
  12. 12.
    Broomhead, D.S., Lowe, D.: Multivariable Functional Interpolation and Adaptive Networks. Complex Syst. 2, 321–355 (1988)MATHMathSciNetGoogle Scholar
  13. 13.
    Kennedy, J., Eberhart, R.C.: Particle Swarm Optimization. In: Proceedings of IEEE International Conference on Neural Networks, Piscataway, NJ, pp. 1942–1948 (1995)Google Scholar
  14. 14.
    Qin, Z., Chen, J.Y., Liu, Y., Lu, J.: Evolving RBF Neural Networks for Pattern Classification. In: Hao, Y., Liu, J., Wang, Y.-P., Cheung, Y.-m., Yin, H., Jiao, L., Ma, J., Jiao, Y.-C. (eds.) CIS 2005. LNCS (LNAI), vol. 3801, pp. 957–964. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    Blake, C., Merz, C.J.: UCI Repository of Machine Learning Databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Junying Chen
    • 1
  • Zheng Qin
    • 1
    • 2
  1. 1.Department of Computer ScienceXian JiaoTong UniversityXianP.R. China
  2. 2.School of SoftwareTsinghua UniversityBeijingP.R. China

Personalised recommendations