Advertisement

Data Mining and Knowledge Discovery

, Volume 8, Issue 3, pp 203–225 | Cite as

Maximum and Minimum Likelihood Hebbian Learning for Exploratory Projection Pursuit

  • Emilio Corchado
  • Donald MacDonald
  • Colin Fyfe
Article

Abstract

In this paper, we review an extension of the learning rules in a Principal Component Analysis network which has been derived to be optimal for a specific probability density function. We note that this probability density function is one of a family of pdfs and investigate the learning rules formed in order to be optimal for several members of this family. We show that, whereas we have previously (Lai et al., 2000; Fyfe and MacDonald, 2002) viewed the single member of the family as an extension of PCA, it is more appropriate to view the whole family of learning rules as methods of performing Exploratory Projection Pursuit. We illustrate this on both artificial and real data sets.

exploratory projection pursuit artificial neural networks 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bell, J.F., Owensby, P.D., Hawke, B.R., and Gaffey, M.J. 1988. The 52 colour asteroid survey: Final results and interpretation. Lunar Planet Sci. Conf., XIX:57 (abstract).Google Scholar
  2. Bishop, C.M. 1995. Neural Networks for Pattern Recognition. Oxford.Google Scholar
  3. Diaconis, P. and Freedman, D. 1984. Asymptotics of graphical projections. The Annals of Statistics, 12(3):793-815.Google Scholar
  4. Fyfe, C. 1993. PCA properties of interneurons. From Neurobiology to Real World Computing, Proceedings of International Conference on Artificial on Artificial Neural Networks, ICAAN 93, pp. 183–188.Google Scholar
  5. Fyfe, C. and Baddeley, R. 1995. Non-linear data structure extraction using simple Hebbian networks. Biological Cybernetics, 72(6):533–541.Google Scholar
  6. Fyfe, C. 1995. Negative feedback as an organising principle for artificial neural networks. Ph.D. Thesis, Strathclyde University.Google Scholar
  7. Fyfe, C. and MacDonald, D. 2002. ε-insensitive Hebbian learning. Neurocomputing (Accepted for publication).Google Scholar
  8. Howell, E.S., Merenyi, E., and Lebofsky, L.A., 1994. Using neural networks to classify asteroid spectra. Journal of Geophysical Research, 99(E5):10847–10865.Google Scholar
  9. Hyvärinen, A. 2001. Complexity pursuit: Separating interesting components from time series. Neural Computation, 13:883–898.Google Scholar
  10. Hyvärinen, A., Karhunen, J., and Oja, E. 2002. Independent Component Analysis. Wiley, ISBN 0-471-40540-X.Google Scholar
  11. Karhunen, J. and Joutsensalo, J. 1994. Representation and separation of signals using non-linearPCAtype learning. Neural Networks, 7:113–127.Google Scholar
  12. Kashyap, R.L., Blaydon, C.C., and Fu, K.S. 1994. Stochastic approximation. In a Prelude to Neural Networks: Adaptive and Learning Systems, M. Jerry (Ed.), Mendel, Prentice Hall, ISBN 0-13-147448–0.Google Scholar
  13. Lai, P.L., Charles, D., and Fyfe, C. 2000. Seeking independence using biologically inspired artificial neural networks. In Developments in Artificial Neural Network Theory: Independent Component Analysis and Blind Source Separation, M.A. Girolami (Ed.), Springer Verlag.Google Scholar
  14. MacDonald, D., McGlinchey, S., Kawala, J., and Fyfe, C. 1999. Comparison ofKohonen, scale-invariant and GTM self-organising maps for interpretation of spectral data. European Symposium on Artificial Neural Networks (ESANN' 99), 117–122.Google Scholar
  15. Oja, E. 1989. Neural networks, principal components and subspaces. International Journal of Neural Systems, 1:61–68.Google Scholar
  16. Oja, E., Ogawa, H., and Wangviwattana, J. 1992. Principal components analysis by homogeneous neural networks, Part 1. The Weighted Subspace Criterion. IEICE Transaction on Information and Systems, E75D:366–375.Google Scholar
  17. Smola, A.J. and Scholkopf, B. 1998. A tutorial on support vector regression. Technical Report NC2-TR-1998-030, NeuroCOLT2 Technical Report Series.Google Scholar
  18. Tholen, D. 1994. Asteroid taxonomy from cluster analysis of photometry. Ph.D. dissertation, University of Arizona.Google Scholar
  19. Xu, L. 1993. Least mean square error reconstruction for self-organizing nets. Neural Networks, 6:627–648.Google Scholar
  20. Zellner, B., Tholen, D.J., and Tedesco, E.F. 1985. The eight-colour asteroid survey: Results fro 589 minor planets. Icarus, 355–416.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Emilio Corchado
    • 1
  • Donald MacDonald
    • 1
  • Colin Fyfe
    • 1
  1. 1.Applied Computational Intelligence Research UnitThe University of PaisleyScotland

Personalised recommendations