Advertisement

In-Situ Learning in Multi-net Systems

  • Matthew Casey
  • Khurshid Ahmad
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3177)

Abstract

Multiple classifier systems based on neural networks can give improved generalisation performance as compared with single classifier systems. We examine collaboration in multi-net systems through in-situ learning, exploring how generalisation can be improved through the simultaneous learning in networks and their combination. We present two in-situ trained systems; first, one based upon the simple ensemble, combining supervised networks in parallel, and second, a combination of unsupervised and supervised networks in sequence. Results for these are compared with existing approaches, demonstrating that in-situ trained systems perform better than similar pre-trained systems.

Keywords

Generalisation Performance Trained System Multiple Classifier System Negative Correlation Learning Single Layer Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Blake, C.L., Merz, C.J.U.: Repository of Machine Learning Databases. Irvine, CA.: University of California, Irvine, Department of Information and Computer Sciences (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  2. 2.
    Bottou, L., Gallinari, P.: A Framework for the Cooperation of Learning Algorithms. In: Lippmann, R.P., Moody, J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems, vol. 3, pp. 781–788 (1991)Google Scholar
  3. 3.
    Casey, M.C.: Integrated Learning in Multi-net Systems. Unpublished doctoral thesis. University of Surrey, Guildford (2004)Google Scholar
  4. 4.
    Drucker, H.: Boosting Using Neural Networks. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp. 51–78. Springer, London (1999)Google Scholar
  5. 5.
    Jacobs, R.A., Tanner, M.: Mixtures of X. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp. 267–295. Springer, Heidelberg (1999)Google Scholar
  6. 6.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  7. 7.
    Kohonen, T.: Self-Organized Formation of Topologically Correct Feature Maps. Biological Cybernetics 43, 59–69 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Liu, Y., Yao, X.: Ensemble Learning via Negative Correlation. Neural Networks 12(10), 1399–1404 (1999)CrossRefGoogle Scholar
  9. 9.
    Liu, Y., Yao, X., Zhao, Q., Higuchi, T.: An Experimental Comparison of Neural Network Ensemble Learning Methods on Decision Boundaries. In: Proceedings of the 2002 International Joint Conference on Neural Networks (IJCNN 2002), vol. 1, pp. 221–226. IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
  10. 10.
    Partridge, D., Griffith, N.: Multiple Classifier Systems: Software Engineered, Automatically Modular Leading to a Taxonomic Overview. Pattern Analysis and Applications 5(2), 180–188 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Prechelt, L.: Early Stopping - But When? In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, pp. 55–69. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  12. 12.
    Sharkey, A.J.C.: Multi-Net Systems. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp. 1–30. Springer, London (1999)Google Scholar
  13. 13.
    Thrun, S.B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., De Jong, K., Dzeroski, S., Fahlman, S.E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R.S., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., van de Welde, W., Wenzel, W., Wnek, J., Zhang, J.: The MONK’s Problems: A Performance Comparison of Different Learning Algorithms. Technical Report CMU-CS-91-197. Pittsburgh, PA.: Carnegie-Mellon University, Computer Science Department (1991)Google Scholar
  14. 14.
    Wanas, N.M., Hodge, L., Kamel, M.S.: Adaptive Training Algorithm for an Ensemble of Networks. In: Proceedings of the 2001 International Joint Conference on Neural Networks (IJCNN 2001), vol. 4, pp. 2590–2595. IEEE Computer Society Press, Los Alamitos (2001)Google Scholar
  15. 15.
    Wolberg, W.H., Mangasarian, O.L.: Multisurface Method of Pattern Separation for Medical Diagnosis Applied to Breast Cytology. Proceedings of the National Academy of Sciences, USA 87(23), 9193–9196 (1990)zbMATHCrossRefGoogle Scholar
  16. 16.
    Xu, L., Krzyzak, A., Suen, C.Y.: Several Methods for Combining Multiple Classifiers and Their Applications in Handwritten Character Recognition. IEEE Transactions on Systems, Man, and Cybernetics 22(3), 418–435 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Matthew Casey
    • 1
  • Khurshid Ahmad
    • 1
  1. 1.Department of Computing, School of Electronics and Physical SciencesUniversity of SurreyGuildford, SurreyUK

Personalised recommendations