ART-Based Fusion of Multi-modal Information for Mobile Robots

  • Elmar Berghöfer
  • Denis Schulze
  • Marko Tscherepanow
  • Sven Wachsmuth
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 363)


Robots operating in complex environments shared with humans are confronted with numerous problems. One important problem is the identification of obstacles and interaction partners. In order to reach this goal, it can be beneficial to use data from multiple available sources, which need to be processed appropriately. Furthermore, such environments are not static. Therefore, the robot needs to learn novel objects. In this paper, we propose a method for learning and identifying obstacles based on multi-modal information. As this approach is based on Adaptive Resonance Theory networks, it is inherently capable of incremental online learning.


sensor data fusion incremental learning Adaptive Resonance Theory 


  1. 1.
    Anagnostopoulos, G.C., Georgiopoulos, M.: Hypersphere ART and ARTMAP for unsupervised and supervised incremental learning. In: Proceedings of the International Joint Conference on Neural Networks, vol. 6, pp. 59–64 (2000)Google Scholar
  2. 2.
    Asfour, Y., Carpenter, G., Grossberg, S., Lesher, G.: Fusion ARTMAP: An adaptive fuzzy network for multi-channel classification. In: Proceedings of the International Conference on Industrial Fuzzy Control and Intelligent Systems, pp. 155–160 (1993)Google Scholar
  3. 3.
    Carpenter, G.A., Grossberg, S.: A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing 37(1), 54–115 (1987)CrossRefGoogle Scholar
  4. 4.
    Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks 3(5), 698–713 (1992)CrossRefGoogle Scholar
  5. 5.
    Carpenter, G.A., Grossberg, S., Rosen, D.B.: Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4, 759–771 (1991)CrossRefGoogle Scholar
  6. 6.
    Cutler, R., Davis, L.: Look who’s talking: speaker detection using video and audio correlation. In: Proceedings of the International Conference on Multimedia and Expo, pp. 1589–1592. IEEE, Los Alamitos (2000)Google Scholar
  7. 7.
    Fritsch, J., Kleinehagenbrock, M., Lang, S., Plötz, T., Fink, G.A., Sagerer, G.: Multi-modal anchoring for human-robot interaction. Robotics and Autonomous Systems 43(2-3), 133–147 (2003)CrossRefGoogle Scholar
  8. 8.
    Harrison, R., Borges, J.: Fusion ARTMAP: Clarification, implementation and developments. Tech. Rep. 589, The University of Sheffield, Department of Automatic Control and Systems Engineering, Mappin Street, Sheffield S1 3JD (1995)Google Scholar
  9. 9.
    Jacobsson, H., Hawes, N., Kruijff, G.J., Wyatt, J.: Crossmodal content binding in information-processing architectures. In: Proceedings of the International Conference on Human Robot Interaction, pp. 81–88. ACM, New York (2008)CrossRefGoogle Scholar
  10. 10.
    Martens, S., Gaudiano, P., Carpenter, G.: Mobile robot sensor integration with fuzzy ARTMAP. In: Proceedings of the International Symposium on Intelligent Control, pp. 307–312 (1998)Google Scholar
  11. 11.
    Stork, D., Wolff, G., Levine, E.: Neural network lipreading system for improved speech recognition. In: Proceedings of the International Joint Conference on Neural Networks, pp. 289–295. IEEE, Los Alamitos (1992)Google Scholar
  12. 12.
    Tscherepanow, M.: TopoART: A topology learning hierarchical ART network. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010. LNCS, vol. 6354, pp. 157–167. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Tscherepanow, M., Jensen, N., Kummert, F.: An incremental approach to automated protein localisation. BMC Bioinformatics 9(445) (2008)Google Scholar
  14. 14.
    Vakil-Baghmisheh, M.T., Pavešić, N.: A fast simplified fuzzy ARTMAP network. Neural Processing Letters 17, 273–316 (2003)CrossRefGoogle Scholar
  15. 15.
    Viola, P., Jones, M.: Robust real-time object detection. In: Second International Workshop on Statistical an Computational Theories of Vision (2001)Google Scholar
  16. 16.
    Wachsmuth, S., Siepmann, F., Schulze, D., Swadzba, A.: ToBI – Team of Bielefeld: The human-robot interaction system for RoboCup@Home 2010. Tech. rep., Bielefeld University, Applied Informatics (2010),

Copyright information

© International Federation for Information Processing 2011

Authors and Affiliations

  • Elmar Berghöfer
    • 1
  • Denis Schulze
    • 1
    • 2
  • Marko Tscherepanow
    • 1
  • Sven Wachsmuth
    • 1
    • 2
  1. 1.Applied Informatics, Faculty of TechnologyBielefeld UniversityBielefeldGermany
  2. 2.CITEC, Cognitive Interaction Technology, Center of ExcellenceBielefeld UniversityBielefeldGermany

Personalised recommendations