A Novel Framework for Data Registration and Data Fusion in Presence of Multi-modal Sensors

  • Hadi Aliakbarpour
  • Joao Filipe Ferreira
  • Kamrad Khoshhal
  • Jorge Dias
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 314)


This article presents a novel framework to register and fuse heterogeneous sensory data. Our approach is based on geometrically registration of sensory data onto a set of virtual parallel planes and then applying an occupancy grid for each layer. This framework is useful in surveillance applications in presence of multi-modal sensors and can be used specially in tracking and human behavior understanding areas. The multi-modal sensors set in this work comprises of some cameras, inertial measurement sensors (IMU), laser range finders (LRF) and a binaural sensing system. For registering data from each one of these sensors an individual approach is proposed. After registering multi-modal sensory data on various geometrically parallel planes, a two-dimensional occupancy grid (as a layer) is applied for each plane.


Multi-modality data registration data fusion Occupancy grid and Homography 


  1. 1.
    Smith, D., Singh, S.: Approaches to multisensor data fusion in target tracking: A survey. IEEE Transactions on Knowledge and Data Engineering 18, 1696–1710 (2006)CrossRefGoogle Scholar
  2. 2.
    Armesto, L., Tornero, J.: On multi-rate fusion for non-linear sampled-data systems: Application to a 6d tracking system. Robotics and Autonomous Systems. Elsevier, Amsterdam (2007)Google Scholar
  3. 3.
    Bellotto, N., Hu, H.: Vision and laser data fusion for tracking people with a mobile robot. In: Proceedings of the 2006 IEEE International Conference on Robotics and Biomimetics, Kunming, China (2006)Google Scholar
  4. 4.
    Chakravarty, J.: Panoramic vision and laser range finder fusion for multiple person tracking. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS (2006)Google Scholar
  5. 5.
    Khan, S.M., Yan, P., Shah, M.: A homographic framework for the fusion of multi-view silhouettes. In: IEEE 11th International Conference on Computer Vision, ICCV 2007 (2007)Google Scholar
  6. 6.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  7. 7.
    Aliakbarpour, H., Nunez, P., Prado, J., Khoshhal, K., Dias, J.: An efficient algorithm for extrinsic calibration between a 3d laser range finder and a stereo camera for surveillance. In: 14th International Conference on Advanced Robotics, ICAR 2009 (2009)Google Scholar
  8. 8.
    Ferreira, J.F., Pinho, C., Dias, J.: Implementation and calibration of a Bayesian binaural system for 3d localisation. In: IEEE International Conference on Robotics and Biomimetics (ROBIO 2008), Bangkok, Tailand, December 2008, pp. 14–17 (2008)Google Scholar
  9. 9.
    Pinho, C., Ferreira, J.F., BessiÚre, P., Dias, J.: A Bayesian binaural system for 3d sound-source localisation. In: International Conference on Cognitive Systems (CogSys 2008), pp. 109–114 (2008)Google Scholar
  10. 10.
    Mirisola, L.G.B., Dias, J.: Tracking from a moving camera with attitude estimates. In: ICR 2008 (2008)Google Scholar
  11. 11.
    Mirisola, L.G.B.: Exploiting attitude sensing in vision-based navigation, mapping and tracking including results from an airship. PhD thesis (2009)Google Scholar
  12. 12.
    Lobo, J., Dias, J.: Relative pose calibration between visual and inertial sensors. International Journal of Robotics Research, Special Issue 2nd Workshop on Integration of Vision and Inertial Sensors 26, 561–575 (2007)Google Scholar
  13. 13.
    Criminisi, A.: Accurate visual metrology from single and multiple uncalibrated images. PhD thesis, Oxford (1999)Google Scholar
  14. 14.
    Mekhnacha, K., Mao, Y., Raulo, D., Laugier, C.: Bayesian occupancy filter based fast clustering-tracking algorithm. In: IROS 2008 (2008)Google Scholar
  15. 15.
    Fleuret, F., Jerome Berclaz, R.L., Fua, P.: Multi-camera people tracking with a probabilistic occupancy map. IEEE Transactions on Pattern Analysis and Machine Intelligence (2008)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2010

Authors and Affiliations

  • Hadi Aliakbarpour
    • 1
    • 2
  • Joao Filipe Ferreira
    • 1
  • Kamrad Khoshhal
    • 1
  • Jorge Dias
    • 1
  1. 1.Institute of Systems and Robotics (ISR)University of CoimbraPortugal
  2. 2.IEEE student memberPortugal

Personalised recommendations