Fusion of a Camera and a Laser Range Sensor for Vehicle Recognition

  • Shirmila Mohottala
  • Shintaro Ono
  • Masataka Kagesawa
  • Katsushi Ikeuchi
Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1)


Fusing different sensory data into a singular data stream is not a recent idea, but with the diffusion of various simple and compact sensors, multi-sensor fusion has inspired new research initiatives. Sensor fusion improves measurement precision and perception, offering greater benefits than using each sensor individually. In this chapter we present a system that fuses information from a vision sensor and a laser range sensor for detection and classification. Although the laser range sensors are good at localizing objects accurately, vision images contain more useful features to classify the object. By fusing these two sensors, we can obtain 3D information about the target object, together with its textures, with high reliability and robustness to outdoor conditions. To evaluate the performance of the system, it is applied to recognition of on-street parked vehicles from a moving probe vehicle. The evaluation experiments show obviously successful results, with a detection rate of 100% and accuracy over 95% in recognizing four vehicle classes.


Sensor fusion Vehicle classification 


  1. 1.
    Blanco, J., Burgard, W., Sanz, R., Fernandez, J.L.: Fast face detection for mobile robots by integrating laser range data with vision. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 625–631 (2003)Google Scholar
  2. 2.
    Song, X., Cui, J., Zhao, H., Zha, H.: Bayesian fusion of laser and vision for multiple people detection and tracking. In: Proceedings of the SICE Annual Conference on 2008 (SICE08), pp. 3014–3019 (2008)Google Scholar
  3. 3.
    Baltzkis, H., Argyros, A., Trahanias, P.: Fusion of laser and visual data for robot motion planning and collision avoidance. Mach. Vis. Appl. 15, 92–100 (2003)CrossRefGoogle Scholar
  4. 4.
    Pagnottelli, S., Taraglio, S., Valigi, P., Zanela, A.: Visual and laser sensory data fusion for outdoor robot localisation and navigation. In: Proceedings of the 12th International Conference on Advanced Robotics, 18–20 July 2005, pp. 171–177 (2005)Google Scholar
  5. 5.
    Wender, S., Clemen, S., Kaempchen, N., Dietmayer, K.C.J.: Vehicle detection with three dimensional object models. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Heidelberg, Germany, September (2006)Google Scholar
  6. 6.
    Ono, S., Kagesawa, M., Ikeuchi, K.: Recognizing vehicles in a panoramic range image. In: Meeting on Image Recognition and Understanding (MIRU), pp. 183–188 (2002)Google Scholar
  7. 7.
    Mei, C., Rives, P.: Calibration between a central catadioptric camera and a laser range finder for robotic applications. In: Proceedings of ICRA06, Orlando, May (2006)Google Scholar
  8. 8.
    Wasielewski, S., Strauss, O.: Calibration of a multi-sensor system laser rangefinder/camera. In: Proceedings of the Intelligent Vehicles ’95 Symposium, pp. 472–477 (1995)Google Scholar
  9. 9.
    Jokinen, O.: Self-calibration of a light striping system by matching multiple 3-d profile maps. In: Proceedings of the 2nd International Conference on 3D Digital Imaging and Modeling, pp. 180–190 (1999)Google Scholar
  10. 10.
    Zhang, Q., Pless, R.: Extrinsic calibration of a camera and laser range finder (improves camera calibration). In: Intelligent Robots and Systems (IROS), (2004)Google Scholar
  11. 11.
    Unnikrishnan, R., Hebert, M.: Fast extrinsic calibration of a laser rangefinder to a camera. Technical Report, CMU-RI-TR-05-09, Robotics Institute, Carnegie Mellon University, July (2005)Google Scholar
  12. 12.
    Greig, D., Porteous, B., Seheult, A.: Exact maximum a posteriori estimation for binary images. J. R. Stat. Soc. 51(2), 271–279 (1989)Google Scholar
  13. 13.
    Yoshida, T., Mohottala, S., Kagesawa, M., Tomonaka, T., Ikeuchi, K.: Vehicle classification system with local-feature based algorithm using cg model images. IEICE Trans. Inf. Syst. E85D(11), 1745–1752 (2002)Google Scholar
  14. 14.
    Krumm, J.: Object detection with vector quantized binary features. In: Proceedings of Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 179–185 (1997)Google Scholar
  15. 15.
    Ohba, K., Ikeuchi, K.: Detectability, uniqueness, and reliability of eigen windows for stable verification of partially occluded objects. IEEE Trans. Pattern Anal. Mach. Intell. 19(9), 1043–1048 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Shirmila Mohottala
    • 1
  • Shintaro Ono
    • 1
  • Masataka Kagesawa
    • 1
  • Katsushi Ikeuchi
    • 1
  1. 1.Institute of Industrial ScienceUniversity of TokyoMeguro-kuJapan

Personalised recommendations