Skip to main content
Log in

Cooperative Fusion for Multi-Obstacles Detection With Use of Stereovision and Laser Scanner

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We propose a new cooperative fusion approach between stereovision and laser scanner in order to take advantage of the best features and cope with the drawbacks of these two sensors to perform robust, accurate and real time-detection of multi-obstacles in the automotive context. The proposed system is able to estimate the position and the height, width and depth of generic obstacles at video frame rate (25 frames per second). The vehicle pitch, estimated by stereovision, is used to filter laser scanner raw data. Objects out of the road are removed using road lane information computed by stereovision. Various fusion schemes are proposed and one is experimented. Results of experiments in real driving situations (multi-pedestrians and multi-vehicles detection) are presented and stress the benefits of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bar-Shalom, Y. 2000. Multitarget-Multisensor Tracking: Applications and Advances vol. III. Artech House.

  • Bertozzi, M. and Broggi, A. 1998. GOLD: A Parallel Real-Time Stereo Vision System for Generic Obstacle and Lane Detection. IEEE Transaction on Image Processing, 7(1).

  • Blackman, S. and Popoli, R. 1999. Modern Tracking Systems. Artech.

  • Coulombeau, P. and Laurgeau, C. 2002. Vehicle Yaw, Pitch, Roll and 3D Road Shape Recovery by Vision. In IEEE Intelligent Vehicles Symposium, Versailles.

  • Franke, U. and Joos, A. 2000. Real-time stereovision for urban traffic scene understanding. In IEEE Intelligent Vehicle Symposium, Dearborn, USA.

  • Gruyer, D. and Berge-Cherfaoui, V. 1999. Multi-objects association in perception of dynamical situation. In UAI’99, Stockholm, Sweden, 30 juillet-1.

  • Royère, C., Gruyer, D., and Cherfaoui, V. 2000. Data association with believe theory. In FUSION’2000, Paris.

  • Gruyer, D., Royere, C., Labayrade, R., and Aubert, D. 2003. Credibilistic multi-sensor fusion for real time embedded application, applied to the near obstacle detection and tracking. In Proceedings IEEE ICAR 2003, Coimbra, Portugal.

  • Hattori, H. and Maki, A. 2000. Stereo without depth search and metric calibration. Research & Development centre, IEEE TOSHIBA Corporation, Kawasaki 212-8582, Japan.

    Google Scholar 

  • Ieng, S.-S., Tarel, J.-P., and Labayrade, R. 2003. On the design of a single lane-markings detector regardless the on-board camera’s position. In Proceedings of IEEE Intelligent Vehicle Symposium (IV’2003), Colombus, OH, USA.

  • Koller, D., Luong, T., and Malik, J. 1994. Binocular stereopsis and lane marker flow for vehicle navigation: Lateral and longitudinal control. Technical Report UCB/CSD 94-804, University of California at Berkeley, Computer Science Division.

  • Labayrade, R., Aubert, D., and Tarel, J.-P. 2002. Real time obstacles detection on non flat road geometry through “v-disparity’ representation. In IEEE Intelligent Vehicles Symposium, Versailles.

  • Labayrade, R. and Aubert, D. 2002. Robust and fast stereovision based road Obstacles Detection for Driving Safety Assistance. In MVA 2002, Nara, Japan.

  • Labayrade, R. and Aubert, D. 2003. In-Vehicle Obstacles Detection and Characterization by Stereovision. IEEE In-Vehicle Cognitive Computer Vision Systems, Graz, Austria.

  • Okutomi, M. and Noguchi, S. 1998. Extraction of road region using stereo images. In Proc. ICPR, pp. 853–856.

  • Rombaut, M. 1998. Decision in multi-obstacle matching process using theory of belief. In AVCS’98, Amiens, pp. 63–68.

  • Shafer, G. 1976. A Mathematical Theory of Evidence. Princeton University Press.

  • Scharstein, D., Szeliski, R., and Zabih, R. 2001. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision, Kauai, HI.

  • Tarel, J.P. and Guichard, F. 2000. Combined Dynamic Tracking of Lane Markings and Boundaries in Curved Road. In Proceedings of IEEE International Conference on Image Processing (ICIP’00).

  • Toulminet, G., and Bensrhair, A. 2001. Systlsquo;éme de stéréovision pour la détection d’obstacles et de véhicule temps reel. In Proceedings 18th Symposium GRETSI’01 on Signal and Image Processing, Toulouse, France.

  • Wang, R., Xu, Y., Libin, and Zhao, Y. 2002. A Vision-Based Road Edge Detection Algorithm. In Proceedings of IEEE Intelligent Vehicules Symposium, Versailles.

  • Williamson, T.-A. 1998. A high-performance stereo vision system for obstacle detection, Phd, Carnegie Mellon University.

  • Zhang, Z., Weiss, R., and Hanson, R.-A. 1994. Qualitative obstacle detection. In Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 554–559.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Raphaël Labayrade.

Additional information

Raphaël Labayrade was born in France, in 1976. He received the M.S. degree in 2000 from the university of Saint Etienne, and he was also graduate from the ENTPE engineer school in 2000. In 2004 he received the Ph.D. degree from the university Pierre et Marie Curie Paris VI. In his thesis he proposed a new approach for detecting road obstacles using stereovision in a generic, fast and robust manner.

He is currently a researcher at INRETS since 2004 in the perception team of the LIVIC department and works on automated highway and on on-board driving assistance systems. His main work deals with obstacles detection using data fusion but he is also interested in road lane recognition. He is involved for vision tasks in various european and french projects dealing with intelligent vehicles (Carsense, Micado, Arcos). He teaches at Jussieu (Paris VI), Ecole Nationale des Ponts et Chaussées, University of Versailles.

He is the author and co-author of several technical papers.

Cyril Royere was born in France, in 1972. He received the M.S. degree in 1995 from the university of Reims. In 2002 he received the Ph.D. degree from the university of Technology of Compiegne. In his thesis, he describes the origins of the conflict which appears when combining of various sources of imperfect information within the framework of the belief theory. Since 2002, he is a researcher at INRETS, into the perception team of the LIVIC department (Laboratory on interactions between vehicles, Infrastructure and drivers) and works on automated highway and on on-board driving assistance systems. His main work deals with obstacles detection using data fusion. He is involved for multi-sensor fusion tasks in several European and French projects dealing with intelligent vehicles (CARSENSE, MICADO, ARCOS).

He is the author and co-author of several technical papers.

Dominique Gruyer was born in France, in 1969. He received the M.S. and Ph.D. degree respectively in 1995 and 1999 from the university of Technology of Compiëgne.

Since 2001, he is a researcher at INRETS, into the perception team of the LIVIC department (Laboratory on interactions between vehicles, Infrastructure and drivers) and he works on the study and the development of multi-sensor/sources association, combination and fusion. His works enter into the conception of on-board driving assistance systems and more precisely on the carry out of multi-obstacle detection and tracking, extended perception, accurate localization, anti-collision system, collision mitigation. He is involved for multi-sensor fusion tasks in several European and French projects dealing with intelligent vehicles (CARSENSE, MICADO, ARCOS). He is a multi-sensor fusion expert for several companies, teaches at Orsay (Paris XI), Ecole Nationale des Ponts et Chaussées and University of Technologie of Compiëgne.

He is the author and co-author of several technical papers.

Didier Aubert was born in France, in 1963. He received the M.S. and Ph.D. degree respectively in 1985 and 1989 from the university of Grenoble. From 1989–1990, he worked as a research scientist on the development of an automatic road following system for the NAVLAB at Carnegie Mellon University. From 1990–1994, he worked in the research department of a private company (ITMI). During this period he was project leader of several projects dealing with computer vision (Multi-resolution, color, motion detection, 3D reconstruction, 3D location, Shape recognition, automatic shape modelling, object tracking), mobile robotic (calibration, roads following, free space computation) and manipulator robotic (calibration, automatic surface tracking). He was also working as an expert for companies on the face recognition, 3D location and roads following topics. He is currently a researcher at INRETS since 1995, manages the perception team of the LIVIC department and works on car traffic monitoring, crowd monitoring, incidents detection, automated highway and on on-board driving assistance systems. He is an image processing expert for several compagnies, teaches at Jussieu (Paris VI), Ecole Nationale des Ponts et Chaussées, Ecole Nationale Supérieure des Télécommunications, Orsay (Paris XI) and is at the editorial board of RTS (Research-Transport-Safety).

He is the author and co-author of several technical papers and has participated to the redaction of the books named “Robotique mobile” and “la route automatisée.”

Rights and permissions

Reprints and permissions

About this article

Cite this article

Labayrade, R., Royere, C., Gruyer, D. et al. Cooperative Fusion for Multi-Obstacles Detection With Use of Stereovision and Laser Scanner. Auton Robot 19, 117–140 (2005). https://doi.org/10.1007/s10514-005-0611-7

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-005-0611-7

Keywords

Navigation