Skip to main content
SpringerLink
Log in

3D reconstruction and classification of natural environments by an autonomous vehicle using multi-baseline stereo

  • Special Issue
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

In natural outdoor settings, advanced perception systems and learning strategies are major requirement for an autonomous vehicle to sense and understand the surrounding environment, recognizing artificial and natural structures, topology, vegetation and drivable paths. Stereo vision has been used extensively for this purpose. However, conventional single-baseline stereo does not scale well to different depths of perception. In this paper, a multi-baseline stereo frame is introduced to perform accurate 3D scene reconstruction from near range up to several meters away from the vehicle. A classifier that segments the scene into navigable and non-navigable areas based on 3D data is also described. It incorporates geometric features within an online self-learning framework to model and identify traversable ground, without any a priori assumption on the terrain characteristics. The ground model is automatically retrained during the robot motion, thus ensuring adaptation to environmental changes. The proposed strategy is of general applicability for robot’s perception and it can be implemented using any range sensor. Here, it is demonstrated for stereo-based data acquired by the multi-baseline device. Experimental tests, carried out in a rural environment with an off-road vehicle, are presented. It is shown that the use of a multi-baseline stereo frame allows for accurate reconstruction and scene segmentation at a wide range of visible distances, thus increasing the overall flexibility and reliability of the perception system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Hague T, Marchant JA, Tillett ND (2000) Ground-based sensing systems for autonomous agricultural vehicles. Comput Electron Agric 25(1–2):11–28

    Article  Google Scholar 

  2. Rankin A, Huertas A, Matthies L (2005) Evaluation of stereo vision obstacle detecion algorithms for off-road autonomous navigation. In: Proceedings of the 32nd AUVSI symposium on unmanned systems, June 2005

  3. Rovira-Más F, Zhang Q, Reid JF (2008) Stereo vision three-dimensional terrain maps for precision agriculture. Comput Electron Agric 60(2):133–143

    Article  Google Scholar 

  4. Konolige K, Agrawal M, Bolles RC, Cowan C, Fischler M, Gerkey B (2008) Outdoor mapping and navigation using stereo vision. Exp Robot Springer Tracts Adv Robot 39:179–190

    Article  Google Scholar 

  5. Milella A, Reina G, Siegwart R (2006) Computer vision methods for improved mobile robot state estimation in challenging terrains. J Multimed 1(7):49–61

    Article  Google Scholar 

  6. Reina G, Milella A (2012) Towards autonomous agriculture: automatic ground detection using trinocular stereovision. Sensors 12(9):12405–12423

    Article  Google Scholar 

  7. Okutomi M, Kanade T (1993) A multiple-baseline stereo. IEEE Trans Pattern Anal Mach Intell 15(4):353–363

    Google Scholar 

  8. Gallup D, Frahm JM, Mordohai P, Pollefeys M (2008) Variable baseline/resolution stereo. In: IEEE conference on computer vision and pattern recognition, Anchorage, AK, 23–28 June 2008, pp 1–8. doi:10.1109/CVPR.2008.4587671

  9. Milella A, Reina G, Foglia M (2013) A multi-baseline stereo system for scene segmentation in natural environments. In: IEEE international conference on technologies for practical robot applications (TePRA), Woburn, MA, 22–23 Apr 2013, pp 1–6. doi:10.1109/TePRA.2013.6556370

  10. Broggi A, Cappalunga A, Caraffi C, Cattani S, Ghidoni S, GrisleriP P, Porta P, Posterli M, Zani P (2010) TerraMax vision at the urban challenge 2007. IEEE Trans Intell Transp Syst 11(1):194–205

    Article  Google Scholar 

  11. Olson CF, Abi-Rached H (2010) Wide-baseline stereo vision for terrain mapping. Mach Vis Appl 21:713–725

    Article  Google Scholar 

  12. http://www.irstea.fr/en/institute. Accessed 26 Feb 2014

  13. Broggi A, Caraffi C, Fedriga RI, Grisleri P (2005) Obstacle detection with stereo vision for off-road vehicle navigation. In: IEEE computer society conference on computer vision and pattern recognition, workshop, San Diego, CA, USA, 25 June 2005, p 65. doi:10.1109/CVPR.2005.503

  14. Kelly A, Stentz A (1998) Stereo vision enhancements for low-cost outdoor autonomous vehicles. In: International conference on robotics and automation, workshop WS-7, navigation of outdoor autonomous vehicles (ICRA’98), May 1998

  15. Manduchi R, Castano A, Talukder A, Matthies L (2003) Obstacle detection and terrain classification for autonomous off-road navigation. Auton Robots 18:81–102

    Article  Google Scholar 

  16. Moravec A (1981) Rover visual obstacle avoidance. In: Proceedings of the 7th international joint conference on artificial intelligence, Vancouver, British Columbia, pp 785–790

  17. Klarquist W, Bovik A (1997) Adaptive variable baseline stereo for vergence control. In: Proceedings of the 1997 IEEE international conference on robotics and automation, vol. 3, pp 1952–1959

  18. Nakabo Y, Mukai T, Hattori Y, Takeuchi Y, Ohnishi N (2005) Variable baseline stereo tracking vision system using high-speed linear slider. In: Proceedings of the 2005 IEEE international conference on robotics and automation, pp 1567–1572

  19. Milella A, Reina G, Underwood J, Douillard B (2011) Combining radar and vision for self-supervised ground segmentation in outdoor environments. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 255–260

  20. Milella A, Reina G, Underwood J, Douillard B (2014) Visual ground segmentation by radar supervision. Robot Auton Syst doi:10.1016/j.robot.2012.10.001 (in press)

  21. Milella A, Reina G, Underwood J (2014) A self-learning framework for statistical ground classification using radar and monocular vision. J Field Robot (in press)

  22. Stavens D, Thrun S (2006) A self-supervised terrain roughness estimator for offroad autonomous driving. In: Proceedings of the conference on uncertainty in AI (UAI), pp 13–16

  23. Zhou S, Xi J, McDaniel MW, Nishihata T, Salesses P, Iagnemma K (2012) Self-supervised learning to visually detect terrain surfaces for autonomous robots operating in forested terrain. J Field Robot 29(2):277–297

    Google Scholar 

  24. Reina G, Milella A, Underwood J (2012) Self-learning classification of radar feautures for scene understanding. Robot Auton Syst 60(11):1377–1388

    Article  Google Scholar 

  25. Vernaza P, Taskar B, Lee DD (2008) Online, self-supervised terrain classification via discriminatively trained submodular Markov random fields. In: Proceedings of IEEE international conference on robotics and automation, pp 2750–2757

  26. Hadsell R, Sermanet P, Ben J, Erkan A, Scoffier M, Kavukcuoglu K, Muller U, LeCun Y (2009) Learning long-range vision for autonomous off-road driving. J Field Robot 26(2):120–144

    Article  Google Scholar 

  27. Konolige K, Agrawal M, Blas MR, Bolles RC, Gerkey BP, Solá J, Sundaresan A (2009) Mapping, navigation, and learning for off-road traversal. J Field Robot 26(1):88–113

    Article  MATH  Google Scholar 

  28. Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, USA

    Google Scholar 

  29. Kuthirummal S, Das A, Samarasekera S (2011) A graph traversal based algorithm for obstacle detection using lidar or stereo. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), San Francisco, CA, 25–30 Sept 2011, pp 3874–3880. doi:10.1109/IROS.2011.6094685

  30. Duda EO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  31. Mardia K, Kent J, Bibby J (1979) Multivariate analysis. Academic Press, London

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Intelligent Systems for Automation (ISSIA), National Research Council (CNR), Via Amendola 122/D-O, 70126 , Bari, Italy

    Annalisa Milella

  2. Department of Engineering for Innovation, University of Salento, Via Arnesano, 73100 , Lecce, Italy

    Giulio Reina

Authors
  1. Annalisa Milella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giulio Reina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Annalisa Milella.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Milella, A., Reina, G. 3D reconstruction and classification of natural environments by an autonomous vehicle using multi-baseline stereo. Intel Serv Robotics 7, 79–92 (2014). https://doi.org/10.1007/s11370-014-0146-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-014-0146-x

Keywords

Search

Navigation