Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation

  • 812 Accesses

  • 149 Citations


NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi-autonomous scenario; however, recent work at JPL and other laboratories has produced stereo systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross-country robotic traverses (of up to 100 meters) to use stereo vision, with all computing on-board the vehicle. Here, we describe the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementation builds bandpass image pyramids and produces disparity maps from the 60×64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done in one 68020 augmented with Datacube image processing boards. We argue that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging. We close with a discussion of practical and theoretical issues involved in evaluating and extending the performance of the stereo system.

This is a preview of subscription content, log in to check access.


  1. 1.

    P. Anandan, Computing dense displacement fields with confidence measures in scenes containing occlusion, Proc. DARPA Image Understanding Workshop, pp. 236–246. SAIC, 1984.

  2. 2.

    N.Ayache, and O.D.Faugeras, Building, registrating, and fusing noisy visual maps, Intern. J. Robot. Res. 7(6):45–65, December 1988.

  3. 3.

    N. Ayache and F. Lustman, Fast and reliable passive stereovision using three cameras, Proc. Intern. Workshop on Industrial Applications of Machine Vision and Machine Intelligence, pp. 186–191, February 1987.

  4. 4.

    H.H. Baker, Depth from edge and intensity based stereo. Ph.D. thesis, Stanford University, 1982.

  5. 5.

    H.H. Baker and T.O. Binford, Depth from edge and intensity based stereo, Proc. 7th Intern. Joint conf. Artif. Intell., pp. 631–636, August 1981.

  6. 6.

    S.T.Barnard, Stochastic stereo matching over scale, Intern. J. Comput. Vision 3(1):17–32, May 1989.

  7. 7.

    P.J. Burt, C.H. Anderson, J.O. Sinniger, and G.S. van der Wal. A pipelined pyramid machine. In Pyramidal Systems for Computer Vision. Vol. NATO ASI Series, vol. F25, pp. 133–152. Springer-Verlag, 1986.

  8. 8.

    P.J.Burt, Fast filter transforms for image processing, Comput. Graph. Image Process. 16:20–51, 1981.

  9. 9.

    L. Chen and T. Boult, An integrated approach to stereo matching, surface reconstruction and depth segmentation using consistent smoothness assumptions, Proc. DARPA Image Understanding Workship, Washington, pp. 166–176, Morgan Kaufmann Publishers, April 1988.

  10. 10.

    J.L.Crowley and R.M.Stern, Fast computation of the difference of low-pass transform, IEEE Trans. Patt. Anal. Mach. Intell. 6(2):212–222, March 1984.

  11. 11.

    A.Elfes, Sonar-based real-world mapping and navigation, IEEE J. Robot. Autom. 3(3):249–265, June 1987.

  12. 12.

    W. Forstner, Personal communication, 1988.

  13. 13.

    W.Forstner and A.Pertl. Photogrammetric standard methods and digital image matching techniques for high precision surface measurements. In E.S.Gelsema and L.N.Kanal, ed. Pattern Recognition in Practice II, Elsevier Science Publishers: New York, pp. 57–72, 1986.

  14. 14.

    E. Gat, M.G. Slack, D.P. Miller, and R.J. Firby, Path planning and execution monitoring for a planetary rover, Proc. IEEE Intern. Conf. Robot. Autom. IEEEE Computer Society Press, pp. 20–25, May 1990.

  15. 15.

    D. Gennery. Personal communication, 1990.

  16. 16.

    D.B. Gennery. Modeling the environment of an exploring vehicle by means of stereo vision. Ph.D. thesis, Stanford University, June 1980.

  17. 17.

    D.B.Gennery, Visual terrain matching for a Mars rover, Proc. Conf. Comput. Vision Patt. Recog., San Diego, IEEE Computer Society Press, pp. 483–491, June 1989.

  18. 18.

    F.A.Graybill, Matrices with Applications in Statistics. Wadsworth International Group: Belmont, CA, 1983.

  19. 19.

    B.K.P.Horn, Robot Vision. MIT Press: Cambridge, MA 1986.

  20. 20.

    A.K.Jain. Fundamentals of Digital Image Processing. Prentice Hall: Englewood Cliffs, NJ, 1989.

  21. 21.

    J.R. Jordan and A.C. Bovik, Computational stereo vision using color, IEEE Cont. Syst. Mag. pp. 31–36, June 1988.

  22. 22.

    A.E. Kayaalp, High speed machine perception using passive sensing technology, Proc. SPIE Conf. 1195, Mobile Robots IV, pp. 62–74, November 1989.

  23. 23.

    D.Marr and T.Poggio, Cooperative computation of stereo disparity, Science, 194:283–287, 1976.

  24. 24.

    J.Marroquin, S.Mitter, and T.Poggio. Probabilistic solution of ill-posed problems in computational vision, J. Amer. Stat. Assoc., 82(397):76–89, March 1987.

  25. 25.

    J.L. Marroquin. Probabilistic Solution of Inverse Problems. Ph.D. thesis, MIT, September 1985.

  26. 26.

    L.H. Matthies. Dynamic Stereo Vision. Ph.D. thesis, Carnegie Mellon University, October 1989.

  27. 27.

    L.H. Matthies, Toward stochastic modeling of obstacle detectability in passive stereo range imagery, Proc. Conf. Comput. Vision Patt. Recog., IEEE Computer Society, June 1992.

  28. 28.

    L.H. Matthies and M. Okutomi. A Bayesian foundation for active stereo vision. In P. Schenker Proc. SPIE Conf. 1198, Sensor Fusion II: Human and Machine Strategies, pp. 62–74, November 1989.

  29. 29.

    L.H.Matthies and S.A.Shafer, Error modeling in stereo navigation, IEEE J. Robot. Autom., RA-3(3):239–248, June 1987.

  30. 30.

    L.H.Matthies, R.Szeliski, and T.Kanade, Kalman filter-based algorithms for estimating depth from image sequences. Intern. J. Comput. Vision, 3:209–236, 1989.

  31. 31.

    P.S.Maybeck, Stochastic Models, Estimation, and Control, vol.. 1, Academic Press: New York, 1979.

  32. 32.

    V.J. Milenkovic and T. Kanade, Trinocular vision using photometric and edge orientation constraints, Proc. DARPA Image Understanding Workship, December 1985.

  33. 33.

    H.P. Moravec. Obstacle avoidance and navigation in the real world by a seeing robot rover. Ph.D. thesis, Stanford University, September 1980.

  34. 34.

    H.K.Nishihara, Practical real-time imaging stereo matcher, Optical Engineering 23(5):536–545, October 1984.

  35. 35.

    H.K. Nishihara, RTVS-3: Real-time binocular stereo and optical flow measurement system. System description manuscript, July 1990.

  36. 36.

    Y.Ohta and T.Kanade, Stereo by intra- and inter-scanline search using dynamic programming, IEEE Trans. Patt. Anal. Mach. Intell., 7(2):139–154, March 1985.

  37. 37.

    T.Poggio, V.Torre, and C.Koch, Computational vision and regularization theory, Nature 317(n):314–319, September 1985.

  38. 38.

    K.Prazdny, Detection of binocular disparities, Biological Cybernetics 52:93–99, 1985.

  39. 39.

    L.H. Quam, Hierarchical warp stereo, Proc. Image Understanding Workshop. Science Applications International, 1984.

  40. 40.

    T.W.Ryan, R.T.Gray, and B.R.Hunt, Prediction of correlation errors in stereo-pair images, Optical Engineering, 19(3):312–322, May/June 1980.

  41. 41.

    D.Sankoff and J.B.Kruskal, Time Warps, String Edits, and Macromolecules: the Theory and Practice of Sequence Comparison. Addison-Wesley: Reading, MA, 1983.

  42. 42.

    R.Sedgewick, Algorithms. Addison-Wesley: Reading, MA, 1983.

  43. 43.

    M.G. Slack. Situationally driven local navigation for mobile robots. Technical report, Jet Propulsion Laboratory, April 1990.

  44. 44.

    C.C.Slama, ed. Manual of Photogrammetry, 4th ed. American Society of Photogrammetry: Falls Church, A, 1980.

  45. 45.

    C.V. Stewart and C.R. Dyer, The trinocular general support algorithm: a three-camera stereo algorithm for overcoming binocular matching errors, Proc. 2nd Itern. Conf. Comput. Vision, Tarpon Springs, FL, pp. 134–138, December 1988.

  46. 46.

    R.Szeliski. Cooperative algorithms for solving random-dot stereograms. Technical Report CMU-CS-8–133, Computer Science Department, Carnegie Mellon University: Pittsburgh, PA, June 1986.

  47. 47.

    R. Szeliski, Regularization uses fractal priors, Proc. AAAI-87: 6th Natl. Conf. Artif. Intell., Morgan Kaufmann Publishers, pp. 749–754, 1987.

  48. 48.

    R. Szeliski, bayesian Modeling of Uncertainty in Low Level Vision. Ph.D. thesis, Carnegie Mellon University, August 1988.

  49. 49.

    R. Szeliski and G. Hinton, Solving random-dot stereograms using the heat equation, Proc. Conf. Comput. Vision Patt. Recog., San Francisco, pp. 284–288, 1985.

  50. 50.

    D.Terzopoulos, Multilevel computational processes for visual surface reconstruction, Comput. Vision, Graph, Image Process. 24:52–96, 1983.

  51. 51.

    D.Terzopoulos, Regularization of inverse visual problems involving discontinuities, Trans. Patt. Anal. Mach. Intell. 8(4):413–424, July 1986.

  52. 52.

    H.L.VanTrees, Detection, Estimation, and Modulation Theory, Part I. Wiley: New York, 1968.

  53. 53.

    A.Witkin, D.Terzopoulos, and M.Kass, Signal matching through scale space, Intern. J. Comput. Vision 1(2):133–144, 1987.

  54. 54.

    G. Xu, S. Tsuji, and M. Asada, Coarse-to-fine control strategy for matching motion stereo pairs, Proc. 9th Intern. Joint Conf. Artif. Intell., Los Angeles, pp. 892–894, 1985.

Download references

Author information

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Matthies, L. Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation. Int J Comput Vision 8, 71–91 (1992).

Download citation


  • Obstacle Avoidance
  • Stereo Vision
  • Stereo Camera
  • Image Pyramid
  • Random Field Model