Journal of Real-Time Image Processing

, Volume 10, Issue 1, pp 163–174 | Cite as

Efficient stereo vision algorithms for resource-limited systems

  • Beau Tippetts
  • Dah Jye Lee
  • Kirt Lillywhite
  • James Archibald
Original Research Paper
  • 438 Downloads

Abstract

In most circumstances, determining an acceptable trade-off between speed and accuracy when selecting a stereo vision algorithm for implementation is dependent on the target application. This work attempts to provide a perspective on the efficiency of existing real-time stereo vision algorithms in terms of this trade-off. This work also provides an example of modifying an existing highly accurate stereo vision algorithm to increase its runtime performance while trying to limit the loss in accuracy. The modifications can be used to increase efficiency of several other local stereo vision algorithms due to sharing some common components. Such an example demonstrates the challenge of making efficient trade-offs in accuracy for runtime performance. It is shown that the modifications resulted in an 8X speedup over the original algorithm, with accuracy results comparable to existing real-time algorithms.

Keywords

Dense disparity stereo vision Stereo image-processing for resource-limited systems Intensity profile shape matching Efficient stereo vision algorithm 

References

  1. 1.
    Ambrosch, K., Zinner, C., Leopold, H.: A miniature embedded stereo vision system for automotive applications. In: Proceedings of the IEEE 26th Convention of Electrical and Electronics Engineers in Israel (IEEEI), pp. 786–789 (2010)Google Scholar
  2. 2.
    Bhusnurmath, A,, Taylor, C.J.: Solving stereo matching problems using interior point methods. In: Fourth International Symposium on 3D Data Processing, Visualization and Transmission, pp. 321–329 (2008)Google Scholar
  3. 3.
    Bleyer, M., Gelautz, M.: A layered stereo matching algorithm using image segmentation and global visibility constraints. ISPRS J. Photogramm. Remote Sens.59(3), 128–150 (2005)CrossRefGoogle Scholar
  4. 4.
    Bleyer, M., Gelautz, M., Rother, C., Rhemann, C.: A stereo approach that handles the matting problem via image warping. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009 (CVPR 2009) pp. 501–508 (2009)Google Scholar
  5. 5.
    Bleyer, M., Rother, C., Kohli, P.: Surface stereo with soft segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 1570–1577 (2010)Google Scholar
  6. 6.
    Bleyer, M., Rhemann, C., Rother, C.: Patchmatch stereo–stereo matching with slanted support windows. In: British Machine Vision Conference (BMVC) (2011)Google Scholar
  7. 7.
    Bleyer, M., Rother, C., Kohli, P., Scharstein, D., Sinha, S.: Object stereo–joint stereo matching and object segmentation. In: IEEE Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  8. 8.
    Bobick, A.F., Intille, S.S.: Large occlusion stereo. Int. J. Comput. Vis. 33, 181–200 (1999). doi:10.1023/A:1008150329890 CrossRefGoogle Scholar
  9. 9.
    Brockers, R: Cooperative stereo matching with color-based adaptive local support. In: Proceedings of the 13th International Conference on Computer Analysis of Images and Patterns, Springer, Berlin, CAIP ’09, pp. 1019–1027 (2009) aCM ID: 1618054Google Scholar
  10. 10.
    Bruch, M., Lum, J., Yee, S., Tran, N.: Advances in autonomy for small UGVs. SPIE Proc 5804: Unmanned Ground Vehicle Technology VII, Orlando, FL (2005)Google Scholar
  11. 11.
    Cassisa, C.: Local vs global energy minimization methods: application to stereo matching. In: Proceedings of the IEEE International Progress in Informatics and Computing (PIC) Conference, vol. 2, pp. 678–683 (2010)Google Scholar
  12. 12.
    Chan, S.O., Wong, Y., Daniel, J.K.: Dense stereo correspondence based on recursive adaptive size multi-windowing. In: Proceedings of the Image and Vision Computing New Zealand, vol. 1, pp. 256–260 (2003)Google Scholar
  13. 13.
    Chang, N.Y., Tsai, T., Hsu, B., Chen, Y., Chang, T.: Algorithm and architecture of disparity estimation with mini-census adaptive support weight. IEEE Trans. Circuits Syst. Video Technol. 20(6), 792–805 (2010)CrossRefGoogle Scholar
  14. 14.
    De-Maeztu, L., Mattoccia, S., Villanueva, A., Cabeza, R.: Linear stereo matching. In: A13th International Conference on Computer Vision (ICCV2011) (2011)Google Scholar
  15. 15.
    De-Maeztu, L., Villanueva, A., Cabeza, R.: Stereo matching using gradient similarity and locally adaptive support-weight. Pattern Recognition Letters 32(13), 1643–1651 (2011)CrossRefGoogle Scholar
  16. 16.
    Demoulin, C., Droogenbroeck, M.V.: A method based on multiple adaptive windows to improve the determination of disparity maps. In: Proceedings of the IEEE Workshop on Circuit, Systems and Signal Processing (2005)Google Scholar
  17. 17.
    Desouza, G.N., Kak, A.C.: Vision for mobile robot navigation: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(2), 237–267 (2002)Google Scholar
  18. 18.
    Einecke, N., Eggert, J.: A two-stage correlation method for stereoscopic depth estimation. In: Digital Image Computing: Techniques and Applications, IEEE Computer Society, Los Alamitos, CA, USA, vol. 0, pp. 227–234 (2010)Google Scholar
  19. 19.
    Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70(1), 41–54 (2006)CrossRefGoogle Scholar
  20. 20.
    Gehrig, S., Franke, U.: Improving sub-pixel accuracy for long range stereo. In: ICCV VRML workshop (2007)Google Scholar
  21. 21.
    Gerrits, M., Bekaert, P.: Local stereo matching with segmentation-based outlier rejection. In: Proceedings of the 3rd Canadian Conference Computer and Robot Vision (2006)Google Scholar
  22. 22.
    Gong, M., Yang, R.: Image-gradient-guided real-time stereo on graphics hardware. In: Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling 3DIM 2005, pp. 548–555 (2005)Google Scholar
  23. 23.
    Gupta, R., Cho, S.: A correlation-based approach for real-time stereo matching. In: Advances in Visual Computing. Lecture Notes in Computer Science, vol. 6454, pp. 129–138. Springer, Berlin (2010) doi:10.1007/978-3-642-17274-8_13
  24. 24.
    Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005 (CVPR 2005), vol. 2, pp. 807–814 (2005)Google Scholar
  25. 25.
    Hosni, A., Bleyer, M., Gelautz, M., Rhemann, C.: Local stereo matching using geodesic support weights. In: 16th IEEE International Conference on Image Processing (ICIP), 2009, pp. 2093–2096 (2009)Google Scholar
  26. 26.
    Howard, A.: Real-time stereo visual odometry for autonomous ground vehicles. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS 2008, 3946–3952 (2008)Google Scholar
  27. 27.
    Humenberger, M., Engelke, T., Kubinger, W.: A census-based stereo vision algorithm using modified semi-global matching and plane fitting to improve matching quality. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2010, pp. 77–84 (2010)Google Scholar
  28. 28.
    Humenberger, M., Zinner, C., Weber, M., Kubinger, W., Vincze, M.: A fast stereo matching algorithm suitable for embedded real-time systems. Comput. Vis. Image Underst. 114(11), 1180–1202 (2010)CrossRefGoogle Scholar
  29. 29.
    Kang, S.B., Szeliski, R., Chai, J.: Handling occlusions in dense multi-view stereo. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2001, vol. 1 (2001)Google Scholar
  30. 30.
    Kim, J.C., Lee, K.M., Choi, B.T., Lee, S.U.: A dense stereo matching using two-pass dynamic programming with generalized ground control points. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2005, vol. 2, pp. 1075–1082 (2005)Google Scholar
  31. 31.
    Klaus, A., Sormann, M., Karner, K.: Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th International Conference on Pattern Recognition 2006 (ICPR 2006), vol. 3, pp. 15–18 (2006)Google Scholar
  32. 32.
    Konolige, K.: Small vision systems: hardware and implementation. In: 8th International Symposium on Robotics Research, pp. 111–116 (1997)Google Scholar
  33. 33.
    Kosov, S., Thormahlen. T., Seidel, H.: Accurate real-time disparity estimation with variational methods. In: ISVC ’09 Proceedings of the 5th International Symposium on Advances in Visual Computing: Part I (2009)Google Scholar
  34. 34.
    Lei, C., Selzer, J., Yang, Y.: Region-tree based stereo using dynamic programming optimization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, vol. 2, pp. 2378–2385 (2006)Google Scholar
  35. 35.
    van der Mark, W., Gavrila, D.M.: Real-time dense stereo for intelligent vehicles. IEEE Trans. Intell. Transp. Syst. 7(1), 38–50 (2006)CrossRefGoogle Scholar
  36. 36.
    Mattoccia, S.: A locally global approach to stereo correspondence. In: IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), 2006, pp. 1763–1770 (2009)Google Scholar
  37. 37.
    Mei, X., Sun, X., Zhou, M., Jiao, S., Wang, H., Zhang, X.: On building an accurate stereo matching system on graphics hardware. Tech. rep, Samsung Advanced Institute of Technology (2011)Google Scholar
  38. 38.
    Min, D., Sohn, K.: Cost aggregation and occlusion handling with WLS in stereo matching. IEEE Trans. Image Process. 17(8), 1431–1442 (2008)Google Scholar
  39. 39.
    Papadakis, N., Caselles, V.: Multi-label depth estimation for graph cuts stereo problems. J. Math. Imaging Vis. 38(1), 70–82 (2010)CrossRefMathSciNetGoogle Scholar
  40. 40.
    PassMark Software: CPU benchmarks (2012) http://www.cpubenchmark.net/cpu_ist.php
  41. 41.
    Scharstein, D.: Middlebury stereo evaluation (2002) http://vision.middlebury.edu/stereo/eval/
  42. 42.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)CrossRefMATHGoogle Scholar
  43. 43.
    Sun, J., Li, Y., Kang, S., Shum, H.: Symmetric stereo matching for occlusion handling. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005 (CVPR 2005), vol. 2, pp. 399–406 (2005)Google Scholar
  44. 44.
    Sun, X., Mei, X., Jiao, S., Zhou, M., Wang, H.: Stereo matching with reliable disparity propagation. In: Proceedings of the International 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT) Conference, pp. 132–139Google Scholar
  45. 45.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)CrossRefGoogle Scholar
  46. 46.
    Tippetts, B.J., Lee, D., Archibald, J.K., Lillywhite, K.D.: Dense disparity real-time stereo vision algorithm for resource-limited systems. IEEE Trans. Circuits Syst. Video Technol. 21(10), 1547–1555 (2011)CrossRefGoogle Scholar
  47. 47.
    Tombari , F., Mattoccia, S., Stefano, L.D.: Segmentation-based adaptive support for accurate stereo correspondence. In: Proceedings of the 2nd Pacific Rim conference on Advances in Image and Video Technology, PSIVT’07, pp. 427–438. Springer, BerlinGoogle Scholar
  48. 48.
    Tombari, F., Mattoccia, S., Di Stefano, L., Addimanda, E.: Classification and evaluation of cost aggregation methods for stereo correspondence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition CVPR 2008, pp. 1–8 (2008)Google Scholar
  49. 49.
    Tombari, F., Mattoccia, S., Stefano, L.D., Addimanda, E.: Near real-time stereo based on effective cost aggregation. In: 19th International Conference on Pattern Recognition 2008 (ICPR 2008), pp. 1–4 (2008)Google Scholar
  50. 50.
    Veksler, O.: Fast variable window for stereo correspondence using integral images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1 (2003)Google Scholar
  51. 51.
    Wang, L., Yang, R.: Global stereo matching leveraged by sparse ground control points. In: IEEE Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  52. 52.
    Wang, Z., Zheng, Z.: A region based stereo matching algorithm using cooperative optimization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 0, pp.1–8 (2008)Google Scholar
  53. 53.
    Xu, Y., Wang, D., Feng, T., Shum, H.: Stereo computation using radial adaptive windows. In: Proceedings of the 16th International Pattern Recognition Conference, vol. 3, pp. 595–598 (2002)Google Scholar
  54. 54.
    Yang, Q., Wang, L., Ahuja, N.: A constant-space belief propagation algorithm for stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1458–1465 (2010)Google Scholar
  55. 55.
    Yoon, K., Kweon, I.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 650–656 (2006)Google Scholar
  56. 56.
    Zhang, K., Lu, J., Lafruit, G.: Cross-based local stereo matching using orthogonal integral images. IEEE Trans. Circuits Syst. Video Technol. 19(7), 1073–1079 (2009)Google Scholar
  57. 57.
    Zitnick, C.L., Kang, S.B.: Stereo for image-based rendering using image over-segmentation. Int. J. Comput. Vis. 75(1), 49–65 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Beau Tippetts
    • 1
  • Dah Jye Lee
    • 1
  • Kirt Lillywhite
    • 1
  • James Archibald
    • 1
  1. 1.Brigham Young UniversityProvoUSA

Personalised recommendations