Advertisement

Machine Vision and Applications

, Volume 23, Issue 3, pp 513–525 | Cite as

An FPGA-based RGBD imager

  • Lei Chen
  • Yunde Jia
  • Mingxiang Li
Original Paper

Abstract

This paper describes a trinocular stereo vision system using a single chip of FPGA to generate the composite color (RGB) and disparity data stream at video rate, called the RGBD imager. The system uses the triangular configuration of three cameras for synchronous image capture and the trinocular adaptive cooperative algorithm based on local aggregation for smooth and accurate dense disparity mapping. We design a fine-grain parallel and pipelining architecture in FPGA for implementation to achieve a high computational and real-time throughput. A binary floating-point format is customized for data representation to satisfy the wide data range and high computation precision demands in the disparity calculation. Memory management and data bit-width control are applied in the system to reduce the hardware resource consumption and accelerate the processing speed. The system is able to produce dense disparity maps with 320 × 240 pixels in a disparity search range of 64 pixels at the rate of 30 frames per second.

Keywords

RGBD Imager Trinocular stereo vision Cooperative algorithm FGPA 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    An, L., Jia, Y., Wang, J., Zhang, X., Li, M.: An efficient rectification method for trinocular stereovision. In: International Conference on Pattern Recognition, vol. 4, pp. 56–59 (2004)Google Scholar
  2. 2.
    Bansal, M., Jain, A., Camus, T., Das, A.: Towards a practical stereo vision sensor. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 63–69 (2005)Google Scholar
  3. 3.
    Brown M.Z., Burschka D., Hager G.D.: Advances in computational stereo. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 993–1008 (2003)CrossRefGoogle Scholar
  4. 4.
    Darabiha A., MacLean W., Rose J.: Reconfigurable hardware implementation of a phase-correlation stereo algorithm. Mach. Vis. Appl. 17(2), 116–132 (2006)CrossRefGoogle Scholar
  5. 5.
    Dhond U., Aggarwal J.: Structure from stereo: a review. IEEE Trans. Syst. Man Cybern. 19(6), 1489–1510 (1989)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Forstmann, S., Kanou, Y., Ohya, J., Thuering, S., Schmitt, A.: Real-time stereo by using dynamic programming. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, pp. 29–36 (2004)Google Scholar
  7. 7.
    Gokturk, S., Yalcin, H., Bamji, C.: A time-of-flight depth sensor system description, issues and solutions. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop (2004)Google Scholar
  8. 8.
    Gong M.: Real-time joint disparity and disparity flow estimation on programmable graphics hardware. Comput. Vis. Image Underst. 113(1), 90–100 (2009)CrossRefGoogle Scholar
  9. 9.
    Gong M., Yang R., Wang L., Gong M.: A performance study on different cost aggregation approaches used in real-time stereo matching. Int. J. Comput. Vis. 75(2), 283–296 (2007)CrossRefGoogle Scholar
  10. 10.
    Hirschmuller H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  11. 11.
    Jia, Y., Xu, Y., Liu, W., Yang, C., Zhu, Y., Zhang, X., An, L.: A miniature stereo vision machine for real-time dense depth mapping. In: International Conference on Computer Vision Systems, pp. 268–277 (2003)Google Scholar
  12. 12.
    Jia, Y., Zhang, X., Li, M., An, L.: A miniature stereo vision machine (msvm-iii) for dense disparity mapping. In: International Conference on Pattern Recognition, pp. 728–731 (2004)Google Scholar
  13. 13.
    Jin S., Cho J., Dai Pham X., Lee K., Park S., Kim M., Jeon J.: Fpga design and implementation of a real-time stereo vision system. IEEE Trans. Circuits Syst. Video Technol. 20(1), 15–26 (2010)CrossRefGoogle Scholar
  14. 14.
    Kanade, T., Kano, H., Kimura, S., Yoshida, A., Oda, K.: Development of a video-rate stereo machine. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 95–100 (1995)Google Scholar
  15. 15.
    Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for video-rate dense depth mapping and its new applications. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 196–202 (1996)Google Scholar
  16. 16.
    Kimura, S., Shinbo, T., Yamaguchi, H., Kawamura, E., Nakano, K.: A convolver-based real-time stereo machine (sazan). In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1457–1463 (1999)Google Scholar
  17. 17.
    Konolige, K.: Small vision systems: Hardware and implementation. In: International Symposium on Robotics Research, vol. 8, pp. 203–212 (1997)Google Scholar
  18. 18.
    Leung C., Appleton B., Sun C.: Iterated dynamic programming and quadtree subregioning for fast stereo matching. Image Vis. Comput. 26(10), 1371–1383 (2008)CrossRefGoogle Scholar
  19. 19.
    Li, M., Jia, Y.: Stereo vision system on programmable chip (svsoc) for small robot navigation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1359–1365 (2006)Google Scholar
  20. 20.
    Marr D., Poggio T.: Cooperative computation of stereo disparity. Science 194(4262), 283–287 (1976)CrossRefGoogle Scholar
  21. 21.
    Marr D., Poggio T.: A computational theory of human stereo vision. R. Soc. Lond. 204(1156), 301–328 (1979)CrossRefGoogle Scholar
  22. 22.
    Okutomi M., Kanade T.: A multiple-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993)CrossRefGoogle Scholar
  23. 23.
    Scharstein D., Szeliski R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)zbMATHCrossRefGoogle Scholar
  24. 24.
    Sizintsev M., Wildes R.: Coarse-to-fine stereo vision with accurate 3d boundaries. Image Vis. Comput. 28(3), 352–366 (2010)CrossRefGoogle Scholar
  25. 25.
    Woodfill, J., Von Herzen, B.: Real-time stereo vision on the parts reconfigurable computer. In: IEEE Symposium on FPGAs for Custom Computing Machines, pp. 201–210 (1997)Google Scholar
  26. 26.
    Woodfill, J.I., Gordon, G., Jurasek, D., Brown, T., Buck, R.: The tyzx deepsea g2 vision system, a taskable, embedded stereo camera. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, pp. 126–132 (2006)Google Scholar
  27. 27.
    Yang Q., Wang L., Yang R., Stewenius H., Nister D.: Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 492–504 (2009)CrossRefGoogle Scholar
  28. 28.
    Yang, R., Pollefeys, M., Li, S.: Improved real-time stereo on commodity graphics hardware. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, pp. 36–43 (2004)Google Scholar
  29. 29.
    Zitnick C., Kanade T.: A cooperative algorithm for stereo matching and occlusion detection. IEEE Trans. Pattern Anal. Mach. Intell. 22(7), 675–684 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Beijing Laboratory of Intelligent Information Technology, School of Computer ScienceBeijing Institute of TechnologyBeijingPeople’s Republic of China

Personalised recommendations