Skip to main content
SpringerLink
Log in

An FPGA-based RGBD imager

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper describes a trinocular stereo vision system using a single chip of FPGA to generate the composite color (RGB) and disparity data stream at video rate, called the RGBD imager. The system uses the triangular configuration of three cameras for synchronous image capture and the trinocular adaptive cooperative algorithm based on local aggregation for smooth and accurate dense disparity mapping. We design a fine-grain parallel and pipelining architecture in FPGA for implementation to achieve a high computational and real-time throughput. A binary floating-point format is customized for data representation to satisfy the wide data range and high computation precision demands in the disparity calculation. Memory management and data bit-width control are applied in the system to reduce the hardware resource consumption and accelerate the processing speed. The system is able to produce dense disparity maps with 320 × 240 pixels in a disparity search range of 64 pixels at the rate of 30 frames per second.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. An, L., Jia, Y., Wang, J., Zhang, X., Li, M.: An efficient rectification method for trinocular stereovision. In: International Conference on Pattern Recognition, vol. 4, pp. 56–59 (2004)

  2. Bansal, M., Jain, A., Camus, T., Das, A.: Towards a practical stereo vision sensor. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 63–69 (2005)

  3. Brown M.Z., Burschka D., Hager G.D.: Advances in computational stereo. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 993–1008 (2003)

    Article  Google Scholar 

  4. Darabiha A., MacLean W., Rose J.: Reconfigurable hardware implementation of a phase-correlation stereo algorithm. Mach. Vis. Appl. 17(2), 116–132 (2006)

    Article  Google Scholar 

  5. Dhond U., Aggarwal J.: Structure from stereo: a review. IEEE Trans. Syst. Man Cybern. 19(6), 1489–1510 (1989)

    Article  MathSciNet  Google Scholar 

  6. Forstmann, S., Kanou, Y., Ohya, J., Thuering, S., Schmitt, A.: Real-time stereo by using dynamic programming. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, pp. 29–36 (2004)

  7. Gokturk, S., Yalcin, H., Bamji, C.: A time-of-flight depth sensor system description, issues and solutions. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop (2004)

  8. Gong M.: Real-time joint disparity and disparity flow estimation on programmable graphics hardware. Comput. Vis. Image Underst. 113(1), 90–100 (2009)

    Article  Google Scholar 

  9. Gong M., Yang R., Wang L., Gong M.: A performance study on different cost aggregation approaches used in real-time stereo matching. Int. J. Comput. Vis. 75(2), 283–296 (2007)

    Article  Google Scholar 

  10. Hirschmuller H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)

    Article  Google Scholar 

  11. Jia, Y., Xu, Y., Liu, W., Yang, C., Zhu, Y., Zhang, X., An, L.: A miniature stereo vision machine for real-time dense depth mapping. In: International Conference on Computer Vision Systems, pp. 268–277 (2003)

  12. Jia, Y., Zhang, X., Li, M., An, L.: A miniature stereo vision machine (msvm-iii) for dense disparity mapping. In: International Conference on Pattern Recognition, pp. 728–731 (2004)

  13. Jin S., Cho J., Dai Pham X., Lee K., Park S., Kim M., Jeon J.: Fpga design and implementation of a real-time stereo vision system. IEEE Trans. Circuits Syst. Video Technol. 20(1), 15–26 (2010)

    Article  Google Scholar 

  14. Kanade, T., Kano, H., Kimura, S., Yoshida, A., Oda, K.: Development of a video-rate stereo machine. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 95–100 (1995)

  15. Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for video-rate dense depth mapping and its new applications. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 196–202 (1996)

  16. Kimura, S., Shinbo, T., Yamaguchi, H., Kawamura, E., Nakano, K.: A convolver-based real-time stereo machine (sazan). In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1457–1463 (1999)

  17. Konolige, K.: Small vision systems: Hardware and implementation. In: International Symposium on Robotics Research, vol. 8, pp. 203–212 (1997)

  18. Leung C., Appleton B., Sun C.: Iterated dynamic programming and quadtree subregioning for fast stereo matching. Image Vis. Comput. 26(10), 1371–1383 (2008)

    Article  Google Scholar 

  19. Li, M., Jia, Y.: Stereo vision system on programmable chip (svsoc) for small robot navigation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1359–1365 (2006)

  20. Marr D., Poggio T.: Cooperative computation of stereo disparity. Science 194(4262), 283–287 (1976)

    Article  Google Scholar 

  21. Marr D., Poggio T.: A computational theory of human stereo vision. R. Soc. Lond. 204(1156), 301–328 (1979)

    Article  Google Scholar 

  22. Okutomi M., Kanade T.: A multiple-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993)

    Article  Google Scholar 

  23. Scharstein D., Szeliski R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)

    Article  MATH  Google Scholar 

  24. Sizintsev M., Wildes R.: Coarse-to-fine stereo vision with accurate 3d boundaries. Image Vis. Comput. 28(3), 352–366 (2010)

    Article  Google Scholar 

  25. Woodfill, J., Von Herzen, B.: Real-time stereo vision on the parts reconfigurable computer. In: IEEE Symposium on FPGAs for Custom Computing Machines, pp. 201–210 (1997)

  26. Woodfill, J.I., Gordon, G., Jurasek, D., Brown, T., Buck, R.: The tyzx deepsea g2 vision system, a taskable, embedded stereo camera. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, pp. 126–132 (2006)

  27. Yang Q., Wang L., Yang R., Stewenius H., Nister D.: Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 492–504 (2009)

    Article  Google Scholar 

  28. Yang, R., Pollefeys, M., Li, S.: Improved real-time stereo on commodity graphics hardware. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, pp. 36–43 (2004)

  29. Zitnick C., Kanade T.: A cooperative algorithm for stereo matching and occlusion detection. IEEE Trans. Pattern Anal. Mach. Intell. 22(7), 675–684 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beijing Laboratory of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing, 100081, People’s Republic of China

    Lei Chen, Yunde Jia & Mingxiang Li

Authors
  1. Lei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunde Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingxiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunde Jia.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, L., Jia, Y. & Li, M. An FPGA-based RGBD imager. Machine Vision and Applications 23, 513–525 (2012). https://doi.org/10.1007/s00138-011-0334-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-011-0334-z

Keywords

Search

Navigation