Machine Vision and Applications

, Volume 17, Issue 2, pp 116–132 | Cite as

Reconfigurable hardware implementation of a phase-correlation stereoalgorithm

Original Paper

Abstract

This paper describes the implementation of a stereo-vision system using Field Programmable Gate Arrays (FPGAs). Reconfigurable hardware, including FPGAs, is an attractive platform for implementing vision algorithms due to its ability to exploit parallelism often found in these algorithms, and due to the speed with which applications can be developed as compared to hardware. The system outputs 8-bit, subpixel disparity estimates for 256× 360 pixel images at 30,fps. A local-weighted phase correlation algorithm for stereo disparity [Fleet, D. J.: {Int. Conf. Syst. Man Cybernetics 1:48–54 (1994)] is implemented. Despite the complexity of performing correlations on multiscale, multiorientation phase data, the system runs as much as 300 times faster in hardware than its software implementation. This paper describes the hardware platform used, the algorithm, and the issues encountered during its hardware implementation. Of particular interest is the implementation of multiscale, steerable filters, which are widely used in computer vision algorithms. Several trade-offs (reducing the number of filter orientations from three to two, using fixed-point computation, changing the location of one localized low-pass filter, and using L1 instead of L2 norms) were required to both fit the design into the available hardware and to achieve video-rate processing. Finally, results from the system are given both for synthetic data sets as well as several standard stereo-pair test images.

Keywords

Stereo disparity estimation Frame rate implementation Field Programmable Gate Arrays (FPGAs) Reconfigurable hardware implementation Phase correlation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fleet, D.J.: Disparity from local weighted phase correlation. Int. Conf. Syst. Man Cybernetics 1, 48–54 (1994)Google Scholar
  2. Xilinx Inc.: Design tools center, cited May 3, 2005. [Online]. Available: http://www.xilinx.com/products/design_resources/design_tool/index.htm
  3. Altera Corporation: DSP Builder, cited May 3, 2005 [Online]. Available: http://www.altera.com/products/software/products/dsp/dsp-builder.html
  4. Fung, J., Mann, S.: Using multiple graphics cards as a general purpose parallel computer: Applications to computer vision. In: Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, pp. 805–808 (2004) online: http://www.eyetap.org/papers/docs/procicpr2004/
  5. Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Proceedings of the 2003 IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, Madison, Wisconsin, pp. 211–218 (2003) [Online]. Available: http://citeseer.ist.psu.edu/article/yang03multiresolution.html
  6. van Ierssel, M.: TM-3 documentation (2002) [Online]. Available: http://www.eecg.toronto.edu/tm3/
  7. van Ierssel, M., Galloway, D., Chow, P., Rose, J.: The Transmogrifier 3-a: Hardware and software for a 3 million gate rapid prototyping system. In: Micronet Annual Workshop (2001)Google Scholar
  8. Xilinx Inc.: Xilinx data sheets (2002) [Online]. Available: http://direct.xilinx.com/bvdocs/publications/ds022-1.pdf
  9. Faugeras, O., Hotz, B., Mathieu, H., Viéville, T., Zhang, Z., Fua, P., Théron, E., Moll, L., Berry, G., Vuillemin, J., Bertin, P., Proy, C.: Real time correlation-based stereo: Algorithm, implementations and applications. INRIA Sophia Antipolis, Tech. Rep. Research Report 2013 (1993) [Online]. Available: ftp://ftp.inria.fr/INRIA/publication/publi-ps-gz/RR/RR-2013.ps.gz
  10. Hou, K.M., Belloum, A.: A reconfigurable and flexible parallel 3d vision system for a mobile robot. In: IEEE Workshop on Computer Architecture for Machine Perception New Orleans, Louisiana (1993)Google Scholar
  11. Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for video-rate dense depth mapping and its new applications. In: Proceedings of the 15th IEEE Computer Vision & Pattern Recognition Conference, pp. 196–202. San Francisco (1996)Google Scholar
  12. Woodfill, J., Herzen, B.V.: Real time stereo vision on the parts reconfigurable computer. In: 5th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, pp. 201–210 (1997)Google Scholar
  13. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the 3rd European Conference on Computer Vision, pp. 150–158 (1994) http://www.cs.cornell.edu/rdz/Papers/Archive/neccv.ps, http://www.cs.cornell.edu/rdz/Papers/Archive/nplt-journal.ps.gz
  14. Burt, P.J.: A pyramid-based front-end processor for dynamic vision applications. In: Proceedings of the IEEE 90(7), 1188–1200 (2002)Google Scholar
  15. van der Wal, G., Hansen, M., Piacentino, M.: The acadia vision processor.In: Proceedings of the Fifth IEEE International Workshop on Computer Architecture for Machine Perception, pp. 31–40. Padua, Italy (2000)Google Scholar
  16. Hirschmüller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. Int. J. Comput. Vis. 47(1–3), 229–246 (2002), stereo, intensity correlation, MMX, fastGoogle Scholar
  17. Mühlmann, K., Maier, D., Hesser, J., Anner, R.M.: Calculating dense disparity maps from color stereo images, an efficient implementation. Int. J. Comput. Vis. 47(1–3), 79–88 (2002), stereo, intensity correlation, MMX, fastGoogle Scholar
  18. Mulligan, J., Isler, V., Daniilidis, K.: Trinocular stereo: A real-time algorithm and its evaluation. Int. J. Comput. Vis. 47(1–3), 51–61 (2002), stereo, trinocular, DFW-V500Google Scholar
  19. Sun, C.: Fast stereo matching using rectangular subregioning and 3d maximum-surface techniques. Int. J. Comput. Vis. 47(1–3), 99–117 (2002)CrossRefMATHGoogle Scholar
  20. Veksler, O.: Stereo correspondence with compact windows via minimum ratio cycle. IEEE Trans. Pattern Anal. Machine Intell. 24(12), 1654–1660 (2002)CrossRefGoogle Scholar
  21. Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 31, 532–540 (1983)CrossRefGoogle Scholar
  22. Meerbergen, G.V., Vergauwen, M., Pollefeys, M., Gool, L.V.: A hierarchical symmetric stereo algorithm using dynamic programming. Int. J. Comput. Vis. 47(1–3), 275–285 (2002)CrossRefMATHGoogle Scholar
  23. Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. Int. J. Comput. Vis. 35(3), 269–293 (1999)CrossRefGoogle Scholar
  24. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3), 7–42 (2002)CrossRefMATHGoogle Scholar
  25. Point Grey Research: Point Grey Research homepage. cited May 3, 2005 [Online]. Available: http://www.ptgrey.com/
  26. Videre Design: SRI small vision system, cited May 3, 2005 [Online]. Available: http://www.videredesign.com/svs.htm
  27. Konolige, K.: Small vision systems: Hardware and implentation. In: Proceedings of the Eighth International Symposium on Robotics Research (Robotics Research 8), Hayama, Japan, pp. 203–212 (1997) [Online].Available: citeseer.ist.psu.edu/konolige97small.htmlGoogle Scholar
  28. Canesta Inc.: Canesta ep development kit. cited May 3, 2005. [Online]. Available: http://www.canesta.com/devkit.htm
  29. Baker, H.H., Binford, T.O.: Depth from edges and intensity based stereo. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp. 631–636. Vancouver (1981)Google Scholar
  30. Ayache, N., Faverjon, B.: Efficient registration of stereo images by matching graph descriptions of edge segments. Int. J. Comput. Vis. 1(2), 107–131 (1987)CrossRefGoogle Scholar
  31. Fleet, D.J., Jepson, A.D., Jenkin, M.R.M.: Phase-based disparity measurement. CVGIP: Image Understanding 53(2), 198–210 (1991)CrossRefMATHGoogle Scholar
  32. Wang, J.J.: Image matching using the windowed fourier phase. Int. J. Comput. Vis. 11(3), 211–236 (1993), readGoogle Scholar
  33. Freeman, W.T., Adelson, E.H.: The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13(9), 891–906 (1991)CrossRefGoogle Scholar
  34. Trucco, E., Verri, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall (1998)Google Scholar
  35. Chang, M.L., Hauck, S.: Precis: A design-time precision analysis tool. In: 10th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, pp. 229–283 (2002) [Online]. Available: http://www.ee.washington.edu/faculty/hauck/publications/PrecisFCCM2002
  36. Darabiha, A.: Video-rate stereo vision on reconfigurable hardware. Master's thesis, Department of Electrical & Computer Engineering, University of Toronto (2003)Google Scholar
  37. Andraka, R.: A survey of CORDIC algorithms for FPGAs. In:Proceedings of the 1998 ACM/SIGDA Sixth International Symposium on Field Programmable Gate Arrays (FPGA '98), Monterey, CA, pp. 191–200 (1998) [Online]. Available: http://www.andraka.com/files/crdcsrvy.pdf
  38. Valls, J., Kuhlmann, M., Parhi, K.K.: Evaluation of CORDIC algorithms for FPGA design. J. VLSI Signal Process. 32, 207–222 (2002)CrossRefMATHGoogle Scholar
  39. Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. & Appl. 6(1), 35–49 (1993), INRIA Research Report 1369Google Scholar
  40. Gluckman, J., Nayar, S.K.: Rectified catadioptric stereo sensors. IEEE Trans. Pattern Anal. Machine Intell. 24(2), 224–236 (2002), stereo, mirros, rectifiedGoogle Scholar
  41. Fleet, D.J., Black, M.J., Jepson, A.D.: Motion feature detection using steerable flow fields. In: Proceedings of the 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 274–281. Santa Barbara (1998)Google Scholar
  42. Fleet, D.J., Jepson, A.D.: Computation of component image velocity from local phase information. Int. J. Comput. Vis. 5(1), 77–104 (1990)CrossRefGoogle Scholar
  43. Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust on-line appearance models for visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 415–422 Kauai, (2001) [Online]. Available: http://www.cs.toronto.edu/tem/cvpr01.pdf
  44. Carneiro, G., Jepson, A.D.: Local phase-based features. In: Proceedings of the 2002 European Conference on Computer Vision, vol. 1, pp. 282–296. Copenhagen, Denmark (2002)Google Scholar
  45. Masrani, D.K., MacLean, W.J.: Expanding disparity range in an FPGA stereo system while keeping resource utilization low. In: The First IEEE Workshop on Embedded Computer Vision, CVPR 2005 (2005)Google Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  • Ahmad Darabiha
    • 1
  • W. James MacLean
    • 1
  • Jonathan Rose
    • 1
  1. 1.Department of Electrical and Computer EngineeringUniversity of TorontoTorontoCanada

Personalised recommendations