Advertisement

FPGA implementation of an efficient similarity-based adaptive window algorithm for real-time stereo matching

  • Madaín Pérez-Patricio
  • Abiel Aguilar-González
Original Research Paper

Abstract

Stereo matching is one of the most widely used algorithms in real-time image processing applications such as positioning systems for mobile robots, three-dimensional building mapping and both recognition, detection and three-dimensional reconstruction of objects. In area-based algorithms, the similarity between one pixel of the left image and one pixel of the right image is measured using a correlation index computed on vicinities of these pixels called correlation windows. To preserve edges, small windows need to be used. On the other hand, for homogeneous areas, large windows are required. Due to only local information is used, matching between primitives is difficult. In this article, FPGA implementing of an efficient similarity-based adaptive window algorithm for dense disparity maps estimation in real-time is described. To evaluate the proposed algorithm's performance, the developed FPGA architecture was simulated via ModelSim-Altera 6.6c using different synthetic stereo pairs and different sizes for correlation window. In addition, the FPGA architecture was implemented in an FPGA Cyclone IIEP2C35F672C6 embedded in an Altera development board DE2. The disparity maps are computed at a rate of 76 frames per second for stereo pairs of 1280 \(\times\) 1024 pixel resolution and a maximum expected disparity equal to 15. The developed FPGA architecture offers better results with respect to most of the real-time area-based stereo matching algorithms reported in the literature, allows increasing the processing speed up to 93,061,120 pixels per second and enables it to be implemented in the majority of the medium gamma FPGA devices.

Keywords

Adaptive window Stereo matching Disparity map FPGA. 

References

  1. 1.
    Aguilar-González, A., Pérez-Patricio, M., Arias-Estrada, M., Camas-Anzueto, J.L., Hernández-deLeón, H.R., Sánchez-Alegría, A.: An FPGA correlation-edge distance approach for disparity map. In: Proceedings of IEEE International Conference on Electronics, Communications and Computers (CONIELECOMP15), Chulula, pp. 21–28 (2015)Google Scholar
  2. 2.
    Alba, A., Arce-Santana, E., Aguilar-Ponce, R.M., Campos-Delgado, D.U.: Phase-correlation guided area matching for realtime vision and video encoding. J Real-Time Image Proc. 9, 621–633 (2014)CrossRefGoogle Scholar
  3. 3.
    Asadi, E., Bottasso, C.L.: Delayed fusion for real-time vision-aided inertial navigation. J Real-Time Image Proc. (2013). doi: 10.1007/s11554-013-0376-8
  4. 4.
    Bartczak, B., Koeser, K., Woelk, F., Koch, R.: Extraction of 3d freeform surfaces as visual landmarks for real-time tracking. J Real-Time Image Proc. 2, 81–101 (2007)CrossRefGoogle Scholar
  5. 5.
    Darabiha, A., MacLean, W., Rose, J.: Reconfigurable hardware implementation of a phase-correlation stereo algorithm. J. Mach. Vis. Appl. 17, 116–132 (2006)CrossRefGoogle Scholar
  6. 6.
    Diaz, J., Ros, E., Pelayo, F., Ortigosa, E., Mota, S.: FPGA based real-time opticalflow system. IEEE Trans. Circuits Syst. Video Technol. 16, 274–279 (2006)CrossRefGoogle Scholar
  7. 7.
    Feiyang, C., Hong, Z., Ding, Y., Mingui, S.: Stereo matching by using the global edge constraint. Neurocomputing 131(11), 217–226 (2013)Google Scholar
  8. 8.
    Fusiello, A., Roberto, V., Trucco, E.: Symmetric stereo with multiple windowing. Int. J. Pattern Recognit. Artif. Intel. 8(14), 1053–1066 (2000)Google Scholar
  9. 9.
    Georgoulas, C., Andreadis, I.: Fpga based disparity map computation with vergence control. Microprocess. Microsyst. 34, 259–273 (2010)CrossRefGoogle Scholar
  10. 10.
    Georgoulas, C., Andreadis, I.: A real-time fuzzy hardware structure for disparity map computation. J Real-Time Image Proc. 6, 257–273 (2011)CrossRefGoogle Scholar
  11. 11.
    Georgoulas, C., Kotoulas, L., Sirakoulis, G.C., Andreadis, I., Gasteratos, A.: Real-time disparity map computation module. Microprocess. Microsyst. 32, 159–170 (2008)CrossRefGoogle Scholar
  12. 12.
    Gong, M., Yang, Y.H.: Near real-time reliable stereo matching using programmable graphics hardware. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1, 924–931 (2005)Google Scholar
  13. 13.
    Granados, S., Barranco, F., Mota, S., Díaz, J., Ros, E.: On-chip semidense representation map for dense visual features driven by attention processes. J Real-Time Image Proc. 9, 171–185 (2014)CrossRefGoogle Scholar
  14. 14.
    Guerra-Filho, G.: An optimal timespace algorithm for dense stereo matching. J Real-Time Image Proc. 7, 69–86 (2012)CrossRefGoogle Scholar
  15. 15.
    Hirschmuller, H.: Improvements in real-time correlation-based stereo vision. In: Proceedings of IEEE workshop on Stereoand Multi-Baseline Vision, Kauai, pp. 141–148 (2001)Google Scholar
  16. 16.
    Jin, M., Maruyama, T.: A fast and high quality stereo matching algorithm on fpga. Int. Conf. Field Program. Log. Appl. (FPL) 22(31), 507–5010 (2012)Google Scholar
  17. 17.
    Jin, S., Cho, J., Pham, X.D., Lee, K.M., Park, S.K., Kim, M., Jeon, J.W.: Fpga design and implementation of a real-time stereo vision system. IEEE Trans. Circuits Syst. Video Technol. 20, 15–26 (2010)CrossRefGoogle Scholar
  18. 18.
    Isakova, N., Basak, S., Sonmez, A.C.: FPGA design and implementation of a real-time stereo vision system. Int. Symp. Innov. Intell. Syst. Appl. (INISTA) 125, 15–26 (2012)Google Scholar
  19. 19.
    Jung, H.Y., Park, H., Park, I.K., Lee, K.M., Lee, S.U.: Stereo reconstruction using high-order likelihoods. Comput. Vis. Image Underst. 125(21), 223–236 (2014)CrossRefGoogle Scholar
  20. 20.
    Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: theory and experiment. In: Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, vol. 16, pp. 920–932 (1991)Google Scholar
  21. 21.
    Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for videorate dense depth mapping and its new applications. In: IEEE Computer Society Conference on Vision and Pattern Recognition, San Francisco, vol. 15, pp. 196–202 (1996)Google Scholar
  22. 22.
    Lee, S., Yi, J., Kim, J.: Real-time stereo vision on a reconfigurable system. Lect. Notes Comput. Sci. Embed. Comput. Syst. 3553, 299–307 (2005)CrossRefGoogle Scholar
  23. 23.
    Lotti, J., Giraudon, G.: Correlation algorithm with adaptive window for aerial image in stereo vision. In: Proceedings of the Image and Signal Processing for Remote Sensing, EUROPTO’94, Rome, vol. 1, pp. 701–703 (1994)Google Scholar
  24. 24.
    Madeo, S., Pelliccia, R., Salvadori, C., del Rincon, J.M., Nebel, J.C.: An optimized stereo vision implementation for embedded systems: application to rgb and infra-red images. J Real-Time Image Proc. (2014). doi: 10.1007/s11554-014-0461-7
  25. 25.
    Martin, H., Christian, Z., Vincze, M.: A fast stereo matching algorithm suitable for embedded real-time systems. Comput. Vis. Image Underst. 114, 1180–1202 (2010)Google Scholar
  26. 26.
    Masrani, D., MacLean, W.: A real-time large disparity range stereo-system using fpgas. In: IEEE International Conference on Computer Vision Systems, pp. 13–19 (2006)Google Scholar
  27. 27.
    Min, D., Lu, J., Do, M.N.: Joint histogram-based cost aggregation for stereo matching. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2539–2545 (2013)CrossRefGoogle Scholar
  28. 28.
    Murphy, C., Lindquist, D., Cecil, A.M.R.T., Leavitt, S., Chang, M.L.: Low-cost stereo vision on an FPGA. Int. Symp. Field-Program. Cust. Comput. Mach. 15(21), 333–334 (2007)Google Scholar
  29. 29.
    Niitsuma, H., Maruyama, T.: High-speed computation of the optical flow. Lect. Notes Comput. Sci. Image Anal. Process. 3617, 287–295 (2005)CrossRefGoogle Scholar
  30. 30.
    Roh, C., Ha, T., Kim, S., Kim, J.: Symmetrical dense disparity estimation: algorithms and fpgas implementation. In: IEEE International Symposium on Consumer Electronics, pp. 452–456 (2004)Google Scholar
  31. 31.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)zbMATHCrossRefGoogle Scholar
  32. 32.
    Stefania Perri, P.C., Cocorullo, G.: Adaptive census transform: a novel hardware-oriented stereovision algorithm. Comput. Vis. Image Underst. 117, 29–41 (2013)CrossRefGoogle Scholar
  33. 33.
    Stefano, L.D., Marchionni, M., Mattoccia, S.: A fast area-based stereo matching algorithm. Image Vis. Comput. 22(22), 983–1005 (2004)CrossRefGoogle Scholar
  34. 34.
    Ttofis, C., Hadjitheophanous, S., Georghiades, A.S., Theocharides, T.: Edge-directed hardware architecture for real-time disparity map computation. IEEE Trans. Comput. 62, 690–704 (2013)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Veksler, O.: Stereo matching by compact windows via minimum ratio cycle. IEEE Trans. Pattern Anal. Mach. Intell. 24(12), 1654 (2002)CrossRefGoogle Scholar
  36. 36.
    Woodfill, J., Herzen, B.V.: Real time stereo vision on the parts reconfigurable computer. IEEE Symp. Field-Program. Cust. Comput. Mach. 5, 201–210 (1997a)Google Scholar
  37. 37.
    Yang, Q.: A non-local cost aggregation method for stereo matching. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’12), Providence, vol. 1, pp. 1–8 (2012)Google Scholar
  38. 38.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the Third European Conference on Computer Vision (ECCV’94), Stockholm, pp. 151–158 (1994)Google Scholar
  39. 39.
    Zhang, Y., Kambhamettu, C.: Stereo matching with segmentation-based cooperation. In: Proceedings of the Seventh European Conference on Computer Vision (ECCV’94), Copenhagen, pp. 556–571 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Division of Graduate Studies and ResearchInstituto Tecnológico de Tuxtla Gutiérrez (ITTG)Tuxtla GutiérrezMexico

Personalised recommendations