Advertisement

Computational Visual Media

, Volume 5, Issue 1, pp 59–71 | Cite as

Real-time stereo matching on CUDA using Fourier descriptors and dynamic programming

  • Mohamed HallekEmail author
  • Fethi Smach
  • Mohamed Atri
Open Access
Research Article
  • 54 Downloads

Abstract

Computation of stereoscopic depth and disparity map extraction are dynamic research topics. A large variety of algorithms has been developed, among which we cite feature matching, moment extraction, and image representation using descriptors to determine a disparity map. This paper proposes a new method for stereo matching based on Fourier descriptors. The robustness of these descriptors under photometric and geometric transformations provides a better representation of a template or a local region in the image. In our work, we specifically use generalized Fourier descriptors to compute a robust cost function. Then, a box filter is applied for cost aggregation to enforce a smoothness constraint between neighboring pixels. Optimization and disparity calculation are done using dynamic programming, with a cost based on similarity between generalized Fourier descriptors using Euclidean distance. This local cost function is used to optimize correspondences. Our stereo matching algorithm is evaluated using the Middlebury stereo benchmark; our approach has been implemented on parallel high-performance graphics hardware using CUDA to accelerate our algorithm, giving a real-time implementation.

Keywords

generalized Fourier descriptors stereo matching dynamic programming CUDA 

References

  1. [1]
    Scharstein, D.; Szeliski, R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision Vol. 47, Nos. 1–3, 7–42, 2002.CrossRefzbMATHGoogle Scholar
  2. [2]
    Wang, M.; Zhang, X.-J.; Liang, J.-B.; Zhang, S.-H.; Martin, R. R. Comfort-driven disparity adjustment for stereoscopic video. Computational Visual Media Vol. 2, No. 1, 3–17, 2016.CrossRefGoogle Scholar
  3. [3]
    Barnes, C.; Zhang, F.-L. A survey of the state-of-the-art in patch-based synthesis. Computational Visual Media Vol. 3, No. 1, 3–20, 2017.CrossRefGoogle Scholar
  4. [4]
    Zhang, F.-L.; Wang, J.; Shechtman, E.; Zhou, Z.-Y.; Shi, J.-X.; Hu, S.-M. PlenoPatch: Patch-based plenoptic image manipulation. IEEE Transactions on Visualization and Computer Graphics Vol. 23, No. 5, 1561–1573, 2017.CrossRefGoogle Scholar
  5. [5]
    Brown, M. Z.; Burschka, D.; Hager, G. D. Advances in computational stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 25, No. 8, 993–1008, 2003.CrossRefGoogle Scholar
  6. [6]
    Hamzah, R. A.; Ibrahim, H. Literature survey on stereo vision disparity map algorithms. Journal of Sensors Vol. 2016, 1–23, 2016.CrossRefGoogle Scholar
  7. [7]
    Bhat, D. N.; Nayar, S. K. Ordinal measures for image correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 20, No. 4, 415–423, 1998.CrossRefGoogle Scholar
  8. [8]
    Lucas, B. D.; Kanade, T. An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, 674–679, 1981.Google Scholar
  9. [9]
    Banks, J.; Corke, P. Quantitative evaluation of matching methods and validity measures for stereo vision. The International Journal of Robotics Research Vol. 20, No. 7, 512–532, 2001.CrossRefGoogle Scholar
  10. [10]
    Li, Z. Y.; Song, L. M.; Xi, J. T.; Guo, Q. H.; Zhu, X. J.; Chen, M. L. A stereo matching algorithm based on SIFT feature and homography matrix. Optoelectronics Letters Vol. 11, No. 5, 390–394, 2015.CrossRefGoogle Scholar
  11. [11]
    Saygili, G.; van der Maaten, L.; Hendriks, E. A. Improving segment based stereo matching using SURF key points. In: Proceedings of the 19th IEEE International Conference on Image Processing, 2973–2976, 2012.Google Scholar
  12. [12]
    Gonidis, P.; Kotoulas, L.; Andreadis, I. A new hardware module for stereo matching using Zernike moments. In: Proceedings of the 3rd International Conference on Autonomic and Autonomous Systems, 33, 2007.Google Scholar
  13. [13]
    Altantawy, D. A.; Obbaya, M.; Kishk, S. A fast non-local based stereo matching algorithm using graph cuts. In: Proceedings of the 9th International Conference on Computer Engineering & Systems, 130–135, 2014.Google Scholar
  14. [14]
    Yang, Q.; Wang, L.; Yang, R.; Stewénius, H.; Nistér, D. Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 31, No. 3, 492–504, 2009.CrossRefGoogle Scholar
  15. [15]
    Veksler, O. Stereo correspondence by dynamic programming on a tree. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 384–390, 2005.Google Scholar
  16. [16]
    Congote, J.; Barandiaran, J.; Barandiaran, I.; Ruiz, O. Realtime dense stereo matching with dynamic programming in CUDA. In: Proceedings of the 19th Spanish Congress of Graphical Informatics, 231–234, 2009.zbMATHGoogle Scholar
  17. [17]
    Mattoccia, S.; Tombari, F.; di Stefano, L. Stereo vision enabling precise border localization within a scanline optimization framework. In: Computer Vision—ACCV 2007. Lecture Notes in Computer Science, Vol. 4844. Yagi, Y.; Kang, S. B.; Kweon, I. S.; Zha, H. Eds. Springer Berlin Heidelberg, 517–527, 2007.CrossRefGoogle Scholar
  18. [18]
    Kordelas, G. A.; Alexiadis, D. S.; Daras, P.; Izquierdo, E. Content-based guided image filtering, weighted semi-global optimization, and efficient disparity refinement for fast and accurate disparity estimation. IEEE Transactions on Multimedia Vol. 18, No. 2, 155–170, 2016.CrossRefGoogle Scholar
  19. [19]
    Sabihuddin, S.; Islam, J.; MacLean, W. J. Dynamic programming approach to high frame-rate stereo correspondence: A pipelined architecture implemented on a field programmable gate array. In: Proceedings of the Canadian Conference on Electrical and Computer Engineering, 1461–1466, 2008.Google Scholar
  20. [20]
    Kowalczuk, J.; Psota, E. T.; Perez, L. C. Realtime stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Transactions on Circuits and Systems for Video Technology Vol. 23, No. 1, 94–104, 2013.CrossRefGoogle Scholar
  21. [21]
    Zhang, D. S.; Lu, G. J. Shape-based image retrieval using generic Fourier descriptor. Signal Processing: Image Communication Vol. 17, 825–848, 2002.Google Scholar
  22. [22]
    Smach, F.; Lemaître, C.; Gauthier, J.-P.; Miteran, J.; Atri, M. Generalized Fourier descriptors with applications to objects recognition in SVM context. Journal of Mathematical Imaging and Vision Vol. 30, No. 1, 43–71, 2008.MathSciNetCrossRefGoogle Scholar
  23. [23]
    Smach, F.; Miteran, J.; Atri, M.; Dubois, J.; Abid, M.; Gauthier, J.-P. An FPGA-based accelerator for Fourier descriptors computing for color object recognition using SVM. Journal of Real-Time Image Processing Vol. 2, No. 4, 249–258, 2007.CrossRefGoogle Scholar
  24. [24]
    Yang, Q. Q.; Ji, P.; Li, D. X.; Yao, S. J.; Zhang, M. Fast stereo matching using adaptive guided filtering. Image and Vision Computing Vol. 32, No. 3, 202–211, 2014.CrossRefGoogle Scholar
  25. [25]
    Hosni, A.; Rhemann, C.; Bleyer, M.; Rother, C.; Gelautz, M. Fast cost-volume filtering for visual correspondence and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 2, 504–511, 2013.CrossRefGoogle Scholar
  26. [26]
    Ohta, Y.; Kanade, T. Stereo by intra- and inter-scanline search using dynamic programming. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 7, No. 2, 139–154, 1985.CrossRefGoogle Scholar
  27. [27]
    Psarakis, E. Z.; Evangelidis, G. D. An enhanced correlation-based method for stereo correspondence with subpixel accuracy. In: Proceedings of the 10th IEEE International Conference on Computer Vision, Vol. 1, 907–912, 2005.Google Scholar
  28. [28]
    Salmen, J.; Schlipsing, M.; Edelbrunner, J.; Hegemann, S.; Lüke, S. Real-time stereo vision: Making more out of dynamic programming. In: Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, Vol. 5702. Jiang, X.; Petkov, N. Eds. Springer Berlin Heidelberg, 1096–1103, 2009.CrossRefGoogle Scholar
  29. [29]
    Wang, L.; Yang, R. G.; Gong, M. L.; Liao, M. Real-time stereo using approximated joint bilateral filtering and dynamic programming. Journal of Real-Time Image Processing Vol. 9, No. 3, 447–461, 2014.CrossRefGoogle Scholar
  30. [30]
    Martins, J. A.; Rodrigues, J. M. F.; du Buf, H. Luminance, colour, viewpoint and border enhanced disparity energy model. PLoS One Vol. 10, No. 6, e0129908, 2015.CrossRefGoogle Scholar
  31. [31]
    Michael, M.; Salmen, J.; Stallkamp, J.; Schlipsing, M. Real-time stereo vision: Optimizing semi-global matching. In: Proceedings of the IEEE Intelligent Vehicles Symposium, 1197–1202, 2013.Google Scholar
  32. [32]
    LeGendre, C.; Batsos, K.; Mordohai, P. High-resolution stereo matching based on sampled photoconsistency computation. In: Proceedings of the British Machine Vision Conference, 2017.Google Scholar
  33. [33]
    Kitagawa, M.; Shimizu, I.; Sara, R. High accuracy local stereo matching using DoG scale map. In: Proceedings of the 15th IAPR International Conference on Machine Vision Applications, 258–261, 2017.Google Scholar
  34. [34]
    Zhang, K.; Li, J.; Li, Y.; Hu, W.; Sun, L.; Yang, S. Binary stereo matching. In: Proceedings of the 21st International Conference on Pattern Recognition, 356–359, 2012.Google Scholar
  35. [35]
    Shahbazi, M.; Sohn, G.; Théau, J.; Ménard, P. Revisiting intrinsic curves for efficient dense stereo matching. In: Proceedings of the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. III-3, 123–130, 2016.CrossRefGoogle Scholar
  36. [36]
    Hirschmuller, H. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 2, 328–341, 2008.CrossRefGoogle Scholar
  37. [37]
    Haythem, B.; Mohamed, H.; Marwa, C.; Fatma, S. A. Fast generalized Fourier descriptor for object recognition of image using CUDA. In: Proceedings of the World Symposium on Computer Applications and Research, 1–5, 2014.Google Scholar
  38. [38]
    Richardt, C.; Orr, D.; Davies, I.; Criminisi, A.; Dodgson, N. A. Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid. In: Computer Vision—ECCV 2010. Lecture Notes in Computer Science, Vol. 6313. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 510–523, 2010.CrossRefGoogle Scholar
  39. [39]
    Wang, L.; Liao, M.; Gong, M.; Yang, R.; Nister, D. High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: Proceedings of the 3rd International Symposium on 3D Data Processing, Visualization, and Transmission, 798–805, 2006.Google Scholar
  40. [40]
    Gong, M.; Yang, Y.-H. Near real-time reliable stereo matching using programmable graphics hardware. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 924–931, 2005.Google Scholar
  41. [41]
    Yu, W.; Chen, T.; Franchetti, F.; Hoe, J. C. High performance stereo vision designed for massively data parallel platforms. IEEE Transactions on Circuits and Systems for Video Technology Vol. 20, No. 11, 1509–1519, 2010.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit https://doi.org/creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from https://doi.org/www.springer.com/journal/41095. To submit a manuscript, please go to https://doi.org/www.editorialmanager.com/cvmj.

Authors and Affiliations

  1. 1.Faculty of Sciences of MonastirMonastirTunisia
  2. 2.Technologies et services de l’informationParisFrance

Personalised recommendations