Skip to main content
SpringerLink
Log in

High-speed dense matching algorithm for high-resolution aerial image based on CPU-FPGA

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The paper proposes a new initial disparity estimation algorithm based on sparse local feature matching and one-dimensional dynamic programming. The new algorithm solves the problems of long calculation time, high memory consumption and unknown search range of disparity in the classical pyramid SGM dense matching algorithm. Based on the initial disparity, the final accurate disparity can be got by using SGM algorithm within fixed little search range. The new algorithm is more accurate than the traditional SGM algorithm (by testing on a public data set, the accuracy of new algorithm is 1.57% higher than traditional SGM algorithm), it does not need to artificially estimate the search range of disparity, and it is suitable for implementation on FPGA. Based on the new algorithm, the CPU-FPGA collaborative high-speed dense matching for high-resolution aerial images is realized. The average speed of dense matching on CPU-FPGA is 31.81times faster than the original SGM algorithm and 6.46 times faster than pyramid SGM algorithm. The speed of accurate matching by FPGA is 4.75 times faster than running on GPU.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  2. Gehrke, S., Morin, K., Downey, M., Boehrer, N., Fuchs, T.: Semi-global matching: an alternative to LIDAR for DSM generation. In: 2010 ISPRS Congress, Calgary, AB, Canada, 15–18 June 2010

  3. Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Article  Google Scholar 

  4. Hannah M.J.: Computer matching of areas in stereo images. Ph.D. thesis, Stanford University (1974)

  5. Zhang, Y.J., Zhang, Z.X.: Generalized photogrammetry of spaceborne, airborne and terrestrial multi-source remote sensing datasets. Acta Geod. Cartogr. Sin. 50(a), 1–11 (2021)

    Google Scholar 

  6. Zhang, K., Lu, J., Lafruit, G.: Scalable stereo matching with locally adaptive polygon approximation. In: 15th IEEE International Conference on Image Processing, 2008. ICIP, pp. 313–316 (2008)

  7. Huang, C., Zhao, H.Z.: Semi-global stereo matching with adaptive window based on grayscale value. J. Image Graph. 24(08), 1381–1390 (2019)

    Google Scholar 

  8. Lukezic, A., Vojir, T., Zajc, L.C., et al. : Discriminative correlation filter with channel and spatial reliability. In: IEEE Conference on Computer Vision & Pattern Recognition, pp. 4847–4856 (2017)

  9. He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013)

    Article  Google Scholar 

  10. Rhemann, C., Hosni, A., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. In: IEEE Conference on Computer Vision & Pattern Recognition (2011)

  11. Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)

    Article  MATH  Google Scholar 

  12. Boykov, Y., Veksler, O., Zabin, R.: Efficient approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  13. Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 807–814 (2005)

  14. Hirschmüller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1582–1599 (2009)

    Article  Google Scholar 

  15. Rothermel, M.: Development of a SGM-Based Multi-view Reconstruction Framework for Aerial Imagery. University of Stuttgart, Stuttgart (2017)

    Google Scholar 

  16. Rothermei, M., Wenzei, K., Fritsch, D., et al.: SURE: photogrammetric surface reconstruction from imagery. In: Proceedings of LC3D Workshop. Berlin (2012)

  17. Liu, Y., Pan, J., Su, Z., et al.: Robust dense correspondence using deep convolutional features. Vis. Comput. 36(4), 827–841 (2020)

    Article  Google Scholar 

  18. Li, Y., Luo, F., Li, W., et al.: Self-supervised monocular depth estimation based on image texture detail enhancement. Vis. Comput. 37(2), 2567–2580 (2021)

    Article  MathSciNet  Google Scholar 

  19. Shi, J., Sun, Y., Bai, S., et al.: A self-supervised method of single-image depth estimation by feeding forward information using max-pooling layers. Vis. Comput. 37(4), 815–829 (2020)

    Article  Google Scholar 

  20. Lee, C.A., Gasster, S.D., Plaza, A.: Recent developments in high performance computing for remote sensing: a review. IEEE. J. Sel. Top. Appl. Earth. Observ. Remote. Sens. 4(3), 508–527 (2011)

    Article  Google Scholar 

  21. Nurvitadhi, E., Sheffield. D., Sim. J.: Accelerating binarized neural networks: comparison of FPGA, CPU, GPU, and ASIC. In: International Conference on Field-Programmable Technology, pp. 77–84 (2017)

  22. Puglia, L., Vigliar, M., Raiconi, G.: Real-time low-power FPGA architecture for stereo vision. IEEE. Trans. Circuits Syst. II Exp. Briefs. 64(1), 1307–1311 (2017)

    Google Scholar 

  23. Yao, P., Feng, J.: Ensemble learning with advanced fast image filtering features for semi-global matching. Mach. Vis. Appl. 32(4), 1–15 (2021)

    Article  Google Scholar 

  24. Passalis, N., Tefas, A.: Deep supervised hashing using quadratic spherical mutual information for efficient image retrieval. Signal Process. Image Commun. 93(3), 116–146 (2021)

    Google Scholar 

  25. Gehrig, S.K., Eberli, F., Meyer, T.: A real-time low-power stereo vision engine using semi-global matching. In: Computer Vision Systems. Springer, pp. 134–143 (2009)

  26. Banz, C., Hesselbarth, S., Flatt, H., Blume, H., Pirsch, P.: Real-time stereo vision system using semi-global matching disparity estimation: architecture and FPGA-implementation. In: IEEE International Conference on Embedded Computer Systems (SAMOS), pp. 93–101 (2010)

  27. Hermann, S., Klette, R.: Evaluation of a new coarse-to-fine strategy for fast semi-global stereo matching. In: Advances in Image and Video Technology-pacific Rim Symposium, pp. 395–406 (2012)

  28. Qamar, A., Muslim, F.B., et al.: High-level synthesis for semi-global matching: is the juice worth the squeeze? IEEE Access 6(5), 8419–8432 (2017)

    Article  Google Scholar 

  29. Cambuim, L.F., Barbosa, J.P., Barros, E.N.: Hardware module for low-resource and real-time stereo vision engine using semi-global matching approach. In: 2017 30th Symposium on Integrated Circuits and Systems Design (SBCCI), Fortaleza, pp. 53–58 (2017)

  30. Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 650–656 (2006)

    Article  Google Scholar 

  31. Ttofis, C., Kyrkou, C., Theocharides, T.: A hardware-efficient architecture for accurate real-time disparity map estimation. ACM Trans. Embed. Comput. Syst. 14(2), 1–26 (2015)

    Article  MATH  Google Scholar 

  32. Ttofis, C., Kyrkou, C., Theocharides, T.: A Low-cost real-time embedded stereo vision system for accurate disparity estimation based on guided image filtering. IEEE Trans. Comput. 65(9), 1–11 (2015)

    MathSciNet  MATH  Google Scholar 

  33. Scharstein, D., Hirschmüller, H., Kitajima, Y., et.al.: High-resolution stereo datasets with subpixel-accurate ground truth. In: German Conference on Pattern Recognition (GCPR 2014), Münster, Germany, pp. 31–42, September 2014

  34. Shahbazi, M., Sohn, G., Theau, J.: High-density stereo image matching using intrinsic curves. ISPRS J. Photogramm. Remote. Sens. 146(12), 373–388 (2018)

    Article  Google Scholar 

  35. Spangenberg, R., Langner, T., Adfeldt, S., Rojas, R.: Large scale semi-global matching on the CPU. In: Proceedings of IEEE Symposium on Intelligent Vehicles Proceedings, pp. 195–201. IEEE, Dearborn (2014)

  36. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: IEEE Proceedings of International Conference on Computer Vision, pp. 508–515. IEEE, Vancouver (2001)

  37. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70, 41–54 (2006)

    Article  Google Scholar 

Download references

Funding

This study was funded by Natural Science Foundation of Guangxi Province (2020GXNSFAA159091), Guangxi Key laboratory for optoelectronics information Processing (GD18108), Innovation Project of Guangxi Graduate Education (JGY2022131) and Graduate education innovation program of Guilin University of Electronic technology (2022YCXS157).

Author information

Authors and Affiliations

  1. School of Optoelectronic Engineering, Guilin University of Electronic Technology, Guilin, 541004, China

    Zhiyong Peng, Lei Wu & Bo Xiao

  2. Guangxi Key Laboratory for Optoelectronics Information Processing, Guilin, 541004, China

    Zhiyong Peng

Authors
  1. Zhiyong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiyong Peng.

Ethics declarations

Conflict of interest

We declare that we have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, Z., Wu, L. & Xiao, B. High-speed dense matching algorithm for high-resolution aerial image based on CPU-FPGA. Vis Comput 39, 5263–5278 (2023). https://doi.org/10.1007/s00371-022-02658-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02658-0

Keywords

Search

Navigation