Skip to main content
SpringerLink
Log in

A novel fast search method to find disparity vectors in multiview video coding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In multiview/3D video, the amount of data to be transmitted to the decoder increases proportionally with the number of cameras. One way to efficiently compress such video is to use Multiview Video Coding (MVC) which simultaneously reduces temporal and spatial redundancy within the same view and among multiple views. But, existing disparity estimation methods used to extract the redundancy among views are typically too computationally complex. In this paper, we propose a method for fast disparity estimation in multiview/3D video. First, for each block of the predicted frame, the view-dependent geometry and the depth information are used to find the corresponding block in the reference views. Then, a fast and greedy search method is proposed to search the surrounding areas to find the most similar block. Simulation results show that our proposed method achieves better bitrate and quality with much lower computational complexity compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Afonso V, Susin A, Perleberg M, Conceição R, Corrêa G, Agostini L, Zatt B, Porto M (2018) Hardware-friendly unidirectional disparity-search algorithm for 3D-HEVC. IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy, pp. 1–5. https://doi.org/10.1109/ISCAS.2018.8351350

  2. Afonso V, Conceição RA, Saldanha MRF, Braatz LA, Perleberg MR, Corrêa GR, Porto MS, Agostini LV, Zatt B, Susin AA (2019) Energy-aware motion and disparity estimation system for 3D-HEVC with run-time adaptive memory hierarchy. IEEE Trans Circuits Syst Video Technol 29(6):1878–1892. https://doi.org/10.1109/TCSVT.2018.2847633

    Article  Google Scholar 

  3. Aksehir Y, Erdayandi K, Ozcan TZ, Hamzaoglu I (2013) A low energy adaptive motion estimation hardware for H.264 multiview video coding. J Real-Time Image Proc 15:3–12. https://doi.org/10.1007/s11554-013-0383-9

    Article  Google Scholar 

  4. Atienza R (2018) Fast disparity estimation using dense networks. 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, pp. 3207–3212. https://doi.org/10.1109/ICRA.2018.8463172.

  5. Bjøntegaard G (2001) Calculation of average PSNR differences between R-D curves. Proceedings of the ITU–Telecommunications Standardization Sector STUDY GROUP 16 Video Coding Experts Group (VCEG), 13th Meeting, Austin, TX, USA, pp. 2–4

  6. Boonthep N, Chamnongthai K (2019) A method of motion-estimation-based H.264 video coding using optimal search-range. Wirel Pers Commun. https://doi.org/10.1007/s11277-019-06766-4

  7. Chen Y, Zhao X, Zhang L, Kang J-W (2016) Multiview and 3D Video Compression Using Neighboring Block Based Disparity Vectors. IEEE Trans Multimedia 18(4):576–589. https://doi.org/10.1109/TMM.2016.2525010

  8. Duggal S, Wang S, Ma W-C, Hu R, Urtasun R (2019) DeepPruner: learning efficient stereo matching via differentiable PatchMatch. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Korea (South), pp. 4383–4392. https://doi.org/10.1109/ICCV.2019.00448

  9. Fan D-P, Lin Z, Zhang Z, Zhu M, Cheng M-M (2020) Rethinking RGB-D salient object detection: models, data sets, and large-scale benchmarks. IEEE Trans Neural Networks Learning Syst:1–1. https://doi.org/10.1109/TNNLS.2020.2996406

  10. Gonzales C, Yeo H, Kuo CJ (1999) Requirements for motion-estimation search range in MPEG-2 coded video. IBM J Res Dev 43(4):453–470. https://doi.org/10.1147/rd.434.0453

  11. Han CH, Lee SW, Choi H (2016) Residual DPCM in HEVC transform skip mode for screen content coding. IEEE Trans Smart Process Comput 5:323–326. https://doi.org/10.5573/IEIESPC.2016.5.5.323

    Article  Google Scholar 

  12. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press. https://doi.org/10.1017/CBO9780511811685

  13. Researcher tools: code & datasets http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload. Accessed 3 May 2020

  14. ITU-T and ISO/IEC JTC, ERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS, Infrastructure of audiovisual services – Coding of moving video (2013) ITU-T Recommendation H.265 and ISO/IEC 23008–2 (HEVC)

  15. Jiang C, Nooshabadi S (2016) A scalable massively parallel motion and disparity estimation scheme for multiview video coding. IEEE Trans Circuits SystVideo Technol 26:346–359. https://doi.org/10.1109/TCSVT.2015.2402853

    Article  Google Scholar 

  16. Jiang C, Nooshabadi S (2016) Decision zone-based parallel fast motion and disparity estimation scheme for multiview coding. Data compression conference (DCC). Snowbird, UT, USA, pp. 609–609. https://doi.org/10.1109/DCC.2016.20

  17. Jin N, Li F, Lai X (2011) Disparity estimation with disparity field correlation and epipolar geometry constraint for multiview video coding. 2nd International Conference on Intelligent Control and Information Processing. Harbin, China, pp 260–263. https://doi.org/10.1109/ICICIP.2011.6008244

  18. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11 (2011) HEVC Test Model (HM) 5.0 Reference Software. https://hevc.hhi.fraunhofer.de/HM-doc, Accessed 3 May 2020

  19. Kaup A, Fecker U (2006) Analysis of multi-reference block matching for multi-view video coding. 7th Workshop Digital Broadcasting, Erlangen, Germany, pp. 33–39

  20. Lin C-L, Tsai P-H (2018) Low-complexity encoding scheme for multiview video content. Microsyst Technol 24:4057–4066. https://doi.org/10.1007/s00542-017-3620-5

    Article  Google Scholar 

  21. Micallef BW, Debono CJ, Farrugia RA (2010) Exploiting depth information for fast multi-view video coding. Picture coding symposium (PCS), Nagoya, Japan, pp. 1–4. https://doi.org/10.1109/3DTV.2011.5877170

  22. Micallef BW, Debono CJ, Farrugia RA (2014) Reducing 3D video coding complexity through more efficient disparity estimation. IEEE Trans Consum Electron 60:74–82. https://doi.org/10.1109/TCE.2014.6780928

    Article  Google Scholar 

  23. Nie G-Y, Cheng M-M, Liu Y, Liang Z, Fan D-P, Liu Y, Wang Y (2019) Multi-level context ultra-aggregation for stereo matching. IEEE/CVF conference on computer vision and pattern recognition (CVPR), Long Beach, CA, USA, pp. 3278–3286. https://doi.org/10.1109/CVPR.2019.00340

  24. Ohm J-R (1999) Stereo/multiview video encoding using the mpeg family of standards.Proc. SPIE 3639, Stereoscopic Displays and Virtual Reality Systems VI, San Jose, CA, United States, pp. 242–253. https://doi.org/10.1117/12.349385

  25. Qiao Z, Li X, Zhao D, Liu Y, Gao W (2011) Fast disparity estimation utilizing depth information for multiview video coding. IEEE Int Symposium Circuits Syst. Rio de Janeiro, Brazil, pp. 2805–2808. https://doi.org/10.1109/ISCAS.2011.5938188

  26. Shen L, Liu Z, Yan T, Zhang Z, An P (2010) View-adaptive motion estimation and disparity estimation for low complexity multiview video coding. IEEE Trans Circuits Syst Video Technol 20:925–930. https://doi.org/10.1109/TCSVT.2010.2045910

    Article  Google Scholar 

  27. Song Y, Jia K (2013) Multiview video coding algorithm based on HBP prediction structure. Ninth international conference on intelligent information hiding and Multimedia signal processing. Beijing, China, pp. 173–176. https://doi.org/10.1109/IIH-MSP.2013.52

  28. Nagoya University Multi-view Sequences Download List http://www.fujii.nuee.nagoya-u.ac.jp/multiview-data. Accessed 3 May 2020

  29. Wang Y, Yang J, Mo Y, Xiao C, An W (2018) Disparity estimation for camera arrays using reliability guided disparity propagation. IEEE Access 6:21840–21849. https://doi.org/10.1109/ACCESS.2018.2827085

    Article  Google Scholar 

  30. Zakeri FS, Bätz M, Jaschke T, Keinert J, Chuchvara A (2019) Benchmarking of several disparity estimation algorithms for light field processing. In: Proceeding of SPIE 11172, fourteenth international conference on quality control by artificial vision, p 111721C. https://doi.org/10.1117/12.2521747

    Chapter  Google Scholar 

  31. Zhang C, Li Z, Cheng Y, Cai R, Chao H, Rui Y (2015) MeshStereo: a global stereo model with mesh alignment regularization for view interpolation. International conference on computer vision (ICCV), Santiago, Chile, pp. 2057–2065. https://doi.org/10.1109/ICCV.2015.238

  32. Zhaoqing P, Zhang Y, Kwong S (2015) Efficient motion and disparity estimation optimization for low complexity multiview video coding. IEEE Trans Broadcast 61:166–176. https://doi.org/10.1109/TBC.2015.2419824

    Article  Google Scholar 

  33. Zhu W, Tian X, Zhou F, Chen Y (2010) Fast disparity estimation using spatio-temporal correlation of disparity field for multiview video coding. IEEE Trans Consum Electron 56:957–964. https://doi.org/10.1109/TCE.2010.5506026

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. K. N. Toosi University of Technology, Tehran, Iran

    Ghane Zandi & Hoda Roodaki

  2. Distributed and Collaborative Virtual Environments Research Laboratory (DISCOVER Lab), University of Ottawa, Ottawa, Canada

    Shervin Shirmohammadi

Authors
  1. Ghane Zandi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hoda Roodaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shervin Shirmohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoda Roodaki.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zandi, G., Roodaki, H. & Shirmohammadi, S. A novel fast search method to find disparity vectors in multiview video coding. Multimed Tools Appl 80, 10821–10837 (2021). https://doi.org/10.1007/s11042-020-10260-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10260-6

Keywords

Search

Navigation