Journal of Electrical Engineering & Technology

, Volume 14, Issue 6, pp 2619–2624 | Cite as

Objective Evaluation of Image Decomposition Algorithms for Depth Map Upsampling

  • Chanho Jung
  • Seungkwon Shin
  • Jiwon Lee
  • Wonjun KimEmail author
Original Article


Depth map upsampling plays an essential role in various three-dimensional (3D) image and video applications such as multi-view rendering and 3D scene modeling. Most of existing depth map upsampling methods have suggested to use a color image as a guide. Recently, in our previous work, the use of a structure component obtained from image decomposition rather than the color image has proven to be very powerful in the task of depth map upsampling. In this paper, to determine how image decomposition algorithms can be best used for depth map upsampling, we conducted a comprehensive comparative study. More precisely, the purpose of this study is to present an “objective” evaluation of recent promising image decomposition methods in terms of the performance of depth map upsampling. This is, to the best of our knowledge, the first experimental comparative demonstration on the performance of depth map upsampling enhanced with several different image decomposition models. We investigated eight different promising recent image decomposition approaches under the same experimental setup. From our quantitative comparison, we can obtain novel and valuable insights into the image decomposition-based depth map upsampling: (1) the best image decomposition solution for depth map upsampling depends on the scaling factor of upsampling, (2) the guided filter-based image decomposition method gives rise to the best performance for lower scaling factors, whereas the tree filter-based image decomposition method leads to the best upsampling performance for higher scaling factors, and (3) the performance of image decomposition-based depth upsampling is not sensitive to image features. We believe that this comprehensive comparative study serves as a reference point and guide for developers and practitioners in choosing an appropriate image decomposition technique adopted for building depth map upsampling systems.


Depth map upsampling Image decomposition Comparative study 



This research was supported by a grant from R&D Program of the Korea Railroad Research Institute, Republic of Korea. This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (no. 2019-0-00524, Development of precise content identification technology based on relationship analysis for maritime vessel/structure).


  1. 1.
    Muller K, Merkle P, Wiegand T (2011) 3-D video representation using depth maps. Proc IEEE 99(4):643–656CrossRefGoogle Scholar
  2. 2.
    Zhao J, Zhao X, Zhang W, Wang F, Zhang Q (2016) An efficient depth modeling mode decision algorithm for 3D-HEVC depth map coding. Optik Int J Light Electron Opt 127(24):12048–12055CrossRefGoogle Scholar
  3. 3.
    Wu X, Mao X, Chen L, Xue Y, Rovetta A (2015) Depth image-based hand tracking in complex scene. Optik Int J Light Electron Opt 126(20):2757–2763CrossRefGoogle Scholar
  4. 4.
    Kopf J, Cohen MF, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. ACM Trans Graph 26(3):96CrossRefGoogle Scholar
  5. 5.
    Min D, Lu J, Do MN (2012) Depth video enhancement based on weighted mode filtering. IEEE Trans Image Process 21(3):1176–1190MathSciNetCrossRefGoogle Scholar
  6. 6.
    Jung SW, Choi O (2014) Learning-based filter selection scheme for depth image super resolution. IEEE Trans Circuits Syst Video Technol 24(10):1641–1650CrossRefGoogle Scholar
  7. 7.
    Jung C, Joo S, Ji SM (2015) Depth map upsampling with image decomposition. Electron Lett 51(22):1782–1784CrossRefGoogle Scholar
  8. 8.
    Subr K, Soler C, Durand F (2009) Edge-preserving multiscale image decomposition based on local extrema. ACM Trans Graph 28(5):147CrossRefGoogle Scholar
  9. 9.
    Buades A, Le TM, Morel JM, Vese LA (2010) Fast cartoon + texture image filters. IEEE Trans Image Process 19(8):1978–1986MathSciNetCrossRefGoogle Scholar
  10. 10.
    Xu L, Lu C, Xu Y, Jia J (2011) Image smoothing via L0 gradient minimization. ACM Trans Graph 30(6):174Google Scholar
  11. 11.
    Xu L, Yan Q, Xia Y, Jia J (2012) Structure extraction from texture via relative total variation. ACM Trans Graph 31(6):139Google Scholar
  12. 12.
    He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397–1409CrossRefGoogle Scholar
  13. 13.
    Karacan L, Erdem E, Erdem A (2013) Structure-preserving image smoothing via region covariances. ACM Trans Graph 32(6):176CrossRefGoogle Scholar
  14. 14.
    Bao L, Song Y, Yang Q, Yuan H, Wang G (2014) Tree filtering: efficient structure-preserving smoothing with a minimum spanning tree. IEEE Trans Image Process 23(2):555–569MathSciNetCrossRefGoogle Scholar
  15. 15.
    Zhang Q, Shen X, Xu L, Jia J (2014) Rolling guidance filter. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Proceedings of European conference on computer vision—ECCV 2014. Lecture notes in computer science, vol 8691. Springer, Cham, pp 815–830CrossRefGoogle Scholar
  16. 16.
    Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1–3):7–42CrossRefGoogle Scholar

Copyright information

© The Korean Institute of Electrical Engineers 2019

Authors and Affiliations

  • Chanho Jung
    • 1
  • Seungkwon Shin
    • 2
  • Jiwon Lee
    • 3
  • Wonjun Kim
    • 4
    Email author
  1. 1.Department of Electrical EngineeringHanbat National UniversityDaejeonKorea
  2. 2.Korea Railroad Research InstituteUiwang-siKorea
  3. 3.Electronics and Telecommunications Research InstituteDaejeonKorea
  4. 4.Department of Electronics EngineeringKonkuk UniversitySeoulKorea

Personalised recommendations