Abstract
Recovery of correct depth values in missing regions of depth maps captured using consumer depth cameras is a challenging problem in the field of computer vision. Applications like robotic vision, automatic navigation and imperfection of captured data with existing sensors, hole-filling in depth maps is a significant research area. Imprecision of estimated depth values of missing regions increases especially at depth discontinuities. To overcome this remarkable complication we model the estimated depth map as a non-local extension of discontinuity-adaptive MRF. The non-convex energy function can be optimized by exploiting the graduated non-convexity algorithm. Experiments with both synthetic and real-world data demonstrate the superiority of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
M. Stommel, M. Beetz, W. Xu, Inpainting of missing values in the Kinect sensor’s depth maps based on background estimates. IEEE Sensor J. 14(4), 1107–1116 (2013)
J. Yang, X. Ye, K. Li, C. Hou, Y. Wang, Color-guided depth recovery from RGB-D data using an adaptive autoregressive model. IEEE Trans. Image Process. 23(8), 3443–3458 (2014)
K. Matsuo, Y. Aoki, Depth image enhancement using local tangent plane approximations, in Computer Vision and Pattern Recognition (2015), pp. 3574–3583
X. Liu, D. Zhai, R. Chen, X. Ji, D. Zhao, W. Gao, Depth restoration from RGB-D data via joint adaptive regularization and thresholding on manifolds. IEEE Trans. Image Process. 28(3), 1068–1079 (2018)
F. Ma, G.V. Cavalheiro, S. Karaman, Self-supervised sparseto-dense: self-supervised depth completion from lidar and monocular camera, in International Conference on Robotics and Automation (IEEE, 2019), pp. 3288–3295
Z. Huang, J. Fan, S. Cheng, S. Yi, X. Wang, H. Li, Hmsnet: Hierarchical multi-scale sparsity-invariant network for sparse depth completion. IEEE Trans. Image Process. 29, 3429–3441 (2019)
Y. Zhang, T. Funkhouser, Deep depth completion of a single RGB-D image, in Computer Vision and Pattern Recognition (2018), pp. 175–185
K. Ramnath, A.N. Rajagopalan, Discontinuity-adaptive shape from focus using a non-convex prior, in Joint Pattern Recognition Symposium (Springer, Berlin, 2009), pp. 181–190
D. Scharstein, C. Pal, Learning conditional random fields for stereo, in Computer Vision and Pattern Recognition, IEEE 1–8 (2007)
S. Song, S.P. Lichtenberg, J. Xiao, Sun RGB-D: a RGB-D scene understanding benchmark suite, in Computer Vision and Pattern Recognition (2015), pp. 567–576
S.Z. Li, Markov Random Field Modeling in Image Analysis (Springer Science and Business Media, 2009)
A. Buades, B. Coll, J.-M. Morel, A non-local algorithm for image denoising, in Computer Vision and Pattern Recognition, vol 2 (IEEE, 2005), pp. 60–65
D.H. Salvadeo, N.D. Mascarenhas, A.L. Levada, Nonlocal Markovian models for image denoising. J. Electronic Imaging 25(1) (2016)
S. Jonna, S. Satapathy, R.R. Sahay, Super-resolution image defencing using a nonlocal nonconvex prior. Appl. Optics 57(2), 322–333 (2018)
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, Slic superpixels. Tech. Rep. (2010)
H. Xue, S. Zhang, D. Cai, Depth image inpainting: improving low rank matrix completion with low gradient regularization. IEEE Trans. Image Process. 26(9), 4311–4320 (2017)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Satapathy, S. (2022). Hole-Filling Method Using Nonlocal Non-convex Regularization for Consumer Depth Cameras. In: Tiwari, R., Mishra, A., Yadav, N., Pavone, M. (eds) Proceedings of International Conference on Computational Intelligence. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-16-3802-2_19
Download citation
DOI: https://doi.org/10.1007/978-981-16-3802-2_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-3801-5
Online ISBN: 978-981-16-3802-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)