Abstract
Depth map generated by the Kinect may have some pixels lost due to echo attenuation of infra- red light and mutual interference between neighboring pixels, which can cause pervasive problems when utilizing Kinect cameras as depth sensors. In this work, we propose a 2-step inpainting algorithm to infill the holes. First, a naive Bayesian estimation is conducted as preliminary inpainting scheme, utilizing neighboring pixels of the missing ones, and corresponding pixels in the color image as prior knowledge. After that, an optimization is implemented to improve the depth map, where the false edges in mistakenly inpainted regions are detected, then iteratively propelled to their true positions under total variation framework. Experimental results are included to show effectiveness of the proposed algorithm.
Similar content being viewed by others
References
Camplani, M., Salgado, L.: Efficient spatio-temporal hole filling strategy for Kinect depth maps. In: Proc. of 3DIP and Applications, Jan., vol. 8290, pp. 1–10 (2012)
Qi, F., Han, J., Wang, P., Shi, G.M., Li, F.: Structure guided fusion for depth map inpainting. Pattern Recognit. Lett. 34(1), 70–76 (2013)
Liu, J.Y., Gong, X.J., Liu, J.L.: Guided inpainting and filtering for Kinect depth maps. In: Proc. of International Conference on Pattern Recognition, pp. 2055–2058 (2012)
Telea, A.: An image inpainting technique based on the fast marching method. J. Graph. GPU Game Tools 9(1), 23–34 (2004)
Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)
Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the 27th Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’00), July, pp. 417–424 (2000)
Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: Proc. of International Conference on Computer Vision and Pattern Recognition, pp. 721–728 (2003)
Chan, T.F., Kang, S.H., Shen, J.: Euler’s elastica and curvature based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)
Schonlieb, C., Bertozzi, A.: Unconditionally stable schemes for higher order inpainting. Commun. Math. Sci. 9(2), 413–457 (2011)
Xu, L., Lu, C.W., Xu, Y., Jia, J.Y.: Image smoothing via L 0 gradient minimization. ACM Trans. Graph. (SIGGRAPH Asia 2011) 30, 174 (2011)
Kou, F., Li, Z.G., Wen, C.Y., Chen, W.H., Zhao, C.C., Wang, J.H.: Perceptual based content adaptive L 0 smoothing. In: Advances in Multimedia Information Processing (PCM 2013). Lecture Notes in Computer Science. Springer, Berlin/Heidelberg (2013, accepted)
Wang, Y.L., Yang, J.F., Yin, W.T., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)
Lai, K., Bo, L., Ren, X., Fox, D.: Detection-based object labeling in 3D scenes. In: Proc. of International Conference on Robotics and Automation, pp. 1330–1337 (2012)
Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Minneapolis, MN (2007)
Acknowledgements
This work has been supported by National Nature Science Foundation of China under the research project 61075075, 61175108, and by Beijing Municipal of Science and Technical Commission under Major Program D121104002812001. Here, we also express our gratitude to Shaoping Bai for his great help in revising the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ding, K., Chen, W. & Wu, X. Optimum inpainting for depth map based on L 0 total variation. Vis Comput 30, 1311–1320 (2014). https://doi.org/10.1007/s00371-013-0888-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-013-0888-z