Abstract
This paper presents a general refinement procedure that enhances any given depth map obtained by passive or active sensing. Given a depth map, either estimated by triangulation methods or directly provided by the sensing system, and its corresponding 2-D image, we correct the depth values by separately treating regions with undesired effects such as empty holes, texture copying or edge blurring due to homogeneous regions, occlusions, and shadowing. In this work, we use recent depth enhancement filters intended for Time-of-Flight cameras, and adapt them to alternative depth sensing modalities, both active using an RGB-D camera and passive using a dense stereo camera. To that end, we propose specific masks to tackle areas in the scene that require a special treatment. Our experimental results show that such areas are satisfactorily handled by replacing erroneous depth measurements with accurate ones.
This work was supported by the National Research Fund, Luxembourg, under the CORE project C11/BM/1204105/FAVE/Ottersten.
Chapter PDF
References
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2003)
Salvi, J., Fernandez, S., Pribanic, T., Llado, X.: A state of the art in structured light patterns for surface profilometry. Pattern Recognition 43, 2666–2680 (2010)
Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, ECCVW (2008)
Garcia, F., Mirbach, B., Ottersten, B., Grandidier, F., Cuesta, A.: Pixel Weighted Average Strategy for Depth Sensor Data Fusion. In: International Conference on Image Processing (ICIP), pp. 2805–2808 (2010)
Garcia, F., Aouada, D., Mirbach, B., Solignac, T., Ottersten, B.: A New Multi-lateral Filter for Real-Time Depth Enhancement. In: Advanced Video and Signal-Based Surveillance, AVSS (2011)
Min, D., Lu, J., Minh, N.D.: Depth Video Enhancement Based on Weighted Mode Filtering. IEEE Transactions on Image Processing (TIP) 21, 1176–1190 (2012)
Kopf, J., Cohen, M., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. In: SIGGRAPH 2007: ACM SIGGRAPH 2007 papers, p. 96. ACM, New York (2007)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV, pp. 839–846 (1998)
Crabb, R., Tracey, C., Puranik, A., Davis, J.: Real-time foreground segmentation via range and color imaging. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–5 (2008)
Benhimane, S., Malis, E.: Real-time image-based tracking of planes using efficient second-order minimization. In: IEEE IROS, vol. 1, pp. 943–948 (2004)
Hirschmuller, H.: Real-time correlation-based stereo vision with reduced border errors. IJCV 47(1/2/3), 229–246 (2002)
Yang, Q., Tan, K.H., Ahuja, N.: Real-time O(1) bilateral filtering. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 557–564 (2009)
Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. International Journal of Computer Vision 81, 24–52 (2009)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47, 7–42 (2002)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. In: IEEE TIP, vol. 13-4, pp. 600–612 (2004)
Middlebury Stereo Datasets (2011), http://vision.middlebury.edu/stereo/data/
Point Grey Research, Inc. (2011), http://www.ptgrey.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Garcia, F., Aouada, D., Abdella, H.K., Solignac, T., Mirbach, B., Ottersten, B. (2012). Depth Enhancement by Fusion for Passive and Active Sensing. In: Fusiello, A., Murino, V., Cucchiara, R. (eds) Computer Vision – ECCV 2012. Workshops and Demonstrations. ECCV 2012. Lecture Notes in Computer Science, vol 7585. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33885-4_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-33885-4_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33884-7
Online ISBN: 978-3-642-33885-4
eBook Packages: Computer ScienceComputer Science (R0)