Abstract
In the paper, an approach is proposed for the problem of consistency in depth maps estimation from binocular stereo video sequence. The consistent method includes temporal consistency and spatial consistency to eliminate the flickering artifacts and smooth inaccuracy in depth recovery. So the improved global stereo matching based on graph cut and energy optimization is implemented. In temporal domain, the penalty function with coherence factor is introduced for temporal consistency, and the factor is determined by Lucas-Kanade optical flow weighted histogram similarity constraint (LKWHSC). In spatial domain, the joint bilateral truncated absolute difference (JBTAD) is proposed for segmentation smoothing. The method can smooth naturally and uniformly in low-gradient region and avoid over-smoothing as well as keep edge sharpness in high-gradient discontinuities to realize spatial consistency. The experimental results show that the algorithm can obtain better spatial and temporal consistent depth maps compared with the existing algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
KHOSHABEH R, CHAN S H, NGUYEN T Q. Spatio-temporal consistency in video disparity estimation [C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Prague, Czech Republic: IEEE, 2011: 885–888.
CIGLA C, ALATAN A A. Temporally consistent dense depth map estimation via belief propagation [C]//IEEE 3DTV Conference on the True Vision—–Capture, Transmission and Display of 3D Video. Potsdam, Germany: IEEE, 2009: 1–4.
GARCIA F, AOUADA D, MIRBACH B, et al. A new multi-lateral filter for real-time depth enhancement [C]//IEEE International Conference on Advanced Video and Signal-Based Surveillance. Klagenfurt, Austria: IEEE, 2011: 42–47.
RICHARDT C, ORR D, DAVIES I, et al. Realtime spatiotemporal stereo matching using the dualcross-bilateral grid [C]//Proceedings of the European Conference on Computer Vision. Hersonissos, Greece: Springer-Verlag, 2010, 6313: 510–523.
LEE S B, HO Y S. Temporally consistent depth map estimation for 3D video generation and coding [J]. China Communications, 2013, 10(5): 39–49.
PHAM C C, NGUYEN V D, JEON J W. Efficient spatio-temporal local stereo matching using information permeability filtering [C]//IEEE International Conference on Image Processing. Orlando, USA: IEEE, 2012: 2965–2968.
MIN D B, LU J B, DO M N. Depth video enhancement based on weighted mode filtering [J]. IEEE Transactions on Image Processing, 2012, 21(3): 1176–1190.
FUSIELLO A, IRSARA L. Quasi-Euclidean uncalibrated epipolar rectification [C]//19th International Conference on Pattern Recognition. Tampa, USA: IEEE, 2008: 1–4.
HEO Y S, LEE K M, LEE S U. Joint depth map and color consistency estimation for stereo images with different illuminations and cameras [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(5): 1094–1106.
BOYKOV Y, VEKSLER O, ZABIH R. Fast approximate energy minimization via graph cuts [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(11): 1222–1239.
QIN S, XIE G. LW-PGD method and fusion feature based real-time pedestrian detection in video [J]. Journal of Computational Information Systems, 2014, 10(6): 2273–2281.
JIANG L C, SHEN G Q, ZHANG G X. An image retrieval algorithm based on HSV color segment histograms [J]. Mechanical & Electrical Engineering Magazine, 2009, 26(11): 54–57 (in Chinese).
ZHANG Y, ZHANG J W, YANG G Q, et al. Video de-hazing using spatial-temporal coherence optimization [J]. Application Research of Computers, 2011, 28(10): 3983–3985 (in Chinese).
BUADES A, COLL B, MOREL J M. Nonlocal image and movie denoising [J]. International Journal of Computer Vision, 2008, 76(2): 123–139.
FERREIRA L, ASSUNCAO P, DA SILVA CRUZ L A. 3D video shot boundary detection based on clustering of depth-temporal features [C]//2013 11th International Workshop on Content-based Multimedia Indexing. Veszprem, Hungary: IEEE, 2013: 1–6.
MA G H, WANG C, LIU P, et al. Sequential similarity detection algorithm based on image edge feature [J]. Journal of Shanghai Jiaotong University (Science), 2014, 19(1): 79–83.
ZHANG G F, JIA J Y, WONG T T, et al. Consistent depth maps recovery from a video sequence [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(6): 974–988.
YANG Q X, YANG R G, DAVIS J, et al. Spatialdepth super resolution for range images [C]//IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA: IEEE, 2007: 1–8.
Author information
Authors and Affiliations
Corresponding author
Additional information
Foundation item: the Science and Technology Innovation Project of Ministry of Culture of China (No. 2014KJCXXM08), the National Key Technology Research and Development Program of the Ministry of Science and Technology of China (No. 2012BAH37F02), and the National High Technology Research and Development Program (863) of China (No. 2011AA01A107)
Rights and permissions
About this article
Cite this article
Duan, F. Consistent depth maps estimation from binocular stereo video sequence. J. Shanghai Jiaotong Univ. (Sci.) 21, 184–191 (2016). https://doi.org/10.1007/s12204-016-1710-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12204-016-1710-7
Keywords
- consistent depth maps
- binocular stereo video sequence
- Lucas-Kanade optical flow weighted histogram similarity constraint (LKWHSC)
- joint bilateral truncated absolute difference (JBTAD)