Abstract
Foreground matting is an elementary image processing problem. It is challenged especially in the case of complicated background. Recently, light field camera has been employed to help improve computer vision algorithms. Light field image can record angular information of rays, which enable us to analyses the 3D scene from the aspects of defocus and correspondence. In this paper, we develop an improved light field algorithm to separate the foreground automatically and accurately. Focal stack and epipolar plane image generated from light field are used to calculate focusness cue and correspondence cue. We then use k-means to integrate all the cues of regions with an ensemble vote and get a hierarchical region boundary. Finally, markov random field is used to assign foreground labels to regions. We show that our light field matting method can overcome the camouflage color or cluttered background problem. Our algorithm is evaluated on a public light field dataset. Compared to some state-of-the-art segmentation algorithms based on two dimension images, our method outputs a more accurate result.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2281 (2012)
Bigun, J., Granlund, G.H.: Optimal orientation detection of linear symmetry. In: Proceedings of the IEEE First International Conference on Computer Vision, pp. 433–438 (1987)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1124–1137 (2004)
Chen, X., Dai, F., Ma, Y., Zhang, Y.: Automatic foreground segmentation using light field images. In: 2015 Visual Communications and Image Processing, VCIP 2015, pp. 1–4 (2016)
Cho, D., Kim, S., Tai, Y.-W.: Consistent matting for light field images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 90–104. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_7
Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2, pp. 264–271 (2001)
Garcia Ugarriza, L., Saber, E., Vantaram, S.R., Amuso, V., Shaw, M., Bhaskar, R.: Automatic image segmentation by dynamic region growth and multiresolution merging. IEEE Trans. Image Process. 18, 2275–2288 (2009)
Wanner, S., Goldluecke, B.: Globally consistent depth labeling of 4D light fields. In: 2012 IEEE Conference on Computer Vision Pattern Recognition, pp. 41–48 (2012)
Graf, F., Kriegel, H.P., Weiler, M.: Robust segmentation of relevant regions in low depth of field images. In: Proceedings - International Conference on Image Processing, ICIP, pp. 2861–2864 (2011)
He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: CVPR 2011, pp. 2049–2056. IEEE (2011)
Kim, C.: Segmenting a low-depth-of-field image using morphological filters and region merging. IEEE Trans. Image Process. 14, 1503–1511 (2005)
Li, F., Porikli, F.: Harmonic variance: a novel measure for in-focus segmentation. In: Proceedings of the British Machine Vision Conference 2013, pp. 33.1–33.11. British Machine Vision Association (2013)
Li, H., Ngan, K.N.: Unsupervised video segmentation with low depth of field. IEEE Trans. Circuits Syst. Video Technol. 17, 1742–1751 (2007)
Li, H., Ngan, K.N.: Learning to extract focused objects from low DOF images. IEEE Trans. Circuits Syst. Video Technol. 21, 1571–1580 (2011)
Li, N., Ye, J., Ji, Y., Ling, H., Yu, J.: Saliency detection on light field. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2806–2813 (2014)
Mei, J., Si, Y., Gao, H.: A curve evolution approach for unsupervised segmentation of images with low depth of field. IEEE Trans. Image Process. 22, 4086–4095 (2013)
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Comput. Sci. Tech. Rep. 20, 1–11 (2005)
Rhemann, C., Rother, C., Gelautz, M.: Improving color modeling for alpha matting. In: Proceedings on the British Machine Vision Conference 2008, pp. 115.1–115.10 (2008)
Joshi, N., Matusik, W., Avidan, S., Pfister, H., Freeman, W.T.: Exploring defocus matting. IEEE Comput. Graph. Appl. 27, 43–52 (2007)
Varnousfaderani, E.S., Rajan, D.: Weighted color and texture sample selection for image matting. IEEE Trans. Image Process. 22, 4260–4270 (2013)
Tao, M.W., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 673–680 (2013)
Tsai, D.-M., Wang, H.-J.: Segmenting focused objects in complex visual images Pattern Recogn. Lett. 19, 929–940 (1998)
Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: Proceedings of the IEEE Computer Society Conference on Computer Vision Pattern Recognition, pp. 1–8 (2007)
Wanner, S., Straehle, C., Goldluecke, B.: Globally consistent multi-label assignment on the ray space of 4D light fields. In: Proceedings of the IEEE Computer Society Conference on Computer Vision Pattern Recognition, pp. 1011–1018 (2013)
Zhang, C., Zhang, H.: An unsupervised approach to determination of main subject regions in images with low depth field. In: Proceedings of the 2008 IEEE 10th Workshop on Multimedia Signal Processing, MMSP 2008, pp. 650–653 (2008)
Zhang, J., Wang, M., Gao, J., Wang, Y., Zhang, X., Wu, X.: Saliency detection with a deeper investigation of light field. In: IJCAI International Joint Conference on Artificial Intelligence, pp. 2212–2218 (2015)
Zhang, K., Lu, H., Wang, Z., Zhao, Q., Duan, M.: A fuzzy segmentation of salient region of interest in low depth of field image. In: Cham, T.-J., Cai, J., Dorai, C., Rajan, D., Chua, T.-S., Chia, L.-T. (eds.) MMM 2007. LNCS, vol. 4351, pp. 782–791. Springer, Heidelberg (2006). https://doi.org/10.1007/978-3-540-69423-6_76
McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. ACM Trans. Graph. 24, 567 (2005)
Acknowledgments
This work is supported by Beijing Advanced Innovation Center for Imaging Technology (BAICIT-2016009), National Nature Science Foundation of China (61371194, 61672361, 61402440, 61771458, 61702479), Beijing Natural Science Foundation (4152012) and the Key Research Program of the Chinese Academy of Sciences, Grant NO. KFZD-SW-407.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Zhou, J., Naren, T., Chen, X., Ma, Y., Liu, J., Dai, F. (2018). Light Field Foreground Matting Based on Defocus and Correspondence. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)