Skip to main content
SpringerLink
Log in

Local complexity difference matting based on weight map and alpha mattes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image matting is an essential image processing technology in computer vision with significant diverse practical applications, including image synthesis, webcasting, and movie production. Although some methods have been proposed to extract the alpha mattes, these methods are not sensitive enough to local complex regions with large differences between the input image and the ground-truth alpha matte. In this paper, we design a weight map generation algorithm, which extracts the local complex region of each image by measuring the local complex differences between the input image and the ground-truth alpha matte. For the weight consistency problem of the pixel-level loss function, we propose a loss function based on local complexity differences, which can strengthen the training on local regions of large complexity differences. Moreover, we design a local complexity difference matting approach on the basis of the presented loss function and weight map generation algorithm to improve the matting accuracy of local complexity difference images. To verify the validity of the proposed matting method, experiments were conducted on the composition-1 k matting evaluation data set produced by Adobe. Experimental results show that the proposed weight map generation algorithm can effectively extract the local complex regions. Our proposed matting method outperforms state-of-the-art matting methods in the cases of locally complexity difference images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Aksoy Y, Aydin TO, Pollefeys M, Zürich E, Zürich DR (2017) Designing effective inter-pixel information flow for natural image matting. In: IEEE conference on computer vision and pattern recognition (CVPR), 2017, IEEE, pp 228—236

  2. Barron JT (2019) A general and adaptive robust loss function. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2019, IEEE, pp 4326–4334

  3. Cai S, Zhang X, Fan H, Huang H, Liu J, Liu J, Liu J, Wang J, Sun J (2019) Disentangled image matting. In: IEEE international conference on computer vision (CV), 2019, IEEE, pp 8819–8828

  4. Chen Q, Li D, Tang C (2013) KNN matting. IEEE Trans Pattern Anal Mach Intell 35(9):2175–2188

    Article  Google Scholar 

  5. Cho D, Kim S, Tai YW, Kweon IS (2016) Automatic trimap generation and consistent matting for light-field images. IEEE Trans Pattern Anal Mach Intell 39(8):1504–1517

    Article  Google Scholar 

  6. Cho D, Tai Y, Kweon IS, Daejeon K, Korea S (2016) Natural image matting using deep convolutional neural networks. Proceedings of European Conference on Computer Vision (ECCV) 2016:626–643

    Google Scholar 

  7. Cho D, Tai Y, Kweon IS (2019) Deep convolutional neural network for natural image matting using initial alpha mattes. IEEE Trans Image Process 28(3):1054–1067

    Article  MathSciNet  Google Scholar 

  8. Feng X, Liang X, Zhang Z (2016) A cluster sampling method for image matting via sparse coding. In: Proceedings of european conference on computer vision (ECCV), 2016, pp. 204–219

  9. Feng F, Huang H, Wu Q, Ling X, Liang Y, Chai Z (2020) An alpha matting algorithm based on collaborative swarm optimization for high-resolution images. Sci Sin Inform 50(3):424–437

    Article  Google Scholar 

  10. Gong M, Qian Y, Cheng L (2015) Integrated foreground segmentation and boundary matting for live videos. IEEE Trans Image Process 24(4):1356–1370

    Article  MathSciNet  MATH  Google Scholar 

  11. He K, Rhemann C, Rother C, Tang X, Sun J (2011) A global sampling method for alpha matting. In: IEEE conference on computer vision and pattern recognition (CVPR), 2011, IEEE, pp 2049–2056

  12. Huang H, Liang Y, Yang X, Hao Z (2019) Pixel-level discrete multiobjective sampling for image matting. IEEE Trans Image Process 28(8):3739–3751

    Article  MathSciNet  MATH  Google Scholar 

  13. Karacan L, Erdem A, Erdem E (2017) Alpha matting with KL-divergence-based sparse sampling. IEEE Trans Image Process 26(9):4523–4536

    Article  MathSciNet  MATH  Google Scholar 

  14. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of International Conference on Learning Representations (LR), 2015, pp 1–15

  15. Lee Y, Yang S (2018) Parallel block sequential closed-form matting with fan-shaped partitions. IEEE Trans Image Process 27(2):594–605

    Article  MathSciNet  MATH  Google Scholar 

  16. Levin A, Lischinski D, Weiss Y (2008) A closed-form solution to natural image matting. IEEE Trans Pattern Anal Mach Intell 30(2):228–242

    Article  Google Scholar 

  17. Li L, Cai H, Zhang Y, Lin W, Kot AC, Sun X (2016) Sparse representation-based image quality index with adaptive sub-dictionaries. IEEE Trans Image Process 25(8):3775–3786

    MathSciNet  MATH  Google Scholar 

  18. Liang Y, Huang H, Cai Z, Hao Z (2019) Multiobjective evolutionary optimization based on fuzzy multicriteria evaluation and decomposition for image matting. IEEE Trans Fuzzy Syst 27(5):1100–1111

    Article  Google Scholar 

  19. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition (CVPR), 2015, IEEE, pp 3431–3440

  20. Lu H, Dai Y, Shen C, Xu S (2021) Index networks. IEEE Trans Pattern Anal Mach Intell 44(1):242–255

    Article  Google Scholar 

  21. Lutz S, Amplianitis K, Smolić A (2018) AlphaGAN: generative adversarial networks for natural image matting. In: IEEE conference on computer vision and pattern recognition (CVPR), 2018, IEEE, pp 3050–3058

  22. Porter T, Duff T (1984) Compositing digital images. In: Proceedings of computer graphics and interactive techniques (CGIT), 1984, pp 253–259

  23. Tang J, Aksoy Y, Oztireli C, Gross M, Aydin TO (2019) Learning-based sampling for natural image matting. In: IEEE conference on computer vision and pattern recognition (CVPR), 2019, IEEE, pp 3050–3058

  24. Wang Y, Niu Y, Duan P, Lin J, Zheng Y (2018) Deep propagation-based image matting. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), 2018, pp 999–1006

  25. Xu N, Price B, Cohen S, Huang T (2017) Deep image matting. In: IEEE conference on computer vision and pattern recognition (CVPR), 2017, IEEE, pp 311–320

  26. Yoon D, Park J, Cho D (2020) Lightweight deep CNN for natural image matting via similarity-preserving knowledge distillation. IEEE Signal Process Lett 27:2139–2143

    Article  Google Scholar 

  27. Yu J, Lin Z, Yang J, Shen X, Huang TS (2019) Generative image inpainting with contextual attention. In: IEEE conference on computer vision and pattern recognition (CVPR), 2018, IEEE, pp 5505–5514

  28. Zhang T, Liu X, Gong L, Wang S, Niu X, Shen L (2021) Late fusion multiple kernel clustering with local kernel alignment maximization. IEEE Trans Multimedia:1–15

  29. Zhao M, Li D, Shi Z, Du S, Li P, Hu J (2019) Blur feature extraction plus automatic KNN matting: a novel two stage blur region detection method for local motion blurred images. IEEE Access 7:181142–181151

    Article  Google Scholar 

  30. Zou D, Chen X, Cao G, Wang X (2020) Unsupervised video matting via sparse and low-rank representation. IEEE Trans Pattern Anal Mach Intell 42(6):1501–1514

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (no. 61876207, no. 62002053 and no. 62162012), the Natural Science Foundation of Guizhou Province (no. QKHJCZK2022YB195), the Natural Science Foundation of Guizhou Minzu University (no. GZMUZK[2021]YB24), the Natural Science Foundation of Guangdong Province (no. 2022A1515011491 and no. 2021A0101180005), the Fundamental Research Funds for the Central Universities (no. 2020ZYGXZR014), the Youth Science and Technology Talents Cultivating Object of Guizhou Province (no. QJHKY2021104), the Science and Technology Support Program of Guizhou (no. QKHZC2021YB531), the Guangdong Basic and Applied Basic Research Foundation (no. 2019A1515111082), the Zhongshan Science and Technology Research Project of Social welfare (no. 2019B2010), the University Young Innovative Talent Project of Guangdong Province (no. 2019KQNCX186), the Key Research and Development Program of Zhongshan (no. 2019A4018).

Author information

Authors and Affiliations

  1. Guizhou Key Laboratory of Pattern Recognition and Intelligent System, Guizhou Minzu University, Guiyang, 550025, China

    Fujian Feng

  2. School of Software Engineering, South China University of Technology, Guangzhou, 510006, China

    Fujian Feng, Han Huang & Di Liu

  3. School of Computer Science, University of Electronic Science and Technology of China, Zhongshan Institute, Zhongshan, 528400, China

    Yihui Liang

Authors
  1. Fujian Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Di Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yihui Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Di Liu.

Ethics declarations

Conflict of interests

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, F., Huang, H., Liu, D. et al. Local complexity difference matting based on weight map and alpha mattes. Multimed Tools Appl 81, 43357–43372 (2022). https://doi.org/10.1007/s11042-022-13223-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13223-1

Keywords

Search

Navigation