Skip to main content
SpringerLink
Log in

EW-Fisher: A Novel Loss Function for Deep Learning-Based Image Co-Segmentation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The loss function is an important factor for the success of machine learning. This paper proposes a new loss function for deep learning-based image co-segmentation. It aims to maximize the inter-class difference between the foreground and the background and at the same time minimize the two intra-class variances. This idea has some similarity to the Fisher criterion in pattern recognition. We further embed an edge weighting strategy into this form of Fisher-like criterion to let the pixels near the foreground edges be paid more attentions in the training process for achieving the finer segmentation. The resultant loss function is called EW-Fisher (Edge-Weighted Fisher). We apply the proposed EW-Fisher loss to image co-segmentation and evaluate it on commonly used datasets. In the experiments, the EW-Fisher stably outperforms the most-widely used cross-entropy loss and Dice loss as well as the recently presented edge agreement loss and Hausdorff distance loss. The comparison results and the ablation studies prove the values of our Fisher-like learning criterion and edge weighting strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Hladka B, Holub M (2015) A gentle introduction to machine learning for natural language processing: How to start in 16 practical steps. Lang Lingus Compass 9(2):55–76

    Article  Google Scholar 

  2. Qing H, Jinfeng S, Hui D, Xiaodong W, Guangzhi W (2018) Robust liver vessel extraction using 3D U-Net with variant dice loss function. Comput Biol Med 101:S0010482518302385

    Google Scholar 

  3. Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 11–19

  4. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Paper presented at: international conference on medical image computing and computer-assisted intervention

  5. Zhou X-Y, Shen M, Riga C, Yang G-Z, Lee S-L (2017) Focal fcn: towards small object segmentation with limited training data. arXiv preprint arXiv:1711.01506

  6. Tran PV (2016) A fully convolutional neural network for cardiac segmentation in short-axis MRI. arXiv preprint arXiv:1604.00494

  7. Milletari F, Navab N, Ahmadi S-A (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV). IEEE, pp 565–571

  8. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 658–666

  9. Al Arif SMR, Knapp K, Slabaugh G (2017) Shape-aware deep convolutional neural network for vertebrae segmentation. In: International workshop and challenge on computational methods and clinical applications in musculoskeletal imaging. Springer, pp 12–24

  10. Kervadec H, Bouchtiba J, Desrosiers C, Granger E, Dolz J, Ayed IB (2019) Boundary loss for highly unbalanced segmentation. In: International conference on medical imaging with deep learning, pp 285–296

  11. Chen X, Williams BM, Vallabhaneni SR, Czanner G, Williams R, Zheng Y (2019) Learning active contour models for medical image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 11632–11640

  12. Kim B, Ye JC (2019) Multiphase level-set loss for semi-supervised and unsupervised segmentation with deep learning. arXiv preprint arXiv:1904.02872

  13. Karimi D, Salcudean SE (2019) Reducing the Hausdorff distance in medical image segmentation with convolutional neural networks. arXiv preprint arXiv:1904.10030

  14. Liu S, Ding W, Liu C, Liu Y, Wang Y, Li H (2018) ERN: edge loss reinforced semantic segmentation network for remote sensing images. Remote Sens 10(9):1339

    Article  Google Scholar 

  15. Zimmermann RS, Siems JN (2019) Faster training of mask r-CNN by focusing on instance boundaries. Comput Vis Image Underst 188:102795

    Article  Google Scholar 

  16. Zhu Y, Sapra K, Reda FA, Shih KJ, Newsam S, Tao A, Catanzaro B (2019) Improving semantic segmentation via video propagation and label relaxation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8856–8865

  17. Zhen M, Wang J, Zhou L, Fang T, Quan L (2019) Learning fully dense neural networks for image semantic segmentation. In: Proceedings of the IEEE Transactions on Medical Imaging, pp 499–513

  18. Li Y, Zhao W, Pan J (2017) Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Trans Autom Sci Eng 14(2):1256–64

    Article  Google Scholar 

  19. Schmidt FR, Boykov Y (2012) Hausdorff distance constraint for multi-surface segmentation. In: Proceedings of the IEEE Transactions on Medical Imaging, pp 499–513

  20. Li W, Jafari OH, Rother C (2018) Deep object co-segmentation. In: Asian conference on computer vision. Springer, pp 638–653

  21. Gong X, Liu X, Li Y, Li H (2020) A novel co-attention computation block for deep learning based image co-segmentation. Image Vis Comput 101:103973

    Article  Google Scholar 

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  23. Han J, Quan R, Zhang D, Nie F (2017) Robust object co-segmentation using background prior. IEEE Trans Image Process 27(4):1639–1651

    Article  MathSciNet  Google Scholar 

  24. Shotton J, Winn J, Rother C, Criminisi A (2006) Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: European conference on computer vision. Springer, pp 1–15

  25. Rubinstein M, Joulin A, Kopf J, Liu C (2013) Unsupervised joint object discovery and segmentation in internet images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1939–1946

  26. Batra D, Kowdle A, Parikh D, Luo J, Chen T (2010) iCoseg: Interactive co-segmentation with intelligent scribble guidance. In: 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 3169–3176

  27. Zhou G, Nebgen B, Lubbers N, Malone WF, Tretiak S (2020) Gpu-accelerated semi-empirical born oppenheimer molecular dynamics using pyTorch. J Chem Theory Comput 59(1):206–14

    Google Scholar 

  28. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  29. Faktor A, Irani M, Co-segmentation by composition. In: 2013 IEEE international conference on computer vision

  30. Quan R, Han J, Zhang D, Nie F (2016) Object co-segmentation via graph optimized-flexible manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 687–695

  31. Jerripothula KR, Cai J, Yuan J (2016) Image co-segmentation via saliency co-fusion. IEEE Trans Multimed 18(9):1896–1909

    Article  Google Scholar 

  32. Yuan Z-H, Lu T, Wu Y (2017) Deep-dense conditional random fields for object co-segmentation. In: IJCAI, pp 3371–3377

Download references

Acknowledgements

This work was supported in part by Beijing Municipal Science and Technology Project [Grant Number Z181100001918002].

Author information

Authors and Affiliations

  1. Beijing Lab of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing, 100081, China

    Xiaopeng Gong, Xiabi Liu, Xin Duan & Yushuo Li

Authors
  1. Xiaopeng Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiabi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yushuo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiabi Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, X., Liu, X., Duan, X. et al. EW-Fisher: A Novel Loss Function for Deep Learning-Based Image Co-Segmentation. Neural Process Lett 52, 2399–2413 (2020). https://doi.org/10.1007/s11063-020-10354-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10354-0

Keywords

Search

Navigation