Skip to main content
Log in

FRNet: an end-to-end feature refinement neural network for medical image segmentation

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Medical image segmentation is a crucial but challenging task for computer-aided diagnosis. In recent years, fully convolutional network-based methods have been widely applied to medical image segmentation. U-shape-based approaches are one of the most successful structures in this medical field. However, the consecutive down-sampling operations in the encoder lead to the loss of spatial information, which is important for medical image segmentation. In this paper, we present a novel lightweight end-to-end feature refinement network (FRNet) to address this issue. The structure of our model is simple and efficient. Specifically, the network adopts an encoder-decoder network as backbone, where two additional paths, spatial refinement path and semantic refinement path, are applied on the encoder and decoder, respectively, to improve the detailed representation ability and discriminative ability of our model. In addition, we introduce a feature adaptive fusion block (FAF block) that effectively combines features of different depths. The proposed FRNet can be trained in an end-to-end way. We have evaluated our method on three different medical image segmentation tasks. Experimental results show that FRNet has better performance than the state-of-the-art approaches. It achieves a high average accuracy without any post-processing of 0.968 and 0.936 for blood vessel segmentation and skin lesion segmentation, respectively. We further demonstrate that our method can be easily applied to other network structures to improve their performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Adal, K.M., van Etten, P.G., Martinez, J.P., Rouwen, K.W., Vermeer, K.A., van Vliet, L.J.: An automated system for the detection and classification of retinal changes due to red lesions in longitudinal fundus images. IEEE Trans. Biomed. Eng. 65, 1382–1390 (2018)

    Article  Google Scholar 

  2. Thaha, M.M., Kumar, K.P.M., Murugan, B.S., Dhanasekeran, S., Vijayakarthick, P., Selvi, A.S.: Brain tumor segmentation using convolutional neural networks in MRI images. J. Med. Syst. 43, 294 (2019)

    Article  Google Scholar 

  3. Paulano, F., Jimenez, J.J., Pulido, R.: 3D segmentation and labeling of fractured bone from CT images. Vis. Comput. 30, 939–948 (2014)

    Article  Google Scholar 

  4. Nguyen, U.T.V., Bhuiyan, A., Park, L.A.F., Ramamohanarao, K.: An effective retinal blood vessel segmentation method using multi-scale line detection. Pattern Recogn. 46, 703–715 (2013)

    Article  Google Scholar 

  5. Lee, Y., Hara, T., Fujita, H., Itoh, S., Ishigaki, T.: Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique. IEEE Trans. Med. Imaging 20, 595–604 (2001)

    Article  Google Scholar 

  6. Hoover, A.D., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19, 203–210 (2002)

    Article  Google Scholar 

  7. Taşcı, E., Uğur, A.: Shape and texture based novel features for automated juxtapleural nodule detection in lung CTs. J. Med. Syst. 39, 46 (2015)

    Article  Google Scholar 

  8. Zakeri, F.S., Behnam, H., Ahmadinejad, N.: Classification of benign and malignant breast masses based on shape and texture features in sonography images. J. Med. Syst. 36, 1621–1627 (2012)

    Article  Google Scholar 

  9. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1409.1556 (2014)

  10. He, K., Zhang, X., Ren, S., Jian, S.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

  11. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2015)

  12. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  13. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 834–848 (2018)

    Article  Google Scholar 

  14. Szegedy, C., Ioffe, S., Vanhoucke, V.: Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI Conference on Artificial Intelligence (2016)

  15. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition (2015)

  16. Deng, W., Shi, Q., Luo, K., Yang, Y., Ning, N.: Brain tumor segmentation based on improved convolutional neural network in combination with non-quantifiable local texture feature. J. Med. Syst. 43, 152 (2019)

    Article  Google Scholar 

  17. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: A Nested U-Net Architecture for Medical Image Segmentation, pp. 3–11. Springer, Cham (2018)

    Google Scholar 

  18. Li, X., Chen, H., Qi, X., Dou, Q., Fu, C., Heng, P.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37, 2663–2674 (2018)

    Article  Google Scholar 

  19. Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for Real-Time Semantic Segmentation on High-Resolution images: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part III edn, pp. 418–434 (2018)

  20. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: European Conference on Computer Vision (ECCV), Cham, pp. 334–349 (2018)

  21. Kai, H., Zhang, Z., Niu, X., Yuan, Z., Cao, C., Xiao, F., et al.: Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing 309, 179–191 (2018)

    Article  Google Scholar 

  22. Riaz, F., Rehman, S., Ajmal, M., Hafiz, R., Hassan, A., Aljohani, N., et al.: Gaussian mixture model based probabilistic modeling of images for medical image segmentation. IEEE Access 8, 16848–16856 (2020)

    Google Scholar 

  23. Lei, B., Kim, J., Kumar, A., Fulham, M., Feng, D.: Stacked fully convolutional networks with multi-channel learning: application to medical image segmentation. Vis. Comput. 33, 1061–1071 (2017)

    Article  Google Scholar 

  24. Jebaseeli, T.J., Deva Durai, C.A., Peter, J.D.: Retinal blood vessel segmentation from diabetic retinopathy images using tandem PCNN model and deep learning based SVM. Optik 199, 163328 (2019)

    Article  Google Scholar 

  25. Joes, S., Abràmoff, M.D., Meindert, N., Viergever, M.A., Bram, V.G.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23, 501–509 (2004)

    Article  Google Scholar 

  26. Codella, N.C.F., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., et al.: Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC). CoRR arXiv:1710.05006 (2017)

  27. Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., Tomancak, P., Hartenstein, V.: An integrated micro- and macro architectural analysis of the Drosophila brain by computer assisted serial section electron microscopy. PLoS Boil. 8, e1000502 (2010). https://doi.org/10.1371/journal.pbio.1000502

    Article  Google Scholar 

  28. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation. arXiv:1802.06955 (2018)

  29. Gu, Z., Cheng, J., Fu, H., Zhou, K., Hao, H., Zhao, Y., et al.: CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans. Med. Imaging 38, 2281–2292 (2019)

    Article  Google Scholar 

  30. Li, Y., Shen, L.: Skin lesion analysis towards melanoma detection using deep learning network. arXiv:1703.00577 (2017)

  31. Stollenga, M.F., Byeon, W., Liwicki, M., Schmidhuber, J.: Parallel Multi- Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation. arXiv:1506.07452 (2015)

Download references

Acknowledgements

This work is supported by the Nature Science Foundation of Guangdong province, No. 2016A030313520.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Guoqing Hu.

Ethics declarations

Conflict of interest

No conflict of interest exists in the submission of this manuscript and all the authors have agreed to have seen and approved the manuscript for submission.

Human participants or animals

The paper does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Hu, G. & Lyu, C. FRNet: an end-to-end feature refinement neural network for medical image segmentation. Vis Comput 37, 1101–1112 (2021). https://doi.org/10.1007/s00371-020-01855-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01855-z

Keywords

Navigation