Skip to main content
Log in

SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation

  • Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript


Inspired by classic Generative Adversarial Networks (GANs), we propose a novel end-to-end adversarial neural network, called SegAN, for the task of medical image segmentation. Since image segmentation requires dense, pixel-level labeling, the single scalar real/fake output of a classic GAN’s discriminator may be ineffective in producing stable and sufficient gradient feedback to the networks. Instead, we use a fully convolutional neural network as the segmentor to generate segmentation label maps, and propose a novel adversarial critic network with a multi-scale L1 loss function to force the critic and segmentor to learn both global and local features that capture long- and short-range spatial relationships between pixels. In our SegAN framework, the segmentor and critic networks are trained in an alternating fashion in a min-max game: The critic is trained by maximizing a multi-scale loss function, while the segmentor is trained with only gradients passed along by the critic, with the aim to minimize the multi-scale loss function. We show that such a SegAN framework is more effective and stable for the segmentation task, and it leads to better performance than the state-of-the-art U-net segmentation method. We tested our SegAN method using datasets from the MICCAI BRATS brain tumor segmentation challenge. Extensive experimental results demonstrate the effectiveness of the proposed SegAN with multi-scale loss: on BRATS 2013 SegAN gives performance comparable to the state-of-the-art for whole tumor and tumor core segmentation while achieves better precision and sensitivity for Gd-enhance tumor core segmentation; on BRATS 2015 SegAN achieves better performance than the state-of-the-art in both dice score and precision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others


  1. Although the pixel value ranges of medical images can vary, one can always normalize them to a certain value range such as [0,1], so it is compact.


  • Adams, R, & Bischof, L. (1994). Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(6), 641–647.

    Article  Google Scholar 

  • Arjovsky, M., Chintala, S., Bottou, L. (2017). Wasserstein gan. arXiv:170107875.

  • Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (6), 679–698.

  • Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K. (2015). Semantic image segmentation with deep convolutional nets and fully connected crfs. In ICLR. arXiv:1412.7062.

  • Cobzas, D., Birkbeck, N., Schmidt, M., Jagersand, M. (2007). Murtha A (2007) 3d variational brain tumor segmentation using a high dimensional feature set. In IEEE 11th international conference on computer vision. ICCV 2007 (pp. 1–8). IEEE.

  • Comaniciu, D., & Meer, P. (2002). Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.

    Article  Google Scholar 

  • Geremia, E., Clatz, O, Menze, B. H., Konukoglu, E., Criminisi, A., Ayache, N. (2011). Spatial decision forests for ms lesion segmentation in multi-channel magnetic resonance images. NeuroImage, 57(2), 378–390.

    Article  PubMed  Google Scholar 

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672–2680).

  • Gooya, A., Biros, G., Davatzikos, C. (2011). Deformable registration of glioma images using em algorithm and diffusion reaction modeling. IEEE Transactions on Medical Imaging, 30(2), 375–390.

    Article  PubMed  Google Scholar 

  • Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P. M., Larochelle, H. (2017). Brain tumor segmentation with deep neural networks. Medical Image Analysis, 35, 18–31.

    Article  PubMed  Google Scholar 

  • Isola, P., Zhu, J. Y., Zhou, T., Efros, A. A. (2016). Image-to-image translation with conditional adversarial networks. arXiv:161107004.

  • Kamnitsas, K., Ledig, C., Newcombe, V. F., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., Glocker, B. (2017). Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical Image Analysis, 36, 61–78.

    Article  PubMed  Google Scholar 

  • Kass, M., Witkin, A., Terzopoulos, D. (1988). Snakes: active contour models. International Journal of Computer Vision, 1(4), 321–331.

    Article  Google Scholar 

  • Lee, C. H., Wang, S., Murtha, A., Brown, M., Greiner, R. (2008). Segmenting brain tumors using pseudo–conditional random fields. In Medical image computing and computer-assisted intervention–MICCAI 2008 (pp. 359–366).

  • Lefohn, A., Cates, J., Whitaker, R. (2003). Interactive, gpu-based level sets for 3d segmentation. In Medical image computing and computer-assisted intervention-MICCAI 2003 (pp. 564–572).

  • Lin, G., Shen, C., van den Hengel, A., Reid, I. (2016). Efficient piecewise training of deep structured models for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3194–3203).

  • Long, J., Shelhamer, E., Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440).

  • Luc, P., Couprie, C., Chintala, S., Verbeek, J. (2016). Semantic segmentation using adversarial networks. arXiv:161108408.

  • Malladi, R., Sethian, J. A., Vemuri, B. C. (1995). Shape modeling with front propagation: a level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(2), 158–175.

    Article  Google Scholar 

  • Manjunath, B., & Chellappa, R. (1991). Unsupervised texture segmentation using markov random field models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(5), 478–482.

    Article  Google Scholar 

  • Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015). The multimodal brain tumor image segmentation benchmark (brats). IEEE Transactions on Medical Imaging, 34(10), 1993–2024.

    Article  PubMed  Google Scholar 

  • Mumford, D., & Shah, J. (1989). Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42(5), 577–685.

    Article  Google Scholar 

  • Noh, H., Hong, S., Han, B. (2015). Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision (pp. 1520–1528).

  • Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9 (1), 62–66.

    Article  Google Scholar 

  • Pereira, S., Pinto, A., Alves, V., Silva, C. A. (2016). Brain tumor segmentation using convolutional neural networks in mri images. IEEE Transactions on Medical Imaging, 35(5), 1240–1251.

    Article  PubMed  Google Scholar 

  • Radford, A., Metz, L., Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434.

  • Ronneberger, O, Fischer, P, Brox, T. (2015). U-net: convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention. Springer (pp. 234–241).

  • Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X. (2016). Improved techniques for training gans. In Advances in neural information processing systems (pp. 2226–2234).

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.

    Article  Google Scholar 

  • Wels, M., Carneiro, G., Aplas, A., Huber, M., Hornegger, J., Comaniciu, D. (2008). A discriminative model-constrained graph cuts approach to fully automated pediatric brain tumor segmentation in 3-d mri. In Medical image computing and computer-assisted intervention–MICCAI 2008 (pp. 67–75).

  • Zhang, H., Xu, T., Li, H., Zhang, S., Huang, X., Wang, X., Metaxas, D. (2017). Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. IEEE Int. Conf. Comput. Vision (ICCV) 5907–5915.

Download references


This research was supported in part by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM), and Lister Hill National Center for Biomedical Communications (LHNCBC), under Contract HHSN276201500692P.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Yuan Xue.

Additional information

Yuan Xue and Tao Xu are Co-first Authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, Y., Xu, T., Zhang, H. et al. SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation. Neuroinform 16, 383–392 (2018).

Download citation

  • Published:

  • Issue Date:

  • DOI: