Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images

  • Christoph BaurEmail author
  • Benedikt Wiestler
  • Shadi Albarqouni
  • Nassir Navab
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11383)


Reliably modeling normality and differentiating abnormal appearances from normal cases is a very appealing approach for detecting pathologies in medical images. A plethora of such unsupervised anomaly detection approaches has been made in the medical domain, based on statistical methods, content-based retrieval, clustering and recently also deep learning. Previous approaches towards deep unsupervised anomaly detection model local patches of normal anatomy with variants of Autoencoders or GANs, and detect anomalies either as outliers in the learned feature space or from large reconstruction errors. In contrast to these patch-based approaches, we show that deep spatial autoencoding models can be efficiently used to capture normal anatomical variability of entire 2D brain MR slices. A variety of experiments on real MR data containing MS lesions corroborates our hypothesis that we can detect and even delineate anomalies in brain MR images by simply comparing input images to their reconstruction. Results show that constraints on the latent space and adversarial training can further improve the segmentation performance over standard deep representation learning.



We thank our clinical partners from Klinikum Rechts der Isar for providing us with their dataset.


  1. 1.
    An, J., Cho, S.: Variational autoencoder based anomaly detection using reconstruction probability. In: Special Lecture on IE, vol. 2, pp. 1–18 (2015)Google Scholar
  2. 2.
    Chong, Y.S., Tay, Y.H.: Abnormal Event Detection in Videos using Spatiotemporal Autoencoder. CoRR (2017)Google Scholar
  3. 3.
    Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS (2014)Google Scholar
  4. 4.
    Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 733–742. IEEE (2016)Google Scholar
  5. 5.
    Iglesias, J.E., Liu, C.Y., Thompson, P.M., Tu, Z.: Robust brain extraction across datasets and comparison with publicly available methods. IEEE Trans. Med. Imaging 30(9), 1617–1634 (2011)CrossRefGoogle Scholar
  6. 6.
    Iheme, L.O., et al.: Concordance between computer-based neuroimaging findings and expert assessments in dementia grading. In: SIU, pp. 1–4 (2013)Google Scholar
  7. 7.
    Kingma, D.P., Welling, M.: Auto-Encoding Variational Bayes. CoRR (2013)Google Scholar
  8. 8.
    Larsen, A.B.L., Sønderby, S.K., Winther, O.: Autoencoding beyond pixels using a learned similarity metric. CoRR cs.LG (2015)Google Scholar
  9. 9.
    Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2536–2544. IEEE (2016)Google Scholar
  10. 10.
    Rohlfing, T., Zahr, N.M., Sullivan, E.V., Pfefferbaum, A.: The SRI24 multichannel atlas of normal adult human brain structure. Hum. Brain Mapp. 31(5), 798–819 (2009)CrossRefGoogle Scholar
  11. 11.
    Rosca, M., Lakshminarayanan, B., Warde-Farley, D., Mohamed, S.: Variational approaches for auto-encoding generative adversarial networks. arXiv preprint arXiv:1706.04987 (2017)
  12. 12.
    Sabokrou, M., Fathy, M., Hoseini, M.: Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder. Electron. Lett. 52(13), 1122–1124 (2016)CrossRefGoogle Scholar
  13. 13.
    Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. CoRR cs.CV (2017)Google Scholar
  14. 14.
    Seeböck, P., et al.: Identifying and Categorizing Anomalies in Retinal Imaging Data. CoRR cs.LG (2016)Google Scholar
  15. 15.
    Sethian, J.A., et al.: Level set methods and fast marching methods. J. Comput. Inf. Technol. 11(1), 1–2 (2003)CrossRefGoogle Scholar
  16. 16.
    Shiee, N., Bazin, P.L., Ozturk, A., Reich, D.S., Calabresi, P.A., Pham, D.L.: A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions. NeuroImage 49(2), 1524–1535 (2010)CrossRefGoogle Scholar
  17. 17.
    Taboada-Crispi, A., Sahli, H., Hernandez-Pacheco, D., Falcon-Ruiz, A.: Anomaly detection in medical image analysis. In: Handbook of Research on Advanced Techniques in Diagnostic Imaging and Biomedical Applications, pp. 426–446. IGI Global (2009)Google Scholar
  18. 18.
    Vaidhya, K., Thirunavukkarasu, S., Alex, V., Krishnamurthi, G.: Multi-modal brain tumor segmentation using stacked denoising autoencoders. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 181–194. Springer, Cham (2016). Scholar
  19. 19.
    Weiss, N., Rueckert, D., Rao, A.: Multiple sclerosis lesion segmentation using dictionary learning and sparse coding. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 735–742. Springer, Heidelberg (2013). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Christoph Baur
    • 1
    Email author
  • Benedikt Wiestler
    • 2
  • Shadi Albarqouni
    • 1
  • Nassir Navab
    • 1
    • 3
  1. 1.Computer Aided Medical Procedures (CAMP)TU MunichMunichGermany
  2. 2.Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der IsarTU MunichMunichGermany
  3. 3.Whiting School of EngineeringJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations