Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery

  • Thomas SchleglEmail author
  • Philipp Seeböck
  • Sebastian M. Waldstein
  • Ursula Schmidt-Erfurth
  • Georg Langs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10265)


Obtaining models that capture imaging markers relevant for disease progression and treatment monitoring is challenging. Models are typically based on large amounts of data with annotated examples of known markers aiming at automating detection. High annotation effort and the limitation to a vocabulary of known markers limit the power of such approaches. Here, we perform unsupervised learning to identify anomalies in imaging data as candidates for markers. We propose AnoGAN, a deep convolutional generative adversarial network to learn a manifold of normal anatomical variability, accompanying a novel anomaly scoring scheme based on the mapping from image space to a latent space. Applied to new data, the model labels anomalies, and scores image patches indicating their fit into the learned distribution. Results on optical coherence tomography images of the retina demonstrate that the approach correctly identifies anomalous images, such as images containing retinal fluid or hyperreflective foci.


Optical Coherence Tomography Latent Space Anomaly Detection Query Image Image Patch 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Del Giorno, A., Bagnell, J.A., Hebert, M.: A discriminative framework for anomaly detection in large videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 334–349. Springer, Cham (2016). doi: 10.1007/978-3-319-46454-1_21 CrossRefGoogle Scholar
  2. 2.
    Matteoli, S., Diani, M., Theiler, J.: An overview of background modeling for detection of targets and anomalies in hyperspectral remotely sensed imagery. IEEE J. Selected Top. Appl. Earth Obs. Remote Sens. 7(6), 2317–2336 (2014)CrossRefGoogle Scholar
  3. 3.
    Carrera, D., Boracchi, G., Foi, A., Wohlberg, B.: Detecting anomalous structures by convolutional sparse models. In: 2015 International Joint Conference on Neural Networks (IJCNN), IEEE, pp. 1–8 (2015)Google Scholar
  4. 4.
    Erfani, S.M., Rajasegarar, S., Karunasekera, S., Leckie, C.: High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognit. 58, 121–134 (2016)CrossRefGoogle Scholar
  5. 5.
    Pimentel, M.A., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Signal Process. 99, 215–249 (2014)CrossRefGoogle Scholar
  6. 6.
    Venhuizen, F.G., van Ginneken, B., Bloemen, B., van Grinsven, M.J., Philipsen, R., Hoyng, C., Theelen, T., Sánchez, C.I.: Automated age-related macular degeneration classification in OCT using unsupervised feature learning. In: SPIE Medical Imaging, International Society for Optics and Photonics, p. 94141I (2015)Google Scholar
  7. 7.
    Schlegl, T., Waldstein, S.M., Vogl, W.-D., Schmidt-Erfurth, U., Langs, G.: Predicting semantic descriptions from medical images with convolutional neural networks. In: Ourselin, S., Alexander, D.C., Westin, C.-F., Cardoso, M.J. (eds.) IPMI 2015. LNCS, vol. 9123, pp. 437–448. Springer, Cham (2015). doi: 10.1007/978-3-319-19992-4_34 CrossRefGoogle Scholar
  8. 8.
    Seeböck, P., Waldstein, S., Klimscha, S., Gerendas, B.S., Donner, R., Schlegl, T., Schmidt-Erfurth, U., Langs, G.: Identifying and categorizing anomalies in retinal imaging data. In: NIPS 2016 MLHC Workshop. Preprint arXiv:1612.00686 (2016)
  9. 9.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  10. 10.
    Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1486–1494 (2015)Google Scholar
  11. 11.
    Donahue, J., Krähenbühl, P., Darrell, T.: Adversarial feature learning. arXiv:1605.09782 (2016)
  12. 12.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434 (2015)
  13. 13.
    Yeh, R., Chen, C., Lim, T.Y., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with perceptual and contextual losses. arXiv:1607.07539 (2016)
  14. 14.
    Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems, pp. 2226–2234 (2016)Google Scholar
  15. 15.
    Garvin, M.K., Abràmoff, M.D., Wu, X., Russell, S.R., Burns, T.L., Sonka, M.: Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images. IEEE Trans. Med. Imaging 28(9), 1436–1447 (2009)CrossRefGoogle Scholar
  16. 16.
    Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: Feature learning by inpainting. CoRR abs/1604.07379 (2016)Google Scholar
  17. 17.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. arXiv:1412.6980 (2014)
  18. 18.
    Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). Software available from

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Thomas Schlegl
    • 1
    • 2
    Email author
  • Philipp Seeböck
    • 1
    • 2
  • Sebastian M. Waldstein
    • 2
  • Ursula Schmidt-Erfurth
    • 2
  • Georg Langs
    • 1
  1. 1.Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided TherapyMedical University ViennaViennaAustria
  2. 2.Christian Doppler Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and OptometryMedical University ViennaViennaAustria

Personalised recommendations