Visualizing image content to explain novel image discovery


The initial analysis of any large data set can be divided into two phases: (1) the identification of common trends or patterns and (2) the identification of anomalies or outliers that deviate from those trends. We focus on the goal of detecting observations with novel content, which can alert us to artifacts in the data set or, potentially, the discovery of previously unknown phenomena. To aid in interpreting and diagnosing the novel aspect of these selected observations, we recommend the use of novelty detection methods that generate explanations. In the context of large image data sets, these explanations should highlight what aspect of a given image is new (color, shape, texture, content) in a human-comprehensible form. We propose DEMUD-VIS, the first method for providing visual explanations of novel image content by employing a convolutional neural network (CNN) to extract image features, a method that uses reconstruction error to detect novel content, and an up-convolutional network to convert CNN feature representations back into image space. We demonstrate this approach on diverse images from ImageNet, freshwater streams, and the surface of Mars. Finally, we evaluate the utility of the visual explanations with a user study.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20


  1. 1.

    Class definitions provided in code repository.

  2. 2.

    See code repository.

  3. 3.

  4. 4.

    We excluded one survey B response from our analysis that had a duration of 1,437,922 s (16 days) on the grounds that this likely did not represent continuous effort. Since the survey assesses novelty, it is important that it is completed in one session.


  1. Angelino E, Larus-Stone N, Alabi D, Seltzer M, Rudin C (2018) Certifiably optimal rule lists for categorical data. J Mach Learn Res 19:1–79

    MATH  Google Scholar 

  2. Bau D, Zhou B, Khosla A, Oliva A, Torralba A (2017) Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6541–6549

  3. Biran O, Cotton C (2017) Explanation and justification in machine learning: a survey. In: Proceedings of the IJCAI-17 workshop on explainable AI, pp 8–13

  4. Bradski G (2000) The OpenCV library. Dr Dobb’s J Softw Tools 25:120–125

    Google Scholar 

  5. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3): Article No. 15.

  6. Dabkowski P, Gal Y (2017) Real time image saliency for black box classifiers. In: Proceedings of the 31st conference on neural information processing systems, pp 6970–6979

  7. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 886–893

  8. Dang XH, Assent I, Ng RT, Zimek A, Schubert E (2014) Discriminative features for identifying and interpreting outliers. In: 2014 IEEE 30th international conference on data engineering, pp 88–99.

  9. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) DeCAF: a deep convolutional activation feature for generic visual recognition. In: Xing EP, Jebara T (eds) Proceedings of the 31st international conference on machine learning, PMLR, Bejing, China, pp 647–655

  10. Dosovitskiy A, Brox T (2016a) Generating images with perceptual similarity metrics based on deep networks. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems 29, Curran Associates, Inc., pp 658–666

  11. Dosovitskiy A, Brox T (2016b) Inverting visual representations with convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4829–4837

  12. Duan L, Tang G, Pei J, Bailey J, Campbell A, Tang C (2015) Mining outlying aspects on numeric data. Data Min Knowl Discov 29(5):1116–1151.

    MathSciNet  Article  MATH  Google Scholar 

  13. Gunning D, Aha DW (2019) DARPA’s explainable artificial intelligence program. AI Mag 4(2):44–58

    Article  Google Scholar 

  14. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. Mach Learn 3:1157–1182

    MATH  Google Scholar 

  15. He J, Carbonell J (2009) Prior-free rare category detection. In: Proceedings of the 2009 SIAM international conference on data mining, SIAM, pp 155–163

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  17. Huang H, He Q, Chiew K, Qian F, Ma L (2013) CLOVER: a faster prior-free approach to rare-category detection. Knowl Inf Syst 35(3):713–736

    Article  Google Scholar 

  18. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. arXiv:1408.5093

  19. Kerner HR, Wellington DF, Wagstaff KL, Bell JF, Amor HB (2019) Novelty detection for multispectral images with application to planetary exploration. In: Proceedings of the innovative applications in artificial intelligence conference, pp 9484–9491

  20. Kim B, Khanna R, Koyejo O (2016) Examples are not enough, learn to criticize! Criticism for interpretability. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems 29, pp 2280–2288

  21. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  22. Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, Chicago

    Google Scholar 

  23. Lim J, Ross D, Lin RS, Yang MH (2005) Incremental learning for visual tracking. In: Advances in neural information processing systems 17, MIT Press, pp 793–800

  24. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110.

    Article  Google Scholar 

  25. Lytle DA, Martínez-Muñoz G, Zhang W, Larios N, Shapiro L, Paasch R, Moldenke A, Mortensen EA, Todorovic S, Dietterich TG (2010) Automated processing and identification of benthic invertebrate samples. J N Am Benthol Soc 29(3):867–874

    Article  Google Scholar 

  26. Macha M, Akoglu L (2018) Explaining anomalies in groups with characterizing subspace rules. Data Min Knowl Discov 32:1444–1480.

    MathSciNet  Article  MATH  Google Scholar 

  27. Mahendran A, Vedaldi A (2015) Understanding deep image representations by inverting them. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5188–5196

  28. Martínez-Muñoz G, Zhang W, Payet N, Todorovic S, Larios N, Yamamuro A, Lytle D, Moldenke A, Mortensen E, Paasch R, Shapiro L, Dietterich T (2009) Dictionary-free categorization of very similar objects via stacked evidence trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 549–556

  29. Mathieu M, Couprie C, LeCun Y (2016) Deep multi-scale video prediction beyond mean square error. In: 4th international conference on learning representations

  30. Micenková B, Ng RT, Dang XH, Assent I (2013) Explaining outliers by subspace separability. In: Proceedings of the IEEE 13th international conference on data mining, pp 518–527.

  31. Montavon G, Samek W, Müller KR (2018) Methods for interpreting and understanding deep neural networks. Digital Signal Process 73:1–15.

    MathSciNet  Article  Google Scholar 

  32. Nair V, Hinton GE (2010) Rectified linear units improve Restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning, pp 807–814

  33. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  34. Olah C, Mordvintsev A, Schubert L (2017) Feature visualization. Distill.

  35. Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1717–1724

  36. Pimentel MAF, Clifton DA, Clifton L, Tarassenko L (2014) A review of novelty detection. Signal Process 99:215–249.

    Article  Google Scholar 

  37. Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 806–813

  38. Reed IS, Yu X (1990) Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution. IEEE Trans Acoust Speech Signal Process 38(10):1760–1770

    Article  Google Scholar 

  39. Ribeiro MT, Singh S, Guestrin C (2016) “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd SIGKDD conference on knowledge discovery and data mining, pp 1135–1144

  40. Rudin C (2018) Please stop explaining black box models for high-stakes decisions. In: Proceedings of the NIPS workshop on critiquing and correcting trends in machine learning

  41. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    MathSciNet  Article  Google Scholar 

  42. Siddiqui MA, Fern A, Dietterich TG, Wong WK (2019) Sequential feature explanations for anomaly detection. ACM Trans Knowl Discov Data.

    Article  Google Scholar 

  43. Stanboli A, Wagstaff K (2017) Mars surface image (Curiosity rover) labeled data set (version 1.0.0). Data set on Zenodo.

  44. Wagstaff KL, Lee J (2018) Interpretable discovery in large image data sets. In: Proceedings of the workshop on human interpretability in machine learning (WHI), pp 107–113

  45. Wagstaff KL, Lanza NL, Thompson DR, Dietterich TG, Gilmore MS (2013) Guiding scientific discovery with explanations using DEMUD. In: Proceedings of the twenty-seventh conference on artificial intelligence, pp 905–911

  46. Wagstaff KL, Lu Y, Stanboli A, Grimes K, Gowda T, Padams J (2018) Deep Mars: CNN classification of Mars imagery for the PDS Imaging Atlas. In: Proceedings of the innovative applications in artificial intelligence conference, pp 7867–7872

  47. Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H (2015) Understanding neural networks through deep visualization. In: Proceedings of the ICML deep learning workshop

  48. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of the European conference on computer vision, pp 818–833

Download references


We thank the Planetary Data System Imaging Node for funding this project. Part of this research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. All rights reserved. ©2019 California Institute of Technology. U.S. Government sponsorship acknowledged.

Author information



Corresponding author

Correspondence to Kiri L. Wagstaff.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Responsible editor: Pierre Baldi.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lee, J.H., Wagstaff, K.L. Visualizing image content to explain novel image discovery. Data Min Knowl Disc (2020).

Download citation


  • Novelty detection
  • Explanations
  • Image analysis