The initial analysis of any large data set can be divided into two phases: (1) the identification of common trends or patterns and (2) the identification of anomalies or outliers that deviate from those trends. We focus on the goal of detecting observations with novel content, which can alert us to artifacts in the data set or, potentially, the discovery of previously unknown phenomena. To aid in interpreting and diagnosing the novel aspect of these selected observations, we recommend the use of novelty detection methods that generate explanations. In the context of large image data sets, these explanations should highlight what aspect of a given image is new (color, shape, texture, content) in a human-comprehensible form. We propose DEMUD-VIS, the first method for providing visual explanations of novel image content by employing a convolutional neural network (CNN) to extract image features, a method that uses reconstruction error to detect novel content, and an up-convolutional network to convert CNN feature representations back into image space. We demonstrate this approach on diverse images from ImageNet, freshwater streams, and the surface of Mars. Finally, we evaluate the utility of the visual explanations with a user study.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Class definitions provided in code repository.
See code repository.
We excluded one survey B response from our analysis that had a duration of 1,437,922 s (16 days) on the grounds that this likely did not represent continuous effort. Since the survey assesses novelty, it is important that it is completed in one session.
Angelino E, Larus-Stone N, Alabi D, Seltzer M, Rudin C (2018) Certifiably optimal rule lists for categorical data. J Mach Learn Res 19:1–79
Bau D, Zhou B, Khosla A, Oliva A, Torralba A (2017) Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6541–6549
Biran O, Cotton C (2017) Explanation and justification in machine learning: a survey. In: Proceedings of the IJCAI-17 workshop on explainable AI, pp 8–13
Bradski G (2000) The OpenCV library. Dr Dobb’s J Softw Tools 25:120–125
Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3): Article No. 15. https://doi.org/10.1145/1541880.1541882
Dabkowski P, Gal Y (2017) Real time image saliency for black box classifiers. In: Proceedings of the 31st conference on neural information processing systems, pp 6970–6979
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 886–893
Dang XH, Assent I, Ng RT, Zimek A, Schubert E (2014) Discriminative features for identifying and interpreting outliers. In: 2014 IEEE 30th international conference on data engineering, pp 88–99. https://doi.org/10.1109/ICDE.2014.6816642
Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) DeCAF: a deep convolutional activation feature for generic visual recognition. In: Xing EP, Jebara T (eds) Proceedings of the 31st international conference on machine learning, PMLR, Bejing, China, pp 647–655
Dosovitskiy A, Brox T (2016a) Generating images with perceptual similarity metrics based on deep networks. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems 29, Curran Associates, Inc., pp 658–666
Dosovitskiy A, Brox T (2016b) Inverting visual representations with convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4829–4837
Duan L, Tang G, Pei J, Bailey J, Campbell A, Tang C (2015) Mining outlying aspects on numeric data. Data Min Knowl Discov 29(5):1116–1151. https://doi.org/10.1007/s10618-014-0398-2
Gunning D, Aha DW (2019) DARPA’s explainable artificial intelligence program. AI Mag 4(2):44–58
Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. Mach Learn 3:1157–1182
He J, Carbonell J (2009) Prior-free rare category detection. In: Proceedings of the 2009 SIAM international conference on data mining, SIAM, pp 155–163
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Huang H, He Q, Chiew K, Qian F, Ma L (2013) CLOVER: a faster prior-free approach to rare-category detection. Knowl Inf Syst 35(3):713–736
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. arXiv:1408.5093
Kerner HR, Wellington DF, Wagstaff KL, Bell JF, Amor HB (2019) Novelty detection for multispectral images with application to planetary exploration. In: Proceedings of the innovative applications in artificial intelligence conference, pp 9484–9491
Kim B, Khanna R, Koyejo O (2016) Examples are not enough, learn to criticize! Criticism for interpretability. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems 29, pp 2280–2288
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105
Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, Chicago
Lim J, Ross D, Lin RS, Yang MH (2005) Incremental learning for visual tracking. In: Advances in neural information processing systems 17, MIT Press, pp 793–800
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94
Lytle DA, Martínez-Muñoz G, Zhang W, Larios N, Shapiro L, Paasch R, Moldenke A, Mortensen EA, Todorovic S, Dietterich TG (2010) Automated processing and identification of benthic invertebrate samples. J N Am Benthol Soc 29(3):867–874
Macha M, Akoglu L (2018) Explaining anomalies in groups with characterizing subspace rules. Data Min Knowl Discov 32:1444–1480. https://doi.org/10.1007/s10618-018-0585-7
Mahendran A, Vedaldi A (2015) Understanding deep image representations by inverting them. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5188–5196
Martínez-Muñoz G, Zhang W, Payet N, Todorovic S, Larios N, Yamamuro A, Lytle D, Moldenke A, Mortensen E, Paasch R, Shapiro L, Dietterich T (2009) Dictionary-free categorization of very similar objects via stacked evidence trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 549–556
Mathieu M, Couprie C, LeCun Y (2016) Deep multi-scale video prediction beyond mean square error. In: 4th international conference on learning representations
Micenková B, Ng RT, Dang XH, Assent I (2013) Explaining outliers by subspace separability. In: Proceedings of the IEEE 13th international conference on data mining, pp 518–527. https://doi.org/10.1109/ICDM.2013.132
Montavon G, Samek W, Müller KR (2018) Methods for interpreting and understanding deep neural networks. Digital Signal Process 73:1–15. https://doi.org/10.1016/j.dsp.2017.10.011
Nair V, Hinton GE (2010) Rectified linear units improve Restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning, pp 807–814
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987
Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1717–1724
Pimentel MAF, Clifton DA, Clifton L, Tarassenko L (2014) A review of novelty detection. Signal Process 99:215–249. https://doi.org/10.1016/j.sigpro.2013.12.026
Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 806–813
Reed IS, Yu X (1990) Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution. IEEE Trans Acoust Speech Signal Process 38(10):1760–1770
Ribeiro MT, Singh S, Guestrin C (2016) “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd SIGKDD conference on knowledge discovery and data mining, pp 1135–1144
Rudin C (2018) Please stop explaining black box models for high-stakes decisions. In: Proceedings of the NIPS workshop on critiquing and correcting trends in machine learning
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252
Siddiqui MA, Fern A, Dietterich TG, Wong WK (2019) Sequential feature explanations for anomaly detection. ACM Trans Knowl Discov Data. https://doi.org/10.1145/3230666
Stanboli A, Wagstaff K (2017) Mars surface image (Curiosity rover) labeled data set (version 1.0.0). Data set on Zenodo. https://doi.org/10.5281/zenodo.1049137
Wagstaff KL, Lee J (2018) Interpretable discovery in large image data sets. In: Proceedings of the workshop on human interpretability in machine learning (WHI), pp 107–113
Wagstaff KL, Lanza NL, Thompson DR, Dietterich TG, Gilmore MS (2013) Guiding scientific discovery with explanations using DEMUD. In: Proceedings of the twenty-seventh conference on artificial intelligence, pp 905–911
Wagstaff KL, Lu Y, Stanboli A, Grimes K, Gowda T, Padams J (2018) Deep Mars: CNN classification of Mars imagery for the PDS Imaging Atlas. In: Proceedings of the innovative applications in artificial intelligence conference, pp 7867–7872
Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H (2015) Understanding neural networks through deep visualization. In: Proceedings of the ICML deep learning workshop
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of the European conference on computer vision, pp 818–833
We thank the Planetary Data System Imaging Node for funding this project. Part of this research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. All rights reserved. ©2019 California Institute of Technology. U.S. Government sponsorship acknowledged.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Responsible editor: Pierre Baldi.
About this article
Cite this article
Lee, J.H., Wagstaff, K.L. Visualizing image content to explain novel image discovery. Data Min Knowl Disc (2020). https://doi.org/10.1007/s10618-020-00700-0
- Novelty detection
- Image analysis