Abstract
Achieving or even surpassing human-level accuracy became recently possible in a variety of application scenarios due to the rise of convolutional neural networks (CNNs) trained from large datasets. However, solving supervised visual recognition tasks by discriminating among known categories is only one side of the coin. In contrast to this, novelty detection is still an unsolved task where instances of yet unknown categories need to be identified. Therefore, we propose to leverage the powerful discriminative nature of CNNs to novelty detection tasks by investigating class-specific activation patterns. More precisely, we assume that a semantic category can be described by its extreme value signature, that specifies which dimensions of deep neural activations have largest values. By following this intuition, we show that already a small number of high-valued dimensions allows to separate known from unknown categories. Our approach is simple, intuitive, and can be easily put on top of CNNs trained for vanilla classification tasks. We empirically validate the benefits of our approach in terms of accuracy and speed by comparing it against established methods in a variety of novelty detection tasks derived from ImageNet. Finally, we show that visualizing extreme value signatures allows to inspect class-specific patterns learned during training which may ultimately help to better understand CNN models.
This research was supported by grant DE 735/10-1 of the German Research Foundation (DFG).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We assume that fine-tuning networks for known classes would further improve the overall accuracy since activation patterns are expected to become specific for known classes. However, fine-tuning networks for all evaluated tasks and splits would be too time consuming. Therefore, the used Places-CNN ensures a fair comparison since it was not trained with any involved ImageNet class.
- 2.
Note that the reported results for Local-KNFST differ from [4] since we use CNN features instead of dense SIFT features which results in improved performance.
- 3.
- 4.
Gradient maps are normalized individually for better visualization, hence, the scaling can not be compared directly. Results of uniformly normalized maps are similar.
References
Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 584–599. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_38
Bendale, A., Boult, T.: Towards open world recognition. In: Computer Vision and Pattern Recognition (CVPR) (2015)
Bendale, A., Boult, T.E.: Towards open set deep networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
Bodesheim, P., Freytag, A., Rodner, E., Denzler, J.: Local novelty detection in multi-class recognition problems. In: Winter Conference on Applications of Computer Vision (WACV) (2015)
Bodesheim, P., Freytag, A., Rodner, E., Kemmler, M., Denzler, J.: Kernel null space methods for novelty detection. In: Computer Vision and Pattern Recognition (CVPR) (2013)
Brandenburg, F.J., Gleißner, A., Hofmeier, A.: The nearest neighbor spearman footrule distance for bucket, interval, and partial orders. J. Comb. Optim. 26(2), 310–332 (2013)
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: DeCAF: A deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning (ICML) (2014)
Dosovitskiy, A., Brox, T.: Inverting convolutional networks with convolutional networks. CoRR abs/1506.02753 (2015)
Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006)
Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning. Springer, Heidelberg (2001). doi:10.1007/978-0-387-21606-5
Huang, Y., Sun, X., Lu, M., Xu, M.: Channel-max, channel-drop and stochastic max-pooling. In: Computer Vision and Pattern Recognition Workshop (CVPR-WS) (2015)
Jumutc, V., Suykens, J.A.: Supervised novelty detection. In: Computational Intelligence and Data Mining (CIDM) (2013)
Jumutc, V., Suykens, J.A.: Multi-class supervised novelty detection. Pattern Anal. Mach. Intell. (PAMI) 36(12), 2510–2523 (2014)
Kemmler, M., Rodner, E., Denzler, J.: One-class classification with Gaussian processes. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6493, pp. 489–500. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19309-5_38
Kenk, V.S., Kovačič, S., Kristan, M., Hajdinjak, M., Perš, J., et al.: Visual re-identification across large, distributed camera networks. Image Vis. Comput. 34, 11–26 (2015)
Li, Y., Liu, L., Shen, C., Van Den Hengel, A.: Mining mid-level visual patterns with deep CNN activations. Int. J. Comput. Vis. 121(3), 344–364 (2017). http://dx.doi.org/10.1007/s11263-016-0945-y
Makhzani, A., Frey, B.J.: Winner-take-all autoencoders. In: Neural Information Processing Systems (NIPS) (2015)
Mensink, T., Verbeek, J., Perronnin, F., Csurka, G.: Distance-based image classification: generalizing to new classes at near-zero cost. Pattern Anal. Mach. Intell. (PAMI) 35(11), 2624–2637 (2013)
Pimentel, M.A., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Signal Process. 99, 215–249 (2014)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)
Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. Pattern Anal. Mach. Intell. (PAMI) 35(7), 1757–1772 (2013)
Schölkopf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)
Sharif Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Conference on Computer Vision and Pattern Recognition Workshops (CVPR-WS) (2014)
Simon, M., Rodner, E., Denzler, J.: Part detector discovery in deep convolutional neural networks. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 162–177. Springer, Cham (2015). doi:10.1007/978-3-319-16808-1_12
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: International Conference on Learning Representations Workshop (ICLR-WS) (2014)
Tax, D.M., Duin, R.P.: Support vector data description. Mach. Learn. 54(1), 45–66 (2004)
Vinokurov, N., Weinshall, D.: Novelty detection in multiclass scenarios with incomplete set of class labels. arXiv preprint arXiv:1604.06242 (2016)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_53
Zhou, B., Khosla, A., Lapedriza, A., Torralba, A., Oliva, A.: Places: an image database for deep scene understanding. arXiv preprint arXiv:1610.02055 (2016)
Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Neural Information Processing Systems (NIPS) (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Schultheiss, A., Käding, C., Freytag, A., Denzler, J. (2017). Finding the Unknown: Novelty Detection with Extreme Value Signatures of Deep Neural Activations. In: Roth, V., Vetter, T. (eds) Pattern Recognition. GCPR 2017. Lecture Notes in Computer Science(), vol 10496. Springer, Cham. https://doi.org/10.1007/978-3-319-66709-6_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-66709-6_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66708-9
Online ISBN: 978-3-319-66709-6
eBook Packages: Computer ScienceComputer Science (R0)