Incorporating Privileged Information to Unsupervised Anomaly Detection

  • Shubhranshu ShekharEmail author
  • Leman Akoglu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11051)


We introduce a new unsupervised anomaly detection ensemble called SPI which can harness privileged information—data available only for training examples but not for (future) test examples. Our ideas build on the Learning Using Privileged Information (LUPI) paradigm pioneered by Vapnik et al. [17, 19], which we extend to unsupervised learning and in particular to anomaly detection. SPI (for Spotting anomalies with Privileged Information) constructs a number of frames/fragments of knowledge (i.e., density estimates) in the privileged space and transfers them to the anomaly scoring space through “imitation” functions that use only the partial information available for test examples. Our generalization of the LUPI paradigm to unsupervised anomaly detection shepherds the field in several key directions, including (i) domain-knowledge-augmented detection using expert annotations as PI, (ii) fast detection using computationally-demanding data as PI, and (iii) early detection using “historical future” data as PI. Through extensive experiments on simulated and real datasets, we show that augmenting privileged information to anomaly detection significantly improves detection performance. We also demonstrate the promise of SPI under all three settings (iiii); with PI capturing expert knowledge, computationally-expensive features, and future data on three real world detection tasks. Code related to this paper is available at:



This research is sponsored by NSF CAREER 1452425 and IIS 1408287. Any conclusions expressed in this material are of the authors and do not necessarily reflect the views, expressed or implied, of the funding parties.

Supplementary material

478880_1_En_6_MOESM1_ESM.pdf (377 kb)
Supplementary material 1 (pdf 377 KB)


  1. 1.
    Aggarwal, C.C.: Outlier analysis. Data Mining, pp. 237–263. Springer, Cham (2015). Scholar
  2. 2.
    Burnaev, E., Smolyakov, D.: One-class SVM with privileged information and its application to malware detection. In: ICDM Workshops, pp. 273–280 (2016)Google Scholar
  3. 3.
    Celik, Z.B., McDaniel, P., Izmailov, R., Papernot, N., Swami, A.: Extending detection with forensic information. arXiv:1603.09638 (2016)
  4. 4.
    Chen, J., Liu, X., Lyu, S.: Boosting with side information. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7724, pp. 563–577. Springer, Heidelberg (2013). Scholar
  5. 5.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Emmott, A., Das, S., Dietterich, T., Fern, A., Wong, W.-K.: Systematic construction of anomaly detection benchmarks from real data. In: KDD ODD (2013)Google Scholar
  7. 7.
    Feyereisl, J., Aickelin, U.: Privileged information for data clustering. Inf. Sci. 194, 4–23 (2012)CrossRefGoogle Scholar
  8. 8.
    Fouad, S., Tino, P., Raychaudhury, S., Schneider, P.: Incorporating privileged information through metric learning. IEEE Neural Netw. Learn. Syst. 24(7), 1086–1098 (2013)CrossRefGoogle Scholar
  9. 9.
    Jonschkowski, R., Höfer, S., Brock, O.: Patterns for learning with side information. arXiv:1511.06429 (2015)
  10. 10.
    Lapin, M., Hein, M., Schiele, B.: Learning using privileged information: SVM+ and weighted SVM. Neural Netw. 53, 95–108 (2014)CrossRefGoogle Scholar
  11. 11.
    Liu, F.T., Ting, K.M., Zhou, Z.-H.: Isolation forest. In: ICDM (2008)Google Scholar
  12. 12.
    Marcacini, R.M., Domingues, M.A., Hruschka, E.R., Rezende, S.O.: Privileged information for hierarchical document clustering: a metric learning approach. In: ICPR, pp. 3636–3641 (2014)Google Scholar
  13. 13.
    Niu, L., Li, W., Xu, D.: Exploiting privileged information from web data for action and event recognition. Int. J. Comput. Vis. 118(2), 130–150 (2016)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Ribeiro, B., Silva, C., Chen, N., Vieira, A., das Neves, J.C.: Enhanced default risk models with SVM+. Expert Syst. Appl. 39(11), 10140–10152 (2012)CrossRefGoogle Scholar
  15. 15.
    Sharmanska, V., Quadrianto, N., Lampert, C.H.: Learning to rank using privileged information. In: ICCV, pp. 825–832 (2013)Google Scholar
  16. 16.
    Sharmanska, V., Quadrianto, N., Lampert, C.H.: Learning to transfer privileged information. arXiv:1410.0389 (2014)
  17. 17.
    Vapnik, V., Izmailov, R.: Learning with intelligent teacher: similarity control and knowledge transfer. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds.) SLDS 2015. LNCS (LNAI), vol. 9047, pp. 3–32. Springer, Cham (2015). Scholar
  18. 18.
    Vapnik, V., Izmailov, R.: Knowledge transfer in SVM and neural networks. Ann. Math. Artif. Intell. 81(1–2), 3–19 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Vapnik, V., Vashist, A.: A new learning paradigm: learning using privileged information. Neural Netw. 22(5–6), 544–557 (2009)CrossRefGoogle Scholar
  20. 20.
    Wang, Z., Ji, Q.: Classifier learning with hidden information. In: CVPR (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Heinz College of Information Systems and Public PolicyCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations