Incorporating Privileged Information to Unsupervised Anomaly Detection
We introduce a new unsupervised anomaly detection ensemble called SPI which can harness privileged information—data available only for training examples but not for (future) test examples. Our ideas build on the Learning Using Privileged Information (LUPI) paradigm pioneered by Vapnik et al. [17, 19], which we extend to unsupervised learning and in particular to anomaly detection. SPI (for Spotting anomalies with Privileged Information) constructs a number of frames/fragments of knowledge (i.e., density estimates) in the privileged space and transfers them to the anomaly scoring space through “imitation” functions that use only the partial information available for test examples. Our generalization of the LUPI paradigm to unsupervised anomaly detection shepherds the field in several key directions, including (i) domain-knowledge-augmented detection using expert annotations as PI, (ii) fast detection using computationally-demanding data as PI, and (iii) early detection using “historical future” data as PI. Through extensive experiments on simulated and real datasets, we show that augmenting privileged information to anomaly detection significantly improves detection performance. We also demonstrate the promise of SPI under all three settings (i–iii); with PI capturing expert knowledge, computationally-expensive features, and future data on three real world detection tasks. Code related to this paper is available at: http://www.andrew.cmu.edu/user/shubhras/SPI.
This research is sponsored by NSF CAREER 1452425 and IIS 1408287. Any conclusions expressed in this material are of the authors and do not necessarily reflect the views, expressed or implied, of the funding parties.
- 2.Burnaev, E., Smolyakov, D.: One-class SVM with privileged information and its application to malware detection. In: ICDM Workshops, pp. 273–280 (2016)Google Scholar
- 3.Celik, Z.B., McDaniel, P., Izmailov, R., Papernot, N., Swami, A.: Extending detection with forensic information. arXiv:1603.09638 (2016)
- 6.Emmott, A., Das, S., Dietterich, T., Fern, A., Wong, W.-K.: Systematic construction of anomaly detection benchmarks from real data. In: KDD ODD (2013)Google Scholar
- 9.Jonschkowski, R., Höfer, S., Brock, O.: Patterns for learning with side information. arXiv:1511.06429 (2015)
- 11.Liu, F.T., Ting, K.M., Zhou, Z.-H.: Isolation forest. In: ICDM (2008)Google Scholar
- 12.Marcacini, R.M., Domingues, M.A., Hruschka, E.R., Rezende, S.O.: Privileged information for hierarchical document clustering: a metric learning approach. In: ICPR, pp. 3636–3641 (2014)Google Scholar
- 15.Sharmanska, V., Quadrianto, N., Lampert, C.H.: Learning to rank using privileged information. In: ICCV, pp. 825–832 (2013)Google Scholar
- 16.Sharmanska, V., Quadrianto, N., Lampert, C.H.: Learning to transfer privileged information. arXiv:1410.0389 (2014)
- 20.Wang, Z., Ji, Q.: Classifier learning with hidden information. In: CVPR (2015)Google Scholar