Incremental Anomaly Detection Approach for Characterizing Unusual Profiles

  • Yi Fang
  • Olufemi A. Omitaomu
  • Auroop R. Ganguly
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5840)


The detection of unusual profiles or anomalous behavioral characteristics from sensor data is especially complicated in security applications where the threat indicators may or may not be known in advance. Predictive modeling of massive volumes of historical data can yield insights on usual or baseline profiles, which in turn can be utilized to isolate unusual profiles when new data are observed in real-time. Thus, an incremental anomaly detection approach is proposed. This is a two-stage approach in which the first stage processes the available historical data and develops statistics that are in turn used by the second stage in characterizing the new incoming data for real-time decisions. The first stage adopts a mixture model of probabilistic principal component analyzers to quantify each historical observation by probabilistic measures. The second stage is a chi-square based anomaly detection approach that utilizes the probabilistic measures obtained in the first stage to determine if the incoming data is an anomaly. The proposed anomaly detection approach performs satisfactorily on simulated and benchmark datasets. The approach is also illustrated in the context of detecting commercial trucks that may pose safety and security risk. It is able to consistently identified trucks with anomalous features in the scenarios investigated.


Transportation security radioactive materials incremental knowledge discovery PPCA chi-square statistics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kouzes, R.T., Ely, J.H., Geelhood, B.D., Hansen, R.R., Lepel, E.A., Schweppe, J.E., Siciliano, E.R., Strom, D.J., Warner, R.A.: Naturally Occurring Radioactive Materials and Medical Isotopes at Border Crossings. In: IEEE Nuclear Science Symposium Conference Record, vol. 2, pp. 1448–1452 (2003)Google Scholar
  2. 2.
    Ely, J.H., Kouzes, R.T., Geelhood, B.D., Schweppe, J.E., Warner, R.A.: Overview of Naturally Occurring Radioactive Material in Plastic Scintillator Materials. IEEE Transactions on Nuclear Science 51(4), 1672–1676 (2004)CrossRefGoogle Scholar
  3. 3.
    Brennan, S.M., Mielke, A.M., Torney, D.C., Maccabe, A.B.: Radiation Detection with Distributed Sensor Networks. IEEE Computer 37(8), 57–59 (2004)Google Scholar
  4. 4.
    Brennan, S.M., Mielke, A.M., Torney, D.C.: Radioactive Source Detection by Sensor Networks. IEEE Transactions on Nuclear Science 52(3), 813–819 (2005)CrossRefGoogle Scholar
  5. 5.
    Geelhood, B.D., Ely, J.H., Hansen, R.R., Kouzes, R.T., Schweppeand, J.E., Warner, R.A.: Overview of Portal Monitoring at Border Crossings. In: IEEE Nuclear Science Symposium Conference Record, vol. 2, pp. 513–517 (2003)Google Scholar
  6. 6.
    Valentine, T.E.: Overview of Nuclear Detection Needs for Homeland Security. In: Proceedings of the American Nuclear Society, PHYSOR 2006, Vancouver, BC, Canada, September 10-14 (2006)Google Scholar
  7. 7.
    Omitaomu, O.A., Ganguly, A.R., Patton, B.W., Protopopescu, V.A.: Anomaly Detection in Radiation Sensor Data with Application to Transportation Security. IEEE Transactions on Intelligent Transportation Systems 10(2), 324–334 (2009)CrossRefGoogle Scholar
  8. 8.
    Agovic, A., Banerjee, A., Ganguly, A.R., Protopopescu, V.A.: Anomaly Detection in Transportation Corridors Using Manifold Embedding. In: Proceedings of the 1st International Workshop on Knowledge Discovery from Sensor Data (2007)Google Scholar
  9. 9.
    Everitt, B.S.: An Introduction to Latent Variable Models. Chapman & Hall, London (1984)zbMATHGoogle Scholar
  10. 10.
    Roweis, S., Ghahramani, Z.: A Unifying Review of Linear Gaussian Models. Neural Computaion 11(2), 305–345 (1999)CrossRefGoogle Scholar
  11. 11.
    Tipping, M.E., Bishop, C.M.: Probabilistic Principal Component Analysis. Journal of the Royal Statistical Society B 61, 611–622 (1999a)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Dempster, A., Laird, N., Rubin, D.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Tipping, M.E., Bishop, C.M.: Mixtures of Probabilistic Principal Component Analyzers. Neural Computation 11, 443–482 (1999b)CrossRefGoogle Scholar
  14. 14.
    Naes, T.: Multivariate Calibration When the Error Covariance Matrix is Structured. Technometrics 27, 301–311 (1985)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Hubert, M., Rousseeuw, P., Verboven, S.: A Fast Method for Robust Principal Components with Applications to Chemometrics. Chemometrics and Intelligent Laboratory Systems 60, 101–111 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Yi Fang
    • 1
  • Olufemi A. Omitaomu
    • 2
  • Auroop R. Ganguly
    • 2
  1. 1.Department of Computer SciencePurdue UniversityWest Lafayette
  2. 2.Oak Ridge National LaboratoryOak RidgeUSA

Personalised recommendations