Anomaly Detection Using Ensembles

  • Larry Shoemaker
  • Lawrence O. Hall
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6713)


We show that using random forests and distance-based outlier partitioning with ensemble voting methods for supervised learning of anomaly detection provide similar accuracy results when compared to the same methods without partitioning. Further, distance-based outlier and one-class support vector machine partitioning and ensemble methods for semi-supervised learning of anomaly detection also compare favorably to the corresponding non-ensemble methods. Partitioning and ensemble methods would be required for very large datasets that need distributed computing approaches. ROC curves often show significant improvement from increased true positives in the low false positive range for ensemble methods used on several datasets.


outliers anomalies random forests data partitioning ROC curves 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abe, N., Zadrozny, B., Langford, J.: Outlier detection by active learning. In: KDD 2006: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 504–509. ACM, New York (2006)Google Scholar
  2. 2.
    UCI KDD Archive. Kdd cup 1999 data (accessed on, January 1 2010),
  3. 3.
    Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: A comparison of decision tree ensemble creation techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence, 173–180 (2007)Google Scholar
  4. 4.
    Bay, S.D., Schwabacher, M.: Mining distance-based outliers in near linear time with randomization and a simple pruning rule. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 29–38. ACM Press, New York (2003)CrossRefGoogle Scholar
  5. 5.
    Bradley, A.P.: The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition 30, 1145–1159 (1997)CrossRefGoogle Scholar
  6. 6.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  7. 7.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM Comput. Surv. 41(3), 1–58 (2009)CrossRefGoogle Scholar
  8. 8.
    Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines (accessed on, November 1 2010),
  9. 9.
    Cumming, G., Fidler, F., Vaux, D.L.: Errror bars in experimental biology. The Journal of Cell Biology 177(1), 7–11 (2007)CrossRefGoogle Scholar
  10. 10.
    Cumming, G., Finch, S.: Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist 60(2), 170–180 (2005)CrossRefGoogle Scholar
  11. 11.
    Fawcett, T.: An introduction to roc analysis. Pattern Recognition Letters 27(8), 861–874 (2006), rOC Analysis in Pattern Recognition CrossRefGoogle Scholar
  12. 12.
    Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9, 69–82 (2008), CrossRefGoogle Scholar
  13. 13.
    Hanley, J.A., McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology 143, 29–36 (1982)CrossRefGoogle Scholar
  14. 14.
    Hempstalk, K., Frank, E., Witten, I.H.: One-class classification by combining density and class probability estimation. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 505–519. Springer, Heidelberg (2008),, ISBN: 978-3-540-87478-2, doi:10.1007/978-3-540-87479-951 CrossRefGoogle Scholar
  15. 15.
    John, B.S., Platt, J.C., Shawe-taylor, J., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Computation 13, 1443–1471 (2001)CrossRefzbMATHGoogle Scholar
  16. 16.
    Lazarevic, A.: Feature bagging for outlier detection. In: KDD 2005, pp. 157–166 (2005)Google Scholar
  17. 17.
    Niennattrakul, V., Keogh, E., Ratanamahatana, C.A.: Data editing techniques to allow the application of distance-based outlier detection to streams. In: IEEE International Conference on Data Mining, vol. 0, pp. 947–952 (2010)Google Scholar
  18. 18.
    Perdisci, R., Ariu, D., Fogla, P., Giacinto, G., Lee, W.: Mcpad: A multiple classifier system for accurate payload-based anomaly detection. Computer Networks 53(6), 864–881 (2009),; Traffic Classification and Its Applications to Modern NetworksCrossRefzbMATHGoogle Scholar
  19. 19.
    Shoemaker, L.: Ensemble Learning With Imbalanced Data. Ph.D. thesis. University of South Florida (2010)Google Scholar
  20. 20.
    Tan, P., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley, Reading (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Larry Shoemaker
    • 1
  • Lawrence O. Hall
    • 1
  1. 1.Computer Science and EngineeringUniversity of South FloridaTampaUSA

Personalised recommendations