oddball: Spotting Anomalies in Weighted Graphs

  • Leman Akoglu
  • Mary McGlohon
  • Christos Faloutsos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6119)

Abstract

Given a large, weighted graph, how can we find anomalies? Which rules should be violated, before we label a node as an anomaly? We propose the oddball algorithm, to find such nodes. The contributions are the following: (a) we discover several new rules (power laws) in density, weights, ranks and eigenvalues that seem to govern the so-called “neighborhood sub-graphs” and we show how to use these rules for anomaly detection; (b) we carefully choose features, and design oddball, so that it is scalable and it can work un-supervised (no user-defined constants) and (c) we report experiments on many real graphs with up to 1.6 million nodes, where oddball indeed spots unusual nodes that agree with intuition.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aggarwal, C.C., Yu, P.S.: Outlier detection for high dimensional data. In: SIGMOD, pp. 37–46 (2001)Google Scholar
  2. 2.
    Akoglu, L., McGlohon, M., Faloutsos, C.: Anomaly detection in large graphs. CMU-CS-09-173 Technical Report (2009)Google Scholar
  3. 3.
    Albert, R., Jeong, H., Barabasi, A.-L.: Diameter of the world wide web. Nature (401), 130–131 (1999)Google Scholar
  4. 4.
    Arning, A., Agrawal, R., Raghavan, P.: A linear method for deviation detection in large databases. In: KDD, pp. 164–169 (1996)Google Scholar
  5. 5.
    Barnett, V., Lewis, T.: Outliers in Statistical Data. John Wiley and Sons, Chichester (1994)MATHGoogle Scholar
  6. 6.
    Bay, S., Kumaraswamy, K., Anderle, M.G., Kumar, R., Steier, D.M.: Large scale detection of irregularities in accounting data. In: ICDM (2006)Google Scholar
  7. 7.
    Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: Lof: Identifying density-based local outliers. In: SIGMOD, pp. 93–104 (2000)Google Scholar
  8. 8.
    Chakrabarti, D.: AutoPart: Parameter-free graph partitioning and outlier detection. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS (LNAI), vol. 3202, pp. 112–124. Springer, Heidelberg (2004)Google Scholar
  9. 9.
    Chaoji, V., Hasan, M.A., Salem, S., Zaki, M.J.: Sparcl: Efficient and effective shape-based clustering. In: ICDM (2008)Google Scholar
  10. 10.
    Chau, D.H., Pandit, S., Faloutsos, C.: Detecting fraudulent personalities in networks of online auctioneers. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) PKDD 2006. LNCS (LNAI), vol. 4213, pp. 103–114. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Chaudhary, A., Szalay, A.S., Moore, A.W.: Very fast outlier detection in large multidimensional data sets. In: DMKD (2002)Google Scholar
  12. 12.
    Dasu, T., Johnson, T.: Exploratory Data Mining and Data Cleaning. Wiley Interscience, Hoboken (May 2003)MATHCrossRefGoogle Scholar
  13. 13.
    Eberle, W., Holder, L.B.: Discovering structural anomalies in graph-based data. In: ICDM Workshops, pp. 393–398 (2007)Google Scholar
  14. 14.
    Ghoting, A., Otey, M.E., Parthasarathy, S.: Loaded: Link-based outlier and anomaly detection in evolving data sets. In: ICDM (2004)Google Scholar
  15. 15.
    Ghoting, A., Parthasarathy, S., Otey, M.E.: Fast mining of distance-based outliers in high-dimensional datasets. Data Min. Knowl. Discov. 16(3), 349–364 (2008)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Hawkins, D.: Identification of outliers. Chapman and Hall, Boca Raton (1980)MATHGoogle Scholar
  17. 17.
    Hu, T., Sung, S.Y.: Detecting pattern-based outliers. Pattern Recognition Letters 24(16) (2003)Google Scholar
  18. 18.
    Jin, R., Wang, C., Polshakov, D., Parthasarathy, S., Agrawal, G.: Discovering frequent topological structures from graph datasets. In: KDD (2005)Google Scholar
  19. 19.
    Johnson, R.A., Wichern, D.W.: Applied Multivariate Statistical Analysis. Prentice Hall, Englewood Cliffs (1998)Google Scholar
  20. 20.
    Knorr, E.M., Ng, R.T.: Algorithms for mining distance-based outliers in large datasets. In: VLDB, pp. 392–403 (1998)Google Scholar
  21. 21.
    Leskovec, J., McGlohon, M., Faloutsos, C., Glance, N., Hurst, M.: Cascading behavior in large blog graphs: Patterns and a model. In: Society of Applied and Industrial Mathematics: Data Mining (2007)Google Scholar
  22. 22.
    Liu, C., Yan, X., Yu, H., Han, J., Yu, P.S.: Mining behavior graphs for ”backtrace” of noncrashing bugs. In: SDM (2005)Google Scholar
  23. 23.
    McGlohon, M., Akoglu, L., Faloutsos, C.: Weighted graphs and disconnected components: Patterns and a model. In: ACM SIGKDD (2008)Google Scholar
  24. 24.
    Moonesinghe, H.D.K., Tan, P.-N.: Outrank: a graph-based outlier detection framework using random walk. International Journal on Artificial Intelligence Tools 17(1) (2008)Google Scholar
  25. 25.
    Ng, R.T., Han, J.: Efficient and effective clustering methods for spatial data mining. In: VLDB, pp. 144–155 (1994)Google Scholar
  26. 26.
    Noble, C.C., Cook, D.J.: Graph-based anomaly detection. In: KDD, pp. 631–636 (2003)Google Scholar
  27. 27.
    Papadimitriou, S., Kitagawa, H., Gibbons, P.B., Faloutsos, C.: Loci: Fast outlier detection using the local correlation integral. In: ICDE (2003)Google Scholar
  28. 28.
    Sequeira, K., Zaki, M.J.: Admit: anomaly-based data mining for intrusions. In: KDD (2002)Google Scholar
  29. 29.
    Sun, J., Qu, H., Chakrabarti, D., Faloutsos, C.: Neighborhood formation and anomaly detection in bipartite graphs. In: ICDM (2005)Google Scholar
  30. 30.
    Yan, X., Han, J.: gspan: Graph-based substructure pattern mining. In: ICDM (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Leman Akoglu
    • 1
  • Mary McGlohon
    • 1
  • Christos Faloutsos
    • 1
  1. 1.School of Computer ScienceCarnegie Mellon University 

Personalised recommendations