Web graph similarity for anomaly detection

  • Panagiotis Papadimitriou
  • Ali Dasdan
  • Hector Garcia-Molina
Open Access
Original Paper

Abstract

Web graphs are approximate snapshots of the web, created by search engines. They are essential to monitor the evolution of the web and to compute global properties like PageRank values of web pages. Their continuous monitoring requires a notion of graph similarity to help measure the amount and significance of changes in the evolving web. As a result, these measurements provide means to validate how well search engines acquire content from the web. In this paper, we propose five similarity schemes: three of them we adapted from existing graph similarity measures, and two we adapted from well-known document and vector similarity methods (namely, the shingling method and random projection based method). We empirically evaluate and compare all five schemes using a sequence of web graphs from Yahoo!, and study if the schemes can identify anomalies that may occur due to hardware or other problems.

Keywords

Anomaly detection Graph similarity Locality sensitive hashing 

References

  1. 1.
    Becchetti L, Castillo C (2006) The distribution of PageRank follows a power-law only for particular values of the damping factor. In: WWW. ACM, New York, pp 941–942CrossRefGoogle Scholar
  2. 2.
    Blondel V, Gajardo A, Heymans M, Senellart P, Dooren PV (2004) A measure of similarity between graph vertices: applications to synonym extraction and web searching. SIAM Rev 46(4):647–666MATHMathSciNetCrossRefGoogle Scholar
  3. 3.
    Boncz P, Kersten M (1999) MIL primitives for querying a fragmented world. VLDB J 8(2):101–119CrossRefGoogle Scholar
  4. 4.
    Borodin A, Roberts GO, Rosenthal JS, Tsaparas P (2005) Link analysis ranking: algorithms, theory, and experiments. ACM Trans Internet Technol 5(1):231–297CrossRefGoogle Scholar
  5. 5.
    Broder A, Glassman S, Manasse M, Zweig G (1997) Syntactic clustering of the web. In: WWW, pp 393–404Google Scholar
  6. 6.
    Bunke H, Dickinson PJ, Kraetzl M, Wallis WD (2007) A graph-theoretic approach to enterprise network dynamics. Birkhäuser, BostonMATHGoogle Scholar
  7. 7.
    Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, Chandra T, Fikes A, Gruber RE (2006) Bigtable: a distributed storage system for structured data. In: OSDI. ACM, New York, pp 205–218Google Scholar
  8. 8.
    Charikar M (2002) Similarity estimation techniques from rounding algorithms. In: STOC. ACM, New York, pp 380–388Google Scholar
  9. 9.
    D’Alberto P, Dasdan A (2009) Non-parametric information-theoretic measures of one-dimensional distribution functions from continuous time series. In: SDM, pp 685–696Google Scholar
  10. 10.
    Dasdan A, Papadimitriou P (2007) Methods and apparatus for computing graph similarity via sequence similarity. US Patent Application, No 20090164411Google Scholar
  11. 11.
    Dasdan A, Papadimitriou P (2007) Methods and apparatus for computing graph similarity via signature similarity, US Patent Application, No 20090150381Google Scholar
  12. 12.
    Eiron N, McCurley KS, Tomlin JA (2004) Ranking the web frontier. In: WWW. ACM, New York, pp 309–318Google Scholar
  13. 13.
    Fagin R, Kumar R, Sivakumar D (2003) Comparing top klists. SIAM J Discrete Math 17(1):134–160MATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    Fan J, Yao Q (2005) Nonlinear time series: nonparametric and parametric methods. Springer series in statistics, 2nd edn. Springer, New YorkGoogle Scholar
  15. 15.
    Gibson D, Kumar R, Tomkins A (2005) Discovering large dense subgraphs in massive graphs. In: VLDB. ACM, New York, pp 721–732Google Scholar
  16. 16.
    Henzinger MR. Finding near-duplicate web pages: a large-scale evaluation of algorithms. In: SIGIRGoogle Scholar
  17. 17.
    Jeh G, Widom J (2002) SimRank: a measure of structural-context similarity. In: KDD, CanadaGoogle Scholar
  18. 18.
    Melnik S, Garcia-Molina H, Rahm E (2002) Similarity flooding: a versatile graph matching algorithm and its applications to schema matching. In: ICDEGoogle Scholar
  19. 19.
    Papadimitriou P, Dasdan A, Garcia-Molina H (2008) Web graph similarity for anomaly detection. Tech Rep 2008-1, Stanford UnivGoogle Scholar
  20. 20.
    Papadopoulos A, Manolopoulos Y (1999) Structure-based similarity search with graph histograms. In: DEXA/IWOSS. IEEE, New York, pp 174–178Google Scholar
  21. 21.
    Robles-Kelley A, Hancock ER (2005) Graph edit distance from spectral seriation. IEEE Trans Pattern Anal Mach Intell 27(3):365–378CrossRefGoogle Scholar
  22. 22.
    Seidl T. References for graph similarity. URL: http://www.dbs.informatik.uni-muenchen.de/~seidl/graphs/
  23. 23.
    Xue GR, Yang Q, Zeng HJ, Yu Y, Chen Z (2005) Exploiting the hierarchical structure for link analysis. In: SIGIR, New York, NY, USA, pp 186–193Google Scholar
  24. 24.
    Zager L, Verghese G (2005) Graph similarity. URL: http://lees.mit.edu/lees/presentations/LauraZager.pdf
  25. 25.
    Zhu P, Wilson RC (2005) A study of graph spectra for comparing graphs. In: MBVCGoogle Scholar

Copyright information

© The Brazilian Computer Society 2010

Authors and Affiliations

  • Panagiotis Papadimitriou
    • 1
  • Ali Dasdan
    • 2
  • Hector Garcia-Molina
    • 1
  1. 1.Stanford UniversityStanfordUSA
  2. 2.Yahoo! Inc.SunnyvaleUSA

Personalised recommendations