Analysis and Detection of Web Spam by Means of Web Content

  • Víctor M. Prieto
  • Manuel Álvarez
  • Rafael López-García
  • Fidel Cacheda
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7356)


Web Spam is one of the main difficulties that crawlers have to overcome. According to Gyöngyi and Garcia-Molina it is defined as “any deliberate human action that is meant to trigger an unjustifiably favourable relevance or importance of some web pages considering the pages’ true value”. There are several studies on characterising and detecting Web Spam pages. However, none of them deals with all the possible kinds of Web Spam. This paper shows an analysis of different kinds of Web Spam pages and identifies new elements that characterise it. Taking them into account, we propose a new Web Spam detection system called SAAD, which is based on a set of heuristics and their use in a C4.5 classifier. Its results are also improved by means of Bagging and Boosting techniques. We have also tested our system in some well-known Web Spam datasets and we have found it to be very effective.


Web characterization Web Spam malware data mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amitay, E., Carmel, D., Darlow, A., Lempel, R., Soffer, A.: The connectivity sonar: detecting site functionality by structural patterns. In: Proceedings of the Fourteenth ACM Conference on Hypertext and Hypermedia, pp. 38–47. ACM Press (2003)Google Scholar
  2. 2.
    Benczur, A.A., Csalogany, K., Sarlos, T., Uher, M., Uher, M.: Spamrank - fully automatic link spam detection. In: Proceedings of the First International Workshop on Adversarial Information Retrieval on the Web (AIRWeb) (2005)Google Scholar
  3. 3.
    Breiman, L., Breiman, L.: Bagging predictors. In: Machine Learning, pp. 123–140 (1996)Google Scholar
  4. 4.
    Castillo, C., Davison, B.D.: Adversarial Web Search 4(5), 377–486 (2010)zbMATHGoogle Scholar
  5. 5.
    Chellapilla, K., Maykov, A.: A taxonomy of javascript redirection spam. In: Proceedings of the 3rd International Workshop on Adversarial Information Retrieval on the Web, AIRWeb 2007, pp. 81–88. ACM, New York (2007)CrossRefGoogle Scholar
  6. 6.
    Cova, M., Kruegel, C., Vigna, G.: Detection and analysis of drive-by-download attacks and malicious javascript code. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 281–290. ACM, New York (2010)CrossRefGoogle Scholar
  7. 7.
    Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51, 107–113 (2008)CrossRefGoogle Scholar
  8. 8.
    Fetterly, D., Manasse, M., Najork, M.: Spam, damn spam, and statistics: using statistical analysis to locate spam web pages. In: Proceedings of the 7th International Workshop on the Web and Databases: colocated with ACM SIGMOD/PODS, WebDB 2004, pp. 1–6. ACM, New York (2004)CrossRefGoogle Scholar
  9. 9.
    Fetterly, D., Manasse, M., Najork, M.: Detecting phrase-level duplication on the world wide web. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 170–177. ACM Press (2005)Google Scholar
  10. 10.
    Gonzalez Jesus, B.W., Cristina, A.: Implementacion y evaluacion de un detector masivo de web spam (2009)Google Scholar
  11. 11.
    Gyongyi, Z., Garcia-Molina, H.: Web spam taxonomy. Technical Report 2004-25, Stanford InfoLab (March 2004)Google Scholar
  12. 12.
    Gyöngyi, Z., Garcia-Molina, H.: Link spam alliances. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005, pp. 517–528. VLDB Endowment (2005)Google Scholar
  13. 13.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11, 10–18 (2009)CrossRefGoogle Scholar
  14. 14.
    Henzinger, M.R., Motwani, R., Silverstein, C.: Challenges in web search engines. SIGIR Forum 36, 11–22 (2002)CrossRefGoogle Scholar
  15. 15.
    Hidalgo, J.M.G.: Evaluating cost-sensitive unsolicited bulk email categorization (2002)Google Scholar
  16. 16.
    Jansen, B.J., Spink, A.: An analysis of web documents retrieved and viewed (2003)Google Scholar
  17. 17.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection, pp. 1137–1143. Morgan Kaufmann (1995)Google Scholar
  18. 18.
    Ntoulas, A., Manasse, M.: Detecting spam web pages through content analysis. In: Proceedings of the World Wide Web Conference, pp. 83–92. ACM Press (2006)Google Scholar
  19. 19.
    Quinlan, J.R.: Bagging, boosting, and c4.5. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725–730. AAAI Press (1996)Google Scholar
  20. 20.
    Quinlan, J.R.: Improved use of continuous attributes in c4.5. Journal of Artificial Intelligence Research 4, 77–90 (1996)zbMATHGoogle Scholar
  21. 21.
    Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)Google Scholar
  22. 22.
    Sahami, M., Dumais, S., Heckerman, D., Horvitz, E.: A bayesian approach to filtering junk e-mail (1998)Google Scholar
  23. 23.
    Webb, S.: Introducing the webb spam corpus: Using email spam to identify web spam automatically. In: Proceedings of the 3rd Conference on Email and AntiSpam (CEAS) (2006) (Mountain View)Google Scholar
  24. 24.
  25. 25.
    Wu, B., Davison, B.D.: Cloaking and redirection: A preliminary study (2005)Google Scholar
  26. 26.
    Wu, B., Davison, B.D.: Identifying link farm spam pages. In: Special Interest Tracks and Posters of the 14th International Conference on World Wide Web, WWW 2005, pp. 820–829. ACM, New York (2005)CrossRefGoogle Scholar
  27. 27.
    Yahoo!: Web spam challenge (2011),
  28. 28.
    Yahoo!: Web Spam Detection - Resources for Research on Web Spam (2011),
  29. 29.
    Zhang, H., Goel, A., Govindan, R., Mason, K., Van Roy, B.: Making Eigenvector-Based Reputation Systems Robust to Collusion. In: Leonardi, S. (ed.) WAW 2004. LNCS, vol. 3243, pp. 92–104. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Víctor M. Prieto
    • 1
  • Manuel Álvarez
    • 1
  • Rafael López-García
    • 1
  • Fidel Cacheda
    • 1
  1. 1.Department of Information and Communication TechnologiesUniversity of A CoruñaA CoruñaSpain

Personalised recommendations