Web spam denotes the manipulation of web pages with the sole intent to raise their position in search engine rankings. Since a better position in the rankings directly and positively affects the number of visits to a site, attackers use different techniques to boost their pages to higher ranks. In the best case, web spam pages are a nuisance that provide undeserved advertisement revenues to the page owners. In the worst case, these pages pose a threat to Internet users by hosting malicious content and launching drive-by attacks against unsuspecting victims. When successful, these drive-by attacks then install malware on the victims’ machines. In this paper, we introduce an approach to detect web spam pages in the list of results that are returned by a search engine. In a first step, we determine the importance of different page features to the ranking in search engine results. Based on this information, we develop a classification technique that uses important features to successfully distinguish spam sites from legitimate entries. By removing spam sites from the results, more slots are available to links that point to pages with useful content. Additionally, and more importantly, the threat posed by malicious web sites can be mitigated, reducing the risk for users to get infected by malicious code that spreads via drive-by attacks.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Atkeson C.G., Moore A.W., Schaal S.: Locally weighted learning. Artif. Intell. Rev. 11(1–5), 11–73 (1997)
Bifet, A., Castillo, C., Chirita, P.-A., Weber, I.: An analysis of factors used in search engine ranking. In: Adversarial Information Retrieval on the Web (2005)
Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. In: 7th International World Wide Web Conference (WWW) (1998)
Cacheda, F., Viña, Á.: Experiencies retrieving information in the world wide web. In: Proceedings of the Sixth IEEE Symposium on Computers and Communications (ISCC 2001), pp. 72–79 (2001)
Chellapilla, K., Chickering, D.: Improving cloaking detection using search query popularity and monetizability. In: Adversarial Information Retrieval on the Web (2006)
Evans M.P.: Analysing Google rankings through search engine optimization data. Internet Res. 17(1), 21–37 (2007)
Fetterly, D., Manasse, M., Najork, M.: Spam, damn spam, and statistics: using statistical analysis to locate spam web pages. In: WebDB, pp. 1–6 (2004)
Google. Zeitgeist: Search patterns, trends, and surprises. http://www.google.com/press/zeitgeist.html. Accessed 29 June 2009
Google Keeps Tweaking Its Search Engine. http://www.nytimes.com/2007/06/03/business/yourmoney/03google.html?page%wanted=4&_r=1. Accessed 29 June 2009
Gyöngyi, Z., Garcia-Molina, H.: Web spam taxonomy. In: Adversarial Information Retrieval on the Web (2005)
Hearst M.A.: Support vector machines. IEEE Intell. Syst. 13(4), 18–28 (1998)
Heckerman, D.: A tutorial on learning with bayesian networks. Technical report, Microsoft Research (1995)
John, G.H., Langley, P.: Estimating continuous distributions in bayesian classifiers. In: UAI ’95: Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence, August 18–20, 1995, Montreal, Quebec, Canada, pp. 338–345 (1995)
Kaburlasos V.G., Athanasiadis I.N., Mitkas P.A.: Fuzzy lattice reasoning (flr) classifier and its application for ambient ozone estimation. Int. J. Approx. Reason. 45(1), 152–188 (2007)
Karlberger, C., Bayler, G., Kruegel, C., Kirda, E.: Exploiting redundancy in natural language to penetrate bayesian spam filters. In: First USENIX Workshop on Offensive Technologies (WOOT07) (2007)
MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
Niu, Y., Wang, Y.-M., Chen, H., Ma, M., Hsu, F.: A quantitative study of forum spamming using context-based analysis. In: NDSS (2007)
Ntoulas, A., Najork, M., Manasse, M., Fetterly, D.: Detecting spam web pages through content analysis. In: 15th International World Wide Web Conference (WWW) (2006)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in kernel methods: support vector learning, pp. 185–208. MIT Press, Cambridge. ISBN:0-262-19416-3 (1999)
Provos, N., Mavrommatis, P., Rajab, M.A., Monrose, F.: All your iframes point to us. In: 17th USENIX Security Symposium (2008)
Provos, N., McNamee, D., Mavrommatis, P., Wang, K., Modadugu, N.: The ghost in the browser analysis of web-based malware. In: First Workshop on Hot Topics in Understanding Botnets (HotBots ’07) (2007)
Quinlan, R.: C4.5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)
Rahul Mohandas (McAfee Avert Labs). Analysis of Adversarial Code: The role of Malware Kits! http://clubhack.com/2007/files/Rahul-Analysis_of_Adversarial_Code.pdf, December 2007. Accessed 29 June 2009
Google Search Engine Ranking Factors. http://www.seomoz.org/article/search-ranking-factors. Accessed 29 June 2009
Shi, H.: Best-first decision tree learning. Master’s thesis, University of Waikato, Hamilton, NZ, COMP594 (2007)
Svore, K., Wu, Q., Burges, C., Raman, A.: Improving web spam classification using rank-time features. In: Adversarial Information Retrieval on the Web (2007)
Wang, Y.-M., Ma, M., Niu, Y., Chen, H.: Spam double-funnel: connecting web spammers with advertisers. In: 16th International Conference on World Wide Web (2007)
Witten, I., Frank, E.: Data mining: practical machine learning tools and techniques 2nd edn. Morgan Kaufmann, San Francisco (2005)
Wu, B., Davison, B.: Cloaking and redirection: a preliminary study. In: Adversarial Information Retrieval on the Web (2005)
Wu, B., Davison, B.D.: Identifying link farm spam pages. In: 14th International World Wide Web Conference (WWW) (2005)
About this article
Cite this article
Egele, M., Kolbitsch, C. & Platzer, C. Removing web spam links from search engine results. J Comput Virol 7, 51–62 (2011). https://doi.org/10.1007/s11416-009-0132-6
- Search Engine
- Query Term
- Ranking Algorithm
- Anchor Text
- Test Page