Mingling of Clear and Muddy Water: Understanding and Detecting Semantic Confusion in Blackhat SEO

Abstract

Search Engine Optimization (SEO) is a set of techniques that help website operators increase the visibility of their webpages to search engine users. However, there are also many unethical practices that abuse ranking algorithms of a search engine to promote illegal online content, called blackhat SEO. In this paper, we make the first attempt to systematically investigate a recent trend in blackhat SEO, semantic confusion, which mingles the content of a webpage to deceive existing detection of blackhat SEO. In particular, from a new perspective of content semantics, we propose an effective defense against the semantic confusion based blackhat SEO. We built a prototype of our defense called SCDS, and then we validated its effectiveness based on 4.5 million domains randomly selected from 11 zone files and passive DNS records. Our evaluation results show that SCDS can detect more than 82 thousand blackhat SEO websites with a precision of 98.35%. We further analyzed 57,477 long-tail keywords promoted by blackhat SEO and found more than 157 SEO campaigns. Finally, we deployed SCDS into the gateway of a campus network for ten months and detected 23,093 domains with malicious semantic confusion content, showing the effectiveness of SCDS in practice.

Appendix

A Practices of Semantic Confusion

We further analyzed the method of embedding mingled semantics in SEO pages and identified three main categories: (1) Modify only the $\texttt {<\!title\!>}$ tag, and keep all other parts the same. This is because search engines often pay more attention to the $\texttt {<\!title\!>}$ tag and the text in the title has a higher probability to be indexed. Also, blackhat SEOers do not want to be detected due to the modification of the pages or the mixed illegal content. (2) Embed the same promotion keywords into different paragraphs repeatedly. Appropriate repeats are helpful for search engines to extract keywords and give them a higher rank. These two methods mainly target search engines. (3) Replace the total paragraph with promotion content. This category mainly targets visitors and aims to attract them immediately upon arrival at the webpage. The replaced content is short-lived because it is easily noticed by search engines and webmasters.

B Keyword Promotion in Other Platforms

GitHub. In our study, we found that some blackhat SEOers are promoting GitHub repositories of gambling software development services. Specifically, when we searched “github.com+[gambling keywords]” in Google, the results show many GitHub repositories introducing gambling software, and the descriptions of these repositories include the developer’s contact information (e.g., phone numbers and IM IDs). Another practice is to place a large number of gambling keywords in a repository’s introduction through which search engines can index them. For example, Fig. 12 shows the search results of a gambling keyword “ ” (a dark jargon that means “Live Dealer Casino Games” in underground gambling business) with “github.com” in Google. The top results are mostly GitHub repositories that promote gambling development services.

Fig. 12.

Gambling software development in GitHub.

Fig. 13.

Gambling shop in JingDong (jd.com).

E-Commerce. We also found that the keywords promoted by blackhat SEO pages were not only used in search engines, but they also appeared on E-Commerce websites. For example, when we searched the most frequent keyword, pk10 on jd.com (a well-known E-Commerce website in China), there were shops that provide illegal gambling software development services, as shown in Fig. 13. Therefore, we recommend that E-Commerce websites should also pay attention to the identification and purification of such activities related to the underground business in search results.