Click Fraud Detection: Adversarial Pattern Recognition over 5 Years at Microsoft

  • Brendan KittsEmail author
  • Jing Ying Zhang
  • Gang Wu
  • Wesley Brandi
  • Julien Beasley
  • Kieran Morrill
  • John Ettedgui
  • Sid Siddhartha
  • Hong Yuan
  • Feng Gao
  • Peter Azo
  • Raj Mahato
Part of the Annals of Information Systems book series (AOIS, volume 17)


Microsoft adCenter is the third largest Search advertising platform in the United States behind Google and Yahoo, and services about 10 % of US traffic. At this scale of traffic approximately 1 billion events per hour, amounting to 2.3 billion ad dollars annually, need to be scored to determine if it is fraudulent or bot-generated [32, 37, 41]. In order to accomplish this, adCenter has developed arguably one of the largest data mining systems in the world to score traffic quality, and has employed them successfully over 5 years. The current paper describes the unique challenges posed by data mining at massive scale, the design choices and rationale behind the technologies to address the problem, and shows some examples and some quantitative results on the effectiveness of the system in combating click fraud.


Internet Protocol Filtration System Internet Protocol Address Fraud Detection Investigation Team 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We would like to thank Raj Mahato, Albert Roux, Ron Mills, Brandon Sobottka, Matthew Rice, Sasha Berger, Jigar Mody, Dennis Minium, Kamran Kanany, Tudor Trufinescu, Dinesh Chahlia, Ken Pierce, Hank Hoek, Tao Ma, Karl Reese, Narayanan Madhu, Dimitry Berger, Rageesh Maniyembath, Meena, Joseph Morrison, Kiran Vemulapalli, Anthony Crispo, Matthew Bisson, Igor Chepil, Matthew Ford, Sachin Ghani, Amjad Hussain, Steve Marlar, Bill Morency, Gerry Moses, Steve Sullivan and many others.


  1. 1.
    Boyd, C.: IE used to launch instant messaging and questionable clicks. (2006)
  2. 2.
    Buchanan, B., Shortliffe, E.: Rule-Based Expert Systems. Addison-Wesley, Reading (1984)Google Scholar
  3. 3.
    Buehrer, G., Stokes, J., Chellapilla, K.: A large-scale study of automated web search traffic. Proceedings of the Fourth International Workshop on Adversarial Information Retrieival on the Web (AIRWEB) (2008)Google Scholar
  4. 4.
    Buehrer, G., Stokes, J., Chellapilla, K., Platt, J.: Classification of automated search traffic. In: King, I., Baeza-Yates, R. (eds.) Weaving Services and People on the World Wide Web, pp. 3–26. Springer, Berlin (2008)Google Scholar
  5. 5.
    Bureau, I.A.: Iab/abce international spiders & bots list. (2010)
  6. 6.
    Claburn, T.: Microsoft sues three for click fraud. InformationWeek (June 2009)Google Scholar
  7. 7.
    Court, U.S.D.: Microsoft vs Eric Lam et. al. Civil Case Number CO 9-0815. (2009)
  8. 8.
    Daswani, N., Mysen, C., Rao, V., Weis, S., Gharachorloo, K., Ghosemajumder, S.: Online advertising fraud. In: Crimeware: understanding new attacks and defenses, Chap. 11. Symantec Press (2008)Google Scholar
  9. 9.
    Daswani, N., Stoppelman, M.: The anatomy of clickbot a. Usenix HotBots 2007 (2007)Google Scholar
  10. 10.
    Edelman, B.: The spyware—click-fraud connection—and yahoo’s role revisited. (2006)
  11. 11.
    Fielding, R. et al.: Hypertext transfer protocol – http/1.1. Tech. Rep. RFC 2616, Network Working Group (1999)Google Scholar
  12. 12.
    Gandhi, M., Jakobsson, M., Ratkiewicz, J.: Badvertisements: Stealthy click-fraud with unwitting accessories. In: Online Fraud, Part I J. Digital Forensic Pract., vol. 1, Special Issue 2 (2006)Google Scholar
  13. 13.
    Ghosemajumder, S.: Findings on invalid clicks. (2006)
  14. 14.
    Goodman, J.: Spam filtering: Text classification with an adversary (2003)Google Scholar
  15. 15.
    Google: Google ad traffic quality resource center. (2010)
  16. 16.
    Google: Google iab click measurement description of method. (2010)
  17. 17.
    Google: Google Form 10-Q Quarterly Report Pursuant to Section 13 or 15(d) of the Securities Exchange Act of 1934 (2012)Google Scholar
  18. 18.
    Jackson, C., Barth, A., Bortz, A., Shao, W., Boneh, D.: Protecting browsers from dns rebinding attacks. Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 421–431 (2007)Google Scholar
  19. 19.
    Jansen, B.: The comparative effectiveness of sponsored and non-sponsored results for web ecommerce queries. ACM Trans. Web 1 (2007)Google Scholar
  20. 20.
    Jansen, B., Flaherty, T., Baeza-Yates, R., Hunter, L., Kitts, B., Murphy, J.: The components and impact of sponsored search. Computer 42, 98–101 (2009)CrossRefGoogle Scholar
  21. 21.
    Kitts, B.: Regression trees (2000), unpublished manuscriptGoogle Scholar
  22. 22.
    Kitts, B.: Click fraud protector. US Patent Application, (2006)Google Scholar
  23. 23.
    Kitts, B.: Introducing adcenter clickids. /06/17/introducing-adcenter-clickids.aspx. (June 2009)
  24. 24.
    Kitts, B., Laxminarayan, P., LeBlanc, B.: Cooperative strategies for keyword auctions. First International Conference on Internet Technologies and Applications (2005)Google Scholar
  25. 25.
    Kitts, B., Laxminarayan, P., LeBlanc, B., Meech, R.: A formal analysis of search auctions including predictions on click fraud and bidding tactics. ACM Conference on E-Commerce Workshop on Sponsored Search (2005)Google Scholar
  26. 26.
    Kitts, B., LeBlanc, B.: Optimal bidding on keyword auctions. Electron. Markets Int. J. Electron. Comm. Bus. Media 14 (2004)Google Scholar
  27. 27.
    Kitts, B., LeBlanc, B., Laxminarayan, P.: Click fraud. American Society for Information Science and Technology Bulletin, pp. 20–23 (December 2006)Google Scholar
  28. 28.
    Kitts, B., Najm, T., Burdick, B.: Identifying automated click fraud programs. US Patent Application, (2006)Google Scholar
  29. 29.
    Leyden, J.: Botnet implicated in click fraud scam. (May 2006)
  30. 30.
    Leyden, J.: Click-fraud menace spreads using IM. (Oct 2006)
  31. 31.
    Microsoft: Microsoft adcenter click measurement description of method. ter_live_Std&querytype=topic&query=MOONSHOT_CONC_ClickMethod.htm (2009)
  32. 32.
    Microsoft: Microsoft Form 10-Q Quarterly Report Pursuant to Section 13 or 15(d) of the Security Exchange Act of 1934 (2010)Google Scholar
  33. 33.
    Mungamuru, B., Garcia-Molina, H.: Managing the quality of cpc traffic. Proceedings of the 10th ACM Conference on Electronic Commerce, pp. 215–224 (2008)Google Scholar
  34. 34.
    Mungamuru, B., Garcia-Molinja, H.: Predictive pricing and revenue sharing. Proceedings of the 4th International Workshop on Internet and Network Economics, pp. 53–60 (2008)Google Scholar
  35. 35.
    Mungamuru, B., Weis, S.: Competition and fraud in online advertising markets. In: Tsudik, G. (ed.) Financial Cryptography and Data Security, pp. 187–191. Springer, Berlin (2008)CrossRefGoogle Scholar
  36. 36.
    Mungamuru, B., Weis, S., Garcia-Molina, H.: Should ad networks bother fighting clickfraud (yes, they should.). Technical Report 2008-24, Stanford InfoLab (2008)Google Scholar
  37. 37.
  38. 38.
    Rey, B., Kannan, A.: Conversion rate based bid adjustment for sponsored search auctions. WWW. (April 2010)Google Scholar
  39. 39.
    Schonfeld, E.: The evolution of click fraud: massive Chinese operation DormRing1 uncovered. (2009)
  40. 40.
    Weinberg, N.: Google wins click-fraud case vs auction experts. e-vs-auction-experts. (July 2005)
  41. 41.
    Whitney, L.: Bing grabs 10 percent of search market. (Sept 2009)
  42. 42.
    Wikipedia: Cross-site request forgery. (2012)
  43. 43.
    Woolsey, B., Schulz, M.: Credit card statistics, industry facts, debt statistics. (2010)
  44. 44.
    Wu, G., Kitts, B.: Experimental comparison of scalable online ad serving. In: Fourteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1008–1015 (2008)Google Scholar
  45. 45.
    Yahoo: Yahoo search marketing click measurement guidelines description of method (2009)Google Scholar
  46. 46.
    Yahoo: Yahoo traffic quality center. (2010)

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Brendan Kitts
    • 1
    Email author
  • Jing Ying Zhang
    • 1
  • Gang Wu
    • 1
  • Wesley Brandi
    • 1
  • Julien Beasley
    • 1
  • Kieran Morrill
    • 1
  • John Ettedgui
    • 1
  • Sid Siddhartha
    • 1
  • Hong Yuan
    • 1
  • Feng Gao
    • 1
  • Peter Azo
    • 1
  • Raj Mahato
    • 1
  1. 1.Microsoft CorporationRedmondUSA

Personalised recommendations