Evaluating the Wisdom of Crowds in Assessing Phishing Websites

  • Tyler Moore
  • Richard Clayton
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5143)


We examine the structure and outcomes of user participation in PhishTank, a phishing-report collator. Anyone who wishes may submit URLs of suspected phishing websites, and may vote on the accuracy of other submissions. We find that PhishTank is dominated by its most active users, and that participation follows a power-law distribution, and that this makes it particularly susceptible to manipulation. We compare PhishTank with a proprietary source of reports, finding PhishTank to be slightly less complete and significantly slower in reaching decisions. We also evaluate the accuracy of PhishTank’s decisions and discuss cases where incorrect information has propagated. We find that users who participate less often are far more likely to make mistakes, and furthermore that users who commit many errors tend to have voted on the same URLs. Finally, we explain how the structure of participation in PhishTank leaves it susceptible to large-scale voting fraud which could undermine its credibility. We also discuss general lessons for leveraging the ‘wisdom of crowds’ in taking security decisions by mass participation.


Cumulative Distribution Function Security Mechanism User Participation Inexperienced User Invalid Report 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adar, E., Zhang, L., Adamic, L., Lukose, R.: Implicit structure and the dynamics of blogspace. In: Workshop on the Weblogging Ecosystem, 13th International World Wide Web Conference (WWW) (2004)Google Scholar
  2. 2.
    Albert, R., Jeong, H., Barabási, A.: Error and attack tolerance of complex networks. Nature 406, 378–382 (2000)CrossRefGoogle Scholar
  3. 3.
    Anderson, L., Holt, C.: Information cascades in the laboratory. American Economic Review 87(5), 847–862 (1995)Google Scholar
  4. 4.
    Blaze, M., Feigenbaum, J., Lacy, J.: Decentralized trust management. In: IEEE Symposium on Security and Privacy (S&P), pp. 164–173. IEEE Computer Society, Los Alamitos (1996)Google Scholar
  5. 5.
    Camp, L.J.: Reliable, usable signaling to defeat masquerade attacks. In: Fifth Workshop on the Economics of Information Security (WEIS) (2006)Google Scholar
  6. 6.
    Denning, P., Horning, J., Parnas, D., Weinstein, L.: Wikipedia risks. Communications of the ACM 48(12), 152 (2005)CrossRefGoogle Scholar
  7. 7.
    Douceur, J.R.: The Sybil Attack. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, pp. 251–260. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Hwang, T.: Herdict: a distributed model for threats online. In: Bradbury, D. (ed.) Network Security, pp. 15–18. Elsevier, Oxford (2007)Google Scholar
  9. 9.
    Kahney, L.: Cheaters bow to peer pressure. Wired (February 15, 2001),,1282,41838,00.html
  10. 10.
    Kamvar, S., Schlosser, M., Garcia-Molina, H.: The EigenTrust algorithm for reputation management in P2P networks. In: 12th WWW, pp. 640–651. ACM Press, New York (2003)CrossRefGoogle Scholar
  11. 11.
    Lamport, L., Shostak, R., Pease, M.: The Byzantine Generals Problem. ACM Transactions on Programming Languages and Systems 4(3), 382–401 (1982)zbMATHCrossRefGoogle Scholar
  12. 12.
    Larkin, E.: Online thugs assault sites that specialize in security help. PC World (September 11, 2007),
  13. 13.
    Levien, R.: Attack resistant trust metrics. PhD thesis (draft), University of California at Berkeley (2004)Google Scholar
  14. 14.
    McMillan, R.: ‘Rock Phish’ blamed for surge in phishing. InfoWorld (December 12, 2006),
  15. 15.
    Moore, T., Clayton, R.: Examining the impact of website take-down on phishing. In: Anti-Phishing Working Group eCrime Researcher’s Summit (APWG eCrime), pp. 1–13. ACM Press, New York (2007)CrossRefGoogle Scholar
  16. 16.
    Newman, M.: Power laws, Pareto distributions and Zipf’s law. Contemporary Physics 46(5), 323–351 (2005)CrossRefGoogle Scholar
  17. 17.
    Pan, Y., Ding, X.: Anomaly based web phishing page detection. In: 22nd Annual Computer Security Applications Conference (ACSAC 2006), pp. 381–392. IEEE Computer Society, Los Alamitos (2006)CrossRefGoogle Scholar
  18. 18.
  19. 19.
    Reiter, M., Stubblebine, S.: Toward acceptable metrics of authentication. In: IEEE S&P, pp. 10–20. IEEE Computer Society, Los Alamitos (1997)Google Scholar
  20. 20.
    Ross, B., Jackson, C., Miyake, N., Boneh, D., Mitchell, J.: Stronger password authentication using browser extensions. In: 14th USENIX Security Symposium, USENIX Association, Berkeley, p. 2 (2005)Google Scholar
  21. 21.
    Shi, X., Tseng, B., Adamic, L.: Looking at the blogosphere topology through different lenses. In: International Conference on Weblogs and Social Media (2007)Google Scholar
  22. 22.
    Shirky, C.: Power laws, weblogs, and inequality (2003),
  23. 23.
  24. 24.
  25. 25.
    Surowiecki, J.: The wisdom of crowds: why the many are smarter than the few. Doubleday, New York (2004)Google Scholar
  26. 26.
    Weaver, R., Collins, M.: Fishing for phishes: applying capture-recapture to phishing. In: APWG eCrime, pp. 14–25. ACM Press, New York (2007)CrossRefGoogle Scholar
  27. 27.
    Zhang, J., Ackerman, M., Adamic, L.: Expertise networks in online communities: structure and algorithms. In: 16th WWW, pp. 221–230. ACM Press, New York (2007)Google Scholar
  28. 28.
    Zhang, Y., Egelman, S., Cranor, L., Hong, J.: Phinding phish: evaluating anti-phishing tools. In: 14th Annual Network & Distributed System Security Symposium (NDSS 2007) (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Tyler Moore
    • 1
  • Richard Clayton
    • 1
  1. 1.Computer LaboratoryUniversity of CambridgeCambridgeUnited Kingdom

Personalised recommendations