Breaking reCAPTCHA: A Holistic Approach via Shape Recognition

  • Paul Baecher
  • Niklas Büscher
  • Marc Fischlin
  • Benjamin Milde
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 354)


CAPTCHAs are small puzzles which should be easily solvable by human beings but hard to solve for computers. They build a security cornerstone of the modern Internet service landscape, deployed in essentially any kind of login service, allowing to distinguish authorized human beings from automated attacks. One of the most popular and successful systems today is reCAPTCHA. As many other systems, reCAPTCHA is based on distorted images of words, where the distortion system evolves over time and determines different generations of the system. In this work, we analyze three recent generations of reCAPTCHA and present an algorithm that is capable of solving at least 5% of the challenges generated by these versions. We achieve this by applying a specialized variant of shape contexts proposed by Belongie et al. to match entire words at once. In order to handle the ellipse shaped distortions employed in one of the generations, we propose a machine learning algorithm that virtually eliminates the distortion. Finally, an improved shape matching strategy allows us to use word dictionaries of a reasonable size (with approximately 20,000 entries).


Holistic Approach Character Recognition Black Pixel Shape Context Entire Word 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: reCAPTCHA: Human-based character recognition via web security measures. Science 321(5895), 1465–1468 (2008) Cited on page 1MathSciNetCrossRefGoogle Scholar
  2. 2.
    Belongie, S., Malik, J., Puzicha, J.: Shape context: A new descriptor for shape matching and object recognition. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) NIPS, pp. 831–837. MIT Press, Cambridge (2000) Cited on pages 2 and 4Google Scholar
  3. 3.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679–698 (1986), Cited on page 6CrossRefGoogle Scholar
  4. 4.
    Chellapilla, K., Larson, K., Simard, P.Y., Czerwinski, M.: Building segmentation based human-friendly human interaction proofs (HIPs). In: Baird, H.S., Lopresti, D.P. (eds.) HIP 2005. LNCS, vol. 3517, pp. 1–26. Springer, Heidelberg (2005) Cited on page 4CrossRefGoogle Scholar
  5. 5.
    Chellapilla, K., Larson, K., Simard, P.Y., Czerwinski, M.: Computers beat humans at single character recognition in reading based human interaction proofs (HIPs). In: CEAS (2005) Cited on page 4Google Scholar
  6. 6.
    Govindaraju, V., Krishnamurthy, R.K.: Holistic handwritten word recognition using temporal features derived from off-line images. Pattern Recognition Letters 17(5), 537–540 (1996) Cited on page 5CrossRefGoogle Scholar
  7. 7.
    Houck, C.W.: Decoding recaptcha (2010), Cited on pages 3 and 6
  8. 8.
    Lavrenko, V., Rath, T.M., Manmatha, R.: Holistic word recognition for handwritten historical documents. In: DIAL, pp. 278–287. IEEE Computer Society Press, Los Alamitos (2004) Cited on page 5Google Scholar
  9. 9.
    Lladós, J., Roy, P.P., Rodríguez, J.A., Sánchez, G.: Word spotting in archive documents using shape contexts. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4478, pp. 290–297. Springer, Heidelberg (2007) Cited on page 4CrossRefGoogle Scholar
  10. 10.
    Madhvanath, S., Govindaraju, V.: Contour-based image preprocessing for holistic handwritten word recognition. In: ICDAR, pp. 536–539. IEEE Computer Society Press, Los Alamitos (1997) Cited on page 5Google Scholar
  11. 11.
    Madhvanath, S., Govindaraju, V.: The role of holistic paradigms in handwritten word recognition. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 149–164 (2001) Cited on page 5CrossRefGoogle Scholar
  12. 12.
    Mori, G., Belongie, S., Malik, J.: Shape contexts enable efficient retrieval of similar shapes. In: CVPR, vol. 1, pp. 723–730. IEEE Computer Society Press, Los Alamitos (2001) Cited on page 4Google Scholar
  13. 13.
    Mori, G., Belongie, S.J., Malik, J.: Efficient shape matching using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1832–1837 (2005) Cited on page 9Google Scholar
  14. 14.
    Mori, G., Malik, J.: Recognizing objects in adversarial clutter: Breaking a visual CAPTCHA. In: CVPR, vol. 1, pp. 134–144. IEEE Computer Society Press, Los Alamitos (2003) Cited on page 4Google Scholar
  15. 15.
    Vertanen, K.: Words in 10 lists (2010), Cited on page 10
  16. 16.
    Wilkins, J.: Strong CAPTCHA guidelines v1.2 (2009), Cited on page 3

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Paul Baecher
    • 1
  • Niklas Büscher
    • 1
  • Marc Fischlin
    • 1
  • Benjamin Milde
    • 1
  1. 1.Darmstadt University of TechnologyGermany

Personalised recommendations