Multimedia Tools and Applications

, Volume 76, Issue 4, pp 5117–5140 | Cite as

FATCHA: biometrics lends tools for CAPTCHAs

  • Maria De Marsico
  • Luca Marchionni
  • Andrea Novelli
  • Michael Oertel


This paper presents a novel strategy to implement a CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart). The aim of these tests is to easily and reliably distinguish between real human users and (malicious) bots. The approach underlying FATCHA is to exploit real time capture of human actions instead of human ability to recognize visual or auditory items. The latter approach explicitly requires proposing a challenge difficult for an automatic responder but easy for a human. However, it is often the case that pursuing the first feature takes to lose the second one. Moreover the user task may be hindered by specific disabilities. According to FATCHA approach the system rather asks the user to carry out some trivial gesture, e.g., rotating or moving the head. The webcam, which is available in almost all computers or mobile devices, captures the user gesture, and the server (hosting the service to protect) matches it with the requested one. It is possible to extend the service with a second module that allows the user to authenticate himself by face recognition instead of using a password. On the contrary, FATCHA gesture challenge can be used as a liveliness test to avoid biometric spoofing. Multimodal interaction is the base for both an advanced Human Interactive Proof (HIP) test and for robust/comfortable authentication.


Human Interactive Proofs CAPTCHA BOT Usability Accessibility Multimodal interaction Denial of service Human face detection 


  1. 1.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041CrossRefMATHGoogle Scholar
  2. 2.
    Almazyad AS, Ahmad Y, Kouchay SA (2011) Multi-modal captcha: a user verification scheme. In: International conference on information science and applications (ICISA), 2011, pp 1–7. IEEEGoogle Scholar
  3. 3.
    Bianchini CS, Borgia F, De Marsico M (2012) Swift-a signwriting editor to bridge between deaf world and e-learning. In: IEEE 12th international conference on advanced learning technologies (ICALT), 2012, pp 526–530. IEEEGoogle Scholar
  4. 4.
    Bursztein E, Bethard S, Fabry C, Mitchell JC, Jurafsky D (2010) How good are humans at solving captchas? A large scale evaluation. Accessed: 2015-12-04.
  5. 5.
    Bursztein E, Martin M, Mitchell J (2011) Text-based captcha strengths and weaknesses. In: Proceedings of the 18th ACM conference on computer and communications security, pp 125–138. ACMGoogle Scholar
  6. 6.
    Bushell D (2011) In search of the perfect CAPTCHA. Accessed: 2015-12-04.
  7. 7.
    Datta R, Li J, Wang JZ (2005) Imagination: a robust image-based captcha generation system. In: Proceedings of the 13th annual ACM international conference on multimedia, pp 331– 334. ACMGoogle Scholar
  8. 8.
    De Marsico M, Marchionni L, Novelli A, Oertel M (2015) Fatcha: the captcha are you!. In: Proceedings of the 11th biannual conference on italian SIGCHI chapter, pp 118–125. ACMGoogle Scholar
  9. 9.
    Gossweiler R, Kamvar M, Baluja S (2009) What’s up captcha?: a captcha based on image orientation. In: Proceedings of the 18th international conference on world wide web, pp 841–850. ACMGoogle Scholar
  10. 10.
    Goswami G, Singh R, Vatsa M, Powell B, Noore A (2012) Face recognition captcha. In: IEEE fifth international conference on biometrics: theory, applications and systems (BTAS), 2012, pp 412–417. IEEEGoogle Scholar
  11. 11.
    Hernandez-Castro CJ, Ribagorda A, Saez Y (2010) Side-channel attack on the humanauth captcha. In: Proceedings of the 2010 international conference on security and cryptography (SECRYPT), pp 1–7. IEEEGoogle Scholar
  12. 12.
    May M (2005) Inaccessibility of captcha. Alternatives to visual turing tests on the web. I: W3C (red.), W3C Working Group Note, work in progressGoogle Scholar
  13. 13.
    Misra D, Gaj K (2006) Face recognition captchas. In: International conference on internet and web applications and services/advanced international conference on telecommunications, 2006. AICT-ICIW’06, pp 122–122. IEEEGoogle Scholar
  14. 14.
    Mori G, Malik J (2003) Recognizing objects in adversarial clutter: Breaking a visual captcha. In: IEEE computer society conference on computer vision and pattern recognition, 2003. Proceedings. 2003, vol 1, pp i–134. IEEEGoogle Scholar
  15. 15.
    Pantic M, Rothkrantz LJ (2004) Facial action recognition for facial expression analysis from static face images. IEEE Trans Syst Man Cybern Part B Cybern 34(3):1449–1461CrossRefGoogle Scholar
  16. 16.
    Petrie H, Bevan N (2009) The evaluation of accessibility, usability and user experience. The universal access handbook, pp 10–20Google Scholar
  17. 17.
    Poh N, Blanco-Gonzalo R, Wong R, Sanchez-Reillo R (2016) Blind subjects faces database. IET Biom 5(1):20–27CrossRefGoogle Scholar
  18. 18.
    Power C, Freire A, Petrie H, Swallow D (2012) Guidelines are only half of the story: accessibility problems encountered by blind users on the web. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 433–442. ACMGoogle Scholar
  19. 19.
    Rathgeb C, Uhl A (2011) A survey on biometric cryptosystems and cancelable biometrics. EURASIP J Inf Secur 2011(1):1–25CrossRefGoogle Scholar
  20. 20.
    Roshanbin N, Miller J (2013) A survey and analysis of current captcha approaches. Journal of Web Engineering 12(1–2):1–40Google Scholar
  21. 21.
    Rui Y, Liu Z (2004) Artifacial: Automated reverse turing test using facial features. Multimedia Systems 9(6):493–502CrossRefGoogle Scholar
  22. 22.
    Schapire RE (2003) The boosting approach to machine learning: an overview. In: Nonlinear estimation and classification, pp 149–171. SpringerGoogle Scholar
  23. 23.
    Shirali-Shahreza M, Shirali-Shahreza S (2007) Captcha for blind people. In: IEEE international symposium on signal processing and information technology, 2007, pp 995–998. IEEEGoogle Scholar
  24. 24.
    Shirali-Shahreza M, Shirali-Shahreza S (2008) Motion captcha. In: Conference on human system interactions, 2008, pp 1042–1044. IEEEGoogle Scholar
  25. 25.
    Shirali-Shahreza MH, Shirali-Shahreza M (2007) Localized captcha for illiterate people. In: International conference on intelligent and advanced systems, 2007. ICIAS 2007, pp 675–679. IEEEGoogle Scholar
  26. 26.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1, pp i–511. IEEEGoogle Scholar
  27. 27.
    Von Ahn L, Maurer B, McMillen C, Abraham D, Blum M (2008) Recaptcha: human-based character recognition via web security measures. Science 321(5895):1465–1468MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    W3C - Web Accessibility Initiative: Captcha alternatives and thoughts. (2015). Last Modified: 2015-08-28; Accessed: 2015-12-04
  29. 29.
    Weinland D, Ronfard R, Boyer E (2011) A survey of vision-based methods for action representation, segmentation and recognition. Comput Vis Image Underst 115(2):224–241CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Maria De Marsico
    • 1
  • Luca Marchionni
    • 1
  • Andrea Novelli
    • 1
  • Michael Oertel
    • 1
  1. 1.Sapienza University of RomeRomeItaly

Personalised recommendations