Evaluation Methodologies for Biometric Presentation Attack Detection

  • Ivana Chingovska
  • Amir MohammadiEmail author
  • André Anjos
  • Sébastien Marcel
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


Presentation attack detection (PAD, also known as anti-spoofing) systems, regardless of the technique, biometric mode or degree of independence of external equipment, are most commonly treated as binary classification systems. The two classes that they differentiate are bona-fide and presentation attack samples. From this perspective, their evaluation is equivalent to the established evaluation standards for the binary classification systems. However, PAD systems are designed to operate in conjunction with recognition systems and as such can affect their performance. From the point of view of a recognition system, the presentation attacks are a separate class that need to be detected and rejected. As the problem of presentation attack detection grows to this pseudo-ternary status, the evaluation methodologies for the recognition systems need to be revised and updated. Consequentially, the database requirements for presentation attack databases become more specific. The focus of this chapter is the task of biometric verification and its scope is three-fold: first, it gives the definition of the presentation attack detection problem from the two perspectives. Second, it states the database requirements for a fair and unbiased evaluation. Finally, it gives an overview of the existing evaluation techniques for presentation attacks detection systems and verification systems under presentation attacks.



The authors would like to thank the projects BEAT ( and TABULA RASA ( both funded under the 7th Framework Programme of the European Union (EU) (grant agreement number 284989 and 257289) respectively. The revision of this chapter was supported under the project on Secure Access Control over Wide Area Networks (SWAN) funded by the Research Council of Norway (grant no. IKTPLUSS 248030/O70). and by the Swiss Center for Biometrics Research and Testing.


  1. 1.
    Information technology – Biometric presentation attack detection – Part 3: Testing and reporting. Standard, International Organization for Standardization, Geneva, CH (2017).
  2. 2.
    Rodrigues RN, Ling LL, Govindaraju V (2009) Robustness of multimodal biometric fusion methods against spoofing attacks. J Vis Lang Comput 20(3):169–179Google Scholar
  3. 3.
    Johnson PA, Tan B, Schuckers S (2010) Multimodal fusion vulnerability to non-zero (spoof) imposters. In: IEEE international workshop on information forensics and securityGoogle Scholar
  4. 4.
    Akhtar Z, Fumera G, Marcialis GL, Roli F. Robustness evaluation of biometric systems under spoof attacks. In: 16th international conference on image analysis and processing, pp 159–168Google Scholar
  5. 5.
    Akhtar Z, Fumera G, Marcialis GL, Roli F. Robustness analysis of likelihood ration score fusion rule for multi-modal biometric systems under spoof attacks. In: 45th IEEE international carnahan conference on security technology, pp 237–244Google Scholar
  6. 6.
    Akhtar Z, Fumera G, Marcialis GL, Roli F (2012) Evaluation of serial and parallel multibiometric systems under spoofing attacks. In: 5th IEEE international conference on biometrics: theory, applications and systemsGoogle Scholar
  7. 7.
    Villalba J, Lleida E (2011) Preventing replay attacks on speaker verification systems. In: 2011 IEEE international carnahan conference on security technology (ICCST), pp 1–8Google Scholar
  8. 8.
    Marasco E, Johnson P, Sansone C, Schuckers S (2011) Increase the security of multibiometric systems by incorporating a spoofing detection algorithm in the fusion mechanism. In: Proceedings of the 10th international conference on Multiple classifier systems, pp 309–318CrossRefGoogle Scholar
  9. 9.
    Marasco E, Ding Y, Ross A (2012) Combining match scores with liveness values in a fingerprint verification system. In: 5th IEEE international conference on biometrics: theory, applications and systemsGoogle Scholar
  10. 10.
    Chingovska I, Anjos A, Marcel S (2013) Anti-spoofing in action: joint operation with a verification system. In: Proceedings of IEEE conference on computer vision and pattern recognition, workshop on biometricsGoogle Scholar
  11. 11.
    Pan G, Sun L, Wu Z, Lao S (2007) Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: IEEE 11th international conference on computer vision. ICCV 2007, pp 1–8Google Scholar
  12. 12.
    Bao W, Li H, Li N, Jiang W (2009) A liveness detection method for face recognition based on optical flow field. In: International conference on image analysis and signal processing. IASP 2009, pp. 233–236Google Scholar
  13. 13.
    yan J, Zhang Z, Lei Z, Yi D, Li SZ (2012) Face liveness detection by exploring multiple scenic clues. In: 12th international conference on control, automation, robotics and vision (ICARCV 2012), ChinaGoogle Scholar
  14. 14.
    Tan X, Li Y, Liu J, Jiang L (2010) Face liveness detection from a single image with sparse low rank bilinear discriminative model. In: Proceedings of the european conference on computer vision (ECCV), LNCS 6316. Springer, Berlin, pp 504–517CrossRefGoogle Scholar
  15. 15.
    Galbally J, Alonso-Fernandez F, Fierrez J, Ortega-Garcia J (2012) A high performance fingerprint liveness detection method based on quality related features. Future Gener Comput Syst 28(1):311–321CrossRefGoogle Scholar
  16. 16.
    Yambay D, Ghiani L, Denti P, Marcialis G, Roli F, Schuckers S (2012) LivDet 2011 - fingerprint liveness detection competition 2011. In: 2012 5th IAPR international conference on biometrics (ICB), pp 208–215Google Scholar
  17. 17.
    Mansfield AJ, Wayman JL, Dr A, Rayner D, Wayman JL (2002) Best practices in testing and reporting performanceGoogle Scholar
  18. 18.
    Galbally-Herrero J, Fierrez-Aguilar J, Rodriguez-Gonzalez JD, Alonso-Fernandez F, Ortega-Garcia J, Tapiador M (006) On the vulnerability of fingerprint verification systems to fake fingerprints attacks. In: IEEE international carnahan conference on security technology, pp 169–179Google Scholar
  19. 19.
    Ruiz-Albacete V, Tome-Gonzalez P, Alonso-Fernandez F, Galbally J, Fierrez J, Ortega-Garcia J (2008) Direct attacks using fake images in iris verification. In: Proceedings of the COST 2101 workshop on biometrics and identity management, BIOID, Springer, Berlin, pp 181–190Google Scholar
  20. 20.
    Galbally J, Fierrez J, Alonso-Fernandez F, Martinez-Diaz M (2011) Evaluation of direct attacks to fingerprint verification systems. Telecommun Syst, Spec Issue Biom 47(3):243–254CrossRefGoogle Scholar
  21. 21.
    Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning: data mining, inference, and prediction: with 200 full-color illustrations. Springer, New YorkCrossRefGoogle Scholar
  22. 22.
    Lui YM, Bolme D, Phillips P, Beveridge J, Draper B (2012) Preliminary studies on the good, the bad, and the ugly face recognition challenge problem. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 9–16Google Scholar
  23. 23.
    Marcialis GL, Lewicke A, Tan B, Coli P, Grimberg D, Congiu A, Tidu A, Roli F, Schuckers S (2009) First international fingerprint liveness detection competition – livdet 2009. In: Proceedings of the IAPR International Conference on Image Analysis and Processing (ICIAP), LNCS-5716, pp 12–23 (2009)CrossRefGoogle Scholar
  24. 24.
    Ghiani L, Yambay D, Mura V, Tocco S, Marcialis G, Roli F, Schuckers S (2013) Livdet 2013 - fingerprint liveness detection competition. In: IEEE international conference on biometrics (ICB)Google Scholar
  25. 25.
    Zhiwei Z, Yan J, Liu S, Lei Z, Yi D, Li SZ (2012) A face antispoofing database with diverse attacks. In: Proceedings of the IAPR international conference on biometrics (ICB), pp 26–31Google Scholar
  26. 26.
    Wen D, Han H, Jain AK (2015) Face spoof detection with image distortion analysis. IEEE Trans Inf Forensics Sec 10(4):746–761. Scholar
  27. 27.
    Boulkenafet Z, Komulainen J, Li L, Feng X, Hadid A (2017) OULU-NPU: A mobile face presentation attack database with real-world variations. In: 2017 12th IEEE international conference on automatic face and gesture recognition (FG 2017), IEEE, pp 612–618Google Scholar
  28. 28.
    Chingovska I, Anjos A, Marcel S (2012) On the effectiveness of local binary patterns in face anti-spoofing. In: Proceedings of the IEEE international conference of the biometrics special interest group (BIOSIG), pp 1–7Google Scholar
  29. 29.
    Costa-Pazo A, Bhattacharjee S, Vazquez-Fernandez E, Marcel S (2016) The REPLAY-MOBILE face presentation-attack database. In: 2016 International conference of the biometrics special interest group (BIOSIG), IEEE, pp 1–7.
  30. 30.
    Pinto A, Schwartz WR, Pedrini H, de Rezende Rocha A (2015) Using visual rhythms for detecting video-based facial spoof attacks. IEEE Trans Inf Forensics Sec 10(5):1025–1038CrossRefGoogle Scholar
  31. 31.
    Peixoto B, Michelassi C, Rocha A (2011) Face liveness detection under bad illumination conditions. In: 2011 18th IEEE international conference on image processing (ICIP), pp 3557–3560Google Scholar
  32. 32.
    Vanoni M, Tome P, El Shafey L, Marcel S (2014) Cross-database evaluation with an open finger vein sensor. In: IEEE workshop on biometric measurements and systems for security and medical applications (BioMS).
  33. 33.
    Tome P, Vanoni M, Marcel S (2014) On the vulnerability of finger vein recognition to spoofing. In: IEEE international conference of the biometrics special interest group (BIOSIG).
  34. 34.
    Tome P, Raghavendra R, Busch C, Tirunagari S, Poh N, Shekar BH, Gragnaniello D, Sansone C, Verdoliva L, Marcel S (2015) The 1st competition on counter measures to finger vein spoofing attacks. In: The 8th IAPR international conference on biometrics (ICB).
  35. 35.
    Tome P, Marcel S (2015) On the vulnerability of palm vein recognition to spoofing attacks. In: The 8th IAPR international conference on biometrics (ICB).
  36. 36.
    Kinnunen T, Sahidullah M, Delgado H, Todisco M, Evans N, Yamagishi J, Lee KA (2017) The asvspoof 2017 challenge: assessing the limits of replay spoofing attack detectionGoogle Scholar
  37. 37.
    Ergünay SK, Khoury E, Lazaridis A, Marcel S (2015) On the vulnerability of speaker verification to realistic voice spoofing. In: IEEE international conference on biometrics: theory, applications and systems (BTAS).
  38. 38.
    Korshunov P, Goncalves AR, Violato RP, Simes FO, Marcel S (2018) On the use of convolutional neural networks for speech presentation attack detection. In: International conference on identity, security and behavior analysisGoogle Scholar
  39. 39.
    Poh N, Bengio S (2006) Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication. Pattern Recognition 2006Google Scholar
  40. 40.
    Martin A, Doddington G, Kamm T, Ordowski M (1997) The det curve in assessment of detection task performance. In: Eurospeech, pp 1895–1898Google Scholar
  41. 41.
    Bengio S, Keller M, Mariéthoz J (2003) The expected performance curve. Technical Report Idiap-RR-85-2003, Idiap Research InstituteGoogle Scholar
  42. 42.
    Wang L, Ding X, Fang C (2009) Face live detection method based on physiological motion analysis. Tsinghua Sci Technol 14(6):685–690CrossRefGoogle Scholar
  43. 43.
    Zhang Z, Yi D, Lei Z, Li S (2011) Face liveness detection by learning multispectral reflectance distributions. In: 2011 IEEE international conference on automatic face gesture recognition and workshops (FG 2011), pp 436–441Google Scholar
  44. 44.
    Johnson P, Lazarick R, Marasco E, Newton E, Ross A, Schuckers S (2012) Biometric liveness detection: Framework and metrics. In: International biometric performance conferenceGoogle Scholar
  45. 45.
    Gao X, Tsong Ng T, Qiu B, Chang SF (2010) Single-view recaptured image detection based on physics-based features. In: IEEE international conference on multimedia and expo (ICME). SingaporeGoogle Scholar
  46. 46.
    Tronci R, Muntoni D, Fadda G, Pili M, Sirena N, Murgia G, Ristori M, Ricerche S, Roli F (2011) Fusion of multiple clues for photo-attack detection in face recognition systems. In: Proceedings of the 2011 international joint conference on biometrics, IJCB ’11, IEEE Computer Society, pp 1–6Google Scholar
  47. 47.
    Jain AK, Flynn P, Ross AA (eds) (2008) Handbook of biometrics. Springer, BerlinGoogle Scholar
  48. 48.
    Adler A, Schuckers S (2009) Encyclopedia of biometrics, chap. security and liveness, overview, Springer, Berlin, pp 1146–1152Google Scholar
  49. 49.
    Galbally J, Cappelli R, Lumini A, de Rivera GG, Maltoni D, Fiérrez J, Ortega-Garcia J, Maio D (2010) An evaluation of direct attacks using fake fingers generated from iso templates. Pattern Recogn Lett 31(8):725–732CrossRefGoogle Scholar
  50. 50.
    Rodrigues R, Kamat N, Govindaraju V (2010) Evaluation of biometric spoofing in a multimodal system. In: 2010 Fourth IEEE international conference on biometrics: theory applications and systems (BTAS)Google Scholar
  51. 51.
    Matsumoto T, Matsumoto H, Yamada K, Hoshino S (2002) Impact of artifical gummy fingers on fingerprint systems. In: SPIE proceedings: optical security and counterfeit deterrence techniques, vol 4677Google Scholar
  52. 52.
    Patrick P, Aversano G, Blouet R, Charbit M, Chollet G (2005) Voice forgery using alisp: indexation in a client memory. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing, 2005 (ICASSP ’05), vol 1, pp 17–20Google Scholar
  53. 53.
    Alegre F, Vipperla R, Evans N, Fauve B (2012) On the vulnerability of automatic speaker recognition to spoofing attacks with artificial signals. In: 2012 Proceedings of the 20th european signal processing conference (EUSIPCO), pp 36–40Google Scholar
  54. 54.
    Bonastre JF, Matrouf D, Fredouille C (2007) Artificial impostor voice transformation effects on false acceptance rates. In: INTERSPEECH, pp 2053–2056Google Scholar
  55. 55.
    Chingovska I, Rabello dos Anjos A, Marcel S (2014) Biometrics evaluation under spoofing attacks. IEEE Trans Inf Forensics Sec 9(12):2264–2276 Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ivana Chingovska
    • 1
  • Amir Mohammadi
    • 1
    Email author
  • André Anjos
    • 1
  • Sébastien Marcel
    • 1
  1. 1.Idiap Research InstituteMartignySwitzerland

Personalised recommendations