Skip to main content
SpringerLink
Log in

Evaluation Methodologies for Biometric Presentation Attack Detection

  • Chapter
  • First Online:
Handbook of Biometric Anti-Spoofing

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Presentation attack detection (PAD, also known as anti-spoofing) systems, regardless of the technique, biometric mode or degree of independence of external equipment, are most commonly treated as binary classification systems. The two classes that they differentiate are bona-fide and presentation attack samples. From this perspective, their evaluation is equivalent to the established evaluation standards for the binary classification systems. However, PAD systems are designed to operate in conjunction with recognition systems and as such can affect their performance. From the point of view of a recognition system, the presentation attacks are a separate class that need to be detected and rejected. As the problem of presentation attack detection grows to this pseudo-ternary status, the evaluation methodologies for the recognition systems need to be revised and updated. Consequentially, the database requirements for presentation attack databases become more specific. The focus of this chapter is the task of biometric verification and its scope is three-fold: first, it gives the definition of the presentation attack detection problem from the two perspectives. Second, it states the database requirements for a fair and unbiased evaluation. Finally, it gives an overview of the existing evaluation techniques for presentation attacks detection systems and verification systems under presentation attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Bona-fide are also called real or live samples. Both genuine and zero-effort impostor samples are bona-fide samples. While zero-effort impostors are negative samples in a verification system, they are considered positive samples in a standalone PAD system (since they are not PAs).

  2. 2.

    In this chapter, since we focus on the biometric recognition task, we will only consider PAs aiming to impersonate an identity and not to conceal (hide) an identity.

  3. 3.

    In this chapter, we shall treat examples in a (discriminative) binary classification system one wishes to keep as positive class or simply as positives, and, examples that should be discarded as negative class or negatives.

  4. 4.

    The software to reproduce the plots of this chapter is available in https://gitlab.idiap.ch/bob/bob.hobpad2.chapter20.

References

  1. Information technology – Biometric presentation attack detection – Part 3: Testing and reporting. Standard, International Organization for Standardization, Geneva, CH (2017). https://www.iso.org/standard/67381.html

  2. Rodrigues RN, Ling LL, Govindaraju V (2009) Robustness of multimodal biometric fusion methods against spoofing attacks. J Vis Lang Comput 20(3):169–179

    Google Scholar 

  3. Johnson PA, Tan B, Schuckers S (2010) Multimodal fusion vulnerability to non-zero (spoof) imposters. In: IEEE international workshop on information forensics and security

    Google Scholar 

  4. Akhtar Z, Fumera G, Marcialis GL, Roli F. Robustness evaluation of biometric systems under spoof attacks. In: 16th international conference on image analysis and processing, pp 159–168

    Google Scholar 

  5. Akhtar Z, Fumera G, Marcialis GL, Roli F. Robustness analysis of likelihood ration score fusion rule for multi-modal biometric systems under spoof attacks. In: 45th IEEE international carnahan conference on security technology, pp 237–244

    Google Scholar 

  6. Akhtar Z, Fumera G, Marcialis GL, Roli F (2012) Evaluation of serial and parallel multibiometric systems under spoofing attacks. In: 5th IEEE international conference on biometrics: theory, applications and systems

    Google Scholar 

  7. Villalba J, Lleida E (2011) Preventing replay attacks on speaker verification systems. In: 2011 IEEE international carnahan conference on security technology (ICCST), pp 1–8

    Google Scholar 

  8. Marasco E, Johnson P, Sansone C, Schuckers S (2011) Increase the security of multibiometric systems by incorporating a spoofing detection algorithm in the fusion mechanism. In: Proceedings of the 10th international conference on Multiple classifier systems, pp 309–318

    Chapter  Google Scholar 

  9. Marasco E, Ding Y, Ross A (2012) Combining match scores with liveness values in a fingerprint verification system. In: 5th IEEE international conference on biometrics: theory, applications and systems

    Google Scholar 

  10. Chingovska I, Anjos A, Marcel S (2013) Anti-spoofing in action: joint operation with a verification system. In: Proceedings of IEEE conference on computer vision and pattern recognition, workshop on biometrics

    Google Scholar 

  11. Pan G, Sun L, Wu Z, Lao S (2007) Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: IEEE 11th international conference on computer vision. ICCV 2007, pp 1–8

    Google Scholar 

  12. Bao W, Li H, Li N, Jiang W (2009) A liveness detection method for face recognition based on optical flow field. In: International conference on image analysis and signal processing. IASP 2009, pp. 233–236

    Google Scholar 

  13. yan J, Zhang Z, Lei Z, Yi D, Li SZ (2012) Face liveness detection by exploring multiple scenic clues. In: 12th international conference on control, automation, robotics and vision (ICARCV 2012), China

    Google Scholar 

  14. Tan X, Li Y, Liu J, Jiang L (2010) Face liveness detection from a single image with sparse low rank bilinear discriminative model. In: Proceedings of the european conference on computer vision (ECCV), LNCS 6316. Springer, Berlin, pp 504–517

    Chapter  Google Scholar 

  15. Galbally J, Alonso-Fernandez F, Fierrez J, Ortega-Garcia J (2012) A high performance fingerprint liveness detection method based on quality related features. Future Gener Comput Syst 28(1):311–321

    Article  Google Scholar 

  16. Yambay D, Ghiani L, Denti P, Marcialis G, Roli F, Schuckers S (2012) LivDet 2011 - fingerprint liveness detection competition 2011. In: 2012 5th IAPR international conference on biometrics (ICB), pp 208–215

    Google Scholar 

  17. Mansfield AJ, Wayman JL, Dr A, Rayner D, Wayman JL (2002) Best practices in testing and reporting performance

    Google Scholar 

  18. Galbally-Herrero J, Fierrez-Aguilar J, Rodriguez-Gonzalez JD, Alonso-Fernandez F, Ortega-Garcia J, Tapiador M (006) On the vulnerability of fingerprint verification systems to fake fingerprints attacks. In: IEEE international carnahan conference on security technology, pp 169–179

    Google Scholar 

  19. Ruiz-Albacete V, Tome-Gonzalez P, Alonso-Fernandez F, Galbally J, Fierrez J, Ortega-Garcia J (2008) Direct attacks using fake images in iris verification. In: Proceedings of the COST 2101 workshop on biometrics and identity management, BIOID, Springer, Berlin, pp 181–190

    Google Scholar 

  20. Galbally J, Fierrez J, Alonso-Fernandez F, Martinez-Diaz M (2011) Evaluation of direct attacks to fingerprint verification systems. Telecommun Syst, Spec Issue Biom 47(3):243–254

    Article  Google Scholar 

  21. Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning: data mining, inference, and prediction: with 200 full-color illustrations. Springer, New York

    Chapter  Google Scholar 

  22. Lui YM, Bolme D, Phillips P, Beveridge J, Draper B (2012) Preliminary studies on the good, the bad, and the ugly face recognition challenge problem. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 9–16

    Google Scholar 

  23. Marcialis GL, Lewicke A, Tan B, Coli P, Grimberg D, Congiu A, Tidu A, Roli F, Schuckers S (2009) First international fingerprint liveness detection competition – livdet 2009. In: Proceedings of the IAPR International Conference on Image Analysis and Processing (ICIAP), LNCS-5716, pp 12–23 (2009)

    Chapter  Google Scholar 

  24. Ghiani L, Yambay D, Mura V, Tocco S, Marcialis G, Roli F, Schuckers S (2013) Livdet 2013 - fingerprint liveness detection competition. In: IEEE international conference on biometrics (ICB)

    Google Scholar 

  25. Zhiwei Z, Yan J, Liu S, Lei Z, Yi D, Li SZ (2012) A face antispoofing database with diverse attacks. In: Proceedings of the IAPR international conference on biometrics (ICB), pp 26–31

    Google Scholar 

  26. Wen D, Han H, Jain AK (2015) Face spoof detection with image distortion analysis. IEEE Trans Inf Forensics Sec 10(4):746–761. https://doi.org/10.1109/TIFS.2015.2400395

    Article  Google Scholar 

  27. Boulkenafet Z, Komulainen J, Li L, Feng X, Hadid A (2017) OULU-NPU: A mobile face presentation attack database with real-world variations. In: 2017 12th IEEE international conference on automatic face and gesture recognition (FG 2017), IEEE, pp 612–618

    Google Scholar 

  28. Chingovska I, Anjos A, Marcel S (2012) On the effectiveness of local binary patterns in face anti-spoofing. In: Proceedings of the IEEE international conference of the biometrics special interest group (BIOSIG), pp 1–7

    Google Scholar 

  29. Costa-Pazo A, Bhattacharjee S, Vazquez-Fernandez E, Marcel S (2016) The REPLAY-MOBILE face presentation-attack database. In: 2016 International conference of the biometrics special interest group (BIOSIG), IEEE, pp 1–7. http://ieeexplore.ieee.org/abstract/document/7736936/

  30. Pinto A, Schwartz WR, Pedrini H, de Rezende Rocha A (2015) Using visual rhythms for detecting video-based facial spoof attacks. IEEE Trans Inf Forensics Sec 10(5):1025–1038

    Article  Google Scholar 

  31. Peixoto B, Michelassi C, Rocha A (2011) Face liveness detection under bad illumination conditions. In: 2011 18th IEEE international conference on image processing (ICIP), pp 3557–3560

    Google Scholar 

  32. Vanoni M, Tome P, El Shafey L, Marcel S (2014) Cross-database evaluation with an open finger vein sensor. In: IEEE workshop on biometric measurements and systems for security and medical applications (BioMS). http://publications.idiap.ch/index.php/publications/show/2928

  33. Tome P, Vanoni M, Marcel S (2014) On the vulnerability of finger vein recognition to spoofing. In: IEEE international conference of the biometrics special interest group (BIOSIG). http://publications.idiap.ch/index.php/publications/show/2910

  34. Tome P, Raghavendra R, Busch C, Tirunagari S, Poh N, Shekar BH, Gragnaniello D, Sansone C, Verdoliva L, Marcel S (2015) The 1st competition on counter measures to finger vein spoofing attacks. In: The 8th IAPR international conference on biometrics (ICB). http://publications.idiap.ch/index.php/publications/show/3095

  35. Tome P, Marcel S (2015) On the vulnerability of palm vein recognition to spoofing attacks. In: The 8th IAPR international conference on biometrics (ICB). http://publications.idiap.ch/index.php/publications/show/3096

  36. Kinnunen T, Sahidullah M, Delgado H, Todisco M, Evans N, Yamagishi J, Lee KA (2017) The asvspoof 2017 challenge: assessing the limits of replay spoofing attack detection

    Google Scholar 

  37. Ergünay SK, Khoury E, Lazaridis A, Marcel S (2015) On the vulnerability of speaker verification to realistic voice spoofing. In: IEEE international conference on biometrics: theory, applications and systems (BTAS). https://publidiap.idiap.ch/downloads//papers/2015/KucurErgunay_IEEEBTAS_2015.pdf

  38. Korshunov P, Goncalves AR, Violato RP, Simes FO, Marcel S (2018) On the use of convolutional neural networks for speech presentation attack detection. In: International conference on identity, security and behavior analysis

    Google Scholar 

  39. Poh N, Bengio S (2006) Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication. Pattern Recognition 2006

    Google Scholar 

  40. Martin A, Doddington G, Kamm T, Ordowski M (1997) The det curve in assessment of detection task performance. In: Eurospeech, pp 1895–1898

    Google Scholar 

  41. Bengio S, Keller M, Mariéthoz J (2003) The expected performance curve. Technical Report Idiap-RR-85-2003, Idiap Research Institute

    Google Scholar 

  42. Wang L, Ding X, Fang C (2009) Face live detection method based on physiological motion analysis. Tsinghua Sci Technol 14(6):685–690

    Article  Google Scholar 

  43. Zhang Z, Yi D, Lei Z, Li S (2011) Face liveness detection by learning multispectral reflectance distributions. In: 2011 IEEE international conference on automatic face gesture recognition and workshops (FG 2011), pp 436–441

    Google Scholar 

  44. Johnson P, Lazarick R, Marasco E, Newton E, Ross A, Schuckers S (2012) Biometric liveness detection: Framework and metrics. In: International biometric performance conference

    Google Scholar 

  45. Gao X, Tsong Ng T, Qiu B, Chang SF (2010) Single-view recaptured image detection based on physics-based features. In: IEEE international conference on multimedia and expo (ICME). Singapore

    Google Scholar 

  46. Tronci R, Muntoni D, Fadda G, Pili M, Sirena N, Murgia G, Ristori M, Ricerche S, Roli F (2011) Fusion of multiple clues for photo-attack detection in face recognition systems. In: Proceedings of the 2011 international joint conference on biometrics, IJCB ’11, IEEE Computer Society, pp 1–6

    Google Scholar 

  47. Jain AK, Flynn P, Ross AA (eds) (2008) Handbook of biometrics. Springer, Berlin

    Google Scholar 

  48. Adler A, Schuckers S (2009) Encyclopedia of biometrics, chap. security and liveness, overview, Springer, Berlin, pp 1146–1152

    Google Scholar 

  49. Galbally J, Cappelli R, Lumini A, de Rivera GG, Maltoni D, Fiérrez J, Ortega-Garcia J, Maio D (2010) An evaluation of direct attacks using fake fingers generated from iso templates. Pattern Recogn Lett 31(8):725–732

    Article  Google Scholar 

  50. Rodrigues R, Kamat N, Govindaraju V (2010) Evaluation of biometric spoofing in a multimodal system. In: 2010 Fourth IEEE international conference on biometrics: theory applications and systems (BTAS)

    Google Scholar 

  51. Matsumoto T, Matsumoto H, Yamada K, Hoshino S (2002) Impact of artifical gummy fingers on fingerprint systems. In: SPIE proceedings: optical security and counterfeit deterrence techniques, vol 4677

    Google Scholar 

  52. Patrick P, Aversano G, Blouet R, Charbit M, Chollet G (2005) Voice forgery using alisp: indexation in a client memory. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing, 2005 (ICASSP ’05), vol 1, pp 17–20

    Google Scholar 

  53. Alegre F, Vipperla R, Evans N, Fauve B (2012) On the vulnerability of automatic speaker recognition to spoofing attacks with artificial signals. In: 2012 Proceedings of the 20th european signal processing conference (EUSIPCO), pp 36–40

    Google Scholar 

  54. Bonastre JF, Matrouf D, Fredouille C (2007) Artificial impostor voice transformation effects on false acceptance rates. In: INTERSPEECH, pp 2053–2056

    Google Scholar 

  55. Chingovska I, Rabello dos Anjos A, Marcel S (2014) Biometrics evaluation under spoofing attacks. IEEE Trans Inf Forensics Sec 9(12):2264–2276 http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6879440

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the projects BEAT (http://www.beat-eu.org) and TABULA RASA (http://www.tabularasa-euproject.org) both funded under the 7th Framework Programme of the European Union (EU) (grant agreement number 284989 and 257289) respectively. The revision of this chapter was supported under the project on Secure Access Control over Wide Area Networks (SWAN) funded by the Research Council of Norway (grant no. IKTPLUSS 248030/O70). and by the Swiss Center for Biometrics Research and Testing.

Author information

Authors and Affiliations

  1. Idiap Research Institute, rue Marconi 19, 1920, Martigny, Switzerland

    Ivana Chingovska, Amir Mohammadi, André Anjos & Sébastien Marcel

Authors
  1. Ivana Chingovska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. André Anjos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sébastien Marcel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Mohammadi .

Editor information

Editors and Affiliations

  1. Idiap Research Institute, Martigny, Switzerland

    Sébastien Marcel

  2. University of Southampton, Southampton, UK

    Mark S. Nixon

  3. Universidad Autonoma de Madrid, Madrid, Spain

    Julian Fierrez

  4. EURECOM, Biot Sophia Antipolis, France

    Nicholas Evans

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chingovska, I., Mohammadi, A., Anjos, A., Marcel, S. (2019). Evaluation Methodologies for Biometric Presentation Attack Detection. In: Marcel, S., Nixon, M., Fierrez, J., Evans, N. (eds) Handbook of Biometric Anti-Spoofing. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-92627-8_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-92627-8_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-92626-1

  • Online ISBN: 978-3-319-92627-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation