Curvewise DET Confidence Regions and Pointwise EER Confidence Intervals Using Radial Sweep Methodology

  • Michael E. Schuckers
  • Yordan Minev
  • Andy Adler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4642)

Abstract

One methodology for evaluating the matching performance of biometric authentication systems is the detection error tradeoff (DET) curve. The DET curve graphically illustrates the relationship between false rejects and false accepts when varying a threshold across a genuine and an imposter match score distributions. This paper makes two contributions to the literature on the matching performance evaluation of biometric identification or bioauthentication systems. First, we create curvewise DET confidence regions using radial sweep methods. Second we use this same methodology to create pointwise confidence intervals for the equal error rate (EER). The EER is the rate at which the false accept rate and the false reject rate are identical. We utilize resampling or bootstrap methods to estimate the variability in both the DET and the EER. Our radial sweep is based on converting the false reject and false accept errors to polar coordinates. Application is made of these methods to data from three different biometric modalities and we discuss the results of these applications.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Mansfield, T., Wayman, J.L.: Best practices in testing and reporting performance of biometric devices (2002), on the web at www.cesg.gov.uk/site/ast/biometrics/media/BestPractice.pdf
  2. 2.
    Green, D.M., Swets, J.A.: Signal Detection Theory and Psychophysics. John Wiley & Sons, Chichester (1966)Google Scholar
  3. 3.
    Poh, N., Bengio, S.: Database, protocol and tools for evaluating score-level fusion algorithms in biometric authentication. Pattern Recognition Journal (2005)Google Scholar
  4. 4.
    Zhou, X.-H., McClish, D.K., Obuchowski, N.A.: Statistical Methods in Diagnostic Medicine. John Wiley & Sons, Chichester (2002)MATHGoogle Scholar
  5. 5.
    Hernández-Orallo, J., Ferri, C., Lachiche, N., Flach, P.A. (eds.): ROC Analysis in Artificial Intelligence, 1st Int. Workshop, ROCAI-2004, Valencia, Spain (2004)Google Scholar
  6. 6.
    Macskassy, S.A., Provost, F.J., Rosset, S.: Roc confidence bands: an empirical evaluation. In: De Raedt, L., Wrobel, S. (eds.) ICML, pp. 537–544. ACM, New York (2005)CrossRefGoogle Scholar
  7. 7.
    Campbell, G.: Advance in statistical methodology for the evaluation of diagnostic and laboratory tests. Statistics in Medicine 13, 499–508 (1994)CrossRefGoogle Scholar
  8. 8.
    Ma, G., Hall, W.J.: Confidence bands for receiver operating characteristics curves. Medical Decision Making 13, 191–197 (1993)CrossRefGoogle Scholar
  9. 9.
    Poh, N., Martin, A., Bengio, S.: Performance generalization in biometric authentication using joint user-specific and sample bootstraps. IEEE Transactions on Pattern Analysis and Machine Intelligence (to appear)Google Scholar
  10. 10.
    Adler, A., Schuckers, M.E.: Calculation of a composite DET curve. In: Kanade, T., Jain, A., Ratha, N.K. (eds.) AVBPA 2005. LNCS, vol. 3546, pp. 279–288. Springer, Heidelberg (2005)Google Scholar
  11. 11.
    Dass, S.C., Zhu, Y., Jain, A.K.: Validating a biometric authentication system: Sample size requirements. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(1), 19–30 (2007)Google Scholar
  12. 12.
    Ross, A., Jain, A.K.: Information fusion in biometrics. Pattern Recognition Letters 24(13), 2115–2125 (2003)CrossRefGoogle Scholar
  13. 13.
    Bolle, R.M., Ratha, N.K., Pankanti, S.: Error analysis of pattern recognition systems – the subsets bootstrap. Computer Vision and Image Understanding 93, 1–33 (2004)CrossRefGoogle Scholar
  14. 14.
    Agresti, A., Coull, B.A.: Approximate is better than ”exact” for interval estimation of binomial proportions. The American Statistician 52(2), 119–126 (1998)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Michael E. Schuckers
    • 1
  • Yordan Minev
    • 1
  • Andy Adler
    • 2
  1. 1.Department of Mathematics, Computer Science and Statistics, St. Lawrence University, Canton, NYUSA
  2. 2.Department of Systems and Computer Engineering, Carleton University, Ottawa, ONCanada

Personalised recommendations