BioSecure Multimodal Evaluation Campaign 2007 (BMEC’2007)

  • Aurélien Mayoue
  • Bernadette Dorizzi
  • Lorène Allano
  • Gérard Chollet
  • Jean Hennebert
  • Dijana Petrovska-Delacr´etraz
  • Florian Verdet


This chapter presents the experimental results from the mobile scenario of the BioSecure Multimodal Evaluation Campaign 2007 (BMEC’2007). This competition was organized by the BioSecure Network of Excellence (NoE) and aimed at testing the robustness of monomodal and multimodal biometric verification systems to degraded acquisition conditions. The database used for the evaluation is the large-scale multimodal database acquired in the framework of the BioSecure NoE in mobility conditions. During this evaluation, the BioSecure benchmarking methodology was followed to enable a fair comparison of the submitted algorithms. In this way, we believe that the BMEC’2007 database and results will be useful both to the participants and, more generally, to all practitioners in the field as a benchmark for improving methods and for enabling evaluation of algorithms.


Gaussian Mixture Model Equal Error Rate Fingerprint Image Speaker Verification False Acceptance Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material


  1. 1.
    ALIZE: a free and open tool for speaker recognition.
  2. 2.
    L. Allano, A.C. Morris, H. Sellahewa, S. Garcia-Salicetti, J. Koreman, S. Jassim, B. Ly Van, D. Wu, and B. Dorizzi. Non intrusive multi-biometrics on a mobile device: a comparison of fusion techniques. In Proceedings of SPIE 2006, conference on Biometric Techniques for Human Identification III, Orlando, Florida, USA, April 2006.Google Scholar
  3. 3.
    E. Bailly-Baillière, S. Bengio, F. Bimbot, M. Hamouz, J. Kittler, J. Mariéthoz, J. Matas, K. Messer, V. Popovici, F. Porée, B. Ruiz, and J.-P. Thiran. The BANCA Database and Evaluation Protocol. In 4th International Conference on Audio-and Video-Based Biometric Person Authentication (AVBPA'03), pages 625–638, Surrey, UK, 2003.Google Scholar
  4. 4.
    C. Barras and J.-L. Gauvain. Feature and Score Normalization for Speaker Verification of Cellular Data. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2003), Hong Kong, China, 2003.Google Scholar
  5. 5.
    BioSecure Mutimodal Evaluation Campaign 2007 (BMEC2007).
  6. 6.
    BioSecure Network of Excellence.
  7. 7.
    R.M. Bolle, N.K. Ratha, and S. Pankanti. Evaluation techniques for biometrics-based authentication systems (FRR). In Proc. 15th Internat. Conf. Pattern Recogn., pages 835–842000.Google Scholar
  8. 8.
    R.M. Bolle, N.K. Ratha, and S. Pankanti. Error analysis of pattern recognition systems - the subsets bootstrap. Computer Vision and Image Understanding, 93:1–33, 2004.CrossRefGoogle Scholar
  9. 9.
    J.-F. Bonastre, N. Scheffer, C. Fredouille, and D. Matrouf. NIST'04 speaker recognition evaluation campaign: new LIA speaker detection plateform based on ALIZE toolkit. In 2004 NIST SRE'04 Workshop: speaker detection evaluation campaign, Toledo, Spain, 2004.Google Scholar
  10. 10.
    H. Bredin and G. Chollet. Audio-Visual Speech Synchrony Measure for Talking-Face Identity Verification. In IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP, 2007.Google Scholar
  11. 11.
    N. Brummer, L. Burget, J. Cernocky, O. Glembek, F. Grezl, M. Karafiat, D.A. van Leeuwen, P. Matejka, P. Scwartz, and A. Strasheim. Fusion of heterogeneous speaker recognition systems in the STBU submission for the NIST speaker recognition evaluation 200In IEEE Transactions on Audio, Speech and Signal Processing, 200To appear.Google Scholar
  12. 12.
    N. Brummer and J. du Preez. Application independent evaluation of speaker detection. Computer Speech and Language, 20, pp.230–275, 2006.Google Scholar
  13. 13.
    A.J. Dobson, An Introduction to Generalized Linear Models. CRC Press, 1990.Google Scholar
  14. 14.
    B. Dumas, C. Pugin, J. Hennebert, D. Petrovska-Delacrétaz, A. Humm, F. Evquoz, R. Ingold, and D. Von Rotz. MyIdea - multimodal biometrics database, description of acquisition protocols. In Proc. of Third COST 275 Workshop (COST 275), pages 59–62, Hatfield UK, October 2005.Google Scholar
  15. 15.
    NIST Speaker Recognition Evaluation.
  16. 16.
    Crazy Talk face animation from Reallusion.
  17. 17.
    B. Fauve, N.W.D. Evans, and J.S.D. Mason. Improving the Performance of Text-Independent Short Duration GMM- and SVM-Based Speaker Verification. In Odyssey the Speaker and Language Recognition Workshop, 200To appear.Google Scholar
  18. 18.
    J. Fierrez, J. Ortega-Garcia, D.T. Toledano, and J. Gonzalez-Rodriguez. Biosec baseline corpus: A multimodal biometric database. Pattern Recognition, 40(4):1389–1392, 2007.zbMATHCrossRefGoogle Scholar
  19. 19.
    Y. Freund and R.E. Schapire. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Scences, 55(1):119–139, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    J. Galbally, J. Fierrez, J. Ortega-Garcia, M.R. Freire, F. Alonso-Fernandez, J.A. Siguenza, J. Garrido-Salas, E. Anguiano-Rey, G. Gonzalez de Rivera, R. Ribalda, M. Faundez-Zanuy, J.A. Ortega, V. Cardenoso-Payo. A. Viloria, C.E. Vivaracho, Q.I. Moro, J.J. Igarza, J. Sanchez, I. Hernaez, and C. Orrite-Urunuela. BiosecurID: a Multimodal Biometric Database. In Proc. MADRINET Workshop, pages 68–76, November 2007.Google Scholar
  21. 21.
    S. Garcia-Salicetti, C. Beumier, G. Chollet, B. Dorizzi, J. Leroux les Jardias, J. Lunter, Y. Ni, and D. Petrovska-Delacrétaz. BIOMET: a Multimodal Person Authentication Database Including Face, Voice, Fingerprint, Hand and Signature Modalities. In 4th International Conference on Audio and Video Based Biometric Person Authentification (AVBPA), Guilford UK, June 2003.Google Scholar
  22. 22.
    S. Garcia-Salicetti, J. Fierrez-Aguilar, F. Alonso-Fernandez, C. Vielhauer, R. Guest, L. Allano, T. Doan Trung, T. Scheidat, B. Ly Van, J. Dittmann, B. Dorizzi, J. Ortega-Garcia, J. Gonzalez-Rodriguez, M. Bacile di Castiglione, and M. Fairhurst. Biosecure Reference Systems for Online Signature Verification: a study of complementarity. Annales des Télécommunications, Special Issue on Multimodal Biometrics, pages 36–61, January-February 2007.Google Scholar
  23. 23.
    S. Garcia-Salicetti, M.A. Mellakh, L. Allano, and B. Dorizzi. Multimodal biometric score fusion: the mean rule vs. support vector classifiers. In Proceedings of EUSIPCO 2005, Antalya, Turkey, September 2005.Google Scholar
  24. 24.
    T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, section 4.1, 2001.Google Scholar
  25. 25.
    J. Hennebert, R. Loeffel, A. Humm, and R. Ingold. A New Forgery Scenario Based On Regaining Dynamics Of Signature. pages 366–375, Seoul, Korea, 2007.Google Scholar
  26. 26.
    IV2: Identification par l'Iris et le Visage via la Vidéo.
  27. 27.
    A. K. Jain, K. Nandakumar, and A. Ross. Score normalization in multimodal biometric systems. Pattern Recognition, 38(12):2270–2285, December 2005.CrossRefGoogle Scholar
  28. 28.
    V.I. Levenshtein. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics, 10, pp. 707–710, 1966.MathSciNetGoogle Scholar
  29. 29.
    BECARS Library and Tools for Speaker Verification.
  30. 30.
    R. Lienhart and J. Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. IEEE ICIP, 1:900–903, 2002.Google Scholar
  31. 31.
    A. Martin, G.R. Doddington, T.Kamm, M. Ordowski, and M.A. Przybocki. The DET curve in assessment of detection task performance. In Proceedings of Eurospeech, pages 1895–1898, Rhodes, Greece, 1997.Google Scholar
  32. 32.
    K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre. XM2VTSDB: The extended M2VTS database. In Proc. Second International Conference on Audio- and Video-based Biometric Person Authentication (AVBPA), 1999.Google Scholar
  33. 33.
    J. Ortega-Garcia, J. Fierrez-Aguilar, D. Simon, M.F.J. Gonzalez and V. Espinosa, A. Satue, I. Hernaez, J.J. Igarza , C. Vivaracho, D. Escudero, and Q.I. Moro. MCYT baseline corpus: a bimodal biometric database. IEE Proceedings Vision, Image and Signal Processing, 150(6):391–401, 2003.CrossRefGoogle Scholar
  34. 34.
    D. Petrovska-Delacrétaz, S. Lelandais, J. Colineau, L. Chen, B. Dorizzi, E. Krichen, M.A. Mellakh, A. Chaari, S. Guerfi, J. DHose, M. Ardabilian, and B. Ben Amor. The iv2 multimodal (2d, 3d, stereoscopic face, talking face and iris) biometric database, and the iv2 2007 evaluation campaign. In the proceedings of the IEEE Second International Conference on Biometrics: Theory, Applications (BTAS), Washington DC, USA, September 2008.Google Scholar
  35. 35.
    P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W.Worek. Overview of the Face Recognition Grand Challenge. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 947–954, Seoul, Korea, 2005.Google Scholar
  36. 36.
    D.A. Reynolds, T.F. Quatieri, and R. Dunn. Speaker Verification Using Adapted Gaussian Mixture Models. Digital Signal Processing, 10(1–3):19–41, 2000.CrossRefGoogle Scholar
  37. 37.
    C. Sanderson and K.K. Paliwal. Fast Feature Extraction Method for Robust Face Verification. IEE Electronics Letters, 38(25):1648–1650, 2002.CrossRefGoogle Scholar
  38. 38.
    T. Scheidat, C. Vielhauer, and J. Dittmann. Distance-Level Fusion Strategies for Online Signature Verification. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Amsterdam, The Netherlands, 2005.Google Scholar
  39. 39.
    F. Schiel, S. Steininger, and U. Trk. The SmartKom multimodal corpus at BAS. In Proc. Intl. Conf. on Language Resources and Evaluation, 2002.Google Scholar
  40. 40.
    S. Schimke, C. Vielhauer, and J. Dittmann. Using Adapted Levenshtein Distance for On-Line Signature Authentication. In Proceedings of the ICPR 2004, IEEE 17th International Conference on Pattern Recognition, ISBN 0-7695-2128-2, 2004.Google Scholar
  41. 41.
    Festival Text-To-Speech System.
  42. 42.
    Faculté Polytechnique de Mons TCTS MBrola voices.
  43. 43.
    The Hidden Markov Model Toolkit (HTK).
  44. 44.
    Speech Signal Processing (SPro) Toolkit.
  45. 45.
    Tools for Fusion and Calibration of automatic speaker detection systems (FoCal).
  46. 46.
    M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71–86, 1991.CrossRefGoogle Scholar
  47. 47.
    B. Ly Van, S. Garcia-Salicetti, and B. Dorizzi. On using the Viterbi Path along with HMM Likelihood Information for On-line Signature Verification. In Proceedings of the IEEE Transactions on Systems, Man and Cybernetics, Part B, to appear.Google Scholar
  48. 48.
    C. Watson, M. Garris, E. Tabassi, C. Wilson, R. McCabe, and S. Janet. User's Guide to Fingerprint Image Software 2 - NFIS2 ( NIST 2004.

Copyright information

© Springer-Verlag London Limited 2009

Authors and Affiliations

  • Aurélien Mayoue
    • 1
  • Bernadette Dorizzi
    • 2
  • Lorène Allano
    • 3
  • Gérard Chollet
    • 4
  • Jean Hennebert
    • 5
  • Dijana Petrovska-Delacr´etraz
    • 6
  • Florian Verdet
    • 7
  1. 1.TELECOM SudParis (ex GET-INT)Evry CedexFrance
  2. 2.TELECOM SudParis (ex GET-INT)Evry CedexFrance
  3. 3.TELECOM SudParis (ex GET-INT)EvryFrance
  4. 4.TELECOM ParisTech (ex GET-ENST)ParisFrance
  5. 5.University of FribourgFribourgSwitzerland
  6. 6.TELECOM SudParis (ex GET-INT)Evry CedexFrance
  7. 7.University of FribourgFribourgSwitzerland

Personalised recommendations