Advertisement

Audio, Video and Multimodal Person Identification in a Smart Room

  • Jordi Luque
  • Ramon Morros
  • Ainara Garde
  • Jan Anguita
  • Mireia Farrus
  • Dušan Macho
  • Ferran Marqués
  • Claudi Martínez
  • Verónica Vilaplana
  • Javier Hernando
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4122)

Abstract

In this paper, we address the modality integration issue on the example of a smart room environment aiming at enabling person identification by combining acoustic features and 2D face images. First we introduce the monomodal audio and video identification techniques and then we present the use of combined input speech and face images for person identification. The various sensory modalities, speech and faces, are processed both individually and jointly. It’s shown that the multimodal approach results in improved performance in the identification of the participants.

Keywords

Face Image Gaussian Mixture Model Person Identification Speaker Recognition Speaker Identification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Casas, J., Stiefelhagen, R., et al.: Multi-camera/multi-microphone system design for continuous room monitoring. CHIL-WP4-D4.1-V2.1-2004-07-08-CO, CHIL Consortium Deliverable D4.1, July (2004)Google Scholar
  2. 2.
    Nadeu, C., Macho, D., Hernando, J.: Time and frequency filtering of filter-bank energies for robust HMM speech recognition. Speech Communication 34, 93–114 (2001)zbMATHCrossRefGoogle Scholar
  3. 3.
    Reynolds, D.A.: Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Transactions on Speech and Audio Processing 3(1), 72–83 (1995)CrossRefGoogle Scholar
  4. 4.
    Kirby, M., Sirovich, L.: Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Trans. PAMI 12(1), 103–108 (1990)Google Scholar
  5. 5.
    Bolle, R.M., Connell, J.H., Pankanti, S., Ratha, N.K., Senior, A.W.: Guide to Biometrics. Springer, New York (2004)Google Scholar
  6. 6.
    Brunelli, R., Falavigna, D.: Person Identification Using Multiple Cues. IEEE on PAMI 17(10), 955–966 (1995)Google Scholar
  7. 7.
    Davis, S.B., Mermelstein, P.: Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustic, Speech and Signal Processing 28, 357–366 (1980)CrossRefGoogle Scholar
  8. 8.
    Furui, S.: Speaker independent isolated word recognition using dynamic features of speech spectrum. IEEE Transactions ASSP 34, 52–59 (1986)CrossRefGoogle Scholar
  9. 9.
    Messer, K., Matas, J., Kittler, J.V., Luettin, J., Maitre, G.: XM2VTSDB: The extended M2VTS Database. AVBPA99 (1999)Google Scholar
  10. 10.
    Fox, N.A., Gross, R., Chazal, P., Cohn, J.F., Reilly, R.B.: Person Identification Using Automatic Integration of Speech, Lip and Face Experts. In: Proc. of the ACM SIGMM 2003 Multimedia Biometrics Methods and Applications Workshop, WBMA’03, Berkeley, CA, pp. 25–32. ACM Press, New York (2003)CrossRefGoogle Scholar
  11. 11.
    Indovina, M., Uludag, U., Snelik, R., Mink, A., Jain, A.: Multimodal Biometric Authentication Methods: A COTS Approach. In: Proceedings of the MMUA 2003, Workshop on Multimodal User Authentication, Santa Barbara, CA, pp. 99–106 (2003)Google Scholar
  12. 12.
    Lucey, S., Chen, T.: Improved Audio-visual Speaker Recognition via the Use of a Hybrid Combination Strategy. In: The 4th International Conference on Audio- and Video- Based Biometric Person Authentication, Guilford, U.K. (2003)Google Scholar
  13. 13.
    Wang, Y., Wangm, Y., Tan, T.: Combining Fingerprint and Voiceprint Biometrics for Identity Verification: an Experimental Comparison. In: Proceedings of the ICBA, Hong Kong, China, pp. 663–670 (2004)Google Scholar
  14. 14.
    D. Mostefa, et al.: CLEAR Evaluation Plan v1.1 (2006), http://www.clear-evaluation.org/downloads/chil-clear-v1.1-2006-02-21.pdf

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Jordi Luque
    • 1
  • Ramon Morros
    • 1
  • Ainara Garde
    • 1
  • Jan Anguita
    • 1
  • Mireia Farrus
    • 1
  • Dušan Macho
    • 1
  • Ferran Marqués
    • 1
  • Claudi Martínez
    • 1
  • Verónica Vilaplana
    • 1
  • Javier Hernando
    • 1
  1. 1.Universitat Politècnica de Catalunya, Jordi Girona 1-3, Campus Nord Edifici D5, 08034 BarcelonaSpain

Personalised recommendations