Multimodal biometric scheme for human authentication technique based on voice and face recognition fusion

  • Anter AbozaidEmail author
  • Ayman Haggag
  • Hany Kasban
  • Mostafa Eltokhy


In this paper, an effective multimodal biometric identification approach for human authentication tool based on face and voice recognition fusion is proposed. Cepstral coefficients and statistical coefficients are employed to extract features of voice recognition and these two coefficients are compared. Face recognition features are extracted utilizing different extraction techniques, Eigenface and Principle Component Analysis (PCA) and the results are compared. Voice and face identification modality are performed using different three classifiers, Gaussian Mixture Model (GMM), Artificial Neural Network (ANN), and Support Vector Machine (SVM). The combination of biometrics systems, voice and face, into a single multimodal biometric system is performed using features fusion and scores fusion. The computer simulation experiments reveal that better results are given in case of utilizing for voice recognition the cepstral coefficients and statistical coefficients and in case of face, Eigenface and SVM experiment gives better results for face recognition. Also, in the proposed multimodal biometrics system the scores fusion performs better than other scenarios.


Multimodal biometrics SVM ANN GMM Voice identification Face recognition 



  1. 1.
    Abdel Karim N, Shukur Z (2015) Review of user authentication methods in online examination. Asian Journal of Information Technology 14(5):166–175Google Scholar
  2. 2.
    Abhishree TM, Latha J, Manikantan K, Ramachandran S (2015) Face recognition using Gabor filter based feature extraction with anisotropic diffusion as a pre-processing technique. Procedia Computer Science 45:312–321CrossRefGoogle Scholar
  3. 3.
    Agashe NM, Nimbhorkar S (2015) A survey paper on continuous authentication by multimodal biometric. International Journal of Advanced Research in Computer Engineering & Technology 4(11):4247-4253Google Scholar
  4. 4.
    Baken RJ, Orlikoff RF (2000) Clinical measurement of speech and voice – second edition. Singular Publishing Group, San DiegoGoogle Scholar
  5. 5.
    Burges C (1998) A tutorial on support vector machines for pattern recognition. In: Data mining and knowledge discovery (Volume 2). Kluwer Academic Publishers, Boston, pp 1–43Google Scholar
  6. 6.
    Chetty G, Wagner M (2008) Robust face-voice based speaker identity verification using multilevel fusion. Image Vis Comput 26:1249–1260CrossRefGoogle Scholar
  7. 7.
    Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  8. 8.
    De A, Saha A, Pal MC (2015) A human facial expression recognition model based on Eigen face approach. Procedia Computer Science 45:282–289CrossRefGoogle Scholar
  9. 9.
    Dodangeh P, Jahangir AH (2018) A biometric security scheme for wireless body area networks. Journal of Information Security and Applications 41:62–74. CrossRefGoogle Scholar
  10. 10.
    El-Bendary MAM (2015) Developing security tools of WSN and WBAN networks applications. Springer, JapanCrossRefGoogle Scholar
  11. 11.
    El-Bendary MA (2017) FEC merged with double security approach based on encrypted image steganography for different purpose in the presence of noise and different attack. Multimed Tools Appl 76(24):26463–26501CrossRefGoogle Scholar
  12. 12.
    Elmir Y, Elberrichi Z, Adjoudj R (2014) Multimodal biometric using a hierarchical fusion of a person’s face, voice, and online signature. J Inf Process Syst:555-567Google Scholar
  13. 13.
    Fookes C, Lin F, Chandran V, Sridharan S (2012) Evaluation of image resolution and super-resolution on face recognition performance. J Vis Commun Image Represent 23(1):75–93CrossRefGoogle Scholar
  14. 14.
    Gad R, El-Fishawy N, El-Sayed A, Zorkany M (2015) Multi-biometric systems: a state of the art survey and research directions. Int J Adv Comput Sci Appl 6(6):128-138Google Scholar
  15. 15.
    Galka J, Masior M, Salasa M (2014) Voice authentication embedded solution for secured access control. IEEE Trans Consum Electron 60(4):653–661CrossRefGoogle Scholar
  16. 16.
    Halvia S, Ramapurb N, Rajac KB, Prasadd S (2017) Fusion based face recognition system using 1D transform domains. Procedia Computer Science 115:383–390CrossRefGoogle Scholar
  17. 17.
    Inthavisas K, Lopresti D (2012) Secure speech biometric templates for user authentication. IET Biometrics 1(1):46–54CrossRefGoogle Scholar
  18. 18.
    Jain A, Nandakumar K, Ross A (2005) Score normalisation in multimodal biometric systems. Pattern Recogn 38:2270–2285CrossRefGoogle Scholar
  19. 19.
    Kasban H (2017) A robust multimodal biometric authentication scheme with voice and face recognition. Arab Journal of Nuclear Sciences and Applications 50(3):120–130Google Scholar
  20. 20.
    Kinnunen T, Karpov E, Franti P (2006) Real time speaker identification and verification. IEEE Trans Audio Speech Lang Process 14(1):277–288CrossRefGoogle Scholar
  21. 21.
    Kumar HCS, Janardhan NA (2016) An efficient personnel authentication through multi modal biometric system. International Journal of Scientific Engineering and Applied Science 2(1):534–543Google Scholar
  22. 22.
    Li H, Suen CY (2016) Robust face recognition based on dynamic rank representation. Pattern Recogn 60:13–24CrossRefGoogle Scholar
  23. 23.
    Liu Z, Wang H (2014) A novel speech content authentication algorithm based on Bessel–Fourier moments. Digital Signal Process 24:197–208MathSciNetCrossRefGoogle Scholar
  24. 24.
    Liu T, Mi JX, Liu Y, Li C (2016) Robust face recognition via sparse boosting representation. Neurocomputing 214:944–957CrossRefGoogle Scholar
  25. 25.
    Lumini A, Nanni L (2017) Overview of the combination of biometric matchers. Information Fusion 33:71–85CrossRefGoogle Scholar
  26. 26.
    Morgen B (2012) Voice biometrics for customer authentication. Biom Technol Today 2012(2):8–11CrossRefGoogle Scholar
  27. 27.
    Palanivel S, Yegnanarayana B (2008) Multimodal person authentication using speech, face and visual speech. Comput Vis Image Underst 109:44–55CrossRefGoogle Scholar
  28. 28.
    Poh N, Korczak J (2001) Hybrid biometric person authentication using face and voice features. International Conference, Audio and Video Based Biometric Person Authentication, Halmstad, Sweden, pp 348–353Google Scholar
  29. 29.
    Qia M, Chena J, Chen Y (2018) A secure biometrics-based authentication key exchange protocol for multi-server TMIS using ECC. Comput Methods Prog Biomed 164:101–109CrossRefGoogle Scholar
  30. 30.
    Raghavendra R, Rao A, Kumar GH (2010) Multimodal person verification system using face and speech. Procedia Computer Science 2:181–187CrossRefGoogle Scholar
  31. 31.
    Reynolds DA, Quatieri TF, Dunn RB (2000) Speaker verification using adapted Guassian mixture models. Digital Signal Processing 10:19–41CrossRefGoogle Scholar
  32. 32.
    Soltane M (2015) Greedy expectation maximization tuning algorithm of finite GMM based face, voice and signature multi-modal biometric verification fusion systems. International Journal of Engineering & Technology 15(03):41–52Google Scholar
  33. 33.
    Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86CrossRefGoogle Scholar
  34. 34.
    Vapnik V (1998) Statistical learning theory. Wiley, New YorkzbMATHGoogle Scholar
  35. 35.
    Xuan S, Xiang S, Ma H (2016) Subclass representation-based face-recognition algorithm derived from the structure scatter of training samples. Comput Vis 10(6):493–502CrossRefGoogle Scholar
  36. 36.
    Zheng CH, Hou YF, Zhang J (2016) Improved sparse representation with low-rank representation for robust face recognition. Neurocomputing 198:114–124CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Anter Abozaid
    • 1
    Email author
  • Ayman Haggag
    • 1
  • Hany Kasban
    • 2
  • Mostafa Eltokhy
    • 1
  1. 1.Electronics Technology Department, Faculty of Industrial EducationHelwan UniversityCairoEgypt
  2. 2.Engineering Department, Nuclear Research CenterAtomic Energy AuthorityCairoEgypt

Personalised recommendations