Audiovisual Liveness Detection

  • Aleksandr Melnikov
  • Rasim Akhunzyanov
  • Oleg Kudashev
  • Eugene Luckyanets
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9280)


Although multi-modal (e.g. voice and face) biometric verification systems were in active development and showed impressive performance they need to be protected from spoofing attacks. In this paper we present methods for verifying face liveness based on estimation of synchrony between audio stream and lips movements track during the pronunciation of passphrase. The passphrase consists of a random set of the predetermined English words that are generated dynamically for each verification attempt. Lip movements extraction is performed by using of so-called Constrained Local Model of face shape. Audio stream is used to determine time intervals of pronounced words by means of automatic segmentation. Estimation of synchrony is done by analysis of lip movements for each word by employing a feedforward neural network and a Gaussian naive Bayes classifier. Finally, liveness score assessment is performed by averaging of individual word predictions during verification phrase utterance. For GRID corpus dataset average EER of 4.38% was achieved.


Bimodal Liveness detection Anti-spoofing Voice features Face features 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ali, A., Deravi, F., Hoque, S.: Liveness detection using gaze collinearity. In: 2012 Third International Conference on Emerging Security Technologies (EST), pp. 62–65. IEEE (2012)Google Scholar
  2. 2.
    Baltrusaitis, T., Robinson, P., Morency, L.: 3d constrained local model for rigid and non-rigid facial tracking. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2610–2617. IEEE (2012)Google Scholar
  3. 3.
    Bao, W., Li, H., Li, N., Jiang, W.: A liveness detection method for face recognition based on optical flow field. In: International Conference on Image Analysis and Signal Processing, IASP 2009, pp. 233–236. IEEE (2009)Google Scholar
  4. 4.
    Çetingül, H.E., Erzin, E., Yemez, Y., Tekalp, A.M.: Multimodal speaker/speech recognition using lip motion, lip texture and audio. Signal Processing 86(12), 3549–3558 (2006)CrossRefzbMATHGoogle Scholar
  5. 5.
    Chakraborty, S., Das, D.: An overview of face liveness detection (2014). arXiv preprint arXiv:1405.2227
  6. 6.
    Chetty, G., Wagner, M.: Automated lip feature extraction for liveness verification in audio-video authentication. Proc. Image and Vision Computing, 17–22 (2004)Google Scholar
  7. 7.
    Chetty, G., Wagner, M.: Multi-level liveness verification for face-voice biometric authentication. In: 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference, pp. 1–6. IEEE (2006)Google Scholar
  8. 8.
    Cooke, M., Barker, J., Cunningham, S., Shao, X.: An audio-visual corpus for speech perception and automatic speech recognition. The Journal of the Acoustical Society of America 120(5), 2421–2424 (2006)CrossRefGoogle Scholar
  9. 9.
    Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. In BMVC, vol. 2, pp. 6. Citeseer (2006)Google Scholar
  10. 10.
    Das, D., Chakraborty, S.: Face liveness detection based on frequency and micro-texture analysis. In: 2014 International Conference on Advances in Engineering and Technology Research (ICAETR), pp. 1–4. IEEE (2014)Google Scholar
  11. 11.
    Dean, D., Sridharan, S.: Dynamic visual features for audio-visual speaker verification. Computer Speech & Language 24(2), 136–149 (2010)CrossRefGoogle Scholar
  12. 12.
    Kim, G., Eum, S., Suhr, J.K., Kim, D.I., Park, K.R., Kim, J.: Face liveness detection based on texture and frequency analyses. In: 2012 5th IAPR International Conference on Biometrics (ICB), pp. 67–72. IEEE (2012)Google Scholar
  13. 13.
    Kim, S., Yu, S., Kim, K., Ban, Y., Lee, S.: Face liveness detection using variable focusing. In: 2013 International Conference on Biometrics (ICB), pp. 1–6. IEEE (2013)Google Scholar
  14. 14.
    Kinnunen, T., Wu, Z.-Z., Lee, K.A., Sedlak, F., Chng, E.S., Li, H.: Vulnerability of speaker verification systems against voice conversion spoofing attacks: The case of telephone speech. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4401–4404, March 2012Google Scholar
  15. 15.
    Kollreider, K., Fronthaler, H., Bigun, J.: Evaluating liveness by face images and the structure tensor. In: Fourth IEEE Workshop on Automatic Identification Advanced Technologies, 2005, pp. 75–80. IEEE (2005)Google Scholar
  16. 16.
    Kollreider, K., Fronthaler, H., Bigun, J.: Non-intrusive liveness detection by face images. Image and Vision Computing 27(3), 233–244 (2009)CrossRefGoogle Scholar
  17. 17.
    Kollreider, K., Fronthaler, H., Faraj, M.I., Bigun, J.: Real-time face detection and motion analysis with application in “liveness” assessment. IEEE Transactions on Information Forensics and Security 2(3), 548–558 (2007)CrossRefGoogle Scholar
  18. 18.
    Komulainen, J., Hadid, A., Pietikainen, M.: Context based face anti-spoofing. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2013)Google Scholar
  19. 19.
    Lagorio, A., Tistarelli, M., Cadoni, M., Fookes, C., Sridharan, S.: Liveness detection based on 3d face shape analysis. In: 2013 International Workshop on Biometrics and Forensics (IWBF), pp. 1–4. IEEE (2013)Google Scholar
  20. 20.
    Määttä, J., Hadid, A., Pietikäinen, M.: Face spoofing detection from single images using texture and local shape analysis. IET Biometrics 1(1), 3–10 (2012)CrossRefGoogle Scholar
  21. 21.
    Maatta, J., Hadid, A., Pietikainen, M.: Face spoofing detection from single images using micro-texture analysis. In: 2011 International Joint Conference on Biometrics (IJCB), pp. 1–7. IEEE (2011)Google Scholar
  22. 22.
    Marcel, S., Nixon, M.S., Li, S.Z.: Handbook of Biometric Anti-Spoofing. Springer (2014)Google Scholar
  23. 23.
    Pan, G., Sun, L., Zhaohui, W., Wang, Y.: Monocular camera-based face liveness detection by combining eyeblink and scene context. Telecommunication Systems 47(3–4), 215–225 (2011)CrossRefGoogle Scholar
  24. 24.
    Peixoto, B., Michelassi, C., Rocha, A.: Face liveness detection under bad illumination conditions. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 3557–3560. IEEE (2011)Google Scholar
  25. 25.
    Shchemelinin, V., Topchina, M., Simonchik, K.: Vulnerability of voice verification systems to spoofing attacks by TTS voices based on automatically labeled telephone speech. In: Ronzhin, A., Potapova, R., Delic, V. (eds.) SPECOM 2014. LNCS, vol. 8773, pp. 475–481. Springer, Heidelberg (2014) Google Scholar
  26. 26.
    Slaney, M., Covell, M.: Facesync: a linear operator for measuring synchronization of video facial images and audio tracks. In: NIPS, pp. 814–820 (2000)Google Scholar
  27. 27.
    Sun, L., Pan, G., Wu, Z., Lao, S.: Blinking-based live face detection using conditional random fields. In: Lee, S.-W., Li, S.Z. (eds.) ICB 2007. LNCS, vol. 4642, pp. 252–260. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  28. 28.
    Taigman, Y., Yang, M., Ranzato, M.A., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1701–1708. IEEE (2014)Google Scholar
  29. 29.
    Tan, X., Li, Y., Liu, J., Jiang, L.: Face liveness detection from a single image with sparse low rank bilinear discriminative model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 504–517. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  30. 30.
    Wang, T., Yang, J., Lei, Z., Liao, S., Li, S.Z.: Face liveness detection using 3d structure recovered from a single camera. In: 2013 International Conference on Biometrics (ICB), pp. 1–6. IEEE (2013)Google Scholar
  31. 31.
    Zhizheng, W., Evans, N., Kinnunen, T., Yamagishi, J., Alegre, F., Li, H.: Spoofing and countermeasures for speaker verification: A survey. Speech Communication 66, 130–153 (2015)CrossRefGoogle Scholar
  32. 32.
    Yan, J., Zhang, Z., Lei, Z., Yi, D., Li, S.Z.: Face liveness detection by exploring multiple scenic clues. In: 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV), pp. 188–193. IEEE (2012)Google Scholar
  33. 33.
    Yang, L.: Face liveness detection by focusing on frontal faces and image backgrounds. In: 2014 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), pp. 93–97. IEEE (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Aleksandr Melnikov
    • 2
  • Rasim Akhunzyanov
    • 1
  • Oleg Kudashev
    • 2
  • Eugene Luckyanets
    • 1
    • 2
  1. 1.ITMO UniversitySt. PetersburgRussia
  2. 2.STC-innovations Ltd.St. PetersburgRussia

Personalised recommendations