Text Driven Face-Video Synthesis Using GMM and Spatial Correlation

  • Dereje Teferi
  • Maycel I. Faraj
  • Josef Bigun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4522)


Liveness detection is increasingly planned to be incorporated into biometric systems to reduce the risk of spoofing and impersonation. Some of the techniques used include detection of motion of the head while posing/speaking, iris size in varying illumination, fingerprint sweat, text-prompted speech, speech-to-lip motion synchronization etc. In this paper, we propose to build a biometric signal to test attack resilience of biometric systems by creating a text-driven video synthesis of faces. We synthesize new realistic looking video sequences from real image sequences representing utterance of digits. We determine the image sequences for each digit by using a GMM based speech recognizer. Then, depending on system prompt (sequence of digits) our method regenerates a video signal to test attack resilience of a biometric system that asks for random digit utterances to prevent play-back of pre-recorded data representing both audio and images. The discontinuities in the new image sequence, created at the connection of each digit, are removed by using a frame prediction algorithm that makes use of the well known block matching algorithm. Other uses of our results include web-based video communication for electronic commerce and frame interpolation for low frame rate video.


Motion Vector Gaussian Mixture Model Motion Estimation Video Signal Speech Recognition System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Jain, A.K., Ross, A., Prebhakar, S.: An introduction to biometric recognition. IEEE Transactions on Circuits and Systems for Video Technology (Special Issue on Image- and Video-Based Biometrics) 14(1) (2004)Google Scholar
  2. 2.
    Jain, A.K., Pankanti, S., Prabhakar, S., Hong, L., Ross, A.: Biometrics: A grand challenge. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, pp. 935–942 (2004)Google Scholar
  3. 3.
    Ortega-Garcia, J., Bigun, J., Reynolds, D., Gonzalez-Rodriguez, J.: Authentication gets personal with biometrics. IEEE Signal Processing Magazine 21(2), 50–62 (2004)CrossRefGoogle Scholar
  4. 4.
    Faundez-Zanuy, M.: Biometric security technology. IEEE Aerospace and Electronic Systems Magazine 21(6), 15–26 (2006)CrossRefGoogle Scholar
  5. 5.
    Bigun, J., Fronthaler, H., Kollreider, K.: Assuring liveness in biometric identity authentication by real-time face tracking. In: IEEE international Conference on Computational Intelligence for Homeland Security and Personal Safety, Venice, Italy, July 2004, IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  6. 6.
    Stephanie, A., Schukers, C.: Spoofing and anti-spoofing measures. Information Security Technical Report (2002)Google Scholar
  7. 7.
    Ratha, N.K., Connell, J.H., Bolle, R.M.: Enhancing security and privacy in biometrics-based authentication systems. IBM Systems Journal 40(2), 614–634 (2001)CrossRefGoogle Scholar
  8. 8.
    Kollreider, K., Fronthaler, H., Bigun, J.: Evaluating liveness by face images and the structure tensor. In: AutoID 2005: Fourth Workshop on Automatic Identification Advanced Technologies, October 2005, pp. 75–80. IEEE Computer Society Press, Los Alamitos (2005)CrossRefGoogle Scholar
  9. 9.
    Li, J., Wang, Y., Tan, T., Jain, A.K.: Live face detection based on the analysis of fourier spectra. In: Jain, A.K., Ratha, N.K. (eds.) Biometric Technology for Human Identification, August 2004. Proceedings of the SPIE, vol. 5404, pp. 296–303 (2004)Google Scholar
  10. 10.
    Faraj, M.I., Bigun, J.: Person verification by lip-motion. In: Computer Vision and Pattern Recognition Workshop (CVPRW), June 2006, pp. 37–45 (2006)Google Scholar
  11. 11.
    Messer, K., Matas, J., Kitler, J., Luettin, J., Maitre, G.: XM2VTSDB: The extended M2VTS database. In: 2nd International Conference on Audio and Video-based Biometric Person Authentication (AVBPA’99), pp. 72–77 (1999)Google Scholar
  12. 12.
    Bigun, J.: Vision with Direction: A Systematic Introduction to Image Processing and Computer Vision. Springer, Heidelberg (2006)Google Scholar
  13. 13.
    Jain, J.R., Jain, A.K.: Displacement measurement and its application in interframe image coding. IEEE Transactions on Communication 29, 1799–1808 (1981)CrossRefGoogle Scholar
  14. 14.
    Gyaourova, A., Kamath, C., Cheung, S.C.: Block matching for object tracking. Technical report UCRL-TR-200271, Laurence Livermore Technical Laboratory (October 2003)Google Scholar
  15. 15.
    Cheng, K.W., Chan, S.C.: Fast block matching algorithms for motion estimation. In: ICASSP-96: IEEE International Conference on Acoustic Speech and Signal Processing, vol. 4, May 1996, pp. 2311–2314. IEEE Computer Society Press, Los Alamitos (1996)CrossRefGoogle Scholar
  16. 16.
    Reynolds, D., Rose, R.: Robust text independent speaker identification using gaussian mixture models. IEEE Transactions on Speech and Audio Processing 3(1), 72–83 (1995)CrossRefGoogle Scholar
  17. 17.
    Young, S., Evermann, G., Gales, M., Hein, T., Kershaw, D., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V., Woodland, P.: The htk book. for version 3.3 (April 2005),
  18. 18.
    Veeravalli, A.G., Pan, W., Adhami, R., Cox, P.G.: A tutorial on using hidden markov models for phoneme recognition. In: Thirty-Seventh Southeastern Symposiumon System Theory, SSST (2005)Google Scholar
  19. 19.
    Aly, S.G., Youssef, A.: Real-time motion based frame estimation in video lossy transmission. In: Symposium on Applications and the Internet, January 2001, pp. 139–146 (2001)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Dereje Teferi
    • 1
  • Maycel I. Faraj
    • 1
  • Josef Bigun
    • 1
  1. 1.School of Information Science, Computer, and Electrical Engineering (IDE), Halmstad University, P.O.Box 823, SE-301 18, HalmstadSweden

Personalised recommendations