Lipreading Using n–Gram Feature Vector

  • Preety Singh
  • Vijay Laxmi
  • Deepika Gupta
  • M. S. Gaur
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 85)


The use of n-grams is quite prevalent in the field of pattern recognition. In this paper, we use this concept to build new feature vectors from extracted parameters to be used for visual speech classification. We extract the lip contour using edge detection and connectivity analysis. The boundary is defined using six cubic curves. The visual parameters are used to build n-gram feature vectors. Two sets of classification experiments are performed with the n-gram feature vectors: using the hidden Markov model and using multiple data mining algorithms in WEKA, a tool widely used by researchers. Preliminary results show encouraging results.


Feature Vector Hide Markov Model Speech Recognition True Positive Rate Speech Sample 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alizadeh, S., Boostani, R., Asadpour, V.: Lip Feature Extraction and Reduction for HMM-Based Visual Speech Recognition Systems. In: Proc. 9th International Conference on Signal Processing (ICSP 2008), pp. 561–564 (2008)Google Scholar
  2. 2.
    Eveno, N., Caplier, A., Coulon, P.Y.: Accurate and Quasi-Automatic Lip Tracking. IEEE Transaction on Circuits and Video Technology 14(5), 706–715 (2004)CrossRefGoogle Scholar
  3. 3.
    Goldschen, A.J.: Continuous automatic speech recognition by lipreading. PhD thesis, George Washington University, Washington, DC, USA (1993)Google Scholar
  4. 4.
    HTK Hidden Markov Model Toolkit home page,
  5. 5.
    Matthews, I., Cootes, T.F., Bangham, J.A., Cox, S., Harvey, R.: Extraction of Visual Features for Lipreading. IEEE Trans. Pattern Analysis and Machine Intelligence 24(2), 198–213 (2002)CrossRefGoogle Scholar
  6. 6.
    Silveira, L.G., Facon, J., Borges, D.L.: Visual Speech Recognition: a solution from feature extraction to words classification. In: Proc. 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2003), Sao Carlos, Brazil, pp. 399–405. IEEE Computer Society, Los Alamitos (2003)CrossRefGoogle Scholar
  7. 7.
    Sumby, W.H., Pollack, I.: Visual Contribution to Speech Intelligibility in Noise. Journal of Acoustical Society of America 26(2), 212–215 (1954)CrossRefGoogle Scholar
  8. 8.
    University of Waikato. Open Source Machine Learning Software WEKA,
  9. 9.
    Yau, W.C., Kumar, D.K., Arjunan, S.P.: Voiceless speech recognition using dynamic visual speech features. In: Proceedings of the HCSNet workshop on Use of vision in human-computer interaction (VisHCI 2006), Canberra, Australia, pp. 93–101. Australian Computer Society, Inc. (2006)Google Scholar
  10. 10.
    Yu, K., Jiang, X., Bunke, H.: Lipreading: A classifier combination approach. Pattern Recognition Letters 18(11-13), 1421–1426 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Preety Singh
    • 1
  • Vijay Laxmi
    • 1
  • Deepika Gupta
    • 1
  • M. S. Gaur
    • 1
  1. 1.Department of Computer EngineeringMalaviya National Institute of TechnologyJaipurIndia

Personalised recommendations