Online Speaker Adaptation of an Acoustic Model Using Face Recognition
We have proposed and evaluated a novel approach for online speaker adaptation of an acoustic model based on face recognition. Instead of traditionally used audio-based speaker identification we investigated the video modality for the task of speaker detection. A simulated on-line transcription created by a Large-Vocabulary Continuous Speech Recognition (LVCSR) system for online subtitling is evaluated utilizing speaker independent acoustic models, gender dependent models and models of particular speakers. In the experiment, the speaker dependent acoustic models were trained offline, and are switched online based on the decision of a face recognizer, which reduced Word Error Rate (WER) by 12% relatively compared to speaker independent baseline system.
Keywordsacoustic model speaker adaptation face recognition multimodal processing automatic speech recognition
Unable to display preview. Download preview PDF.
- 1.Pražák, A., Zajíc, Z., Machlica, L., Psutka, J.V.: Fast Speaker Adaptation in Automatic Online Subtitling. In: International Conference on Signal Processing and Multimedia Applications, vol.(1), pp. 126–130 (2009)Google Scholar
- 6.Povey, D.: Discriminative Training for Large Vocabulary Speech Recognition. PhD thesis, Cambridge University, Engineering Department (2003)Google Scholar
- 8.Uřičář, M., Franc, V., Hlaváč, V.: Detector of Facial Landmarks Learned by the Structured Output SVM. In: VISAPP 2012: Proceedings of the 7th International Conference on Computer Vision Theory and Applications1, Rome, Italy, pp. 547–556 (2012)Google Scholar
- 9.Sonnenburg, S., Franc, V.: COFFIN: A Computational Framework for Linear SVMs. Technical Report 1, Center for Machine Perception, Czech Technical University, Prague, Czech Republic (2009)Google Scholar
- 11.El Khoury, E., Sénac, C., Joly, P.: Audiovisual diarization of people in video content. In: Multimedia Tools and Applications, pp. 1–29 (2012)Google Scholar
- 12.Bendris, M., Charlet, D., Chollet, G.: People indexing in TV-content using lip-activity and unsupervised audio-visual identity verification. In: 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI), pp. 139–144. IEEE (2011)Google Scholar