Client and Speech Detection System for Intelligent Infokiosk
Timely attraction of a client and detection of his/her speech message in real noisy conditions are main difficulties at deployment of speech and multimodal interfaces in information kiosks. Combination of sound source localization, voice activity and face detection technologies allowed to determine client mouth coordinates and extract boundaries of speech signal appeared in the kiosk dialogue area. Talking head model based on audio-visual speech synthesis immediately greets the client, when her face is captured in the video-monitoring area, in order to attract him/her to the information service before leaving the interaction area. Client’s face tracking is also used for turning the talking head in direction to the client that significantly improves the naturalness of interaction. The developed infokiosk set in the institute hall provides information about structure and staff of laboratories. Statistics of human-kiosk interaction is accumulated within last six months in 2009.
KeywordsMultimodal interfaces information kiosk sound source localization face tracking talking head voice activity detection
Unable to display preview. Download preview PDF.
- 2.Johnston, M., Bangalore, S.: MATCHkiosk: A Multimodal Interactive City Guide. In: Association of Computational Linguistics ACL 2004, Barcelona, pp. 223–226 (2004)Google Scholar
- 4.Zhang, C., et al.: Boosting-Based Multimodal Speaker Detection for Distributed Meeting Videos. MultMed. 10(8), 1541–1552 (2008)Google Scholar
- 5.Lienhart, R., Maydt, J.: An Extended Set of Haar-like Features for Rapid Object Detection. In: 9th IEEE International Conference on Image Processing ICIP 2002, Rochester, New York, pp. 900–903 (2002)Google Scholar
- 6.Karpov, A., Tsirulnik, L., Krnoul, Z., Ronzhin, A., Lobanov, B., Zelezny, M.: Audio-Visual Speech Asynchrony Modeling in a Talking Head. In: 10th International Conference Interspeech-2009, pp. 2911–2914. ISCA, Brighton (2009)Google Scholar
- 8.Brandstein, M., Ward, D.: Microphone Arrays. Springer, Heidelberg (2000)Google Scholar
- 10.Lathoud, G., McCowan, I.: A Sector-Based Approach for Localization of Multiple Speakers with Microphone Arrays. In: ISCA Workshop on Statistical and Perceptual Audio Processing SAPA 2004. ISCA, Jeiu (2004)Google Scholar
- 11.Omologo, M., Svaizer, P.: Acoustic Event Localization Using a Crosspower-Spectrum Phase Based Technique. In: International Conference on Acoustics, Speech and Signal Processing ICASSP 1994, pp. 273–276. IEEE, Australia (1994)Google Scholar