Abstract
This paper investigates the estimation of fusion weights under varying acoustic noise conditions for audio-visual multi-level hybrid fusion strategy in speaker identification. The multi-level fusion combines model level and decision level fusion via dynamic Bayesian networks (DBNs). A novel methodology known as support vector regression (SVR) is utilized to estimate the fusion weights directly from audio features; Sigma-Pi network sampling method is also incorporated to reduce feature dimensions. Experiments on the homegrown Chinese database and CMU English database both demonstrate that the method improves the accuracies of audio-visual bimodal speaker identification under dynamically varying acoustic noise conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Senior, A., Neti, C., Maison, B.: On the Use of Visual Information for Improving Audiobased Speaker Recognition. In: Audio-visual Speech Processing Conf. (1999) 108–111
Nefian, A.V., Liang, L.H., Fu, T.Y., Liu, X.X.: A Bayesian Approach to Audio-Visual Speaker Identification. In: Proc. 4th Int. Conf. AVBPA, Vol. 2688 (2003) 761–769
Chibelushi, C.C., Deravi, F., Mason, J.S.D.: A Review of Speech-based Bimodal Recognition. IEEE Trans. Multimedia 4 (2002) 23–37
Wu, Z.Y., Cai, L.H., Meng, M.H.: Multi-level Fusion of Audio and Visual Features for Speaker Identification. In: Proc. Int. Conf. Biometrics, LNCS 3832 (2006) 493–499
Scholkopf, B., Smola, A.J., Williamson, R.C., Bartlett, P.L.: New Support Vector Algorithms. Neural Computation 12 (2000) 1083–1121
Gramß, T., Strube, H.W.: Recognition of Isolated Words based on Psychoacoustics and Neurobiology. Speech Communication 9 (1990) 35–40
Chen, T.: Audiovisual Speech Processing. IEEE Trans. Signal Processing 18 (2001) 9–21
Bilmes, J., Zweig, G.: The Graphical Models Toolkit: An Open Source Software System for Speech and Time-series Processing. In: Proc. Int. Conf. ICASSP. (2002) 3916–3919
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Wu, Z., Cai, L., Meng, H.M. (2006). Weight Estimation for Audio-Visual Multi-level Fusion in Bimodal Speaker Identification. In: Huang, DS., Li, K., Irwin, G.W. (eds) Intelligent Computing in Signal Processing and Pattern Recognition. Lecture Notes in Control and Information Sciences, vol 345. Springer, Berlin, Heidelberg . https://doi.org/10.1007/978-3-540-37258-5_144
Download citation
DOI: https://doi.org/10.1007/978-3-540-37258-5_144
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37257-8
Online ISBN: 978-3-540-37258-5
eBook Packages: EngineeringEngineering (R0)