Fusion of Acoustic and Prosodic Features for Speaker Clustering

  • Janez Žibert
  • France Mihelič
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5729)


This work focus on a speaker clustering methods that are used in speaker diarization systems. The purpose of speaker clustering is to associate together segments that belong to the same speakers. It is usually applied in the last stage of the speaker-diarization process. We concentrate on developing of proper representations of speaker segments for clustering and explore different similarity measures for joining speaker segments together. We realize two different competitive systems. The first is a standard approach using a bottom-up agglomerative clustering principle with the Bayesian Information Criterion (BIC) as a merging criterion. In the next approach a fusion speaker clustering system is developed, where the speaker segments are modeled by acoustic and prosody representations. The idea here is to additionally model the speaker prosody characteristics and add it to basic acoustic information estimated from the speaker segments. We construct 10 basic prosody features derived from the energy of the audio signals, the estimated pitch contours, and the recognized voiced and unvoiced regions in speech. In this way we impose higher-level information in the representations of the speaker segments, which leads to improved clustering of the segments in the case of similar speaker acoustic characteristics or poor acoustic conditions.


Bayesian Information Criterion Prosodic Feature Broadcast News Speech Detection Prosodic Phrase 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, S., Gopalakrishnan, P.S.: Speaker, environment and channel change detection and clustering via the Bayesian information criterion. In: Proceedings of the DARPA Speech Recognition Workshop, Lansdowne, Virginia, USA, pp. 127–132 (1998)Google Scholar
  2. 2.
    Delacourt, P., Bonastre, J., Fredouille, C., Merlin, T., Wellekens, C.: A Speaker Tracking System Based on Speaker Turn Detection for NIST Evaluation. In: Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2000), Istanbul, Turkey (June 2006)Google Scholar
  3. 3.
    Fiscus, J.G., Garofolo, J.S., Le, A., Martin, A.F., Pallett, D.S., Przybocki, M.A., Sanders, G.: Results of the Fall 2004 STT and MDE Evaluation. In: Proceedings of the Fall 2004 Rich Transcription Workshop, Palisades, NY, USA (2004)Google Scholar
  4. 4.
    Gallwitz, F., Niemann, H., Noth, E., Warnke, V.: Integrated recognition of words and prosodic phrase boundaries. Speech Communication 36(1-2), 81–95 (2002)CrossRefGoogle Scholar
  5. 5.
    Jain, A., Nandakumar, K., Ross, A.: Score normalization in multimodal biometric systems. Pattern Recognition 38(12), 2270–2285 (2005)CrossRefGoogle Scholar
  6. 6.
    Matsoukas, S., Schwartz, R., Jin, H., Nguyen, L.: Practical Implementations of Speaker-Adaptive Training. In: Proceedings of the 1997 DARPA Speech Recognition Workshop, Chantilly VA, USA (February 1997)Google Scholar
  7. 7.
    Meignier, S., Bonastre, J.-F., Fredouille, C., Merlin, T.: Evolutive HMM for Multi-Speaker Tracking System. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Istanbul, Turkey (2000)Google Scholar
  8. 8.
    Shriberg, E., Ferrer, L., Kajarekar, S., Venkataraman, A., Stolcke, A.: Modeling prosodic feature sequences for speaker recognition. Speech Communication 46(3-4), 455–472 (2005)CrossRefGoogle Scholar
  9. 9.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 2nd edn. Academic Press, Elsevier, USA (2003)Google Scholar
  10. 10.
    Tranter, S., Reynolds, D.: An Overview of Automatic Speaker Diarisation Systems. IEEE Transactions on Speech, Audio and Language Processing, Special Issue on Rich Transcription 14(5), 1557–1565 (2006)CrossRefGoogle Scholar
  11. 11.
    Woodland, P.C.: The development of the HTK Broadcast News transcription system: An overview. Speech Communication 37(1-2), 47–67 (2002)CrossRefGoogle Scholar
  12. 12.
    Žibert, J., Mihelič, F.: Development of Slovenian Broadcast News Speech Database. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal, May 2004, pp. 2095–2098 (2004)Google Scholar
  13. 13.
    Žibert, J.: et al.: The COST278 Broadcast News Segmentation and Speaker Clustering Evaluation - Overview, Methodology, Systems, Results. In: Proceedings of Interspeech 2005, Lisbon, Portugal, pp. 629–632 (2005)Google Scholar
  14. 14.
    Žibert, J., Vesnicer, B., Mihelič, F.: Novel Approaches to Speech Detection in the Processing of Continuous Audio Streams. In: Grimm, M., Kroschel, K. (eds.) Robust Speech Recognition and Understanding, pp. 23–48. I-Tech Education and Publishing, Croatia (2007)Google Scholar
  15. 15.
    Žibert, J., Mihelič, F.: Novel approaches to speaker clustering for speaker diarization in audio broadcast news data. In: Mihelič, F., Žibert, J. (eds.) Speech recognition: technologies and applications. Artificial intelligence series, pp. 341–362 (2008) ISBN 978-953-7619-29-9Google Scholar
  16. 16.
    Zhu, X., Barras, C., Meignier, S., Gauvain, J.-L.: Combining Speaker Identification and BIC for Speaker Diarization. In: Proceedings of Interspeech 2005 - Eurospeech, Lisbon, Portugal, pp. 2441–2444 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Janez Žibert
    • 1
  • France Mihelič
    • 2
  1. 1.Primorska Institute of Natural Sciences and TechnologyUniversity of PrimorskaKoperSlovenia
  2. 2.Faculty of Electrical EngineeringUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations