Skip to main content
SpringerLink
Log in

Speaker Diarization Using Deep Recurrent Convolutional Neural Networks for Speaker Embeddings

  • Conference paper
  • First Online:
Information Systems Architecture and Technology: Proceedings of 38th International Conference on Information Systems Architecture and Technology – ISAT 2017 (ISAT 2017)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 655))

Abstract

In this paper we propose a new method of speaker diarization that employs a deep learning architecture to learn speaker embeddings. In contrast to the traditional approaches that build their speaker embeddings using manually hand-crafted spectral features, we propose to train for this purpose a recurrent convolutional neural network applied directly on magnitude spectrograms. To compare our approach with the state of the art, we collect and release for the public an additional dataset of over 6 h of fully annotated broadcast material. The results of our evaluation on the new dataset and three other benchmark datasets show that our proposed method significantly outperforms the competitors and reduces diarization error rate by a large margin of over 30% with respect to the baseline.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://voicebiometry.org/.

  2. 2.

    http://github.com/cyrta/broadcast-news-videos-dataset.

  3. 3.

    http://yaafe.sourceforge.net/.

References

  1. Miro, X.A., Bozonnet, S., Evans, N.W.D., Fredouille, C., Friedland, G., Vinyals, O.: Speaker diarization: a review of recent research. IEEE Trans. Audio Speech Lang. Process. 20(2), 356–370 (2012)

    Article  Google Scholar 

  2. Gupta, V., Kenny, P., Ouellet, P., Stafylakis, T.: I-vector-based speaker adaptation of deep neural networks for French broadcast audio transcription. In: ICASSP (2014)

    Google Scholar 

  3. Liu, Y., Tian, Y., He, L., Liu, J.: Investigating various diarization algorithms for speaker in the wild (SITW) speaker recognition challenge. In: Interspeech (2016)

    Google Scholar 

  4. Le Lan, G., Meignier, S., Charlet, D., Deleglise, P.: Speaker diarization with unsupervised training framework. In: ICASSP (2016)

    Google Scholar 

  5. Woubie, A., Luque, J., Hernando, J.: Short-and long-term speech features for hybrid hmm-i-vector based speaker diarization system. In: Odyssey (2016)

    Google Scholar 

  6. Bredin, H., Gelly, G.: Improving speaker diarization of tv series using talking-face detection and clustering. In: ACM on Multimedia Conference (2016)

    Google Scholar 

  7. Xu, Y., McLoughlin, I., Song, Y., Wu, K.: Improved i-vector representation for speaker diarization. Circ. Syst. Sig. Process. 35(9), 3393–3404 (2016)

    Article  MathSciNet  Google Scholar 

  8. Ferras, M., Madikeri, S., Motlicek, P., Bourlard, H.: Systemfusion and speaker linking for longitudinal diarization of tv shows. In: ICASSP (2016)

    Google Scholar 

  9. Mermelstein, P.: Distance measures for speech recognition, psychological and instrumental. Pattern Recog. Artif. Intell. 116, 374–388 (1976)

    Google Scholar 

  10. Hermansky, H.: Perceptual linear predictive (PLP) analysis of speech. J. Acoust. Soc. Am. 87(4), 1738–1752 (1990)

    Article  Google Scholar 

  11. Hermansky, H., Morgan, N.: Rasta processing of speech. IEEE Trans. Audio Speech Lang. Process. 2(4), 578–589 (1994)

    Article  Google Scholar 

  12. Sainath, T.N., Kingsbury, B., Mohamed, A., Ramabhadran, B.: Learning filter banks within a deep neural network framework. In: Workshop on Automatic Speech Recognition and Understanding (2013)

    Google Scholar 

  13. Zhu, Z., Engel, J.H., Hannun, A.Y.: Learning multiscale features directly from waveforms. In: Interspeech (2016)

    Google Scholar 

  14. Hoshen, Y., Weiss, R.J., Wilson, K.W.: Speecha coustic modeling from raw multi channel waveforms. In: ICASSP (2015)

    Google Scholar 

  15. Palaz, D., Magimai-Doss, M., Collobert, R.: Analysis of CNN-based speech recognition system using raw speech as input. In: Interspeech (2015)

    Google Scholar 

  16. Lukic, Y., Vogt, C., Dürr, O., Stadelmann, T.: Speaker identification and clustering using convolutional neural networks. In: International Workshop on Machine Learning for Signal Processing (MLSP) (2016)

    Google Scholar 

  17. Zuo, Z., Shuai, B., Wang, G., Liu, X., Wang, X., Wang, B., Chen, Y.: Convolutional recurrent neural networks: learning spatial dependencies for image representation. In: CVPR (2015)

    Google Scholar 

  18. Cakir, E., Adavanne, S., Parascandolo, G., Drossos, K., Virtanen, T.: Convolutional recurrent neural networks for bird audio detection. In: ICASSP (2017)

    Google Scholar 

  19. Snyder, D., Ghahremani, P., Povey, D., Garcia-Romero, D., Carmiel, Y., Khudanpur, S.: Deep neural network-based speaker embeddings for end-to-end speaker verification. In: IEEE Spoken Language Technology Workshop (2016)

    Google Scholar 

  20. Yella, S.H.: Speaker diarization of spontaneous meeting room conversations. Ph.D. dissertation, Ecole Polytechnique Federale de Lausanne (2015)

    Google Scholar 

  21. Sell, G., Garcia-Romero, D.: Speaker diarization with plda i-vector scoring and unsupervised calibration. In: 2014 IEEE Spoken Language Technology Workshop (2014)

    Google Scholar 

  22. Vesnicer, B., Zganec-Gros, J., Dobrisek, S., Struc, V.: Incorporating duration information into i-vector-based speaker recognition systems. In: Odyssey: The Speaker and Language Recognition Workshop, pp. 241–248 (2014)

    Google Scholar 

  23. Mami, Y., Charlet, D.: Speaker identification by location in an optimal space of anchor models. In: Interspeech (2002)

    Google Scholar 

  24. Rouvier, M., Bousquet, P., Favre, B.: Speaker diarization through speaker embeddings. In: 23rd European Signal Process- ing Conference, EUSIPCO (2015)

    Google Scholar 

  25. Bredin, H.: Tristounet: triplet loss for speaker turn embedding. CoRR, abs/1609.04301 (2016)

    Google Scholar 

  26. Garcia-Romero, D., Snyder, D., Sell, G., Povey, D., McCree, A.: Speaker diarization using deep neural networks. In: ICASSP (2017)

    Google Scholar 

  27. Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M.A., Schuller, B., Zafeiriou, S.: Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network. In: ICASSP (2016)

    Google Scholar 

  28. Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: EMNLP (2015)

    Google Scholar 

  29. Choi, K., Fazekas, G., Sandler, M., Cho, K.: Convolutional recurrent neural networks for music classification. arXiv preprint arXiv:1609.04243 (2016)

  30. Ghahremani, P., Manohar, V., Povey, D., Khudanpur, S.: Acoustic modelling from the signal domain using CNNs. In: Interspeech 2016 (2016)

    Google Scholar 

  31. Dieleman, S., Schrauwen, B.: End-to-end learning for music audio. In: ICASSP (2014)

    Google Scholar 

  32. Patterson, R., Nimmo-Smith, I., Holdsworth, J., Rice, P.: An efficient auditory filterbank based on the gammatone function. A meeting of the IOC Speech Group on Auditory Modelling at RSRE, vol. 2(7) (1987)

    Google Scholar 

  33. Brown, J.C.: Calculation of a constantq spectral transform. J. Acoust. Soc. Am. 89(1), 425–434 (1991)

    Article  Google Scholar 

  34. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. CoRR, abs/1502.03167 (2015)

    Google Scholar 

  35. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555 (2014)

    Google Scholar 

  36. Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). CoRR, abs/1511.07289 (2015)

    Google Scholar 

  37. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580 (2012)

    Google Scholar 

  38. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, abs/1412.6980 (2014)

    Google Scholar 

  39. Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kraaij, W., Kronenthal, M., Lathoud, G., Lincoln, M., Lisowska, A., McCowan, I., Post, W., Reidsma, D., Wellner, P.: The ami meeting corpus: a pre-announcement. In: MLMI (2006)

    Google Scholar 

  40. Janin, A., Baron, D., Edwards, J., Ellis, D., Gelbart, D., Morgan, N., Peskin, B., Pfau, T., Shriberg, E., Stolcke, A., Wooters, C.: The ICSI meeting corpus, pp. 364–367 (2003)

    Google Scholar 

  41. Schmidt, L., Sharifi, M., Moreno, I.L.: Large-scale speaker identification. In ICASSP (2014)

    Google Scholar 

  42. Chollet, F.: Keras (2015). https://github.com/fchollet/keras

  43. Al-Rfou, R., et. al.: Theano: a python framework for fast computation of mathematical expressions. CoRR, abs/1605.02688 (2016)

    Google Scholar 

  44. Meignier, S., Merlin, T.: Lium spkdiarization: an open source toolkit for diarization. In: CMU SPUD Workshop (2010)

    Google Scholar 

  45. Hershey, J.R., Chen, Z., Roux, J.L., Watanabe, S.: Deep clustering: discriminative embeddings for segmentation and separation. In: ICASSP (2016)

    Google Scholar 

Download references

Acknowledgment

The authors would like to thank NowThisMedia Inc. for enabling this research.

Author information

Authors and Affiliations

  1. Tooploox, Warsaw, Poland

    Pawel Cyrta, Tomasz Trzciński & Wojciech Stokowiec

  2. Warsaw University of Technology, Warsaw, Poland

    Tomasz Trzciński

  3. Polish-Japanese Academy of Information Technology, Warsaw, Poland

    Wojciech Stokowiec

Authors
  1. Pawel Cyrta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomasz Trzciński
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wojciech Stokowiec
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pawel Cyrta .

Editor information

Editors and Affiliations

  1. Department of Computer Science, Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Leszek Borzemski

  2. Department of Computer Science, Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Jerzy Świątek

  3. Department of Management Systems, Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Zofia Wilimowska

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Cite this paper

Cyrta, P., Trzciński, T., Stokowiec, W. (2018). Speaker Diarization Using Deep Recurrent Convolutional Neural Networks for Speaker Embeddings. In: Borzemski, L., Świątek, J., Wilimowska, Z. (eds) Information Systems Architecture and Technology: Proceedings of 38th International Conference on Information Systems Architecture and Technology – ISAT 2017. ISAT 2017. Advances in Intelligent Systems and Computing, vol 655. Springer, Cham. https://doi.org/10.1007/978-3-319-67220-5_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67220-5_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67219-9

  • Online ISBN: 978-3-319-67220-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation