Knowledge and Information Systems

, Volume 20, Issue 2, pp 137–156 | Cite as

On classification and segmentation of massive audio data streams

  • Charu C. AggarwalEmail author
Regular Paper


In recent years, the proliferation of VOIP data has created a number of applications in which it is desirable to perform quick online classification and recognition of massive voice streams. Typically such applications are encountered in real time intelligence and surveillance. In many cases, the data streams can be in compressed format, and the rate of data processing can often run at the rate of Gigabits per second. All known techniques for speaker voice analysis require the use of an offline training phase in which the system is trained with known segments of speech. The state-of-the-art method for text-independent speaker recognition is known as Gaussian mixture modeling (GMM), and it requires an iterative expectation maximization procedure for training, which cannot be implemented in real time. In many real applications (such as surveillance) it is desirable to perform the recognition process in online time, so that the system can be quickly adapted to new segments of the data. In many cases, it may also be desirable to quickly create databases of training profiles for speakers of interest. In this paper, we discuss the details of such an online voice recognition system. For this purpose, we use our micro-clustering algorithms to design concise signatures of the target speakers. One of the surprising and insightful observations from our experiences with such a system is that while it was originally designed only for efficiency, we later discovered that it was also more accurate than the widely used GMM. This was because of the conciseness of the micro-cluster model, which made it less prone to over training. This is evidence of the fact that it is often possible to get the best of both worlds and do better than complex models both from an efficiency and accuracy perspective. We present experimental results illustrating the effectiveness and efficiency of the method.


Classification Segmentation Audio streams 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aggarwal CC (2007) A framework for classification and segmentation of massive audio data streams. ACM KDD Conference, pp 1013–1017Google Scholar
  2. 2.
    Aggarwal C, Han J, Wang J, Yu P (2003a) A framework for clustering evolving data streams. VLDB Conference, pp 81–92Google Scholar
  3. 3.
    Aggarwal C (2003b) A framework for diagnosing changes in evolving data streams. ACM SIGMOD Conference, pp 575–586Google Scholar
  4. 4.
    Assaleh KT, Mammone RJ (1994) Robust cepstral features for speaker identification. In: Proceedings of the IEEE international conference on acoustics, speech and signal processingGoogle Scholar
  5. 5.
    Ellis D (URL) Semester course: music content analysis by machine learning. URL:
  6. 6.
    Gish H, Schmidt M, Mielke A (1994) A robust segmental method for text independent speaker identification. In: Proceedings of the IEEE international conference on acoustics, speech and signal processingGoogle Scholar
  7. 7.
    Ming J, Stewart D, Vaseghi S (2005) Speaker identification in unknown noisy conditions—a universal compensation approach. In: Proceedings of the international conference on acoustics, speech, and signal processingGoogle Scholar
  8. 8.
    Nabney I (2001) Netlab: algorithms for pattern recognition. Advances in pattern recognition. Springer, Germany. URL:
  9. 9.
    Oates T, Jensen D (1998) Large datasets lead to overly complex models: an explanation and a solution. In: KDD conference proceedings, pp 294–298Google Scholar
  10. 10.
    Prybocki M, Martin A (URL) NIST’s assessment of text independent speaker recognition performance. URL:
  11. 11.
    Reynolds D et al (1995) Robust text independent speaker detection system using Gaussian mixture models. In: IEEE transactions on speech and audio processing, vol 3. No. 1Google Scholar
  12. 12.
    Reynolds D, Quateiri T, Dunn R (2000) Speaker verification using adapted Gaussian mixture models. Digital Signal Process 10: 42–54CrossRefGoogle Scholar
  13. 13.
    Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, LondonzbMATHGoogle Scholar
  14. 14.
    Suhadi S, Stan S, Fingschiedt T, Beaugeant C (2003) An evaluation of VTS and IMM for speaker recognition in noise. Eurospeech, pp 1669–1672Google Scholar
  15. 15.
    Williams C, Utans J (1996) Model complexity, NIPS Workshop. URL:
  16. 16.
    Zhang T, Ramakrishnan R, Livny M (1996) BIRCH: an efficient data clustering method for very large databases, ACM SIGMOD conference, pp 103–114Google Scholar

Copyright information

© Springer-Verlag London Limited 2008

Authors and Affiliations

  1. 1.IBM T. J. Watson Research CenterHawthorneUSA

Personalised recommendations