Structural and Semantic Modeling of Audio for Content-Based Querying and Browsing

  • Mustafa Sert
  • Buyurman Baykal
  • Adnan Yazıcı
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4027)


A typical content-based audio management system deals with three aspects namely audio segmentation and classification, audio analysis, and content-based retrieval of audio. In this paper, we integrate the three aspects of content-based audio management into a single framework and propose an efficient method for flexible querying and browsing of auditory data. More specifically, we utilize two robust feature sets namely MPEG-7 Audio Spectrum Flatness (ASF) and Mel Frequency Cepstral Coefficients (MFCC) as the underlying features in order to improve the content-based retrieval accuracy, since both features have some advantages for distinct types of audio (e.g., music and speech). The proposed system provides a wide range of opportunities to query and browse an audio data by content, such as querying and browsing for a chorus section, sound effects, and query-by-example. In addition, the clients can express their queries in the form of point, range, and k-nearest neighbor, which are particularly significant in the multimedia domain.


Similarity Matrix Range Query Point Query Audio Data Audio Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aigrain, P., Zhang, H., Petkovic, D.: Content-based Representation and Retrieval of Visual Media: A State-of-the-art Review. Multimedia Tools and Applications 3(3), 179–202 (1996)CrossRefGoogle Scholar
  2. 2.
    Chai, W., Vercoe, B.: Music Thumbnailing via Structural Analysis. In: Proceedings of ACM Multimedia Conference (2003)Google Scholar
  3. 3.
    Bartsch, M., Wakefield, G.: Audio Thumbnailing of Popular Music Using Croma-Based Representations. IEEE Transactions on Multimedia 7(1), 96–104 (2005)CrossRefGoogle Scholar
  4. 4.
    Bach, J.R.: The Virage Image Search Engine: An Open Framework for Image Management. In: Proceedings of SPIE 1996, San Jose, California (1996)Google Scholar
  5. 5.
    Niblack, W., Zhu, X., et al.: Updates to the QBIC System. In: Proceedings of SPIE 1998, San Jose, California (1998)Google Scholar
  6. 6.
    Amato, G., Mainetto, G., Savino, P.: An Approach to a Content-Based Retrieval of Multimedia Data. Multimedia Tools and Applications 7(1/2), 9–36 (1998)CrossRefGoogle Scholar
  7. 7.
    Wold, E., Blum, T., Keislar, D., et al.: Content-based Classification, Search, and Retrieval of Audio. IEEE Multimedia 3(3), 27–36 (1996)CrossRefGoogle Scholar
  8. 8.
    Foote, J.: Content-based Retrieval of Music and Audio. In: Proc. of SPIE 1997 (1997)Google Scholar
  9. 9.
    Zhang, T., Jay Kuo, C.-C.: Content-based Classification and Retrieval of Audio. In: Proceedings of SPIE 1998, San Diego (1998)Google Scholar
  10. 10.
    Lu, L., Jiang, H., Zhang, H.: A Robust Audio Classification and Segmentation Method. In: Proc. of the 9th ACM Int. Conf. on Multimedia, Ottawa Canada (2001)Google Scholar
  11. 11.
    Tzanetakis, G., Cook, P.: Multifeature Audio Segmentation for Browsing and Annotation. In: IEEE WASPAA conference, New Paltz, NY (1999)Google Scholar
  12. 12.
    Pfeiffer, S.: Pause Concepts for Audio Segmentation at Different Semantic Levels. In: Proc. of the 9th ACM Int. Conf. on Multimedia, Ottawa, Canada (2001)Google Scholar
  13. 13.
    Chai, W., Vercoe, B.: Structural Analysis of Musical Signals for Indexing and Thumbnailing. In: Proceedings of the 3rd ACM/IEEE-CS joint Conference on Digital Libraries, Houston Texas (2003)Google Scholar
  14. 14.
    Cooper, M., Foote, J.: Summarizing Popular Music via Structural Similarity Analysis. In: IEEE WASPAA conference, New Paltz, NY (2003)Google Scholar
  15. 15.
    Goto, M.: A Chorus-Section Detecting Method for Musical Audio Signals. In: IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Hong Kong, China (2003)Google Scholar
  16. 16.
    MPEG-7.: Information Technology - Multimedia Content Description Interface - Part 4: Audio ISO/IEC JTC 1/SC 29/WG 11 (2000)Google Scholar
  17. 17.
    Lloyd, S.P.: Least Squares Quantization in PCM. IEEE Transaction on Information Theory IT-2, 129–137 (1982)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Wellhausen, J., Crysandt, H.: Temporal Audio Segmentation using MPEG-7 Descriptors. In: Proceedings of SPIE, Santa Clara (CA), USA (2003)Google Scholar
  19. 19.
    Xu, C., Zhu, Y., Tian, Q.: Automatic Music Summarization Based on Temporal, Spectral and Cepstral Features. In: Proceedings of IEEE ICME 2002 (2002)Google Scholar
  20. 20.
    Lu, L., Wang, M., Zhang, H.J.: Repeating Pattern Discovery and Structure Analysis from Acoustic Music Data (MIR 2004), New York USA (2004)Google Scholar
  21. 21.
    Lu, G.: Indexing and Retrieval of Audio: A Survey. Multimedia Tools and Applications 15(3), 269–290 (2001)MATHCrossRefGoogle Scholar
  22. 22.
    Sert, M., Baykal, B., Yazıcı, A.: Generating Expressive Summaries for Speech and Musical Audio using Self-similarity Clues. In: Proc. of IEEE ICME 2006, Toronto, Ontario Canada (to appear, 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mustafa Sert
    • 1
    • 2
  • Buyurman Baykal
    • 3
  • Adnan Yazıcı
    • 4
  1. 1.Department of Computer EngineeringBaşkent UniversityAnkaraTurkey
  2. 2.Faculty of Technical Education, Department of Electronics, and Computer EducationGazi UniversityAnkaraTurkey
  3. 3.Department of Electrical and Electronics EngineeringMiddle East Technical UniversityAnkaraTurkey
  4. 4.Department of Computer EngineeringMiddle East Technical UniversityAnkaraTurkey

Personalised recommendations