Music Thumbnailing Incorporating Harmony- and Rhythm Structure

  • Björn Schuller
  • Florian Dibiasi
  • Florian Eyben
  • Gerhard Rigoll
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5811)


A variety of approaches exist to the automatic retrieval of the key part within a musical piece - its thumbnail. Most of these however do not use adequate modeling with respect to either harmony or rhythm. In this work we therefore introduce thumbnailing that aims at adequate musical feature modeling. The rhythmic structure is extracted to obtain a segmentation based on beats and bars by an IIR comb-filter bank. Further, we extract chroma energy distribution normalized statistics features of the segmented song improving performance with dB(A) and pitch correction. Harmonic similarities are determined by construction and analysis of a similarity matrix based on the normalized scalar product of the feature vectors. Last, thumbnails are found lending techniques from image processing. Extensive test runs on roughly 24 h of music reveal the high effectiveness of our approach.


Similarity Matrix Audio Signal Dynamic Time Warp Musical Piece Rhythm Structure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Abdallah et al., 2005]
    Abdallah, S.A., Noland, K., Sandler, M., Casey, M., Rhodes, C.: Theory and Evaluation of a Bayesian Music Structure Extractor. In: Proc. 6th ISMIR, pp. 420–425 (2005)Google Scholar
  2. [Aucouturier et al., 2005]
    Aucouturier, J.-J., Pachet, F., Sandler, M.: “The way it Sounds”: Timbre Models for Analysis and Retrieval of Music Signals. IEEE Transactions on Multimedia 7(6), 1028–1035 (2005)CrossRefGoogle Scholar
  3. [Aucouturier and Sandler, 2001]
    Aucouturier, J.-J., Sandler, M.: Segmentation of Musical Signals Using Hidden Markov Models. In: Proc. of the Audio Engineering Society 110th Convention (2001)Google Scholar
  4. [Bartsch and Wakefield, 2001]
    Bartsch, M.A., Wakefield, G.H.: To Catch a Chorus: using Chroma-Based Representations for Audiothumbnailing. In: Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 15–18 (2001)Google Scholar
  5. [Berenzweig and Ellis, 2001]
    Berenzweig, A., Ellis, D.: Locating singing voice segments within musical signals. In: Proc. Int. Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), Mohonk, NY, pp. 119–123 (2001)Google Scholar
  6. [Burges et al., 2004]
    Burges, C.J.C., Plastina, D., Platt, J.C., Renshaw, E., Malvar, H.S.: Duplicate Detection and Audio Thumbnails with Audio Fingerprinting. Technical Report MSR-TR-2004-19, Microsoft Research, MSR (2004)Google Scholar
  7. [Chew, 2002]
    Chew, E.: An algorithm for determining key boundaries. In: Proc. 2nd Int. Conf. on Music and Artificial Intelligence (2002)Google Scholar
  8. [Cooper and Foote, 2002]
    Cooper, M., Foote, J.: Automatic Music Summarization via Similarity Analysis. In: Proc. 3rd ISMIR, pp. 81–85 (2002)Google Scholar
  9. [D’Aguanno and Vercellesi, 2008]
    D’Aguanno, A., Vercellesi, G.: Automatic synchronization between audio and partial music score representation. In: Proc. 6th Workshop on Adaptive Multimedia Retrieval (AMR 2008), Berlin, Germany (2008)Google Scholar
  10. [Eyben et al., 2007]
    Eyben, F., Schuller, B., Reiter, S., Rigoll, G.: Wearable Assistance for the Ballroom-Dance Hobbyist - Holistic Rhythm Analysis and Dance-Style Classification. In: Proc. ICME, pp. 92–95 (2007)Google Scholar
  11. [Foote, 1999]
    Foote, J.: Visualizing Music and Audio using Self-Similarity. In: Proc. 7th ACM Int. Conf. on Multimedia (Part 1), pp. 77–80 (1999)Google Scholar
  12. [Goto, 2006]
    Goto, M.: A Chorus Section Detection Method for Musical Audio Signals and its Application to a Music Listening Station. IEEE Transactions on Audio, Speech and Language Processing 14(5), 1783–1794 (2006)CrossRefGoogle Scholar
  13. [Jehan, 2005]
    Jehan, T.: Hierarchical Multi-Class Self Similarities. In: Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 311–314 (2005)Google Scholar
  14. [Lee and Slaney, 2008]
    Lee, K., Slaney, M.: Acoustic chord transcription and key extraction from audio using key-dependent hmms trained on synthesized audio. IEEE Transactions on Audio, Speech and Language Processing 16, 291–301 (2008)CrossRefGoogle Scholar
  15. [Logan and Chu, 2000]
    Logan, B., Chu, S.: Music Summarization Using Key Phrases. In: Proc. ICASSP, vol. 2, pp. 749–752 (2000)Google Scholar
  16. [Müller and Kurth, 2006]
    Müller, M., Kurth, F.: Enhancing Similarity Matrices for Music Audio Analysis. In: Proc. ICASSP, vol. 5, pp. 9–12 (2006)Google Scholar
  17. [Müller et al., 2005]
    Müller, M., Kurth, F., Clausen, M.: Chroma-Based Statistical Audio Features for Audio Matching. In: Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 275–278 (2005)Google Scholar
  18. [Peeters et al., 2002]
    Peeters, G., Burthe, A.L., Rodet, X.: Toward automatic music audio summary generation from signal analysis. In: Proc. 3rd ISMIR, pp. 94–100 (2002)Google Scholar
  19. [Scheirer, 1998]
    Scheirer, E.: Tempo and beat analysis of acoustic musical signals. Journal of the Acoustic Society of America 103(1), 588–601 (1998)CrossRefGoogle Scholar
  20. [Schuller et al., 2007]
    Schuller, B., Eyben, F., Rigoll, G.: Fast and Robust Meter and Tempo Recognition for the Automatic Discrimination of Ballroom Dance Styles. In: Proc. ICASSP, vol. 1, pp. 217–220 (2007)Google Scholar
  21. [Schuller et al., 2008]
    Schuller, B., Eyben, F., Rigoll, G.: Beat-synchronous data-driven automatic chord labeling. In: Proc. DAGA 2008, Dresden, Germany, pp. 555–556. DEGA (2008)Google Scholar
  22. [Schuller et al., 2003]
    Schuller, B., Zobl, M., Rigoll, G., Lang, M.: A Hybrid Music Retrieval System using Belief Networks to Integrate Queries and Contextual Knowledge. In: Proc. ICME, vol. 1, pp. 57–60 (2003)Google Scholar
  23. [Stober and Nürnberger, 2008]
    Stober, S., Nürnberger, A.: Towards user-adaptive structuring and organization of music collections. In: Proc. 6th Workshop on Adaptive Multimedia Retrieval (AMR 2008), Berlin, Germany (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Björn Schuller
    • 1
  • Florian Dibiasi
    • 1
  • Florian Eyben
    • 1
  • Gerhard Rigoll
    • 1
  1. 1.Institute for Human-Machine CommunicationTechnische Universität MünchenMünchenGermany

Personalised recommendations