Advertisement

A Comparison of Attention Mechanisms of Convolutional Neural Network in Weakly Labeled Audio Tagging

  • Yuanbo HouEmail author
  • Qiuqiang Kong
  • Shengchen Li
Conference paper
  • 196 Downloads
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 568)

Abstract

Audio tagging aims to predict the types of sound events occurring in audio clips. Recently, the convolutional recurrent neural network (CRNN) has achieved state-of-the-art performance in audio tagging. In CRNN, convolutional layers are applied on input audio features to extract high-level representations followed by recurrent layers. To better learn high-level representations of acoustic features, attention mechanisms were introduced to the convolutional layers of CRNN. Attention is a learning technique that could steer the model to information important to the task to obtain better performance. The two different attention mechanisms in the CRNN, the Squeeze-and-Excitation (SE) block and gated linear unit (GLU), are based on a gating mechanism, but their concerns are different. To compare the performance of the SE block and GLU, we propose to use a CRNN with a SE block (SE-CRNN) and a CRNN with a GLU (GLU-CRNN) in weakly labeled audio tagging and compare these results with the CRNN baseline. The experiments show that the GLU-CRNN achieves an area under curve score of 0.877 in polyphonic audio tagging, outperforming the SE-CRNN of 0.865 and the CRNN baseline of 0.838. The results show that the performance of attention based on GLU is better than the performance of attention based on the SE block in CRNN for weakly labeled polyphonic audio tagging.

Keywords

Audio tagging Convolutional recurrent neural network (CRNN) Convolutional neural network (CNN) Squeeze-and-Excitation (SE) block Gated linear unit (GLU) 

Notes

Acknowledgement

Qiuqiang Kong was supported by the China Scholarship Council (CSC) No. 201406150082.

References

  1. 1.
    Xu Y, Kong Q, Wang W, Plumbley MD (2018) Large-scale weakly supervised audio classification using gatedconvolutional neural network. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2018, Calgary Canada, 2018, (pp 121–125)Google Scholar
  2. 2.
    Stowell D, Giannoulis D, Benetos E, Lagrange M, Plumbley MD (2015) Detection and classification of acoustic scenes and events. IEEE Trans Multimed 17(10):1733–1746CrossRefGoogle Scholar
  3. 3.
    Dimitrov S, Britz J, Brandherm B, Frey J (2014) Analyzing sounds of home environment for device recognition. In: European Conference on Ambient Intelligence, pp 1–16Google Scholar
  4. 4.
    Kumar A, Raj B (2016) Audio event detection using weakly labeled data. In: ACM on Multimedia Conference, pp 1038–1047Google Scholar
  5. 5.
    Mesaros A, Heittola T, Diment A, Elizalde B, Shah A, Vincent E, Raj B, Virtanen T (2017) DCASE 2017 challenge setup: Tasks, datasets and baseline system. In: Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE) 2017, Munich, Germany, 2017Google Scholar
  6. 6.
    Dauphin YN, Fan A, Auli M, Grangier D (2017) Language modeling with gated convolutional networks. In: Proceedings of International Conference on Machine Learning (ICML), 2017, pp 933–941Google Scholar
  7. 7.
    Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141Google Scholar
  8. 8.
    Mnih V, Heess N, Graves A (2014) Recurrent models of visual attention. In: Advances in neural information processing systems, pp 2204–2212Google Scholar
  9. 9.
    Mesaros A, Heittola T, Eronen A, Virtanen T (2010) Acoustic event detection in real life recordings. In: European Signal Processing Conference. IEEE, pp 1267–1271Google Scholar
  10. 10.
    Lidy T, Schindler A (2016) CQT-based convolutional neural networks for audio scene classification: In: Proceedings of the detection and classification of acoustic scenes and events 2016 workshop 90:1032–1048Google Scholar
  11. 11.
    Krizhevsky A, Sutskever I, Hinton GE (2010) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  12. 12.
    Choi K, Fazekas G, Sandler M (2016) Automatic tagging using deep convolutional neural networks. In: arXiv preprint, arXiv:1606.00298Google Scholar
  13. 13.
    Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp 1254–1259CrossRefGoogle Scholar
  14. 14.
    Xu Y, Kong Q, Huang Q, Wang W, Plumbley MD (2017) Attention and localization based on a deep convolutional recurrent model for weakly supervised audio tagging. In: arXiv preprint, arXiv:1703.06052Google Scholar
  15. 15.
    Xu Y, Huang Q, Wang W, Foster P, Sigtia S, Jackson PJ, Plumbley MD (2017) Unsupervised feature learning based on deep models for environmental audio tagging. In: IEEE/ACM Transactions Audio, Speech, Language Process 25(6):1230–1241CrossRefGoogle Scholar
  16. 16.
    Serizel R, Turpault N, Eghbal-Zadeh H (2018) Large-scale weakly labeled semi-supervised sound event detection in domestic environments. In Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE) 2018, November 2018, Surrey, UKGoogle Scholar
  17. 17.
    Mesaros A, Heittola T, Virtanen T (2016) Metrics for polyphonic sound event detection. Appl Sci 6(6):162CrossRefGoogle Scholar
  18. 18.
    Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1):29–36CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Beijing University of Posts and TelecommunicationsBeijingPeople’s Republic of China
  2. 2.Centre for Vision, Speech and Signal ProcessingUniversity of SurreyGuildfordUK

Personalised recommendations