Deep Representation Learning for Orca Call Type Classification

  • Christian BerglerEmail author
  • Manuel Schmitt
  • Rachael Xi Cheng
  • Hendrik Schröter
  • Andreas Maier
  • Volker Barth
  • Michael Weber
  • Elmar NöthEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11697)


Marine mammals produce a wide variety of vocalizations. There is a growing need for robust automatic classification methods especially in noisy underwater environments in order to access large amounts of bioacoustic signals and to replace tedious and error prone human perceptual classification. In case of the northern resident killer whale (Orcinus orca), echolocation clicks, whistles, and pulsed calls make up its vocal repertoire. Pulsed calls are the most intensively studied type of vocalization. In this study we propose a hybrid call type classification approach outperforming our previous work on supervised call type classification consisting of two components: (1) deep representation learning of killer whale sounds by investigating various autoencoder architectures and data corpora and (2) subsequent supervised training of a ResNet18 call type classifier on a much smaller dataset by using the pre-trained representations. The best semi-supervised trained classification model achieved a test accuracy of 96% and a mean test accuracy of 94% outperforming our previous work by 7% points.


Deep learning Classification Representation learning Bioacoustics Orca Killer whale Call type 


  1. 1.
    Bengio, Y., Courville, A.C., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013)CrossRefGoogle Scholar
  2. 2.
    Bigg, M.A., Olesiuk, P.F., Ellis, G.M., Ford, J.K.B., Balcomb, K.C.: Organization and genealogy of resident killer whales (Orcinus orca) in the coastal waters of British Columbia and Washington State. Int. Whaling Comm. 12, 383–405 (1990)Google Scholar
  3. 3.
    Brown, J., Hodgins-Davis, A., Miller, P.: Classification of vocalizations of killer whales using dynamic time warping. JASA Express Lett. 119(3), 617–628 (2006)Google Scholar
  4. 4.
    Brown, J.C., Smaragdis, P.: Hidden Markov and Gaussian mixture models for automatic call classification. J. Acoust. Soc. Am. 125, 221–224 (2009)CrossRefGoogle Scholar
  5. 5.
    Brown, J.C., Smaragdis, P., Nousek-McGregor, A.: Automatic identification of individual killer whales. J. Acoust. Soc. Am. 128, 93–98 (2010)Google Scholar
  6. 6.
    Deecke, V.B., Janik, V.M.: Automated categorization of bioacoustic signals: avoiding perceptual pitfalls. J. Acoust. Soc. Am. 119, 645–653 (2006)CrossRefGoogle Scholar
  7. 7.
    Filatova, O.A., Samarra, F.I., Deecke, V.B., Ford, J.K., Miller, P.J., Yurk, H.: Cultural evolution of killer whale calls: background, mechanisms and consequences. Behaviour 152, 2001–2038 (2015)CrossRefGoogle Scholar
  8. 8.
    Ford, J., Ellis, G., Balcomb, K.: Killer Whales: The Natural History and Genealogy of Orcinus Orca in British Columbia and Washington. UBC Press, Vancouver (2000)Google Scholar
  9. 9.
    Ford, J.K.B.: A catalogue of underwater calls produced by killer whales (Orcinus orca) in British Columbia. Canadian Data Report of Fisheries and Aquatic Science (633), p. 165 (1987)Google Scholar
  10. 10.
    Ford, J.K.B.: Acoustic behaviour of resident killer whales (Orcinus orca) off Vancouver Island, British Columbia. Can. J. Zool. 67, 727–745 (1989)CrossRefGoogle Scholar
  11. 11.
    Ford, J.K.B.: Vocal traditions among resident killer whales (Orcinus orca) in coastal waters of British Columbia. Can. J. Zool. 69, 1454–1483 (1991)CrossRefGoogle Scholar
  12. 12.
    Garland, E., Castellote, M., Berchok, C.: Beluga whale (Delphinapterus leucas) vocalizations and call classification from the eastern Beaufort sea population. J. Acoust. Soc. of Am. 137, 3054–3067 (2015)CrossRefGoogle Scholar
  13. 13.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)zbMATHGoogle Scholar
  14. 14.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)Google Scholar
  15. 15.
    Ivkovich, T., Filatova, O., Burdin, A., Sato, H., Hoyt, E.: The social organization of resident-type killer whales (Orcinus orca) in Avacha Gulf, Northwest Pacific, as revealed through association patterns and acoustic similarity. Mamm. Biol. 75, 198–210 (2010)CrossRefGoogle Scholar
  16. 16.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  17. 17.
    Mercado, E., Kuh, A.: Classification of humpback whale vocalizations using a self-organizing neural network. In: IEEE International Conference on Neural Networks - Conference Proceedings, pp. 1584–1589, June 1998Google Scholar
  18. 18.
    Miller, P., Bain, D.: Within-pod variation in the sound production of a pod of killer whales, Orcinus orca. Anim. Behav. 60, 617–628 (2000)CrossRefGoogle Scholar
  19. 19.
    Ness, S.: The Orchive: a system for semi-automatic annotation and analysis of a large collection of bioacoustic recordings. Ph.D. thesis (2013)Google Scholar
  20. 20.
    ORCALAB: a whale research station on Hanson Island. Accessed May 2019
  21. 21.
    Paszke, A., et al.: Automatic differentiation in PyTorch. In: NIPS 2017 Workshop, October 2017Google Scholar
  22. 22.
    Schröter, H., Nöth, E., Maier, A., Cheng, R., Barth, V., Bergler, C.: Segmentation, classification, and visualization of orca calls using deep learning. In: International Conference on Acoustics, Speech, and Signal Processing, Proceedings (ICASSP), May 2019Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Christian Bergler
    • 1
    Email author
  • Manuel Schmitt
    • 1
  • Rachael Xi Cheng
    • 2
  • Hendrik Schröter
    • 1
  • Andreas Maier
    • 1
  • Volker Barth
    • 3
  • Michael Weber
    • 3
  • Elmar Nöth
    • 1
    Email author
  1. 1.Department of Computer Science – Pattern Recognition LabFriedrich-Alexander-University Erlangen-NurembergErlangenGermany
  2. 2.Leibniz Institute for Zoo and Wildlife Research (IZW) in the Forschungsverbund Berlin e.V.BerlinGermany
  3. 3.Anthro-MediaBerlinGermany

Personalised recommendations