Skip to main content

Advertisement

SpringerLink
Log in

A deep learning-based pipeline for mosquito detection and classification from wingbeat sounds

  • 1223: Information Technology for Social Good
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Mosquito vector-borne diseases such as malaria and dengue constitute some of the most serious public health burdens in tropical and sub-tropical countries. Effective targeting of disease control efforts requires accurate estimates of mosquito vector population density. The traditional, and still most common, approach to this involves the use of traps along with manual counting and classification of mosquito species. This process is costly and labor-intensive, which hinders its widespread use. In this paper we present a software pipeline for detection and classification of mosquito wingbeat sounds. Since our target platform is low-cost IoT devices, we explore the tradeoff between accuracy and efficiency. When a fast binary mosquito detector precedes the classifier, we can reduce the computational demand compared with use of the classifier alone by a factor of 10. While the accuracy of traditional machine learning model drops from 90% to 64% when reducing the sample rate from 96 kHz to 8 kHz, our deep-learning models maintain an accuracy of almost 83%, even when additionally reducing the bit depth from 24 to 16 bits. We conclude that the combination of an efficient mosquito detector with a convolutional neural network provides for an excellent trade-off between accuracy and efficiency to detect, classify and count mosquitoes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Adavanne S, Pertilä P, Virtanen T (2017) Sound event detection using spatial features and convolutional recurrent neural network. In: 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 771–775. https://doi.org/10.1109/ICASSP.2017.7952260

  2. Alar HS, Fernandez PL (2021) Classifying mosquito presence and genera using median and interquartile values from 26-filter wingbeat acoustic properties. Procedia Computer Science 193:453–463. https://doi.org/10.1016/j.procs.2021.10.047

    Article  Google Scholar 

  3. Arthur BJ, Emr KS, Wyttenbach RA, Hoy RR (2014) Mosquito (Aedes aegypti) flight tones: frequency, harmonicity, spherical spreading, and phase relationships. J Acoust Soc Am 135(2):933–941. https://doi.org/10.1121/1.4861233

    Article  Google Scholar 

  4. Chen Y, Why A, Batista G, Mafra-Neto A, Keogh E (2014) Flying insect classification with inexpensive sensors. J Insect Behav 27(5):657–677. https://doi.org/10.1007/s10905-014-9454-4

    Article  Google Scholar 

  5. Dombois F, Eckel G (2011) Audification. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook, pp 301–324. COST and Logos, Chap. 12. https://sonification.de/handbook/

  6. Fanioudakis E, Geismar M, Potamitis I (2018) Mosquito wingbeat analysis and classification using deep learning. In: 2018 26th European signal processing conference (EUSIPCO). IEEE, pp 2410–2414. https://doi.org/10.23919/EUSIPCO.2018.8553542

  7. Fernandes MS, Cordeiro W, Recamonde-Mendoza M (2021) Detecting aedes aegypti mosquitoes through audio classification with convolutional neural networks. Comput Biol Med 129:104152. https://doi.org/10.1016/j.compbiomed.2020.104152

    Article  Google Scholar 

  8. Genoud AP, Gao Y, Williams GM, Thomas BP (2020) A comparison of supervised machine learning algorithms for mosquito identification from backscattered optical signals. Ecol Inform 58:101090. https://doi.org/10.1016/j.ecoinf.2020.101090

    Article  Google Scholar 

  9. Goodwin A, Padmanabhan S, Hira S, Glancey M, Slinowsky M, Immidisetti R, Scavo L, Brey J, Sudhakar BMMS, Ford T (2021) Mosquito species identification using convolutional neural networks with a multitiered ensemble model for novel species detection. Sci Rep 11(1):1–15. https://doi.org/10.1038/s41598-021-92891-9

    Article  Google Scholar 

  10. Göpfert MC, Briegel H, Robert D (1999) Mosquito hearing: sound-induced antennal vibrations in male and female aedes aegypti. J Exp Biol 202:2727–2738

    Article  Google Scholar 

  11. Joshi A, Miller C (2021) Review of machine learning techniques for mosquito control in urban environments. Eco Inform 61:101241

    Article  Google Scholar 

  12. Kiskin I, Cobb AD, Sinka M, Willis K, Roberts S (2021) Automatic acoustic mosquito tagging with bayesian neural networks. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 351–366. https://doi.org/10.1007/978-3-030-86514-6-22

  13. Kiskin I, Zilli D, Li Y, Sinka M, Willis K, Roberts S (2020) Bioacoustic detection with wavelet-conditioned convolutional neural networks. Neural Comput Applic 32 (4):915–927. https://doi.org/10.1007/s00521-018-3626-7

    Article  Google Scholar 

  14. Kittichai V, Pengsakul T, Chumchuen K, Samung Y, Sriwichai P, Phatthamolrat N, Tongloy T, Jaksukam K, Chuwongin S, Boonsang S (2021) Deep learning approaches for challenging species and gender identification of mosquito vectors. Sci Rep 11(1):1–14. https://doi.org/10.1038/s41598-021-84219-4

    Article  Google Scholar 

  15. Lemon SM, Sparling PF, Hamburg MA, Relman DA, Choffnes ER, Mack A (2008) Vector-borne diseases: understanding the environmental, human health, and ecological connections. workshop summary. In: Vector-borne diseases: understanding the environmental, human health, and ecological connections. Workshop summary. National Academies Press. https://doi.org/10.17226/11950

  16. Lostanlen V, Salamon J, Farnsworth A, Kelling S, Bello JP (2019) Robust sound event detection in bioacoustic sensor networks. PloS one 14 (10):0214168. https://doi.org/10.1371/journal.pone.0214168

    Article  Google Scholar 

  17. Loureiro P, Prandi C, Nisi V, Nunes N (2019) On exploiting acoustic sensing and citizen science in a game for biodiversity monitoring and awareness. In: IEEE INFOCOM 2019-IEEE conference on computer communications workshops (INFOCOM WKSHPS). IEEE, pp 572–577. https://doi.org/10.1109/INFCOMW.2019.8845197

  18. Mauch M, Dixon S (2014) pyin: a fundamental frequency estimator using probabilistic threshold distributions. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 659–663. https://doi.org/10.1109/ICASSP.2014.6853678

  19. Mesaros A, Heittola T, Eronen A, Virtanen T (2010) Acoustic event detection in real life recordings. In: 2010 18th European signal processing conference. IEEE, pp 1267–1271. https://ieeexplore.ieee.org/document/7096611

  20. Morales N, Toledano DT, Hansen JHL, Colas J (2007) Blind feature compensation for time-variant band-limited speech recognition. IEEE Signal Process Lett 14(1):70–73. https://doi.org/10.1109/LSP.2006.881516

    Article  Google Scholar 

  21. Motta D, Santos AÁB, Machado BAS, Ribeiro-Filho OGV, Camargo LOA, Valdenegro-Toro MA, Kirchner F, Badaró R (2020) Optimization of convolutional neural network hyperparameters for automatic classification of adult mosquitoes. Plos one 15(7):0234959. https://doi.org/10.1371/journal.pone.0234959

    Article  Google Scholar 

  22. Mukundarajan H, Hol FJH, Castillo EA, Newby C, Prakash M (2017) Using mobile phones as acoustic sensors for high-throughput mosquito surveillance. eLife 6:27854. https://doi.org/10.7554/eLife.27854

    Article  Google Scholar 

  23. Offenhauser WH, Kahn MC (1949) The sounds of disease-carrying mosquitoes. J Acoust Soc Am 21(3):259–263. https://doi.org/10.1121/1.1917085

    Article  Google Scholar 

  24. Offenhauser WH, Marrison WA (1965) Sound spectrograms of mosquito calls. Acustica 16:238–242

    Google Scholar 

  25. Okayasu K, Yoshida K, Fuchida M, Nakamura A (2019) Vision-based classification of mosquito species: comparison of conventional and deep learning methods. Appl Sci 9(18):3935. https://doi.org/10.3390/app9183935

    Article  Google Scholar 

  26. Potamitis I (2014) Classifying insects on the fly. Eco Inform 21:40–49. https://doi.org/10.1016/j.ecoinf.2013.11.005

    Article  Google Scholar 

  27. Ravi P, Syam U, Kapre N (2016) Preventive detection of mosquito populations using embedded machine learning on low power IoT platforms. In: Proceedings of the 7th annual symposium on computing for development, pp 1–10. https://doi.org/10.1145/3001913.3001917

  28. Salamon J, Bello JP (2017) Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Process Lett 24(3):279–283. https://doi.org/10.1109/ICCSNT47585.2019.8962462

    Article  Google Scholar 

  29. Saraceno C, Leonardi R (1997) Audio as a support to scene change detection and characterization of video sequences. In: 1997 IEEE international conference on acoustics, speech, and signal processing, vol 4. IEEE, pp 2597–2600. https://doi.org/10.1109/ICASSP.1997.595320

  30. Teensy® 4.0 Development Board. https://www.pjrc.com/store/teensy40.html. Accessed 05 March 2022.

  31. Vasconcelos D, Nunes N, Ribeiro M, Prandi C, Rogers A (2019) LOCOMOBIS: a low-cost acoustic-based sensing system to monitor and classify mosquitoes. In: 2019 16th IEEE annual consumer communications & networking conference (CCNC). IEEE, pp 1–6. https://doi.org/10.1109/CCNC.2019.8651767

  32. Vasconcelos D, Yin MS, Wetjen F, Herbst A, Ziemer T, Forster, Ä, Barkowsky T, Nunes N, Haddawy P (2021) Counting mosquitoes in the wild: an internet of things approach. In: Proceedings of the conference on information technology for social good, pp 43–48. https://doi.org/10.1145/3462203.3475914

  33. Yin MS, Haddawy P, Nirandmongkol B, Kongthaworn T, Chaisumritchoke C, Supratak A, Sa-ngamuang C, Sriwichai P (2021) A lightweight deep learning approach to mosquito classification from wingbeat sounds. In: Proceedings of the conference on information technology for social good, pp 37–42. https://doi.org/10.1145/3462203.3475908

  34. Ziemer T, Koch J, Sa-Ngamuang C, Yin MS, Siai M, Berkhausen B, Efe D (2020) A bio-inspired acoustic detector of mosquito sex and species. J Acoust Soc Am 148(4):2480–2480. https://doi.org/10.1121/1.5146873

    Article  Google Scholar 

  35. Ziemer T, Wetjen F, Herbst A (2022) The antenna base plays a crucial role in mosquito courtship behavior. Frontiers in Tropical Diseases 3. https://doi.org/10.3389/fitd.2022.803611

Download references

Acknowledgements

This work was partially supported by a grant from the Mahidol University Office of International Relations to Haddawy in support of the Mahidol-Bremen Medical Informatics Research Unit (MIRU), by a fellowship from the Hanse-Wissenschaftskolleg Institute for Advanced Study to Su Yin, and by a Young Researcher grant from Mahidol University to Su Yin.

Author information

Authors and Affiliations

  1. Faculty of ICT, Mahidol University, 999 Phuttamonthon 4 Road, Salaya, Nakhon Pathom, Thailand

    Myat Su Yin, Peter Haddawy, Akara Supratak, Kanrawee Chiamsakul, Worameth Siritanakorn, Tharit Chantanalertvilai & Chaitawat Sa-ngamuang

  2. Bremen Spatial Cognition Center, University of Bremen, Enrique-Schmidt Str. 5, Bremen, Germany

    Myat Su Yin, Peter Haddawy, Tim Ziemer & Fabian Wetjen

  3. Faculty of Tropical Medicine, Mahidol University, Ratchathewi, Bangkok, Thailand

    Patchara Sriwichai

Authors
  1. Myat Su Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Haddawy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tim Ziemer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fabian Wetjen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Akara Supratak
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kanrawee Chiamsakul
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Worameth Siritanakorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tharit Chantanalertvilai
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Patchara Sriwichai
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Chaitawat Sa-ngamuang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Haddawy.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, M.S., Haddawy, P., Ziemer, T. et al. A deep learning-based pipeline for mosquito detection and classification from wingbeat sounds. Multimed Tools Appl 82, 5189–5205 (2023). https://doi.org/10.1007/s11042-022-13367-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13367-0

Keywords

Search

Navigation