Skip to main content

Automated Behavior Analysis Using a YOLO-Based Object Detection System

  • Protocol
  • First Online:
Behavioral Neurogenetics

Part of the book series: Neuromethods ((NM,volume 181))

Abstract

Object detection methods employing convolutional neural networks have fueled the development of automated behavior analysis, which is traditionally composed of two steps of data processing: tracking and behavior annotation. The former generates the time series of positional data describing the movement of individuals and/or body parts, which is subsequently analyzed in the latter to identify particular behaviors/actions using the mathematically defined criteria. YOLO is a state-of-the-art algorithm for high-speed object detection, which can be used for the tracking of animals in video images. In this section, we extend the use of YOLO for the annotation of various behaviors in a single step. To test the usability of this method, we analyzed the territorial and courtship behavior in a hyper-aggressive fruit fly, Drosophila prolongata. Automated behavior analysis using YOLO successfully identified when and where each fight and courtship event occurred, demonstrating that this YOLO-based behavior analysis system can serve as a versatile tool for behavioral studies because it is easy to handle for researchers who are not experts in computer science.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Protocol
USD 49.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Dell AI, Bender JA, Branson K et al (2014) Automated image-based tracking and its application in ecology. Trends Ecol Evol 29:417–428. https://doi.org/10.1016/j.tree.2014.05.004

    Article  PubMed  Google Scholar 

  2. Egnor SER, Branson K (2016) Computational analysis of behavior. Annu Rev Neurosci 39:217–236. https://doi.org/10.1146/annurev-neuro-070815-013845

    Article  CAS  PubMed  Google Scholar 

  3. Robie AA, Seagraves KM, Egnor SER, Branson K (2017) Machine vision methods for analyzing social interactions. J Exp Biol 220:25–34. https://doi.org/10.1242/jeb.142281

    Article  PubMed  Google Scholar 

  4. Valletta JJ, Torney C, Kings M, Thornton A, Madden J (2017) Applications of machine learning in animal behavior studies. Anim Behav 124:203–220. https://doi.org/10.1016/j.anbehav.2016.12.005

    Article  Google Scholar 

  5. Abbas W, Rodo DM (2019) Computer methods for automatic locomotion and gesture tracking in mice and small animals for neuroscience applications: a survey. Sensors 19:3274. https://doi.org/10.3390/s19153274

    Article  PubMed Central  Google Scholar 

  6. Moulin TC, Covill LE, Itskov PM, Williams MJ, Sciöth HB (2021) Rodent and fly models in behavioral neuroscience: an evaluation of methodological advances, comparative research, and future perspectives. Neurosci Biobehav Rev 120:1–12. https://doi.org/10.1016/j.neubiorev.2020.11.014

    Article  PubMed  Google Scholar 

  7. Hoopfer ED, Jung Y, Inagaki HK, Rubin GM, Anderson DJ (2015) P1 interneurons promote a persistent internal state that enhances inter-male aggression in Drosophila. eLife 4:e11346. https://doi.org/10.7554/eLife.11346

    Article  PubMed  PubMed Central  Google Scholar 

  8. Scaplen KM, Mei NJ, Bounds HA, Song SL, Azanchi R, Kaun KR (2019) Automated real-time quantification of group locomotor activity in Drosophila melanogaster. Sci Rep 9:4427. https://doi.org/10.1038/s41598-019-40952-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Clemens J, Coen P, Roemschied FA, Pereira T, Mazumder D, Aldarondo D, Pacheco D, Murthy M (2018) Discovery of a new song mode in drosophila reveals hidden structure in the sensory and neural drivers of behavior. Curr Biol 28:2400–2412

    Article  CAS  Google Scholar 

  10. Branson K, Robie AA, Bender J, Perona P, Dickinson MH (2009) High-throughput ethomics in large groups of drosophila. Nat Methods 6:451–457

    Article  CAS  Google Scholar 

  11. Dankert H, Wang L, Hoopfer ED, Anderson DJ, Perona P (2009) Automated monitoring and analysis of social behavior in drosophila. Nat Methods 6:297–303

    Article  CAS  Google Scholar 

  12. Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K (2013) JAABA: interactive machine learning for automatic annotation of animal behavior. Nat Methods 10:64–67

    Article  CAS  Google Scholar 

  13. Redmon J, Divvala S, Girshick R, Farhadi A (2015) You only look once: unified, real-time object detection arXiv:1506.02640

    Google Scholar 

  14. Hong S-J, Han Y, Kim S-Y, Lee A-Y, Kim G (2019) Application of deep-learning methods to bird detection using unmanned aerial vehicle imagery. Sensors 19:1651. https://doi.org/10.3390/s19071651

    Article  PubMed Central  Google Scholar 

  15. Lu H, Uemura T, Wang D, Zhu J, Huang Z, Kim H (2020) Deep-sea organisms tracking using dehazing and deep learning. Mobile Networks Appl 25:1008–1015. https://doi.org/10.1007/s11036-018-1117-9

    Article  Google Scholar 

  16. Silvirianti, Shin SV (2019) UAV based search and rescue with honeybee flight behavior in forest. In: Proceedings of 5th international conference on mechatronics and robotics engineering (ICMRE 2019). Italy, Rome. https://doi.org/10.1145/3314493.3314497

    Chapter  Google Scholar 

  17. Wageeh Y, Mohamed HE-D, Fadl A, Anas O, ElMasry N, Nabil A, Atia A (2021) YOLO fish detection with Euclidean tracking in fish farms. J Ambient Intell Humaniz Comput 12:5–12. https://doi.org/10.1007/s12652-020-02847-6

    Article  Google Scholar 

  18. Luh G-C, Wu H-B, Yong Y-T, Lai Y-J, Chen Y-H (2019) Facial expression based emotion recognition employing YOLOv3 deep neural networks. Int Conf Machine Learning Cybernetics (ICMLC). https://doi.org/10.1109/ICMLC48188.2019.8949236

  19. Nguyen CC, Tran GS, Nghiem TP, Burie JC, Luong CM (2019) Real-time smile detection using deep learning. J Comput Sci Cybernetics 35:135–145. https://doi.org/10.15625/1813-9663/35/2/13315

    Article  Google Scholar 

  20. Qian H, Zhou X, Zheng M (2020) Abnormal behavior detection and recognition method based on improved ResNet model. Comput Mat Continua 65:2153–2167. https://doi.org/10.32604/cmc.2020.011843

    Article  Google Scholar 

  21. Thenmozhi M, Saravanan M, Kumar KPM, Suseela S, Deepan S (2020) Improving the prediction rate of unusual behaviors of animal in a poultry using deep learning technique. Soft Comput 24:14491–14502. https://doi.org/10.1007/s00500-020-04801-2

    Article  Google Scholar 

  22. Wang J, Wang N, Li L, Ren Z (2019) Real-time behavior detection and judgement of egg breeders based on YOLO v3. Neural Comput & Applic 32:5471–5481. https://doi.org/10.1007/s00521-019-04645-4

    Article  Google Scholar 

  23. Amino K, Matsuo T (2020) Intra- versus inter-sexual selection on sexually dimorphic traits in Drosophila prolongata. Zool Sci 37:210–216. https://doi.org/10.2108/zs200010

    Article  Google Scholar 

  24. Kudo A, Takamori H, Watabe H, Ishikawa Y, Matsuo T (2015) Variation in morphological and behavioral traits among isofemale strains of Drosophila prolongata (Diptera: Drosophilidae). Entomol Sci 18:221–229. https://doi.org/10.1111/ens.12116

    Article  Google Scholar 

  25. Kudo A, Shigenobu S, Kadota K, Nozawa M, Shibata TF, Ishikawa Y, Matsuo T (2017) Comparative analysis of the brain transcriptome in a hyper-aggressive fruit fly, Drosophila prolongata. Insect Biochem Mol Biol 82:11–20. https://doi.org/10.1016/j.ibmb.2017.01.006

    Article  CAS  PubMed  Google Scholar 

  26. Kaufmann JH (1983) On the definitions and functions of dominance and territoriality. Biol Rev 58:1–20. https://doi.org/10.1111/j.1469-185X.1983.tb00379.x

    Article  Google Scholar 

  27. Setoguchi S, Takamori H, Aotsuka T, Sese J, Ishikawa Y, Matsuo T (2014) Sexual dimorphism and courtship behavior in Drosophila prolongata. J Ethol 32:91–102. https://doi.org/10.1007/s10164-014-0399-z

    Article  Google Scholar 

  28. Hitoshi Y, Ishikawa Y, Matsuo T (2016) Intraspecific variation in heat tolerance of Drosophila prolongata (Diptera: Drosophilidae). Appl Entomol Zool 51:515–520. https://doi.org/10.1007/s13355-016-0425-4

    Article  CAS  Google Scholar 

  29. Matsuo T (2018) Effect of social condition on behavioral development during early adult phase in Drosophila prolongata. J Ethol 36:15–22. https://doi.org/10.1007/s10164-017-0524-x

    Article  PubMed  Google Scholar 

  30. Ando Y, Yoshimizu T, Matsuo T (2020) Food availability reverses the effect of hunger state on copulation rate in Drosophila prolongata females. Anim Behav 166:51–59. https://doi.org/10.1016/j.anbehav.2020.06.003

    Article  Google Scholar 

  31. Setoguchi S, Kudo A, Takanashi T, Ishikawa Y, Matsuo T (2015) Social context-dependent modification of courtship behavior in Drosophila prolongata. Proc R Soc B 282:20151377. https://doi.org/10.1098/rspb.2015.1377

    Article  PubMed  PubMed Central  Google Scholar 

  32. Minekawa K, Miyatake T, Ishikawa Y, Matsuo T (2018) The adaptive role of a species-specific courtship behavior in coping with remating suppression of mated females. Anim Behav 140:29–37. https://doi.org/10.1016/j.anbehav.2018.04.002

    Article  Google Scholar 

  33. Minekawa K, Amino K, Matsuo T (2020) A courtship behavior that makes monandrous females polyandrous. Evolution 74:2483–2493. https://doi.org/10.1111/evo.14098

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Takashi Matsuo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Science+Business Media, LLC, part of Springer Nature

About this protocol

Check for updates. Verify currency and authenticity via CrossMark

Cite this protocol

Amino, K., Matsuo, T. (2022). Automated Behavior Analysis Using a YOLO-Based Object Detection System. In: Yamamoto, D. (eds) Behavioral Neurogenetics. Neuromethods, vol 181. Humana, New York, NY. https://doi.org/10.1007/978-1-0716-2321-3_14

Download citation

  • DOI: https://doi.org/10.1007/978-1-0716-2321-3_14

  • Published:

  • Publisher Name: Humana, New York, NY

  • Print ISBN: 978-1-0716-2320-6

  • Online ISBN: 978-1-0716-2321-3

  • eBook Packages: Springer Protocols

Publish with us

Policies and ethics