Skip to main content
SpringerLink
Log in

Automatic identification of stone-handling behaviour in Japanese macaques using LabGym artificial intelligence

  • Original Article
  • Published:
Primates Aims and scope Submit manuscript

Abstract

The latest advances in artificial intelligence technology have opened doors to the video analysis of complex behaviours. In light of this, ethologists are actively exploring the potential of these innovations to streamline the time-intensive behavioural analysis process using video data. Several tools have been developed for this purpose in primatology in the past decade. Nonetheless, each tool grapples with technical constraints. To address these limitations, we have established a comprehensive protocol designed to harness the capabilities of a cutting-edge artificial intelligence-assisted software, LabGym. The primary objective of this study was to evaluate the suitability of LabGym for the analysis of primate behaviour, focusing on Japanese macaques as our model subjects. First, we developed a model that accurately detects Japanese macaques, allowing us to analyse their actions using LabGym. Our behavioural analysis model succeeded in recognising stone-handling-like behaviours on video. However, the absence of quantitative data within the specified time frame limits the ability of our study to draw definitive conclusions regarding the quality of the behavioural analysis. Nevertheless, to the best of our knowledge, this study represents the first instance of applying the LabGym tool specifically for the analysis of primate behaviours, with our model focusing on the automated recognition and categorisation of specific behaviours in Japanese macaques. It lays the groundwork for future research in this promising field to complexify our model using the latest version of LabGym and associated tools, such as multi-class detection and interactive behaviour analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Codes are available on Github: https://github.com/umyelab/LabGym. Data are available on Zenodo: https://doi.org/10.48550/arXiv.2310.07812.

References

  • Altmann J (1974) Observational study of behaviour: sampling methods. Behaviour 49:227–267

    Article  CAS  PubMed  Google Scholar 

  • Anderson DJ, Perona P (2014) Toward a science of computational ethology. Neuron 84:18–31

    Article  CAS  PubMed  Google Scholar 

  • Barnard CJ (2012) Animal behaviour: ecology and evolution. Springer Science & Business Media

    Google Scholar 

  • Bharati P, Pramanik A (2020) Deep learning techniques–R-CNN to mask R-CNN: a survey. Comput Intell Pattern Recognit Proc CIPR 2019:657–668

    Google Scholar 

  • Bohnslav JP, Wimalasena NK, Clausing KJ et al. (2021) DeepEthogram, a machine learning pipeline for supervised behaviour classification from raw pixels. ELife 10:e63377. https://doi.org/10.7554/eLife.63377

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Brakes P, Dall SRX, Aplin LM et al. (2019) Animal cultures matter for conservation. Science 363:1032–1034. https://doi.org/10.1126/science.aaw3557

    Article  CAS  PubMed  Google Scholar 

  • Chakravarty P, Cozzi G, Dejnabadi H et al. (2020) Seek and learn: automated identification of microevents in animal behaviour using envelopes of acceleration data and machine learning. Methods Ecol Evol 11:1639–1651

    Article  Google Scholar 

  • Charpentier MJE, Harté M, Poirotte C et al. (2020) Same father, same face: deep learning reveals selection for signaling kinship in a wild primate. Sci Adv 6:eaba3274. https://doi.org/10.1126/sciadv.aba3274

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • De Cesarei A, Cavicchi S, Cristadoro G, Lippi M (2021) Do humans and deep convolutional neural networks use visual information similarly for the categorisation of natural scenes? Cogn Sci 45:e13009

    Article  PubMed  PubMed Central  Google Scholar 

  • Dutta A, Gupta A, Zissermann A (2016) VGG image annotator (VIA). URL: https://www.robots.ox.ac.uk/~vgg/software/via/

  • Enquist M, Ghirlanda S (2005) Neural networks and animal behaviour. Princeton University Press

    Google Scholar 

  • Gris KV, Coutu J-P, Gris D (2017) Supervised and unsupervised learning technology in the study of rodent behaviour. Front Behav Neurosci 11:141

    Article  PubMed  PubMed Central  Google Scholar 

  • Gu J, Wang Z, Kuen J et al. (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  • Hardin A, Schlupp I (2022) Using machine learning and DeepLabCut in animal behaviour. Acta Ethologica 25:125–133

    Article  Google Scholar 

  • Hu Y, Ferrario CR, Maitland AD et al. (2023) LabGym: quantification of user-defined animal behaviours using learning-based holistic assessment. Cell Rep Methods. https://doi.org/10.1016/j.crmeth.2023.100415

    Article  PubMed  PubMed Central  Google Scholar 

  • Huffman MA, Quiatt D (1986) Stone handling by Japanese macaques (Macaca fuscata): implications for tool use of stone. Primates 27:413–423

    Article  Google Scholar 

  • Huffman MA, Leca JB, Nahallage CA (2010) Cultured Japanese macaques: a multidisciplinary approach to stone handling behaviour and its implications for the evolution of behavioural tradition in non-human primates. In: Nakagawa N, Nakamichi M, Sugiura H (eds) The Japanese Macaques. Springer, Tokyo

    Google Scholar 

  • Huntingford F (2012) The study of animal behaviour. Springer Science & Business Media

  • Jabir B, Falih N, Rahmani K (2021) Accuracy and efficiency comparison of object detection open-source models. Int J Biomed Eng. https://doi.org/10.3991/ijoe.v17i05.21833

    Article  Google Scholar 

  • Kabra M, Robie AA, Rivera-Alba M et al. (2013) JAABA: interactive machine learning for automatic annotation of animal behaviour. Nat Methods 10:64–67

    Article  CAS  PubMed  Google Scholar 

  • Kleanthous N, Hussain AJ, Khan W et al. (2022) A survey of machine learning approaches in animal behaviour. Neurocomputing 491:442–463

    Article  Google Scholar 

  • LabGym · PyPI. https://pypi.org/project/LabGym/. Accessed 25 Sep 2023

  • Labuguen R, Matsumoto J, Negrete SB et al. (2021) MacaquePose: a novel ‘in the wild’ macaque monkey pose dataset for markerless motion capture. Front Behav Neurosci 14:581154

    Article  PubMed  PubMed Central  Google Scholar 

  • Lauer J, Zhou M, Ye S et al. (2022) Multi-animal pose estimation, identification and tracking with DeepLabCut. Nat Methods 19:496–504

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Leca JB, Gunst N (2023) The exaptive potential of (object) play behaviour. Int J Play 12(1):40–52

    Article  Google Scholar 

  • Leca J-B, Gunst N, Huffman MA (2007) Japanese macaque cultures: inter-and intra-troop behavioural variability of stone handling patterns across 10 troops. Behaviour 144:251–281

    Article  Google Scholar 

  • Leca J, Gunst N, Huffman MA (2008) Food provisioning and stone handling tradition in Japanese macaques: a comparative study of ten troops. Am J Primatol Off J Am Soc Primatol 70:803–813

    Google Scholar 

  • Leca JB, Gunst N, Huffman MA (2010) Indirect social influence in the maintenance of the stone-handling tradition in Japanese macaques, Macaca fuscata. Animal Behav 79(1):117–126

    Article  Google Scholar 

  • Leca J-B, Gunst N, Huffman M (2011) Complexity in object manipulation by Japanese macaques (Macaca fuscata): a cross-sectional analysis of manual coordination in stone handling patterns. J Comp Psychol 125:61

    Article  PubMed  Google Scholar 

  • Leca J-B, Huffman MA, Vasey PL (2012) The monkeys of stormy mountain: 60 years of primatological research on the Japanese macaques of Arashiyama. Cambridge University Press

    Book  Google Scholar 

  • Liang J (2020) Image classification based on RESNET. IOP Publishing, p 012110

  • Lin Q, Ye G, Wang J, Liu H (2022) RoboFlow: a data-centric workflow management system for developing AI-enhanced Robots. Proc Mach Learn, pp 1789–1794

  • Matsuzawa T, McGrew WC (2008) Kinji Imanishi and 60 years of Japanese primatology. Curr Biol 18:R587–R591

    Article  CAS  PubMed  Google Scholar 

  • Nahallage CA, Leca J-B, Huffman MA (2016) Stone handling, an object play behaviour in macaques: welfare and neurological health implications of a bio-culturally driven tradition. Behaviour 153:845–869

    Article  Google Scholar 

  • Nakagawa N, Nakamichi M, Sugiura H (2010) The Japanese macaques. Springer Science & Business Media

    Book  Google Scholar 

  • Naud A, Chailleux E, Kestens Y et al. (2016) Relations between spatial distribution, social affiliations and dominance hierarchy in a semi-free mandrill population. Front Psychol 7:612. https://doi.org/10.3389/fpsyg.2016.00612

    Article  PubMed  PubMed Central  Google Scholar 

  • Noldus LP, Spink AJ, Tegelenbosch RA (2001) EthoVision: a versatile video tracking system for automation of behavioural experiments. Behav Res Methods Instrum Comput 33:398–414

    Article  CAS  PubMed  Google Scholar 

  • Paulet J, Molina A, Beltzung B et al. (2023) Deep Learning for automatic detection and facial recognition in Japanese macaques: illuminating social networks. http://arxiv.org/abs/arXiv:2310.06489.

  • Pelletier AN, Kaufmann T, Mohak S, Milan R, Nahallage CAD, Huffman MA, Gunst N, Rompis A, Wandia IN, Arta Putra IGA, Pellis SM, Leca JB (2017) Behavior systems approach to object play: stone handling repertoire as a measure of propensity for complex foraging and percussive tool use in the genus Macaca. Animal Behav Cogn 4(4):455–473. https://doi.org/10.26451/abc.04.04.05.201

    Article  Google Scholar 

  • Rigoudy N, Dussert G, Benyoub A et al. (2022) The DeepFaune initiative: a collaborative effort toward the automatic identification of the French fauna in camera-trap images. Biorxiv 66:62

    Google Scholar 

  • Roffo G (2017) Ranking to learn and learning to rank: on the role of ranking in pattern recognition applications. arXiv preprint arXiv:170605933

  • Romano V, MacIntosh AJJ, Sueur C (2020) Stemming the flow: information, infection, and social evolution. Trends Ecol Evol 35:849–853. https://doi.org/10.1016/j.tree.2020.07.004

    Article  PubMed  PubMed Central  Google Scholar 

  • Schofield D, Nagrani A, Zisserman A et al. (2019) Chimpanzee face recognition from videos in the wild using deep learning. Sci Adv 5:0736. https://doi.org/10.1126/sciadv.aaw0736

    Article  Google Scholar 

  • Shi J, Tripp B, Shea-Brown E et al. (2022) MouseNet: a biologically constrained convolutional neural network model for the mouse visual cortex. PLoS Comput Biol 18:e1010427

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Sueur C, Fourneret E, Espinosa R (2023) Animal capital: a new way to define human-animal bond in view of global changes. OSF Preprint. https://doi.org/10.31219/osf.io/svg7x

    Article  Google Scholar 

  • Valletta JJ, Torney C, Kings M et al. (2017) Applications of machine learning in animal behaviour studies. Anim Behav 124:203–220

    Article  Google Scholar 

  • Whiten A (2021) The burgeoning reach of animal culture. Science 372:eabe6514. https://doi.org/10.1126/science.abe6514

    Article  CAS  PubMed  Google Scholar 

  • Xu B, Wang W, Falzon G et al. (2020) Automated cattle counting using Mask R-CNN in quadcopter vision system. Comput Electron Agric 171:105300

    Article  Google Scholar 

  • Xu J, Wu Q, Zhang J, Tait A (2021) Automatic sheep behaviour analysis using mask r-cnn. IEEE, pp 01–06

Download references

Acknowledgements

We thank Yujia HU, the developer of LabGym1.0, for his help in applying LabGym to our study. Cédric Sueur was funded by an invited researcher fellowship from Kyoto University.

Funding

Institut écologie et environnement, Comp²A, Cedric Sueur, Kyoto University.

Author information

Authors and Affiliations

  1. Master Biodiversité Ecologie Et Evolution, Université Paris-Saclay, Orsay, France

    Théo Ardoin

  2. Magistère de Biologie, Université Paris-Saclay, Orsay, France

    Théo Ardoin

  3. Université de Strasbourg, IPHC UMR7178, CNRS, Strasbourg, France

    Cédric Sueur

  4. ANTHROPO-LAB, ETHICS EA 7446, Université Catholique de Lille, Lille, France

    Cédric Sueur

  5. Institut Universitaire de France, Paris, France

    Cédric Sueur

Authors
  1. Théo Ardoin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cédric Sueur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cédric Sueur.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 8363 KB)

Supplementary file2 (MP4 1752 KB)

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ardoin, T., Sueur, C. Automatic identification of stone-handling behaviour in Japanese macaques using LabGym artificial intelligence. Primates 65, 159–172 (2024). https://doi.org/10.1007/s10329-024-01123-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10329-024-01123-x

Keywords

Search

Navigation