Abstract
Human–animal interactions may affect the animal welfare and productivity in rearing environments. Previously proposed human–animal-related techniques focus on the manual discrimination of single animal behaviors or simple human–animal interactions. To address the automatic detection and classification of complex animal behaviors and the animals reactions to human, we propose an approach built upon both the visual representation with Fisher vectors and the end-to-end generative hidden Markov model to facilitate the discrimination of both coarse- and fine-grained animal–human interactions. To satisfy the requirement for abundant data samples of the generative approach, we recorded and annotated more than 480 hours of videos featuring eight persons and 210 laying hens during the process of feeding and cleaning. The experimental results show that the proposed method outperforms state-of-the-art approaches. According to the experimental performance of our method on practical videos, our approach can be used to monitor the human–animal interactions or animal behaviors in modern poultry farms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alaqil, A., Zulkifli, I., Hair, B.M., Sazili, A.Q., Rajion, M.A., Somchit, M.N.: Changes in heat shock protein 70, blood parameters, and fear-related behavior in broiler chickens as affected by pleasant and unpleasant human contact. Poult. Sci. 92(1), 33–40 (2013)
Aydin, A.: Development of an early detection system for lameness of broilers using computer vision. Comput. Electron. Agric. 136, 140–146 (2017)
Aydin, A.: Using 3d vision camera system to automatically assess the level of inactivity in broiler chickens. Comput. Electron. Agric. 135, 4–10 (2017)
Cinbis, R.G., Verbeek, J.J., Schmid, C.: Segmentation driven object detection with fisher vectors. 2968–2975 (2013)
Davis, H., Taylor, A.: Discrimination between individual humans by domestic fowl (gallus gallus domesticus). Br. Poult. Sci. 42(2), 276–279 (2001)
Dawkins, M.S.: From an animal’s point of view: motivation, fitness, and animal welfare. Behav. Brain Sci. 13(1), 1–9 (1990)
Forman, G, Scholz, M.B., Rajaram, S.S.,: Feature shaping for linear svm classifiers. 299–308 (2009)
Forney, G.D.: The Viterbi algorithm. Proc. IEEE. 61(3), 268–278 (1973)
Jones, R.B.: Fear and adaptability in poultry: insights, implications and imperatives. Worlds Poult. Sci. J. 52(02), 131–174 (1996)
Ke, Y., Sukthankar, R.: Pca-sift: a more distinctive representation for local image descriptors. 2, 506–513 (2004)
Lao, F., Brown-Brandl, T., Stinn, J.P., Liu, K., Teng, G., Xin, H.: Automatic recognition of lactating sow behaviors through depth image processing. Comput. Electron. Agric. 125, 56–62 (2016)
Linden, D.V.D., Zamansky. A.: Agile with animals: towards a development method. In: IEEE International Requirements Engineering Conference Workshops, 423–426 (2017)
Mancini, C.: Animal-computer interaction: a manifesto. interactions 18(4), 69–73 (2011)
Mehdizadeh, S.A., Neves, D.P., Tscharke, M., Naas, I.D.A., Banhazi, T.: Image analysis method to evaluate beak and head motion of broiler chickens during feeding. Comput. Electron. Agric. 114, 88–95 (2015)
Nakarmi, A.D., Tang, L., Xin, H.: Automated tracking and behavior quantification of laying hens using 3d computer vision and radio frequency identification technologies. Trans. ASABE. 57(5), 1455–1472 (2014)
Nasirahmadi, A., Richter, U., Hensel, O., Edwards, S.A., Sturm, B.: Using machine vision for investigation of changes in pig group lying patterns. Comput. Electron. Agric. 119, 184–190 (2015)
Passill, A.M.D., Rushen, J.: Can we measure human–animal interactions in on-farm animal welfare assessment? some unresolved issues. Appl. Anim. Behav. Sci. 92(3), 193–209 (2005)
Pereira, D.F., Miyamoto, B.C.B., Maia, G.D.N., Sales, G.T., De Magalhaes, M.M., Gates, R.S.: Machine vision to identify broiler breeder behavior. Comput. Electron. Agric. 99, 194–199 (2013)
Perronnin, F., Larlus, D.: Fisher vectors meet neural networks: A hybrid classification architecture, 3743–3752 (2015)
Peters, S.M., Pinter, I.J., Pothuizen, H.H.J., De Heer, R.C., Der Harst, J.E.V., Spruijt, B.M.: Novel approach to automatically classify rat social behavior using a video tracking system. J. Neurosci. Methods. 268, 163–170 (2016)
Pons, P., Jaen, J., Catala, A.: Assessing machine learning classifiers for the detection of animals behavior using depth-based tracking. Expert Syst. Appl. 86, 235–246 (2017)
Sanchez, J., Perronnin, F., Mensink, T., Verbeek, J.J.: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vision. 105(3), 222–245 (2013)
Wang, C., Chen, H., Zhang, X., Meng, C.: Evaluation of a laying-hen tracking algorithm based on a hybrid support vector machine. J. Anim. Sci. Biotechnol. 8(1), 226–235 (2017)
Welch, Lloyd R.: Hidden markov models and the Baum-Welch algorithm. IEEE Inf. Theory Soc. Newsl. 53(2), 194–211 (2003)
Yang, J., Zhang, D., Frangi, A.F., Yang, J.: Two-dimensional pca: a new approach to appearance-based face representation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 131–137 (2004)
Yang, M., Ahuja, N.: Gaussian mixture model for human skin color and its applications in image and video databases. Proc. SPIE. 3656, 458–466 (1998)
Young, S., Evermann, G., Gales, M., Kershaw, D., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V., Woodland, P.: The htk book version 3.4. Cambridge University Engineering Department, 2006
Zamansky, A., Roshier, A., Mancini, C., Collins, E.C., Hall, C., Grillaert, K., Morrison, A., North, S., Wirman, H.: A report on the first international workshop on research methods in animal-computer interaction. In: CHI Conference Extended Abstracts on Human Factors in Computing Systems, 806–815 (2017)
Zhou, H., Yuan, Y., Shi, C.: Object tracking using sift features and mean shift. Comput. Vision Image Underst. 113(3), 345–352 (2009)
Zulkifli, I.: Review of human–animal interactions and their impact on animal productivity and welfare. J. Anim. Sci. Biotechnol. 4(3), 25 (2013)
Zulkifli, I., Azah, A.S.N.: Fear and stress reactions, and the performance of commercial broiler chickens subjected to regular pleasant and unpleasant contacts with human being. Appl. Anim. Behav. Sci. 88(1), 77–87 (2004)
Acknowledgements
The authors sincerely thank the editors and reviewers for their work.
Funding
This work was made possible through support from the Natural Science Foundation of China (61572300), Taishan Scholar Program of Shandong Province in China (TSHW201502038) and SDUST Excellent Teaching Team Construction Plan JXTD20160512.
Author information
Authors and Affiliations
Contributions
Y. Zheng conceived of the study and designed the experiments. JL, WJ and Y. Zhao performed the experiments, MF, DW and SS analyzed the data. JL wrote the paper. All authors helped revise and approved the manuscript.
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses or interpretation of the data; in the writing of the manuscript, or in the decision to publish the results.
Ethical standard
This study was approved by the Animal Care and Use Committee of Qingdao Agricultural University (Qingdao, China).
Availability of data and material
The datasets analyzed during the current study are available from the corresponding authors upon reasonable request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lian, J., Zheng, Y., Jia, W. et al. Automated recognition and discrimination of human–animal interactions using Fisher vector and hidden Markov model. SIViP 13, 993–1000 (2019). https://doi.org/10.1007/s11760-019-01437-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-019-01437-0