Skip to main content
SpringerLink
Log in

Smart Animal Detection and Counting Framework for Monitoring Livestock in an Autonomous Unmanned Ground Vehicle Using Restricted Supervised Learning and Image Fusion

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Automated livestock monitoring is a promising solution for vast and isolated farmlands or cattle stations. The advancement in sensor technology and the rise of unmanned systems have paved the way for the automated systems. In this work, we propose an Unmanned Ground Vehicle (UGV) based livestock detection-counting system for fusion images using restricted supervised learning technique. For image fusion, we propose Dual-scale image Decomposition based Fusion technique (DDF) that fuses visible and thermal images. To reduce the difficulty of ground truth annotation, we introduce Seed Labels focused Object Detector (SLOD) that carefully propagates the annotation to all the object instances in the training images. Further, we propose a novel Restricted Supervised Learning (RSL) technique that produces competitive results with minimal training data. Experimental results show that the proposed RSL is more efficient and accurate when compared to other learning techniques (fully and weakly supervised). On the test data, with only five training images and five seed labels, the restricted supervised learning has improved the average precision from 4.05% (using fully supervised learning) to 80.58% (using restricted supervised learning). With 50 seed labels, the average precision is further boosted to 91.56%. The proposed model is extensively tested on benchmark animal datasets and has achieved an average accuracy of 98.3%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Abbreviations

AG:

Average Gradient

AWA:

Animals with Attributes

CE:

Commission Error

CNN:

Convolutional Neural Network

CPU:

Central Processing Unit

CUDA:

Compute Unified Device Architecture

DDF:

Dual-scale image Decomposition based Fusion technique

FCNN:

Fully Convolutional Neural Network

FLIR:

Forward-Looking InfraRed

GAME:

Grid Average Mean Error

GPU:

Graphics Processing Unit

IE:

Information Entropy

mAE:

Mean Absolute Error

MI:

Mutual Information

MSAC:

M-estimator SAmple Consensus

mSE:

Mean Squared Error

NMS:

Non-Maximum Suppression

OD:

Object Detection

OE:

Omission Error

R-CNN:

Region-based convolutional neural networks

R-FCN:

Region-based Fully Convolutional Network

RSL:

Restricted Supervised Learning

SF:

Spatial Frequency

SGD:

Stochastic Gradient Descent

SLOD:

Seed Labels focused Object Detector

SSD:

Single Shot Detector

UAV:

Unmanned Aerial Vehicles

UGV:

Unmanned Ground Vehicle

VIF:

Visual Information

YOLO:

You Only Look Once

References

  1. Sundaram DM, Loganathan A (2020) FSSCaps-DetCountNet: fuzzy soft sets and CapsNet-based detection and counting network for monitoring animals from aerial images. J Appl Remote Sens 14(2):026521

    Article  Google Scholar 

  2. Wang D, Shao Q, Yue H (2019) Surveying wild animals from satellites, manned aircraft and unmanned aerial systems (UASs): a review. Remote Sens 11(11):1308

    Article  Google Scholar 

  3. Meena D, Agilandeeswari L (2020) Invariant features-based fuzzy inference system for animal detection and recognition using thermal images. Int J Fuzzy Syst 22:1868–1879

    Article  Google Scholar 

  4. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178

    Article  Google Scholar 

  5. Li J, Huo H, Sui C, Jiang C, Li C (2019) Poisson reconstruction-based fusion of infrared and visible images via saliency detection. IEEE Access 7:20676–20688

    Article  Google Scholar 

  6. Zhou Z, Wang B, Li S, Dong M (2016) Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters. Inf Fusion 30:15–26

    Article  Google Scholar 

  7. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  8. Liu Y, Chen X, Cheng J, Peng H, Wang Z (2018) Infrared and visible image fusion with convolutional neural networks. Int J Wavelets Multiresolut Inf Process 16(03):1850018

    Article  MathSciNet  Google Scholar 

  9. Li H, Wu XJ, Kittler J (2018) Infrared and visible image fusion using a deep learning framework. In: 2018 24th international conference on pattern recognition (ICPR), IEEE, pp 2705–2710

  10. Li W, Xie Y, Zhou H, Han Y, Zhan K (2018) Structure-aware image fusion. Optik 172:1–11

    Article  Google Scholar 

  11. Kumar BS (2015) Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 9(5):1193–1204

    Article  Google Scholar 

  12. Bavirisetti DP, Xiao G, Liu G (2017). Multi-sensor image fusion based on fourth order partial differential equations. In: 2017 20th international conference on information fusion (Fusion), IEEE, pp 1–9

  13. Bavirisetti DP, Dhuli R (2016) Two-scale image fusion of visible and infrared images using saliency detection. Infrared Phys Technol 76:52–64

    Article  Google Scholar 

  14. Naidu VPS (2011) Image fusion technique using multi-resolution singular value decomposition. Def Sci J 61(5):479

    Article  MathSciNet  Google Scholar 

  15. Cerra D, Israel M, Datcu M (2009) Parameter-free clustering: application to fawns detection. In 2009 IEEE international geoscience and remote sensing symposium, IEEE, vol. 3, pp III-467

  16. Divya Meena S, Agilandeeswari L (2020) Stacked convolutional autoencoder for detecting animal images in cluttered scenes with a novel feature extraction framework. Soft computing for problem solving. Advances in intelligent systems and computing. Springer, Singapore, pp 513–522. https://doi.org/10.1007/978-981-15-0184-5_44

    Chapter  Google Scholar 

  17. Franke U, Goll B, Hohmann U, Heurich M (2012) Aerial ungulate surveys with a combination of infrared and high—resolution natural color images. Anim Biodivers Conserv 35(2):285–293

    Article  Google Scholar 

  18. Chrétien LP, Théau J, Ménard P (2015) Wildlife multispecies remote sensing using visible and thermal infrared imagery acquired from an unmanned aerial vehicle (UAV). Int Arch Photogramm Remote Sens Spat Inf Sci 40:241–248

    Article  Google Scholar 

  19. Chrétien LP, Théau J, Ménard P (2016) Visible and thermal infrared remote sensing for the detection of white-tailed deer using an unmanned aerial system. Wildl Soc Bull 40(1):181–191

    Article  Google Scholar 

  20. Divya Meena S, Agilandeeswari L (2019) An efficient framework for animal breeds classification using semi-supervised learning and multi-part convolutional neural network (MP-CNN). IEEE Access 7:151783–151802. https://doi.org/10.1109/ACCESS.2019.2947717

    Article  Google Scholar 

  21. Divya Meena S, Agilandeeswari L (2019) Adaboost cascade classifier for classification and identification of wild animals using movidius neural compute stick. Int J Eng Adv Technol 9(1S3):495–499. https://doi.org/10.35940/ijeat.A1089.1291S319

    Article  Google Scholar 

  22. Salberg AB (2015) Detection of seals in remote sensing images using features extracted from deep convolutional neural networks. In: 2015 IEEE international geoscience and remote sensing symposium (IGARSS), IEEE, pp 1893–1896

  23. Meena SD, Loganathan A (2020) Intelligent animal detection system using sparse multi discriminative-neural network (SMD-NN) to mitigate animal-vehicle collision. Environ Sci Pollut Res 27:39619–39634

    Article  Google Scholar 

  24. Rivas A, Chamoso P, González-Briones A, Corchado JM (2018) Detection of cattle using drones and convolutional neural networks. Sensors 18(7):2048

    Article  Google Scholar 

  25. Kellenberger B, Marcos D, Tuia D (2018) Detecting mammals in UAV images: Best practices to address a substantially imbalanced dataset with deep learning. Remote Sens Environ 216:139–153

    Article  Google Scholar 

  26. Chen K, Gong S, Xiang T, Change Loy C (2013) Cumulative attribute space for age and crowd density estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2467–2474

  27. Zhang C, Li H, Wang X, Yang X (2015) Cross-scene crowd counting via deep convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 833–841

  28. Hsieh MR, Lin YL, Hsu WH (2017) Drone-based object counting by spatially regularized regional proposal network. In: Proceedings of the IEEE international conference on computer vision, pp 4145–4153

  29. https://thenextweb.com/plugged/2016/05/23/robot-will-shepherd-keep-livestock-healthy/

  30. Bechar A, Vigneault C (2016) Agricultural robots for field operations: Concepts and components. Biosys Eng 149:94–111

    Article  Google Scholar 

  31. Ribeiro A, Bengochea-Guevara JM, Conesa-Muñoz J, Nuñez N, Cantuña K, Andújar D (2017) 3D monitoring of woody crops using an unmanned ground vehicle. Adv Anim Biosci 8(2):210–215

    Article  Google Scholar 

  32. Subbarao R, Meer P (2006) Subspace estimation using projection based M-estimators over Grassmann manifolds. In: European conference on computer vision, Springer, Berlin, Heidelberg, pp 301–312

  33. Zhang Z, He Z, Cao G, Cao W (2016) Animal detection from highly cluttered natural scenes using spatiotemporal object region proposals and patch verification. IEEE Trans Multimed 18(10):2079–2092

    Article  Google Scholar 

  34. Tabak MA, Norouzzadeh MS, Wolfson DW, Sweeney SJ, VerCauteren KC, Snow NP, Halseth JM, Di Salvo PA, Lewis JS, White MD, Teton B (2019) Machine learning to classify animal species in camera trap images: applications in ecology. Methods Ecol Evol 10(4):585–590

    Article  Google Scholar 

  35. Lampert CH, Nickisch H, Harmeling S (2009). Learning to detect unseen object classes by between-class attribute transfer. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 951–958

  36. Szankin M, Kwaśniewska A, Rumiński J (2019) Influence of thermal imagery resolution on accuracy of deep learning based face recognition

  37. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv, 1804.02767

  38. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

  39. Sundaram DM, Loganathan A (2020) A new supervised clustering framework using multi discriminative parts and expectation–maximization approach for a fine-grained animal breed classification (SC-MPEM). Neural Process Lett 52(1):727–766

    Article  Google Scholar 

  40. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision, Springer, Cham, pp 21–37

  41. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  42. Oishi Y, Matsunaga T (2014) Support system for surveying moving wild animals in the snow using aerial remote-sensing images. Int J Remote Sens 35(4):1374–1394

    Article  Google Scholar 

  43. Wang J, Xiao R, Guo Y, Zhang L (2019) Learning to count objects with few exemplar annotations. arXiv preprint arXiv, 1905.07898

Download references

Acknowledgements

The authors thank VIT for providing the VIT seed grant for procuring the thermal camera for the research.

Author information

Authors and Affiliations

  1. School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, India

    S. Divya Meena & L. Agilandeeswari

Authors
  1. S. Divya Meena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Agilandeeswari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. Agilandeeswari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meena, S.D., Agilandeeswari, L. Smart Animal Detection and Counting Framework for Monitoring Livestock in an Autonomous Unmanned Ground Vehicle Using Restricted Supervised Learning and Image Fusion. Neural Process Lett 53, 1253–1285 (2021). https://doi.org/10.1007/s11063-021-10439-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10439-4

Keywords

Search

Navigation