Skip to main content
SpringerLink
Log in

A deep learning-based framework for object recognition in ecological environments with dense focal loss and occlusion

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In precision agricultural analysis, the remote sensing of geospatial data holds substantial potential for multi-purpose crop surveys, targeting automatic crop area delineation, health monitoring, and yield estimation. Advanced remote sensing methods, when integrated with machine learning techniques, have significantly advanced agricultural analyses. This study introduces a three-tiered framework. Firstly, orchard areas are delineated using the ESA Sentinel-2 multispectral instrument (MSI) at 10 m resolution, employing the normalized differential vegetation index (NDVI). In the second stage, mango tree canopies are detected from hand-annotated true color composite imagery using two variants of a convolutional neural network. The first variant, CanopyNet-1, is built directly over RetinaNet foundational layers, achieving a mean average precision (mAP) of 0.79, a precision of 0.80, and a recall of 0.76. The second variant, CanopyNet-2, builds upon DeepForest, a generalized tree canopy trained model, also using RetinaNet at its base. CanopyNet-2 demonstrates superior performance, achieving a mAP of 0.83, a precision of 0.98, and a recall of 0.96, notably surpassing conventional models such as YOLOv5 and Faster R-CNN. Lastly, the health of the orchard is characterized using 3-m-resolution multispectral imagery. Cumulatively, our framework, with its tiered approach, exhibits high accuracy in both tree canopy delineation and health characterization, suggesting it as a comprehensive solution for large-scale orchard monitoring and yield optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The authors declare that all necessary details regarding the data supporting the findings of this study are available within the paper. Should any raw data files be needed in any relevant format, they are available from the corresponding author upon reasonable request.

References

  1. Azam A, Shafique M (2017) Agriculture in Pakistan and its impact on economy. A Review. Inter J Adv Sci Technol 103:47–60

    Article  Google Scholar 

  2. Usman M (2016) Contribution of agriculture sector in the GDP growth rate of Pakistan. J Global Econ 4(2):1–3

    Google Scholar 

  3. Xue J, Su B et al (2017) Significant remote sensing vegetation indices: a review of developments and applications. J Sens. https://doi.org/10.1155/2017/1353691

    Article  Google Scholar 

  4. Glenn EP, Huete AR, Nagler PL, Nelson SG (2008) Relationship between remotely-sensed vegetation indices, canopy attributes, and plant physiological processes: what vegetation indices can and cannot tell us about the landscape. Sensors 8(4):2136–2160

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  5. Wang S, Fu G (2023) Modelling soil moisture using climate data and normalized difference vegetation index based on nine algorithms in alpine grasslands. Front Environ Sci 11:1130448

    Article  Google Scholar 

  6. Stamford JD, Vialet-Chabrand S, Cameron I, Lawson T (2023) Development of an accurate low cost NDVI imaging system for assessing plant health. Plant Methods 19(1):9

    Article  PubMed  PubMed Central  Google Scholar 

  7. Weiss M, Jacob F, Duveiller G (2020) Remote sensing for agricultural applications: a meta-review. Remote Sens Environ 236:111402

    Article  Google Scholar 

  8. Omia E, Bae H, Park E, Kim MS, Baek I, Kabenge I, Cho B-K (2023) Remote sensing in field crop monitoring: a comprehensive review of sensor systems, data analyses and recent advances. Remote Sens 15(2):354

    Article  ADS  Google Scholar 

  9. Zhang S, Yu J, Xu H, Qi S, Luo J, Huang S, Liao K, Huang M (2023) Mapping the age of subtropical secondary forest using dense landsat time series data: an ensemble model. Remote Sens 15(8):2067

    Article  ADS  Google Scholar 

  10. Gupta P, Sharma R (1990) Application of satellite remote sensing technique in delineation and hectarage estimation of mango orchards in parts of uttar pradesh, india. In: Proceedings of the GIS development, Uttar Pradesh, India, pp 12–14

  11. Stussi N, Liew SC, Kwoh LK, Lim H, Nichol J, Goh KC (1997) Landcover classification using ERS SAR/INSAR data on the coastal region of central sumatra

  12. N’Doume C, Lachenaud P, Hussard A, Nguyen H, Flori A (2000) Etude de faisabilité pour l’élaboration d’une cartographie statistique d’inventaire des vergers café et cacao en côte d’ivoire par télédétection satellitale

  13. Yadav I, Rao NS, Reddy B, Rawal R, Srinivasan V, Sujatha N, Bhattacharya C, Rao PN, Ramesh K, Elango S (2002) Acreage and production estimation of mango orchards using Indian remote sensing (IRS) satellite data. Sci Hortic 93(2):105–123

    Article  Google Scholar 

  14. Palaniswami C, Upadhyay A, Maheswarappa H (2006) Spectral mixture analysis for subpixel classification of coconut. Curr Sci 91:1706–1711

    Google Scholar 

  15. Sharma A, Panigrahy S (2007) Apple orchard characterization using remote sensing and GIS in shimla district of himachal pradesh. In: Proceedings of remote sensing and photogrammetry annual conference 2007, pp 11–14. Citeseer

  16. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  17. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  18. Weinstein BG, Marconi S, Bohlman S, Zare A, White E (2019) Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks. Remote Sens 11(11):1309

    Article  ADS  Google Scholar 

  19. Salovaara KJ, Thessler S, Malik RN, Tuomisto H (2005) Classification of Amazonian primary rain forest vegetation using Landsat ETM+ satellite imagery. Remote Sens Environ 97(1):39–51

    Article  ADS  Google Scholar 

  20. Ke Y, Quackenbush LJ (2011) A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing. Int J Remote Sens 32(17):4725–4747

    Article  Google Scholar 

  21. Larsen M, Eriksson M, Descombes X, Perrin G, Brandtberg T, Gougeon FA (2011) Comparison of six individual tree crown detection algorithms evaluated under varying forest conditions. Int J Remote Sens 32(20):5827–5852

    Article  Google Scholar 

  22. Aubry-Kientz M, Dutrieux R, Ferraz A, Saatchi S, Hamraz H, Williams J, Coomes D, Piboule A, Vincent G (2019) A comparative assessment of the performance of individual tree crowns delineation algorithms from ALS data in tropical forests. Remote Sens 11(9):1086

    Article  ADS  Google Scholar 

  23. Han W, Zhang X, Wang Y, Wang L, Huang X, Li J, Wang S, Chen W, Li X, Feng R et al (2023) A survey of machine learning and deep learning in remote sensing of geological environment: challenges, advances, and opportunities. ISPRS J Photogramm Remote Sens 202:87–113

    Article  ADS  Google Scholar 

  24. Mittal P, Singh R, Sharma A (2020) Deep learning-based object detection in low-altitude UAV datasets: a survey. Image Vis Comput 104:104046

    Article  Google Scholar 

  25. Ahmad A, Saraswat D, El Gamal A (2023) A survey on using deep learning techniques for plant disease diagnosis and recommendations for development of appropriate tools. Smart Agric Technol 3:100083

    Article  Google Scholar 

  26. Li W, Fu H, Yu L, Cracknell A (2017) Deep learning based oil palm tree detection and counting for high-resolution remote sensing images. Remote Sens 9(1):22

    Article  ADS  Google Scholar 

  27. Csillik O, Cherbini J, Johnson R, Lyons A, Kelly M (2018) Identification of citrus trees from unmanned aerial vehicle imagery using convolutional neural networks. Drones 2(4):39

    Article  Google Scholar 

  28. Ayrey E, Hayes DJ (2018) The use of three-dimensional convolutional neural networks to interpret LiDAR for forest inventory. Remote Sens 10(4):649

    Article  ADS  Google Scholar 

  29. Guirado E, Tabik S, Alcaraz-Segura D, Cabello J, Herrera F (2017) Deep-learning versus OBIA for scattered shrub detection with google earth imagery: Ziziphus lotus as case study. Remote Sens 9(12):1220

    Article  ADS  Google Scholar 

  30. Stein M, Bargoti S, Underwood J (2016) Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors 16(11):1915

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  31. Nagaraja A, Sahoo R, Usha K, Gupta V (2017) Estimation of mango growing areas using remote sensing. Indian J Hortic 74(2):184–188

    Article  Google Scholar 

  32. Rahman MM, Robson A, Bristow M (2018) Exploring the potential of high resolution worldview-3 imagery for estimating yield of mango. Remote Sens 10(12):1866

    Article  ADS  Google Scholar 

  33. Agaradahalli Gurumurthy V, Kestur R, Narasipura O (2019) Mango tree net–a fully convolutional network for semantic segmentation and individual crown detection of mango trees. arXiv preprint arXiv:1907.06915

  34. Kestur R, Meduri A, Narasipura O (2019) MangoNet: a deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard. Eng Appl Artif Intell 77:59–69

    Article  Google Scholar 

  35. Huete A, Didan K, Miura T, Rodriguez EP, Gao X, Ferreira LG (2002) Overview of the radiometric and biophysical performance of the modis vegetation indices. Remote Sens Environ 83(1–2):195–213

    Article  ADS  Google Scholar 

  36. Gitelson AA, Viña A, Arkebauer TJ, Rundquist DC, Keydan G, Leavitt B (2003) Remote estimation of leaf area index and green leaf biomass in maize canopies. Geophys Res Lett. https://doi.org/10.1029/2002GL016450

    Article  Google Scholar 

  37. Qi J, Chehbouni A, Huete AR, Kerr YH, Sorooshian S (1994) A modified soil adjusted vegetation index. Remote Sens Environ 48(2):119–126

    Article  ADS  Google Scholar 

  38. Huete AR (1988) A soil-adjusted vegetation index (savi). Remote Sens Environ 25(3):295–309

    Article  ADS  Google Scholar 

  39. Gamon JA, Field CB, Goulden ML, Griffin KL, Hartley AE, Joel G, Penuelas J, Valentini R (1995) Relationships between NDVI, canopy structure, and photosynthesis in three Californian vegetation types. Ecol Appl 5(1):28–41

    Article  Google Scholar 

  40. Huang S, Tang L, Hupy JP, Wang Y, Shao G (2021) A commentary review on the use of normalized difference vegetation index (NDVI) in the era of popular remote sensing. J For Res 32(1):1–6

    Article  Google Scholar 

  41. Buda M, Maki A, Mazurowski MA (2018) A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw 106:249–259

    Article  PubMed  Google Scholar 

  42. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  43. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  44. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  45. Gaiser H, Vries M, Williamson A, Henon Y, Morariu M, Lacatusu V, Liscio E, Fang W, Clark M, Sande M, et al (2019) fizyr/keras-retinanet 0.2

  46. Morad M, Chalmers A, O’regan P (1996) The role of root-mean-square error in the geo-transformation of images in GIS. Int J Geogr Inf Sci 10(3):347–353

    Article  Google Scholar 

  47. Wu W, Liu H, Li L, Long Y, Wang X, Wang Z, Li J, Chang Y (2021) Application of local fully convolutional neural network combined with YOLO v5 algorithm in small target detection of remote sensing image. PLoS ONE 16(10):0259283

    Article  Google Scholar 

  48. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pp 21–37. Springer

  49. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

Download references

Author information

Authors and Affiliations

  1. Institute of Geographical Information Systems, National University of Sciences and Technology, Islamabad, Pakistan

    Muhammad Munir Afsar, Ejaz Hussain & Javed Iqbal

  2. Department of Electrical Engineering, Military College of Signals, National University of Science and Technology, Rawalpindi, Pakistan

    Asim Dilawar Bakhshi

Authors
  1. Muhammad Munir Afsar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Asim Dilawar Bakhshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ejaz Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Javed Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Munir Afsar.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Afsar, M.M., Bakhshi, A.D., Hussain, E. et al. A deep learning-based framework for object recognition in ecological environments with dense focal loss and occlusion. Neural Comput & Applic (2024). https://doi.org/10.1007/s00521-024-09582-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-024-09582-5

Keywords

Search

Navigation