Skip to main content
SpringerLink
Log in

CitrusYOLO: A Algorithm for Citrus Detection under Orchard Environment Based on YOLOv4

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Achieving rapid and accurate detection of tree fruits under natural environments is essential for many precision agriculture application (such as harvesting robots and yield estimation). A real-time citrus recognition method was proposed in this paper by improving the latest YOLOv4 (You Only Look Once version 4) detector for using in orchard environments. The Canopy algorithm and the K-Means + + algorithm were used to automatically select the number and size of prior frames corresponding to the image dataset. Then, an attention mechanism module is added in front of the output layer of each feature of different scales, and a depthwise separable convolution module is added before the upsampling of the network neck to better detect clementines in complex backgrounds. Finally, the network is pruned using the scientific control-based neural network pruning (SCOP) algorithm, and the parameters of the pruned model were fine-tuned to restore some recognition accuracy. Five common used deep learning algorithms including the Faster R-CNN, SSD, YOLOv3, YOLOv4 and Detectron2 were compared to verify the effectiveness of the proposed method. The experimental results showed that our improved YOLOv4 detector works well for detecting different growth periods of citrus in natural orchard environment. The average accuracy increase from 92.89 to 96.15%, the detection time for each image is 0.06s, both are superior the above five algorithms. While the model size was down from 250 MB to 187 MB. This would promise the proposed method being suitable for orchard yield estimation and the development of fruit harvesting robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Bargoti S, Underwood JP (2017) Image segmentation for fruit detection and yield estimation in apple orchards. J Field Robot 34:1039–1060

    Article  Google Scholar 

  2. Bochkovskiy A, Wang C Y, Liao H Y M (2020) YOLOv4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.

  3. Chen P, Li Y, Zhou H, Liu B, Liu P (2020) Detection of small ship objects using anchor boxes cluster and feature pyramid network model for SAR imagery. J Mar Sci Eng 8:112

    Article  Google Scholar 

  4. Dai J, Tong J (2018) Galaxy morphology classification using deep residual networks. Progress in Astronomy 36:384–397

    Google Scholar 

  5. Font D, Palleja T, Tresanchez M, Runcan D, Moreno J, Martinez D, Teixido M, Palacin J (2014) A proposal for automatic fruit harvesting by combining a low-cost stereovision camera and a robotic arm. Sensors 14:11557–11579

    Article  Google Scholar 

  6. Fu C Y, Liu W, Ranga A, Tyagi A, Berg A C (2017) DSSD: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659.

  7. Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K (2015) Sensors and systems for fruit detection and localization: a review. Comput Electron Agric 116:8–19

    Article  Google Scholar 

  8. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H. (2017). Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

  9. Huang X, Li G, Ma C, Yang S (2018) Green peach recognition based on improved discriminative regional feature integration algorithm in similar background. Trans CSAE 34:142–148

    Google Scholar 

  10. Jiang B, Wu Q, Yin X, Wu D, He D (2019) Flyolov3 deep learning for key parts of dairy cow body detection. Comput Electron Agric 166:104982

    Article  Google Scholar 

  11. Kang H, Chen C (2019) Fruit detection and segmentation for apple harvesting using visual sensor in orchards. Sensors 19:4599

    Article  Google Scholar 

  12. Koirala A, Walsh KB, Wang ZL, McCarthy C (2019) Deep learning – method overview and review of use for fruit detection and yield estimation. Comput Electron Agric 162:219–234

    Article  Google Scholar 

  13. Koirala A, Walsh KB, Wang ZL, McCarthy C (2019) Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’. Precision Agric 20:1107–1135

    Article  Google Scholar 

  14. Kurtulmuş F, Kavdir I (2014) Detecting corn tassels using computer vision and support vector machines. Exp Syst Appl 41:7390–7397

    Article  Google Scholar 

  15. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  16. Li F, Jiang Y, Li T, Feng Y, Chen S (2020) Design of a robot end effector with measurement system for precise pick-and-place of square objects. Procedia Manuf 48:172–180

    Article  Google Scholar 

  17. Lin G, Tang Y, Zou X, Li J, Xiong J (2019) In-field citrus detection and localisation based on RGB-D image analysis. Biosyst Eng 186:34–44

    Article  Google Scholar 

  18. Liu W, Anguelov D, Erhan D et al (2016) SSD: single shot multibox detector. European conference on computer vision. Springer, Cham, pp 21–37

    Google Scholar 

  19. Liu Z, Li J, Shen Z, Huang G, Yan S, Zhang C (2017) Learning efficient convolutional networks through network slimming. arXiv preprint arXiv:1708.06519.

  20. Misra D (2019) Mish: a self regularized non-monotonic neural activation function. arXiv preprint arXiv:1908.08681.

  21. Nasiri A, Taheri-Garavand A, Zhang YD (2019) Image-based deep learning automated sorting of date fruit. Postharvest Biol Tec 153:133–141

    Article  Google Scholar 

  22. Osco LP, Arruda MDSD, Junior JM, Silva NBD, Gonalves WN (2020) A convolutional neural network approach for counting and Geolocating Citrus-trees in UAV multispectral imagery. ISPRS J Photogramm 160:97–106

    Article  Google Scholar 

  23. Pham V, Pham C, Dang T (2020) Road damage detection and classification with detectron2 and faster r-cnn. In: 2020 IEEE international conference on big data, pp 5592–5601

    Chapter  Google Scholar 

  24. Redmon J, Divvala S, Girshick R, Farhadi A (2015) You only look once: unified, real-time object detection. arXiv preprint arXiv:1506.02640.

  25. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE T Pattern Anal 39(6):1137–1149

    Article  Google Scholar 

  26. Sengupta S, Lee WS (2014) Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosyst Eng 117:51–61

    Article  Google Scholar 

  27. Takalkar MA, Thuseethan S, Rajasegarar S, Chaczko Z, Xu M, Yearwood J (2021) LGAttNet: automatic micro-expression detection using dual-stream local and global attentions. Knowl-Based Syst 212:106566

    Article  Google Scholar 

  28. Tang Y, Wang Y, Xu Y, Tao D, Xu C, Xu C, Xu C (2020) Scop: scientific control for reliable neural network pruning. Adv Neural Inf Proces Syst 33:10936–10947

    Google Scholar 

  29. Tian Y, Yang G, Wang Z, Wang H, Li E, Liang Z (2019) Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput Electron Agric 157:417–426

    Article  Google Scholar 

  30. Ulzii-Orshikh D, Malrey L, Sang-seok Y (2017) An yield estimation in citrus orchards via fruit detection and counting using image processing. Comput Electron Agric 140:103–112

    Article  Google Scholar 

  31. Vasconez JP, Kantor GA, Cheein FAA (2019) Human–robot interaction in agriculture: a survey and current challenges. Biosyst Eng 179:35–48

    Article  Google Scholar 

  32. Vasconez JP, Delpiano J, Vougioukas S, Auat Cheein F (2020) Comparison of convolutional neural networks in fruit detection and counting: a comprehensive evaluation. Comput Electron Agric 173:105348

    Article  Google Scholar 

  33. Wang C, Li X, Wu Z, Zhou Z, Feng Y (2014) Machine vision detecting potato mechanical damage based on manifold learning algorithm. Trans CSAE 30:245–252

    Google Scholar 

  34. Wu J, Peng B, Huang Z, Xie J (2013) Research on computer vision-based object detection and classification. In: Li D, Chen Y (eds) Computer and computing Technologies in Agriculture VI. Springer, Berlin, Heidelberg, pp 183–188

    Chapter  Google Scholar 

  35. Wu D, Wu Q, Yin X, Jiang B, Song H (2020) Lameness detection of dairy cows based on the yolov3 deep learning algorithm and a relative step size characteristic vector. Biosyst Eng 189:150–163

    Article  Google Scholar 

  36. Xiong J, Zheng Z, Liang J, Zhong Z, Liu B, Sun B (2020) Citrus detection method in night environment based on improved YOLO v3 network. Trans CSAE 51:199–206

    Google Scholar 

  37. Yang L, Zhang RY, Li L, Xie X (2021) Simam: a simple, parameter-free attention module for convolutional neural networks. In: International conference on machine learning, PMLR, pp 11863–11874

  38. Zhuang JJ, Luo SM, Hou CJ, Tang Y, He Y, Xue XY (2018) Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Comput Electron Agric 152:64–73

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (no. 61762013, 62062015 and 61662006), the Science and Technology Foundation of Guangxi Province (no. 2018 AD19339), Innovation Project of Guangxi Graduate Education (No. XYCSZ2021007) and Research Fund of Guangxi Key Lab of Multi-source Information Mining & Security (No. 20-A-02-02). The authors also thank the Guangxi Scholarship Fund of the Guangxi Education Department for the support.

Author information

Authors and Affiliations

  1. College of Computer Science & Engineering, Guangxi Normal University, Guilin, 541004, China

    Wenkang Chen, Shenglian Lu, Ming Chen & Guo Li

  2. Guangxi Key Lab of Multisource Information Mining & Security, 541004, Guilin, China

    Shenglian Lu & Ming Chen

  3. Guangxi Citrus Breeding and Cultivation Engineering Technology Center, Guangxi Academy of Specialty Crops, 541004, Guilin, China

    Binghao Liu

  4. Agricultural Information Institutes of Science and Technology, Shanghai Academy of Agriculture Sciences, 201403, Shanghai, China

    Tingting Qian

Authors
  1. Wenkang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shenglian Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Binghao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tingting Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

W.C. developed the algorithm, trained the models and wrote the manuscript; S.L., M.C. and T.Q. designed the experiments, helped perform the analysis with constructive discussions; B.L. and G.L. carried out the experiments and helped to label data;

Corresponding authors

Correspondence to Shenglian Lu or Tingting Qian.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, W., Lu, S., Liu, B. et al. CitrusYOLO: A Algorithm for Citrus Detection under Orchard Environment Based on YOLOv4. Multimed Tools Appl 81, 31363–31389 (2022). https://doi.org/10.1007/s11042-022-12687-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12687-5

Keywords

Search

Navigation