Skip to main content
SpringerLink
Log in

Detecting Apples in Orchards Using YOLOv3

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2020 (ICCSA 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12249))

Included in the following conference series:

Abstract

A machine vision system for detecting apples in orchards was developed. The system designed for use in harvesting robots is based on a YOLOv3 algorithm modification with pre- and postprocessing. As a result, apples that are blocked by leaves and branches, green apples on a green background, darkened apples are detected. Apple detection time averaged 19 ms with 90.8% Recall (fruit detection rate), and 7.8% FPR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bechar, A., Vigneault, C.: Agricultural robots for field operations: concepts and components. Biosyst. Eng. 149, 94–111 (2016). https://doi.org/10.1016/j.biosystemseng.2016.06.014

    Article  Google Scholar 

  2. Sistler, F.E.: Robotics and intelligent machines in agriculture. IEEE J. Robot. Autom. 3(1), 3–6 (1987). https://doi.org/10.1109/JRA.1987.1087074

    Article  Google Scholar 

  3. Ceres, R., Pons, J., Jiménez, A., Martín, J., Calderón, L.: Design and implementation of an aided fruit-harvesting robot (Agribot). Ind. Robot 25(5), 337–346 (1998). https://doi.org/10.1108/01439919810232440

    Article  Google Scholar 

  4. Edan, Y., Han, S.F., Kondo, N.: Automation in agriculture. In: Springer Handbook of Automation, pp. 1095–1128. Springer, Berlin, Heidelberg (2009). https://doi.org/10.1007/978-3-540-78831-7_63

  5. Grift, T., Zhang, Q., Kondo, N., Ting, K.C.: A review of automation and robotics for the bio-industry. J. Biomechatronics Eng. 1(1), 37–54 (2008). http://journal.tibm.org.tw/wp-content/uploads/2013/06/2.-automation-and-robotics-for-the-bio-industry.pdf. Accessed 19 Apr 2020

  6. Mao, W.H., Ji, B.P., Zhan, J.C., Zhang, X.C., Hu, X.A.: Apple location method for the apple harvesting robot. In: Proceedings of the 2nd International Congress on Image and Signal Processing – CIPE 2009, Tianjin, China, 7–19 October 2009, pp. 17–19 (2009). https://doi.org/10.1109/cisp.2009.5305224

  7. Bulanon, D.M., Kataoka, T.: A fruit detection system and an end effector for robotic harvesting of Fuji apples. Agric. Eng. Int. CIGR J. 12(1), 203–210 (2010). https://cigrjournal.org/index.php/Ejounral/article/view/1285/1319. Accessed 19 Apr 2020

  8. Wei, X., Jia, K., Lan, J., Li, Y., Zeng, Y., Wang, C.: Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot. Optics 125(12), 5684–5689 (2014). https://doi.org/10.1016/j.ijleo.2014.07.001

    Article  Google Scholar 

  9. Zhao, Y.S., Gong, L., Huang, Y.X., Liu, C.L.: A review of key techniques of vision-based control for harvesting robot. Comput. Electron. Agric. 127, 311–323 (2016). https://doi.org/10.1016/j.compag.2016.06.022

    Article  Google Scholar 

  10. Bulanon, D.M., Burks, T.F., Alchanatis, V.: Image fusion of visible and thermal images for fruit detection. Biosyst. Eng. 103(1), 12–22 (2009). https://doi.org/10.1016/j.biosystemseng.2009.02.009

    Article  Google Scholar 

  11. Wachs, J.P., Stern, H.I., Burks, T., Alchanatis, V.: Low and high-level visual feature-based apple detection from multi-modal images. Precision Agric. 11, 717–735 (2010). https://doi.org/10.1007/s11119-010-9198-x

    Article  Google Scholar 

  12. Wachs, J.P., Stern, H.I., Burks, T., Alchanatis, V.: Apple detection in natural tree canopies from multimodal images. In: Proceedings of the 7th Joint International Agricultural Conference – JIAC 2009, Wageningen, Netherlands, 6–8 July 2009, pp. 293–302 (2009). http://web.ics.purdue.edu/~jpwachs/papers/2009/ApplesDetection_FullPaper_revised.pdf. Accessed 19 Apr 2020

  13. Whittaker, A.D., Miles, G.E., Mitchell, O.R.: Fruit location in a partially occluded image. Trans. Am. Soc. Agric. Eng. 30(3), 591–596 (1987). https://doi.org/10.13031/2013.30444

    Article  Google Scholar 

  14. Xie, Z.Y., Zhang, T.Z., Zhao, J.Y.: Ripened strawberry recognition based on Hough transform. Trans. Chin. Soc. Agric. Mach. 38(3), 106–109 (2007)

    Google Scholar 

  15. Xie, Z., Ji, C., Guo, X., Zhu, S.: An object detection method for quasi-circular fruits based on improved Hough transform. Trans. Chin. Soc. Agric. Mach. 26(7), 157–162 (2010). https://doi.org/10.3969/j.issn.1002-6819.2010.7.028

    Article  Google Scholar 

  16. Kelman, E.E., Linker, R.: Vision-based localization of mature apples in tree images using convexity. Biosyst. Eng. 118(1), 174–185 (2014). https://doi.org/10.1016/j.biosystemseng.2013.11.007

    Article  Google Scholar 

  17. Xie, Z., Ji, C., Guo, X., Zhu, S.: Detection and location algorithm for overlapped fruits based on concave spots searching. Trans. Chin. Soc. Agric. Mach. 42(12), 191–196 (2011)

    Google Scholar 

  18. Zhao, J., Tow, J., Katupitiya, J.: On-tree fruit recognition using texture properties and color data. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, Canada, 2–6 August 2005, pp. 263–268 (2005). https://doi.org/10.1109/iros.2005.1545592

  19. Parrish, E.A., Goksel, J.A.K.: Pictorial pattern recognition applied to fruit harvesting. Trans. Am. Soc. Agric. Eng. 20(5), 822–827 (1977). https://doi.org/10.13031/2013.35657

    Article  Google Scholar 

  20. Sites, P.W., Delwiche, M.J.: Computer vision to locate fruit on a tree. Trans. Am. Soc. Agric. Eng. 31(1), 257–263 (1988). https://doi.org/10.13031/2013.30697

    Article  Google Scholar 

  21. Bulanon, D.M., Kataoka, T., Okamoto, H., Hata, S.: Development of a real-time machine vision system for apple harvesting robot. In: Society of Instrument and Control Engineers Annual Conference, Sapporo, Japan, 4–6 August 2004, pp. 595–598 (2004). https://doi.org/10.11499/sicep.2004.0_108_5

  22. Seng, W.C., Mirisaee, S.H.: A new method for fruits recognition system. In: Proceedings of the 2009 International Conference on Electrical Engineering and Informatics – ICEEI 2009, Selangor, Malaysia, 5–7 August 2009, vol. 1, pp. 130–134 (2009). https://doi.org/10.1109/iceei.2009.5254804

  23. Linker, R., Cohen, O., Naor, A.: Determination of the number of green apples in RGB images recorded in orchards. Comput. Electron. Agric. 81(1), 45–57 (2012). https://doi.org/10.1016/j.compag.2011.11.007

    Article  Google Scholar 

  24. Ji, W., Zhao, D., Cheng, F.Y., Xu, B., Zhang, Y., Wang, J.: Automatic recognition vision system guided for apple harvesting robot. Comput. Electr. Eng. 38(5), 1186–1195 (2012). https://doi.org/10.1016/j.compeleceng.2011.11.005

    Article  Google Scholar 

  25. Tao, Y., Zhou, J.: Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking. Comput. Electron. Agric. 142(A), 388–396 (2017). https://doi.org/10.1016/j.compag.2017.09.019

  26. Zhan, W.T., He, D.J., Shi, S.L.: Recognition of kiwifruit in field based on Adaboost algorithm. Trans. Chin. Soc. Agric. Eng. 29(23), 140–146 (2013). https://doi.org/10.3969/j.issn.1002-6819.2013.23.019

    Article  Google Scholar 

  27. Zhao, Y.S., Gong, L., Huang, Y.X., Liu, C.L.: Robust tomato recognition for robotic harvesting using feature images fusion. Sensors 16(2), 173–185 (2016). https://doi.org/10.3390/s16020173

    Article  Google Scholar 

  28. Zhao, Y.S., Gong, L., Huang, Y.X., Liu, C.L.: Detecting tomatoes in greenhouse scenes by combining AdaBoost classifier and colour analysis. Biosyst. Eng. 148(8), 127–137 (2016). https://doi.org/10.1016/j.biosystemseng.2016.05.001

    Article  Google Scholar 

  29. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 – NIPS 2012, Harrahs and Harveys, Lake Tahoe, Canada, 3–8 December 2012, pp. 1–9 (2012). https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf. Accessed 19 Apr 2020

  30. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations – ICLR 2015, San Diego, California, USA, 7–9 May 2015, pp. 1–14 (2015). https://arxiv.org/abs/1409.1556. Accessed 19 Apr 2020

  31. Williams, H.A.M., Jones, M.H., Nejati, M., Seabright, M.J., MacDonald, B.A.: Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms. Biosyst. Eng. 181, 140–156 (2019). https://doi.org/10.1016/j.biosystemseng.2019.03.007

    Article  Google Scholar 

  32. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 29th IEEE Conference on Computer Vision and Pattern Recognition – CVPR 2016, Las Vegas, Nevada, USA, 26 June–1 July 2016, pp. 779–788 (2016). https://arxiv.org/abs/1506.02640. Accessed 19 Apr 2020

  33. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: YOLOv3: an incremental improvement. In: 31th IEEE Conference on Computer Vision and Pattern Recognition – CVPR 2018, Salt Lake City, Utah, USA, 18–22 June 2018, pp. 1–6 (2018). https://arxiv.org/abs/1804.02767. Accessed 19 Apr 2020

  34. Tian, Y., Yang, G., Wang, Z., Wang, H., Li, E., Liang, Z.: Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agric. 157, 417–426 (2019). https://doi.org/10.1016/j.compag.2019.01.012

    Article  Google Scholar 

  35. Tian, Y., Yang, G., Wang, Z., Li, E., Liang, Z.: Detection of apple lesions in orchards based on deep learning methods of CycleGAN and YOLO-V3-Dense. J. Sens. 2019, 1–14 (2019). https://doi.org/10.1155/2019/7630926

    Article  Google Scholar 

  36. Huang, G., Liu, Zh., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 30th IEEE Conference on Computer Vision and Pattern Recognition – CVPR 2017, Honolulu, Hawaii, USA, 22–25 July 2017, pp. 1–9 (2017). https://arxiv.org/abs/1608.06993. Accessed 19 Apr 2020

  37. Kang, H., Chen, C.: Fruit detection, segmentation and 3D visualization of environments in apple orchards. Comput. Electron. Agric. 171, article 105302 (2020). https://doi.org/10.1016/j.compag.2020.105302

  38. Wan, S., Goudos, S.: Faster R-CNN for multi-class fruit detection using a robotic vision system. Comput. Netw. 168, article 107036 (2020). https://doi.org/10.1016/j.comnet.2019.107036

  39. COCO: Common Objects in Context Dataset. http://cocodataset.org/#overview. Accessed 19 Apr 2020

Download references

Author information

Authors and Affiliations

  1. Financial University under the Government of the Russian Federation, 38 Shcherbakovskaya, Moscow, 105187, Russia

    Anna Kuznetsova, Tatiana Maleva & Vladimir Soloviev

Authors
  1. Anna Kuznetsova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tatiana Maleva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Soloviev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Soloviev .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Potenza, Italy

    Beniamino Murgante

  3. Chair- Center of ICT/ICE, Covenant University, Ota, Nigeria

    Sanjay Misra

  4. University of Cagliari, Cagliari, Italy

    Chiara Garau

  5. University of Cagliari, Cagliari, Italy

    Ivan Blečić

  6. Clayton School of Information Technology, Monash University, Clayton, VIC, Australia

    David Taniar

  7. Department of Information Science, Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  8. University of Minho, Braga, Portugal

    Ana Maria A.C. Rocha

  9. Polytechnic University of Bari, Bari, Italy

    Eufemia Tarantino

  10. Polytechnic University of Bari, Bari, Italy

    Carmelo Maria Torre

  11. Department of Neurology, University of Massachusetts Medical School, Worcester, MA, USA

    Yeliz Karaca

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kuznetsova, A., Maleva, T., Soloviev, V. (2020). Detecting Apples in Orchards Using YOLOv3. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2020. ICCSA 2020. Lecture Notes in Computer Science(), vol 12249. Springer, Cham. https://doi.org/10.1007/978-3-030-58799-4_66

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58799-4_66

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58798-7

  • Online ISBN: 978-3-030-58799-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation