Abstract
The paper presents a novel fruit detection algorithm for a plum harvesting robot. At present, the adequate recognition of plum fruits remains a particularly challenging, under-researched task. Difficulties occur due to small plum fruit sizes and dense growth, as well as numerous occlusions in their environment. A harvesting robot operating in such conditions needs to understand which fruits are reachable, in order to avoid collision and end effector damage. This makes a precise and robust visual detection system of crucial importance. Therefore, a lightweight plum detection procedure based on the improved YOLOv7 algorithm has been proposed. Firstly, the images of domestic plums (Prunus domestica L.) were collected in the field, and train, validation and test sets were established. Secondly, light data augmentation was performed. Next, the initial anchor box sizes used by the original YOLOv7 have been updated, based on the plum sizes in the collected dataset. Finally, an SE (Squeeze-and-Excitation) module was added to the backbone network, which helps model the channel interdependencies at almost no computational cost. The Improved-YOLOv7 model was then trained and evaluated on our dataset. The achieved Precision, Recall and mAP were 70.2%, 72.1% and 76.8%, respectively. The model has been compared with other recent models from the YOLO family, and has shown the best accuracy and generalization ability in real, complex environments. Therefore, the proposed plum detection method can provide theoretical and technical support for harvesting robots in real environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Tomić, J., Štampar, F., Glišić, I., Jakopič, J.: Phytochemical assessment of plum (Prunus domesti-ca L.) cultivars selected in Serbia. Food Chem. 299, 125113 (2019). https://doi.org/10.1016/j.foodchem.2019.125113
Zhou, H., Wang, X., Au, W., Kang, H., Chen, C.: Intelligent robots for fruit harvesting: recent developments and future challenges. Precis. Agric. 23(5), 1856–1907 (2022). https://doi.org/10.1007/s11119-022-09913-3
Wang, Z., Xun, Y., Wang, Y., Yang, Q.: Review of smart robots for fruit and vegetable picking in agriculture. Int. J. Agric. Biol. Eng. 15(1), 33–54 (2022). https://doi.org/10.25165/j.ijabe.20221501.7232
Vrochidou, E., Tsakalidou, V.N., Kalathas, I., Gkrimpizis, T., Pachidis, T., Kaburlasos, V.G.: An overview of end effectors in agricultural robotic harvesting systems. Agriculture 12(8), 1240 (2022). https://doi.org/10.3390/agriculture12081240
Koirala, A., Walsh, K.B., Wang, Z., McCarthy, C.: Deep learning – method overview and review of use for fruit detection and yield estimation. Comput. Electron. Agric. 162, 219–234 (2019). https://doi.org/10.1016/j.compag.2019.04.017
Tang, Y., et al.: Recognition and localization methods for vision-based fruit picking robots: a review. Front. Plant Sci. 11, 510 (2020). https://doi.org/10.3389/fpls.2020.00510
Ukwuoma, C.C., Zhiguang, Q., Bin Heyat, M.B., Ali, L., Almaspoor, Z., Monday, H.N.: Recent advancements in fruit detection and classification using deep learning techniques. Math. Probl. Eng. 2022, 1–29 (2022). https://doi.org/10.1155/2022/9210947
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 38(1), 142–158 (2016). https://doi.org/10.1109/TPAMI.2015.2437384
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, Bastian, Matas, Jiri, Sebe, Nicu, Welling, Max (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection (2015). https://doi.org/10.48550/ARXIV.1506.02640
Koirala, A., Walsh, K.B., Wang, Z., McCarthy, C.: Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO.’ Precision Agric. 20(6), 1107–1135 (2019). https://doi.org/10.1007/s11119-019-09642-0
Chen, W., Lu, S., Liu, B., Li, G., Qian, T.: Detecting citrus in orchard environment by using improved YOLOv4. Sci. Program. 2020, 1–13 (2020). https://doi.org/10.1155/2020/8859237
Parico, A.I.B., Ahamed, T.: Real time pear fruit detection and counting using YOLOv4 models and deep SORT. Sensors 21(14), 4803 (2021). https://doi.org/10.3390/s21144803
Yan, B., Fan, P., Lei, X., Liu, Z., Yang, F.: A real-time apple targets detection method for picking robot based on improved YOLOv5. Remote Sens. 13(9), 1619 (2021). https://doi.org/10.3390/rs13091619
Wu, D., et al.: Detection of camellia Oleifera fruit in complex scenes by using YOLOv7 and data augmentation. Appl. Sci. 12(22), 11318 (2022). https://doi.org/10.3390/app122211318
Wang, L., Zhao, Y., Liu, S., Li, Y., Chen, S., Lan, Y.: Precision detection of dense plums in orchards using the improved YOLOv4 Model. Front. Plant Sci. 13, 839269 (2022). https://doi.org/10.3389/fpls.2022.839269
Kim, E., et al.: CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques. Sci. Rep. 12(1), 20796 (2022). https://doi.org/10.1038/s41598-022-25260-9
Redmon, J., Farhadi, A.: YOLOv3: an Incremental improvement (2018). https://doi.org/10.48550/ARXIV.1804.02767
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022). https://doi.org/10.48550/ARXIV.2207.02696
Wang, C. Y., et al.: CSPNet: a new backbone that can enhance learning capability of CNN (2019). https://doi.org/10.48550/ARXIV.1911.11929
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition (2014). https://doi.org/10.48550/ARXIV.1406.4729
Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze -and-excitation networks (2017). https://doi.org/10.48550/ARXIV.1709.01507
Acknowledgements
This work is the result of research and development carried out on the project “Development of a demonstration prototype of a manipulative robot for picking edible plums - proof of concept”, contract no. 623–1-22, innovation voucher no. 1216 co-financed by the Innovation Fund of the Republic of Serbia, 2022.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Šumarac, J., Kljajić, J., Rodić, A. (2023). A Fruit Detection Algorithm for a Plum Harvesting Robot Based on Improved YOLOv7. In: Petrič, T., Ude, A., Žlajpah, L. (eds) Advances in Service and Industrial Robotics. RAAD 2023. Mechanisms and Machine Science, vol 135. Springer, Cham. https://doi.org/10.1007/978-3-031-32606-6_52
Download citation
DOI: https://doi.org/10.1007/978-3-031-32606-6_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-32605-9
Online ISBN: 978-3-031-32606-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)