Skip to main content
SpringerLink
Log in
Book cover

International Conference on Smart and Sustainable Agriculture

SSA 2021: Smart and Sustainable Agriculture pp 84–102Cite as

Comparison of Machine Learning and Deep Learning Methods for Grape Cluster Segmentation

Part of the Communications in Computer and Information Science book series (CCIS,volume 1470)

Abstract

Automatic grape detection is one of the first steps towards automatic yield estimation. This step is often performed with a computer vision algorithm using the classic feature extraction and classification approach. Many grape bunch detection algorithms have been proposed in the last decade and most of them follow this standard approach. An alternative is semantic segmentation with deep learning models. The main objective of this work is to compare existing algorithms based on machine learning to encoder-decoder segmentation models (UNet and PSPNet). The comparison was performed on two challenging datasets of white grape varieties in natural lighting conditions. The UNet model reached better performances on both datasets with up-to 0.76 IoU score (compared to 0.59 IoU for the second best model). UNet was combined to a linear model to estimate the total number of grape bunches in 200 plants and reached 86% counting accuracy. The results show that deep learning models are more robust to white grape detection compared to classic segmentation techniques. This is an important property for early yield estimation before veraison.

Keywords

  • Grape detection
  • Precision viticulture
  • Deep learning
  • Semantic segmentation

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://romeo.univ-reims.fr.

References

  1. Abdelghafour, F., Rosu, R., Keresztes, B., Germain, C., Da Costa, J.P.: A bayesian framework for joint structure and colour based pixel-wise classification of grapevine proximal images. Comput. Electron. Agric. 158, 345–357 (2019). https://doi.org/10.1016/j.compag.2019.02.017

    CrossRef  Google Scholar 

  2. Aquino, A., Millan, B., Diago, M.P., Tardaguila, J.: Automated early yield prediction in vineyards from on-the-go image acquisition. Comput. Electron. Agric. 144, 26–36 (2018). https://doi.org/10.1016/j.compag.2017.11.026

    CrossRef  Google Scholar 

  3. Araya-Alman, M., et al.: A new localized sampling method to improve grape yield estimation of the current season using yield historical data. Precis. Agric. 20(2), 445–459 (2019). https://doi.org/10.1007/s11119-019-09644-y

    CrossRef  Google Scholar 

  4. Behroozi-Khazaei, N., Maleki, M.R.: A robust algorithm based on color features for grape cluster segmentation. Comput. Electron. Agric. 142, 41–49 (2017). https://doi.org/10.1016/j.compag.2017.08.025

    CrossRef  Google Scholar 

  5. Berenstein, R., Shahar, O.B., Shapiro, A., Edan, Y.: Grape clusters and foliage detection algorithms for autonomous selective vineyard sprayer. Intell. Serv. Robot. 3(44), 233–243 (2010). https://doi.org/10.1007/s11370-010-0078-z

    CrossRef  Google Scholar 

  6. Bramley, R., Hamilton, R.: Understanding variability in winegrape production systems. Aust. J. Grape Wine Res. 10(1), 32–45 (2004). https://doi.org/10.1111/j.1755-0238.2004.tb00006.x, https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1755-0238.2004.tb00006.x

  7. Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998). https://doi.org/10.1023/A:1009715923555

    CrossRef  Google Scholar 

  8. Cecotti, H., Rivera, A., Farhadloo, M., Pedroza, M.A.: Grape detection with convolutional neural networks. Expert Syst. Appl. 159, 113588 (2020). https://doi.org/10.1016/j.eswa.2020.113588

  9. Chamelat, R., Rosso, E., Choksuriwong, A., Rosenberger, C., Laurent, H., Bro, P.: Grape detection by image processing. In: IECON 2006–32nd Annual Conference on IEEE Industrial Electronics, pp. 3697–3702 (2006). https://doi.org/10.1109/IECON.2006.347704

  10. Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, pp. 2843–2851. NIPS 2012, Curran Associates Inc., Red Hook, NY, USA (2012)

    Google Scholar 

  11. Clingeleffer, P.R., Martin, S.R., Dunn, G.M., Krstic, M.P.: Crop development, crop estimation and crop control to secure quality and production of major wine grape varieties : a national approach : final report to Grape and Wine Research and Development Corporation/principal investigator, Peter Clingeleffer; [prepared and edited by Steve Martin and Gregory Dunn]. Adelaide, Grape and Wine Research and Development Corporation (2001)

    Google Scholar 

  12. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893 (2005). https://doi.org/10.1109/CVPR.2005.177

  13. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  14. Dunn, G.M., Martin, S.R.: Yield prediction from digital image analysis: a technique with potential for vineyard assessments prior to harvest. Aust. J. Grape Wine Res. 10(33), 196–198 (2004). https://doi.org/10.1111/j.1755-0238.2004.tb00022.x

    CrossRef  Google Scholar 

  15. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997). https://doi.org/10.1006/jcss.1997.1504, https://www.sciencedirect.com/science/article/pii/S002200009791504X

  16. Goldberg, D.E.: Genetic algorithms. Pearson Education India (2006)

    Google Scholar 

  17. Grimm, J., Herzog, K., Rist, F., Kicherer, A., Töpfer, R., Steinhage, V.: An adaptable approach to automated visual detection of plant organs with applications in grapevine breeding. Biosyst. Eng. 183, 170–183 (2019). https://doi.org/10.1016/j.biosystemseng.2019.04.018

    CrossRef  Google Scholar 

  18. Grossetete, M., Berthoumieu, Y., Da Costa, J.P., Germain, C., Lavialle, O., Grenier, G.: Early estimation of vineyard yield: site specific counting of berries by using a smartphone. In: International Conference on Agiculture Engineering (AgEng), pp. tabla137-C1915 (July 2012). https://hal.archives-ouvertes.fr/hal-00950298

  19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  20. Heinrich, K., Roth, A., Breithaupt, L., Möller, B., Maresch, J.: Yield prognosis for the agrarian management of vineyards using deep learning for object counting. Wirtschaftsinformatik 2019 Proceedings, p. 15 (February 2019). https://aisel.aisnet.org/wi2019/track05/papers/3

  21. Keresztes, B., Abdelghafour, F., Randriamanga, D., Da Costa, J.P., Germain, C.: Real-time fruit detection using deep neural networks. In: 14th International Conference on Precision Agriculture (2018). https://hal.archives-ouvertes.fr/hal-02518559

  22. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980

  23. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, pp. 1097–1105. NIPS 2012, Curran Associates Inc., Red Hook, NY, USA (2012)

    Google Scholar 

  24. LeCun, Y., Bengio, Y.: Convolutional Networks for Images, Speech, and Time Series, pp. 255–258. MIT Press, Cambridge, MA, USA (1998)

    Google Scholar 

  25. Liu, L., et al.: On the variance of the adaptive learning rate and beyond. CoRR abs/1908.03265 (2019). http://arxiv.org/abs/1908.03265

  26. Liu, S., Zeng, X., Whitty, M.: A vision-based robust grape berry counting algorithm for fast calibration-free bunch weight estimation in the field. Comput. Electron. Agric. 173, 11 (2020). https://doi.org/10.1016/j.compag.2020.105360

    CrossRef  Google Scholar 

  27. Luo, L., Tang, Y., Zou, X., Wang, C., Zhang, P., Feng, W.: Robust grape cluster detection in a vineyard by combining the adaboost framework and multiple color components. Sensors (Basel, Switzerland) 16(1212), 21 (2016). https://doi.org/10.3390/s16122098, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5191078/

  28. Millan, B., Velasco-Forero, S., Aquino, A., Tardaguila, J.: On-the-go grapevine yield estimation using image analysis and boolean model (December 2018). https://doi.org/10.1155/2018/9634752, https://www.hindawi.com/journals/js/2018/9634752/

  29. Nuske, S., Wilshusen, K., Achar, S., Yoder, L., Singh, S.: Automated visual yield estimation in vineyards. J. Field Robot. 31(55), 837–860 (2014). https://doi.org/10.1002/rob.21541

    CrossRef  Google Scholar 

  30. Reis, M.J.C.S., et al.: Automatic detection of bunches of grapes in natural environment from color images. J. Appl. Log. 10(44), 285–290 (2012). https://doi.org/10.1016/j.jal.2012.07.004

  31. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    CrossRef  Google Scholar 

  32. Rudolph, R., Herzog, K., Töpfer, R., Steinhage, V.: Efficient identification, localization and quantification of grapevine inflorescences in unprepared field images using fully convolutional networks. arXiv:1807.03770 [cs] pp. 95–104 (July 2018). http://arxiv.org/abs/1807.03770, arXiv: 1807.03770

  33. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2018)

    Google Scholar 

  34. Santos, T.T., de Souza, L.L., dos Santos, A.A., Avila, S.: Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. Comput. Electron. Agric. 170, 105247 (2020). https://doi.org/10.1016/j.compag.2020.105247

  35. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017). https://doi.org/10.1109/TPAMI.2016.2572683

    CrossRef  Google Scholar 

  36. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1409.1556

  37. Vazquez-Fernandez, E., Dacal-Nieto, A., Martin, F., Formella, A., Torres-Guijarro, S., Gonzalez-Jorge, H.: A computer vision system for visual grape grading in wine cellars. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 335–344. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04667-4_34

    CrossRef  Google Scholar 

  38. Victorino, G., Maia, G., Queiroz, J., Braga, R., Marques, J., Lopes, C.: Grapevine yield prediction using image analysis - improving the estimation of non-visible bunches. European Federation for Information Technology in Agriculture, Food and the Environment (EFITA), p. 6 (2019)

    Google Scholar 

  39. Wada, K., LabelMe, K.: Image Polygonal Annotation with Python (2016). https://github.com/wkentaro/labelme

  40. Yakubovskiy, P.: Segmentation models (2019). https://github.com/qubvel/segmentation_models

  41. Zabawa, L., Kicherer, A., Klingbeil, L., Töpfer, R., Kuhlmann, H., Roscher, R.: Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS J. Photogramm. Remote. Sens. 164, 73–83 (2020). https://doi.org/10.1016/j.isprsjprs.2020.04.002

    CrossRef  Google Scholar 

  42. Zhang, M.R., Lucas, J., Hinton, G.E., Ba, J.: Lookahead optimizer: k steps forward, 1 step back. CoRR abs/1907.08610 (2019). http://arxiv.org/abs/1907.08610

  43. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. arXiv:1612.01105 [cs] (December 2016). http://arxiv.org/abs/1612.01105, arXiv: 1612.01105

  44. Škrabánek, P.: Deepgrapes: precise detection of grapes in low-resolution images. IFAC-PapersOnLine 51(66), 185–189 (2018). https://doi.org/10.1016/j.ifacol.2018.07.151

    CrossRef  Google Scholar 

  45. Škrabánek, P., Doležel, P.: Robust grape detector based on svms and hog features. Comput. Intell. Neurosci. 2017, 3478602 (2017). https://doi.org/10.1155/2017/3478602

Download references

Acknowledgements

This work has been performed in the project AI4DI: Artificial Intelligence for Digitizing Industry, under grant agreement No 826060. The project is co-funded by grants from Germany, Austria, Finland, France, Norway, Latvia, Belgium, Italy, Switzerland, and the Czech Republic and - Electronic Component Systems for European Leadership Joint Undertaking (ECSEL JU).

We want to thank Vranken-Pommery Monopole, our partner in the AI4DI project, for allowing image collection in their vineyards. We also thank the ROMEO Computing Center (https://romeo.univ-reims.fr) of Université de Reims Champagne-Ardenne, whose Nvidia DGX-1 server allowed us to accelerate the training steps and compare several model approaches.

Author information

Authors and Affiliations

  1. Université de Reims Champagne Ardenne, Laboratoire LICIIS - LRC CEA DIGIT, 51687, Reims Cedex 2, France

    Lucas Mohimont, Mathias Roesler, François Alin & Luiz Angelo Steffenel

  2. Université de Reims Champagne Ardenne, RIBP - EA4707 - USC - INRAE1488, 51687, Reims Cedex 2, France

    Lucas Mohimont, Marine Rondeau & Nathalie Gaveau

Authors
  1. Lucas Mohimont
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathias Roesler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marine Rondeau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nathalie Gaveau
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. François Alin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luiz Angelo Steffenel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lucas Mohimont .

Editor information

Editors and Affiliations

  1. CNAM/CEDRIC, Paris, France

    Selma Boumerdassi

  2. Université Internationale de Rabat, Rabat, Morocco

    Mounir Ghogho

  3. Laboratoire LIGM UMR 8049 CNRS, ESIEE Paris, Noisy-le-Grand, France

    Éric Renault

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mohimont, L., Roesler, M., Rondeau, M., Gaveau, N., Alin, F., Steffenel, L.A. (2021). Comparison of Machine Learning and Deep Learning Methods for Grape Cluster Segmentation. In: Boumerdassi, S., Ghogho, M., Renault, É. (eds) Smart and Sustainable Agriculture. SSA 2021. Communications in Computer and Information Science, vol 1470. Springer, Cham. https://doi.org/10.1007/978-3-030-88259-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88259-4_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88258-7

  • Online ISBN: 978-3-030-88259-4

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation