Abstract
In this paper we present a perception system for agriculture robotics that enables an unmanned ground vehicle (UGV) equipped with a multi spectral camera to automatically perform the crop/weed detection and classification tasks in real-time. Our approach exploits a pipeline that includes two different convolutional neural networks (CNNs) applied to the input RGB+near infra-red (NIR) images. A lightweight CNN is used to perform a fast and robust, pixel-wise, binary image segmentation, in order to extract the pixels that represent projections of 3D points that belong to green vegetation. A deeper CNN is then used to classify the extracted pixels between the crop and weed classes. A further important contribution of this work is a novel unsupervised dataset summarization algorithm that automatically selects from a large dataset the most informative subsets that better describe the original one. This enables to streamline and speed-up the manual dataset labeling process, otherwise extremely time consuming, while preserving good classification performance. Experiments performed on different datasets taken from a real farm robot confirm the effectiveness of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In our setup, one second represents a resonable time constraint in order to enable the robot to actively remove the weeds as soon as they are detected.
- 2.
This is a lower bound: in most of the practical cases the approximated solution ensures much better results.
References
Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). http://tensorflow.org/
Aitkenhead, M., Dalgetty, I., Mullins, C., McDonald, A., Strachan, N.: Weed and crop discrimination using image analysis and artificial intelligence methods. Comput. Electron. Agric. 39(3), 157–171 (2003)
Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc. (2006)
Borregaard, T., Nielsen, H., Nrgaard, L., Have, H.: Cropweed discrimination by line imaging spectroscopy. J. Agric. Eng. Res. 75(4), 389–400 (2000)
Burks, T.F., Shearer, S.A., Gates, R.S., Donohue, K.D.: Backpropagation neural network design and evaluation for classifying weed species using color image texture. Trans. ASAE 43(4), 1029–1037 (2000)
Cerutti, G., Tougne, L., Mille, J., Vacavant, A., Coquin, D.: A model-based approach for compound leaves understanding and identification. In: IEEE International Conference on Image Processing. pp. 1471–1475 (2013)
Che Hussin, N., Jamil, N., Nordin, S., Awang, K.: Plant species identification by using scale invariant feature transform (SIFT) and grid based colour moment (GBCM). In: Proceedings of the IEEE Conference on Open Systems (ICOS), pp. 226–230 (2013)
Feyaerts, F., Gool, L.V.: Multi-spectral vision system for weed detection. Pattern Recognit. Lett. 22(6–7), 667–674 (2001)
Haug, S., Michaels, A., Biber, P., Ostermann, J.: Plant classification system for crop /weed discrimination without segmentation. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV) (2014)
Hemming, J., Rath, T.: PA-precision agriculture: computer-vision-based weed identification under field conditions using controlled lighting. J. Agric. Eng. Res. 78(3), 233–243 (2001)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of a Advances in Neural Information Processing Systems (NIPS), pp. 1106–1114 (2012)
Kumar, N., Belhumeur, P.N., Biswas, A., Jacobs, D.W., Kress, W.J., Lopez, I., Soares, J.V.B.: Leafsnap: A computer vision system for automatic plant species identification. In: The 12th European Conference on Computer Vision (ECCV), pp. 502–516 (2012)
Lee, S.H., Chan, C.S., Wilkin, P., Remagnino, P.: Deep-plant: Plant identification with convolutional neural networks. In: Proceedings of the IEEE International Conference on Image Processing (ICIP), pp. 452–456 (2015)
Lin, H., Bilmes, J.: A class of submodular functions for document summarization. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. vol. 37, pp. 510–520 (2011)
Lottes, P., Hoeferlin, M., Sander, S., Müter, M., Schulze Lammers, P., Stachniss, C.: an effective classification system for separating sugar beets and weeds for precision farming applications. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2016)
Nemhauser, G.L., Wolsey, L.A., Fisher, M.L.: An analysis of approximations for maximizing submodular set functions–i. Math. Program. 14(1), 265–294 (1978)
Parkhi, O.M., Vedaldi, A., Jawahar, C.V., Zisserman, A.: Cats and dogs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3498–3505 (2012)
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)
Reyes, A.K., Caicedo, J.C., Camargo, J.E.: Fine-tuning deep convolutional networks for plant recognition. In: Working Notes of Conference and Labs of the Evaluation forum (CLEF) (2015)
Rouse, Jr., J.W., Haas, R.H., Schell, J.A., Deering, D.W.: Monitoring vegetation systems in the great plains with ERTS. In: Proceedings of the 3rd Earth Resource Technology Satellite (ERTS) Symposium. vol. 1 (1974)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: Integrated recognition, localization and detection using convolutional networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2014)
Shearer, S.A., Holmes, R.G.: Plant identification using color co-occurrence matrices. Trans. ASAE 33(6), 2037–2044 (1990)
Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering objects and their location in images. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2005)
Tellaeche, A., Burgos-Artizzu, X.P., Pajares, G., Ribeiro, A.: A vision-based method for weeds identification through the bayesian decision theory. Pattern Recognit. 41(2), 521–530 (2008)
Vinh, L.T., Lee, S., Park, Y., d’Auriol, B.J.: A novel feature selection method based on normalized mutual information. Appl. Intell. 37(1), 100–120 (2011)
Wang, X.F., shuang Huang, D., xiang Du, J., Xu, H., Heutte, L.: Classification of plant leaf images with complicated background. Appl. Math. Comput. 205(2), 916–926 (2008)
Yao, B., Khosla, A., Fei-Fei, L.: Combining randomization and discrimination for fine-grained image categorization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1577–1584 (2011)
Acknowledgements
We thank Cyrill Stachniss and Philipp Lottes for providing us with the datasets used in this paper.
This work has been supported by the European Commission under the grant number H2020-ICT-644227-FLOURISH.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Potena, C., Nardi, D., Pretto, A. (2017). Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture. In: Chen, W., Hosoda, K., Menegatti, E., Shimizu, M., Wang, H. (eds) Intelligent Autonomous Systems 14. IAS 2016. Advances in Intelligent Systems and Computing, vol 531. Springer, Cham. https://doi.org/10.1007/978-3-319-48036-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-48036-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48035-0
Online ISBN: 978-3-319-48036-7
eBook Packages: EngineeringEngineering (R0)