Abstract
Timely and accurate identification of apple leaf diseases provides an important basis for the early warning and precise control of apple leaf diseases. It was of great significance to reduce the economic losses caused by diseases. In order to improve the accuracy of apple leaf lesion segmentation by using the deep neural networks, this paper proposed a twill pooling method and combined it with strip pooling to propose the double cross pooling method. The pooling method could contract multidirectional and multi-scale spatial context information. Then, on the constructed apple leaf disease dataset, these modules were combined with existing deep semantic segmentation networks to perform disease lesion semantic segmentation. These three constructed modules were added to the fully convolutional network respectively, and the universality of the constructed modules was further verified on other semantic segmentation networks. Experimental results showed that the proposed modules could improve the mean intersection over union of fully convolutional network by 7.69%, up to 80.44% on the collected apple leaf disease dataset. Furthermore, when adding the proposed pooling modules to DeepLabV3 + , PSPNet and U-Net, the mean intersection over union improved by varying degrees when adding any of the proposed modules alone. The performance of the improved fully convolutional network, DeepLabV3 + , PSPNet and U-Net were compared. The DeepLabV3 + with the double cross pooling module had the best segmentation performance whose mean pixel accuracy, mean intersection over union, leaf disease classification accuracy and leaf disease degree diagnosis accuracy were 99.07%, 82.1%, 99.52% and 77.65% respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The data are not publicly available due to restrictions. Their containing information that could compromise the privacy of research participants.
References
Chen, L., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Berlin, Germany: Springer. https://doi.org/10.1007/978-3-030-01234-2_49
Chen, Z., Ting, D., Newbury, R., & Chen, C. (2021). Semantic segmentation for partially occluded apple trees based on deep learning. Computers and Electronics in Agriculture, 181, 105952. https://doi.org/10.1016/j.compag.2020.105952
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904–1916. https://doi.org/10.1109/tpami.2015.2389824
Hou, Q., Zhang, L., Cheng, M., & Feng, J. (2020). Strip pooling: Rethinking spatial pooling for scene parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Piscataway, USA: IEEE Computer Society. https://doi.org/10.1109/cvpr42600.2020.00406
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019). Ccnet: Criss-cross attention for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Piscataway, USA: IEEE Computer Society. https://doi.org/10.1109/iccv.2019.00069
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Piscataway, USA: IEEE Computer Society. https://doi.org/10.1109/cvpr.2015.7298965
Maheswari, P., Raja, P., Apolo-Apolo, O. E., & Pérez-Ruiz, M. (2021). Intelligent fruit yield estimation for orchards using deep learning based semantic segmentation techniques—a review. Frontiers in Plant Science, 12, 684328. https://doi.org/10.3389/fpls.2021.684328
Milioto, A., Lottes, P., & Stachniss, C. (2018). Real-time semantic segmentation of crop and weed for precision agriculture robots leveraging background knowledge in CNNs. In IEEE International Conference on Robotics and Automation (ICRA), Piscataway, USA: IEEE Computer Society. https://doi.org/10.1109/icra.2018.8460962
Ministry of Agriculture of the People's Republic of China. (2006). The operation quality for air-assisted orchard sprayer (NY/T 992-2006).
Peng, Y., Wang, A., Liu, J., & Faheem, M. (2021). A comparative study of semantic segmentation models for identification of grape with different varieties. Agriculture, 11(10), 997. https://doi.org/10.3390/agriculture11100997
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCA), Berlin, Germany: Springer. https://doi.org/10.1007/978-3-319-24574-4_28
Sadeghi-Tehran, P., Virlet, N., Ampe, E. M., Reyns, P., & Hawkesford, M. J. (2019). DeepCount: In-field automatic quantification of wheat spikes using simple linear iterative clustering and deep convolutional neural networks. Frontiers in Plant Science, 10, 1176. https://doi.org/10.3389/fpls.2019.01176
Su, J., Yi, D., Su, B., Mi, Z., Liu, C., & Hu, X. (2020). Aerial visual perception in smart farming: Field study of wheat yellow rust monitoring. IEEE Transactions on Industrial Informatics, 17(3), 2242–2249. https://doi.org/10.1109/tii.2020.2979237
Wang, X., Girshick, R., Gupta, A., & He, K. (2018). Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Piscataway, USA: IEEE Computer Society. https://doi.org/10.1109/cvpr.2018.00813
You, J., Liu, W., & Lee, J. (2020). A DNN-based semantic segmentation for detecting weed and crop. Computers and Electronics in Agriculture, 178, 105750. https://doi.org/10.1016/j.compag.2020.105750
Yu, F., & Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv e-prints, 1511–7122. https://doi.org/10.48550/arXiv.1511.07122
Yu, H., & Son, C. (2020). Leaf spot attention network for apple leaf disease identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Piscataway, USA:, IEEE Computer Society. https://doi.org/10.1109/cvprw50498.2020.00034
Zhang, X., Yang, Y., Li, Z., Ning, X., Qin, Y., & Cai, W. (2021). An improved encoder-decoder network based on strip pool method applied to segmentation of farmland vacancy field. Entropy, 23(4), 435. https://doi.org/10.3390/e23040435
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017). Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Piscataway, USA: IEEE Computer Society. https://doi.org/10.1109/cvpr.2017.660
Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C. C., Lin, D., & Jia, J. (2018). Psanet: Point-wise spatial attention network for scene parsing. In Proceedings of the European conference on computer vision (ECCV), Berlin, Germany: Springer. https://doi.org/10.1007/978-3-030-01240-3_17
Zhong, Y., & Zhao, M. (2020). Research on deep learning in apple leaf disease recognition. Computers and Electronics in Agriculture, 168, 105146. https://doi.org/10.1016/j.compag.2019.105146
Zou, K., Chen, X., Wang, Y., Zhang, C., & Zhang, F. (2021). A modified U-Net with a specific data argumentation method for semantic segmentation of weed images in the field. Computers and Electronics in Agriculture, 187, 106242. https://doi.org/10.1016/j.compag.2021.106242
Acknowledgements
This research was funded by the Research Start-up Fee for PhD of Northwest A&F University under Grant 2452021095, and it was also funded by the Undergraduate Training Program for Innovation and Entrepreneurship of Northwest A&F University under Grant X202010712368 and No. S202110712283 .
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Feng, J., Chao, X., Zhang, Z. et al. A pooling module with multidirectional and multi-scale spatial information and its application on semantic segmentation of leaf lesions. Precision Agric 24, 2416–2437 (2023). https://doi.org/10.1007/s11119-023-10046-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11119-023-10046-4