Abstract
Crop diseases have always been a dilemma as it can cause significant diminution in both quality and quantity of agricultural yields. Thus, automatic recognition and severity estimation of crop diseases on leaves plays a crucial role in agricultural sector. In this paper, we propose a series of automatic image-based crop leaf diseases recognition and severity estimation networks, i.e., BR-CNNs, which can simultaneously recognize crop species, classify crop diseases and estimate crop diseases severity based on deep learning. BR-CNNs based on binary relevance (BR) multi-label learning algorithm and deep convolutional neural network (CNN) approaches succeed in identifying 7 crop species, 10 crop diseases types (including Healthy) and 3 crop diseases severity kinds (normal, general and serious). Compared with LP-CNNs and MLP-CNNs, the overall performance of BR-CNNs is superior. The BR-CNN based on ResNet50 achieves the best test accuracy of 86.70%, which demonstrates the feasibility and effectiveness of our network. The BR-CNN based on the light-weight NasNet also achieves excellent test accuracy of 85.28%, which can provide more possibilities for the development of mobile systems and devices.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atoum Y, Afridi MJ, Liu X, McGrath JM, Hanson LE (2016) On developing and enhancing plant-level disease rating systems in real fields. Pattern Recogn 53:287–299. https://doi.org/10.1016/j.patcog.2015.11.021
Bhowmick PK, Basu A, Mitra P, Prasad A (2009) Multi-label text classification approach for sentence level news emotion analysis. In: International conference on pattern recognition and machine intelligence. https://doi.org/10.1007/978-3-642-11164-8_42
Chen L, Wang R, Yang J, Xue L, Hu M (2019) Multi-label image classification with recurrently learning semantic dependencies. Vis Comput 35(10):1361–1371. https://doi.org/10.1007/s00371-018-01615-0
Chenghai Y, Odvody GN, Fernandez CJ, Landivar JA, Minzenmayer RR, Nichols RL (2015) Evaluating unsupervised and supervised image classification methods for mapping cotton root rot. Precis Agric 16(2):201–215. https://doi.org/10.1007/s11119-014-9370-9
Fan GF, Peng LL, Hong WC (2018) Short term load forecasting based on phase space reconstruction algorithm and bi-square kernel regression model. Appl Energy 224:13–33. https://doi.org/10.1016/j.apenergy.2018.04.075
Ferentinos KP (2018) Deep learning models for plant disease detection and diagnosis. Comput Electron Agric 145:311–318. https://doi.org/10.1016/j.compag.2018.01.009
Guan W, Sun Y, Wang J (2017) Automatic image-based plant disease severity estimation using deep learning. Comput Intell Neurosci 2017:2917536. https://doi.org/10.1155/2017/2917536
He N, Wang T, Chen P, Yan H, Jin Z (2018) An android malware detection method based on deep autoencoder. In: Proceedings of the 2018 artificial intelligence and cloud computing conference, pp 88–93. https://doi.org/10.1145/3299819.3299834
Hiary H, Bani S, Reyalat M, Braik M, Alars Z (2011) Fast and accurate detection and classification of plant diseases. Int J Comput Appl 17(1):31–38. https://doi.org/10.5120/2183-2754
Hong WC, Li MW, Geng J, Zhang Y (2019) Novel chaotic bat algorithm for forecasting complex motion of floating platforms. Appl Math Model 72:425–443. https://doi.org/10.1016/j.apm.2019.03.031
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141. https://doi.org/10.1109/TPAMI.2019.2913372
Huang G, Liu S, Van der Maaten L, Weinberger KQ (2018) Condensenet: an efficient densenet using learned group convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2752–2761
Huang G, Liu Z, Laurens VDM, Weinberger KQ (2016) Densely connected convolutional networks. https://doi.org/10.1109/CVPR.2017.243
Hughes DP, Salathe M (2015) An open access repository of images on plant health to enable the development of mobile disease diagnostics. https://challenger.ai/dataset/pdd2018/. Accessed 10 Nov 2018
Ji M, Zhang L, Wu Q (2019) Automatic grape leaf diseases identification via unitedmodel based on multiple convolutional neural networks. Inf Process Agric. https://doi.org/10.1016/j.inpa.2019.10.003
Kaiming H, Xiangyu Z, Shaoqing R, Jian S (2015) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.2015.7298594
Ke L, Liang G, Yixian H, Chengliang L, Junsong P (2019) Deep learning-based segmentation and quantification of cucumber powdery mildew using convolutional neural network. Front Plant Sci. https://doi.org/10.3389/fpls.2019.00155
Lee J, Seo W, Park JH, Kim DW (2019) Compact feature subset-based multi-label music categorization for mobile devices. Multimed Tools Appl 78(4):4869–4883. https://doi.org/10.1007/s11042-018-6100-8
Liang Q, Xiang S, Hu Y, Coppola G, Zhang D, Sun W (2019) Pd2se-net: computer-assisted plant disease diagnosis and severity estimation network. Comput Electron Agric 157:518–529. https://doi.org/10.1016/j.compag.2019.01.034
Lizbeth HRD, Ramos-Quintana F, Guerrero JJ (2014) Integrating soms and a Bayesian classifier for segmenting diseased plants in uncontrolled environments. Sci World J 2014:214674. https://doi.org/10.1155/2014/214674
Ma N, Zhang X, Zheng HT, Sun J (2018) Shufflenet v2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European conference on computer vision (ECCV), pp 116–131. https://doi.org/10.1007/978-3-030-01264-9_8
Madjarov G, Kocev D, Gjorgjevikj D, Džeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recogn 45(9):3084–3104. https://doi.org/10.1016/j.patcog.2012.03.004
Martinez A (2018) Georgia plant disease loss estimates. http://www.caes.uga.edu/Publications/displayHTML.cfm?pk_id=7762/. Accessed 10 Nov 2018
Mohanty SP, Hughes DP, Salathé M (2016) Using deep learning for image-based plant disease detection. Front Plant Sci 7:1419. https://doi.org/10.3389/fpls.2016.01419
Nikolaev P (2019) Multi-label human activity recognition on image using deep learning. In: 7th Scientific conference on information technologies for intelligent decision making support (ITIDS 2019). https://doi.org/10.2991/itids-19.2019.26
Pouyanfar S, Wang T, Chen SC (2019) A multi-label multimodal deep learning framework for imbalanced data classification. In: 2019 IEEE conference on multimedia information processing and retrieval (MIPR), pp 199–204
Sabatelli M, Kestemont M, Daelemans W, Geurts P (2018) Deep transfer learning for art classification problems. In: Proceedings of the European conference on computer vision (ECCV). https://doi.org/10.1007/978-3-030-11012-3_48
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Comput Sci
Sun H, Wei J, Zhang J, Yang W (2014) A comparison of disease severity measurements using image analysis and visual estimates using a category scale for genetic analysis of resistance to bacterial spot in tomato. Eur J Plant Pathol 139(1):125–136. https://doi.org/10.1007/s10658-013-0371-8
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going deeper with convolutions. https://doi.org/10.1109/CVPR.2016.90
Tian Y, Wang L, Zhou Q (2011) Grading method of crop disease based on image processing. In: International conference on computer and computing technologies in agriculture V, vol 369, pp 427–433. https://doi.org/10.1007/978-3-642-27278-3_45
Wosiak A, Glinka K, Zakrzewska D (2017) Multi-label classification methods for improving comorbidities identification. Comput Biol Med 100:279–288. https://doi.org/10.1007/978-94-009-7798-3_15
Xavier TW, Souto RN, Statella T, Galbieri R, Santos ES, S Suli G, Zeilhofer P (2019) Identification of ramularia leaf blight cotton disease infection levels by multispectral, multiscale UAV imagery. Drones 3(2):33. https://doi.org/10.3390/drones3020033
Zhang ML, Zhou ZH (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837. https://doi.org/10.1109/TKDE.2013.39
Zhang Z, Hong WC (2019) Electric load forecasting by complete ensemble empirical mode decomposition adaptive noise and support vector regression with quantum-based dragonfly algorithm. Nonlinear Dyn 98(2):1107–1136. https://doi.org/10.1007/s11071-019-05252-7
Zhao B, Li X, Lu X, Wang Z (2018) A CNN-RNN architecture for multi-label weather recognition. Neurocomputing 322:47–57. https://doi.org/10.1016/j.neucom.2018.09.048
Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8697–8710
Zufferey D, Hofer T, Hennebert J, Schumacher M, Ingold R, Bromuri S (2015) Performance comparison of multi-label learning algorithms on clinical data for chronic diseases. Comput Biol Med 65:34–43. https://doi.org/10.1016/j.compbiomed.2015.07.017
Acknowledgements
This work was supported by the Public Welfare Industry (Agriculture) Research Projects Level-2 under Grant 201503116-04-06; Postdoctoral Foundation of Heilongjiang Province under Grant LBHZ15020; Harbin Applied Technology Research and Development Program under Grant 2017RAQXJ096; and National Key Application Research and Development Program in China under Grant 2018YFD0300105-2.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ji, M., Zhang, K., Wu, Q. et al. Multi-label learning for crop leaf diseases recognition and severity estimation based on convolutional neural networks. Soft Comput 24, 15327–15340 (2020). https://doi.org/10.1007/s00500-020-04866-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-04866-z