Keywords

1 Introduction

Traditional methods to diagnose crop diseases, which depend on experience of agriculture experts or individual subjective consciousness who refer to some relative books, have an impact on diagnosis accuracy. Recently, the successful application of computer vision system in the area of crop disease image recognition makes up defects of conventional methods. Camargo and Smith [1] compared recognition accuracy rate of three kinds of cotton diseases by extracting different features with support vector machine (SVM), and the highest accuracy rate reached 90%. Li et al. [2] identified wheat stripe rust and leaf rust with SVM, which acquired a good result. Huang [3] applied artificial neural network to identify Clivia soft rot, black rot and leaf spot, and the average recognition rate ultimately got 89%. Based on rough sets theory and BP neural network, Zhang et al. [4] made four kinds of cotton diseases recognition and the average accuracy rate reached 92.72%. Nevertheless, most of these methods depend on two assumptions. First, the training and test data should be under the same distribution. Second, there should be enough labeled training samples. However, these two assumptions cannot be satisfied in many cases. In order to solve this problem, transfer learning [5] is proposed, which can improve a learner by transferring information from a relative domain to a new domain. Most proposed works mainly focused on the text domain [6,7,8].

In the field of image classification, the research results of transfer learning can be summarized to two main categories. The one is heterogeneous transfer learning by using related words to help image classification [9, 10]. And the other one is that usually classifies images with other images from a relative domain [11]. Here, in order to solve the problem with few labeled training samples of crop diseases, combining with digital image processing technology, the paper proposed a method which introduced transfer learning to the crop disease image recognition. Took the images of five kinds of cucumber and rice diseases, for example, firstly, the spot images could be obtained after image preprocessing and segmentation. Then, the TrAdaBoost-based method was used to identify the spot images after color, texture and shape features extraction.

2 Image Acquisition and Features Extraction

2.1 Image Acquisition and Equipment

Images in this paper were collected on sunny days, using the digital single lens reflex camera of the model Canon EOS 6D. For the consideration of time and space efficiency while preserving more image details, the original images were compressed to the resolution of \(600 \times 400\) pixels and the major disease parts were reserved with Photoshop.

2.2 Image Segmentation

Before the lesion spot image recognition, we need to segment the lesion area. There are several image segmentation methods including edge detection [12], graph theory [13] and so on. In this paper, the double Otsu algorithm [14] is used due to its universality on these five kinds of diseases. More concretely, the R component of RGB space of the original color image is selected for the first Otsu segmentation and morphological operation. As a result, the image is divided into background and non-background class. Next, after comparing different color components of the lesion area of non-background class, we carry out the second Otsu on the Cr component of YCbCr space. The spot images obtained by double Otsu are shown in Fig. 1.

Fig. 1.
figure 1

Segmentation results of five kinds of cucumber and rice diseases (cucumber target leaf spot, cucumber downy mildew, cucumber bacterial angular leaf spot, rice blast, rice brown spot) (Color figure online)

Full size image

2.3 Feature Extraction

Color feature extraction. Since it is obviously different between the normal area and the lesion area of leaf in color, the color can be considered as an important feature. The color moment is a common method to extract color features. So we express the color distribution with average, variance and skewness. RGB and HIS are two kinds of common color spaces which consist of the component R, G, B and the component H, I, S, respectively. It is evident for the lesion areas of cucumber and rice leaf at channel B. And the component H is only one dimension value because of a conversion from gray value in RGB space to hue value in HIS space [15]. So we can calculate average, variance and skewness of B and H component according to formulas (1)–(3):

$$\begin{aligned} Average:\mu _{i}=\frac{1}{n}\sum ^{n}_{j=1}P_{i,j}, \end{aligned}$$
(1)
$$\begin{aligned} Variance:\sigma _{i}=(\frac{1}{n}\sum ^{n}_{j=1}(P_{i,j}-\mu _{i})^{2})^{\frac{1}{2}}, \end{aligned}$$
(2)
$$\begin{aligned} Skewness:s_{i}=(\frac{1}{n}\sum ^{n}_{j=1}(P_{i,j}-\mu _{i})^{3})^{\frac{1}{3}}, \end{aligned}$$
(3)

where \(P_{(i,j)}\) is gray value of j pixel point at channel i and n is the number of total pixels.

Texture feature extraction. As an important indicator for the lesion area recognition, texture feature is usually expressed by GLCM (Gray Level Co-occurrence Matrix). We assume that f(x,y) is an \(M \times N\) gray image whose grayscale is \(N_g\). So the element of GLCM can be shown as:

$$\begin{aligned} P(i,j,d,\theta )=\#\{[(x_{1},y_{1}),(x_{2},y_{2})]\in M \times N|f(x_{1},y_{1})=i,f(x_{2},y_{2})=j\}, \end{aligned}$$
(4)

where \(P(i,j,d,\theta )\) is the element of row i and j column, d is the distance between the two pixels, \(\theta \) is the angle between the pixel and the abscissa axis, and \(\#\)(x) is the number of the set x, and \((i,j)\in N_g\times N_g\) [15].

In the paper, we set \(d=1\). When \(\theta \) is 0\(^\circ \), 45\(^\circ \), 90\(^\circ \) and 135\(^\circ \), we respectively calculate the energy, the contrast and the entropy. Afterwards, we obtain their average and standard deviation. Some correlative formulas are given as:

$$\begin{aligned} Energy(Ene):Ene=\sum _{i=0}^{N_g}\sum _{j=0}^{N_g}[p(i,j)]^{2}, \end{aligned}$$
(5)
$$\begin{aligned} Contrast(Con):Con=\sum _{i=0}^{N_g}\sum _{j=0}^{N_g}(i-j)^{2}p(i,j), \end{aligned}$$
(6)
$$\begin{aligned} Entropy(Ent):Ent=\sum _{i=0}^{N_g}\sum _{j=0}^{N_g}p(i,j)log_{2}[p(i,j)]^{2}, \end{aligned}$$
(7)

Shape feature extraction. Because the shape of lesion areas is various for different crop diseases, it plays an important role on the lesion area recognition. We extract the shape features based on the spot boundary. Firstly, we get the binary spot image. Next, we mark every region after its boundary coordinates are found in the binary image. Finally, according to the region shape, seven parameters such as circularity, discrete index [16], inscribed circle radius, the radius ratio between the inscribed circle and circumscribed circle, rectangle, elongation and eccentricity are taken for the shape feature. Several parameters are given as follows:

$$\begin{aligned} Circularity(c):c=\frac{4\pi A}{L^{2}}, \end{aligned}$$
(8)

where A is the area of the spot image [17] and L is the perimeter. Obviously, the range of c is 0–1, so when \(c=1\), the spot shape is circularity.

$$\begin{aligned} Discrete\ Index(d):d=\frac{L^{2}}{A}, \end{aligned}$$
(9)
$$\begin{aligned} Inscribed\ Circle\ Radius(r):r=\frac{2A}{L}, \end{aligned}$$
(10)
$$\begin{aligned} Rectangle(R):R=\frac{A}{A_{R}}, \end{aligned}$$
(11)

where \(A_{R}\) is the area of smallest circumscribed rectangle, the range of R is 0–1 and when \(R=1\), the spot shape is rectangle.

Synthesizing the above features, we extract nineteen parameters as the eigenvector for the crop disease image recognition.

Fig. 2.
figure 2

The overview of disease image recognition system

Full size image

3 Disease Image Recognition Based on Transfer Learning

A frame for disease image recognition is shown in Fig. 2.

The purpose of transfer learning is to use the auxiliary training data under a different distribution from the less target training data to help to establish a reliable classification model [18]. Furthermore, if the distributions of two data sets are similar, the transfer learning based on instance works better [19]. So we introduce the instance-based transfer learning in the paper and identify cucumber and rice diseases with the TrAdaBoost-based method(TrBM).

Algorithm 1 is a holistic description of TrBM. In each iteration, on the one hand, TrBM uses Adaboost to adjust the weight of the target training data. If the sample is wrongly predicted, Adaboost will increase the weight of this sample through multiplying its weight with \(\beta _{t}^{-|h_{t}(x_{i})-c(x_{i})|}\). On the other hand, in order to reduce the effect of the auxiliary training data which is most dissimilar to the target training data, a mechanism [19] is added to decrease its weight by multiplying its weight with \(\beta ^{|h_{t}(x_{i})-c(x_{i})|}\). At the same time, if the weight is less than a fixed value that we set, it will be removed. After several iterations, the target training data which is wrongly predicted and the auxiliary training data that is similar to the target training data will have larger training weights. So they will help to train a better classifier.

4 Experimental Results and Discussion

4.1 Data Sets Description

We conduct the experiments on five kinds of crop diseases, that is, cucumber target leaf spot (We shortly write as c.t.l.s.), downy mildew (c.d.m), bacterial angular leaf spot (c.b.a.l.s), rice brown spot (r.b.s) and rice blast (r.b). In addition, all the algorithms mentioned in this paper are implemented in Matlab2015a and Visual Studio 2013. We take four sets of data in Table 1 to fit transfer learning scenario after repeated attempts.

figure a
Table 1. Data sets description in experiment
Full size table

4.2 Experimental Results and Analysis

It is necessary to normalize four data sets before the experiments. Subsequently, we compare TrBM which uses SVM as the basic learner with other four baseline methods. The descriptions of these five methods are shown in Table 2. Thereinto, a linear kernel is applied in all SVMs and the nearest number of training samples is set to 7 in all KNNs. During the experiments, a target training set \(D_{b}\) and a test set \(D_{t}\) are under the same distribution. Table 3 presents the experimental results of five methods when the ratio between the target training data and the auxiliary training data is 0.04 (Since the number of experimental data is scarce, in order to ensure reliability in experiment, the smallest ratio is set to 0.04). The performance in accuracy rate is the average of 10 repeats by random choice of the target training data and the iteration number of weight adjustment is set to 100.

Table 2. The descriptions of five methods
Full size table
Table 3. Accuracy rates when the ratio is 0.04
Full size table

From Table 3, we can see that the accuracy rates achieved by TrBM are absolutely higher than four other methods. The experimental results on four data sets are presented in Fig. 3 (Because the experimental results with SVM-T and KNN-T are much worse than other three methods, we only show SVM, KNN and TrBM results in Figure). Here, the ratio between the target and the auxiliary training data is gradually increased from 0.04 to 0.2.

In Fig. 3(a) and (d), when the ratio is less than 0.08, the performance of TrBM is better than other methods. However, as the ratio increases, the performance of TrBM is not as good as SVM, but is still better than KNN. As we know, the advantage of SVM is that it can get a better classification model when the number of the training data is not large.

Fig. 3.
figure 3

The experimental results on four data sets

Full size image

In Fig. 3(b) and (c), the performance of TrBM always exceeds the SVM and KNN when the ratio is within 0.2. Meanwhile, as the ratio increases, the results of SVM gradually approach TrBM. We believe that the auxiliary training data contain not only good knowledge, but also noisy data. When there are enough target training data to learn a good classifier, the noisy part of the auxiliary training data affect the learner.

5 Conclusions

(1) The paper proposed the instance-based transfer learning for the crop disease image recognition. After the auxiliary, target and test image pre-processing, we obtained a nineteen dimensional eigenvector for each spot image. Then, we transferred the useful auxiliary data to target data with TrBM. In short, we increased the weight of the target training data that was wrongly predicted, decrease the weight of the auxiliary training data which was most dissimilar to the target training data and remove the weight that is less than the lower limit of weight in each iteration. Finally, we implemented comparison experiments by TrBM and four other methods on cucumber and rice diseases.

(2) The experimental results reveal that transfer learning is beneficial for the crop disease image recognition when the training sample is not enough. Especially, TrBM can implement samples transfer between two domains which are under the different distribution. In the future, we will extend transfer learning to other crop diseases image recognition.