Abstract
Pine wilt disease caused by the pinewood nematode Bursaphelenchus xylophilus has led to the death of a large number of pine trees in China. This destructive disease has the characteristics of bring wide-spread, fast onset, and long incubation time. Most importantly, in China, the fatality rate in pines is as high as 100%. The key to reducing this mortality is how to quickly find the infected trees. We proposed a method of automatically identifying infected trees by a convolution neural network and bounding box tool. This method rapidly locates the infected area by classifying and recognizing remote sensing images obtained by high resolution earth observation Satellite. The recognition accuracy of the test data set was 99.4%, and the remote sensing image combined with convolution neural network algorithm can identify and determine the distribution of the infected trees. It can provide strong technical support for the prevention and control of pine wilt disease.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Pine wilt disease (PWD) caused by pine wood nematode (Bursaphelenchus xylophilus) (Yang 2010) is a global, destructive threat to forest ecosystems. PWD spreads quickly. After being infected by the B. xylophilus nematode, a healthy pine tree dies within 30 d (Fig. 1). After an outbreak of infection, dead trees are fumigated, and felled and burnt to limit the spread of the disease, otherwise the entire pine forest could be infected within 3–5 years (Ye 2019), (Fig. 2). PWD first appeared in North America and was accidentally introduced into China in 1982, and is now is widely distributed in the country. Currently, 18 provinces in China (cities and autonomous regions) have PWD over an area of 1.12 million ha. Considering its destructive potential to forest resources (Zhao et al. 2020) and the environment (Zhan 2014), PWD has been included in the internal and external forest plant quarantine objectives in China.
For PWD, the faster infected trees are found, the less forestry resources will be lost (Qin et al. 2021). Therefore, it is obligatory to develop a feasible and effective approach to detect pine trees in the early stages of infection. Compared with the ground survey conducted by forest rangers, the detection method using satellite images has the advantages of short time, high efficiency and wide detection range. This low-cost and low-risk technology is beneficial for forestry departments to monitor forest diseases in real time and formulate strategies to control the speed and area of disease transmission.
Before introducing the new method, we refer to the research of other scholars on PWD, B. xylophilus and related pest diseases. Yin et al. (2015) used support vector machine (SVM) to detect tomato early blight by hyperspectral remote sensing data. The training accuracy was 84.6%, and the prediction accuracy got 80.7%. Kong et al. (2020) UAV hyperspectral platform to obtain rice panicle neck blast canopy data of different disease grades. The random forest method was used to identify japonica rice panicle neck blast in alpine areas at an accuracy of 90.0%. Li et al. (2020) analyzed the degree of infected trees through an image segmentation algorithm with a combination of an ultra-green feature factor, maximum inter-class variance, and a disaster grade analysis method of remote sensing panorama. The data were collected from the infected area by UAV. The recognition accuracy was 90.4%. Tao et al. (2020) proposed a threshold division method based on hue-saturation-brightness (HSV). This method improved the efficiency of manual investigation of infected pine trees. It provided a method support for using UAV images to monitor infected trees. With the development of artificial neural networks such as VGG (Xu et al. 2021), ResNet (Wu et al. 2019), Faster-R-CNN (Wan and Sotirios 2020), and U-Net (Falk et al. 2019), other networks were proposed. These algorithms have also been used in agricultural and forestry image recognition. Zhang et al. (2020) obtained a visible light image of an infected pine forest in a large area using aerial photography by drones. His team segmented the infected pine on the basis of these images and the U-Net deep learning segmentation network. The training and verification accuracy obtained by the experiment were 98.7 and 97.8%, respectively. Fang et al. (2021) used Faster R-CNN framework to identify pine with different disease degrees, and the correct rate was 83.2%. It realizes the efficient and accurate discrimination of diseased pine in remote sensing images of a forest area, providing a reliable auxiliary means for the prevention and control of the disease. Li et al. (2020) used Faster R-CNN target detection algorithm to automatically identify infected pine trees. After improving the original algorithm, the overall accuracy of infected tree recognition was improved from 75.6 to 82.4%.
A deep learning method was used to automatically detect dead pine trees infected with pine wilt disease in high-resolution satellite remote sensing images. The research used unrelated data sets that contained a large number of dead and healthy trees to train the model. The method can significantly reduce manual work and improve recognition accuracy.
Materials and methods
Requirements and solutions of satellite remote sensing data
Whether human or convolution neural networks, in order to accurately distinguish the objectives in the images, the images need to have specific resolution. The approach proposed requires images with sub-meter resolution and a short monitoring window period. First, the study assumes that the crown is round. Considering that the actual diameter may be 2 –5 m, it is difficult to distinguish infected trees from satellite images with a resolution more than one meter. In addition, PWD occurs briefly and spreads quickly. Therefore, it is necessary to ensure the timeliness of satellite images. Fortunately, the successful launch of the Beijing 2 (BJ-2) satellite developed by China solved both these problems.
BJ-2 is a high-resolution commercial remote sensing satellite, which consists of three optical remote sensing satellites. It operates in a solar synchronous orbit with an altitude of 647 km and an inclination of 97.8°. The three satellites are located in the same orbital plane, and the revisit period to any position of the earth is 1 day. The BJ-2 satellite uses the SSTL-300 S1 satellite platform, and the yaw maneuver range is up to ± 45°. The quality of the satellite is 447 kg, and the design life is 7 years. The satellite carries a high-resolution imager, VHRI-100, with a panchromatic resolution of 1 m, a multispectral resolution of 4 m and a width of 23 km. The satellite has the ability of stereoscopic imaging along or crossing the orbit. The daily image acquisition area of a single satellite is 1,00,000 km2. In the long stripe imaging mode, the stripe imaging distance is up to 4000 km. This lays a foundation for satellite remote sensing image monitoring of pine wood nematode disease. At present, BJ-2 satellite has been widely used in many research fields (Fan et al. 2021; Zhang et al. 2018).
General situation of the research area
Datian County as the study area of this experiment is located in central of Fujian Province, on the west side of the Daiyunshan vein, between 25°29' N–26°10' N and 117°29' E–118°03' E (Chen 2014). Datian County is one of the 12 priority ecological areas in Fujian Province, China, with a forest cover of 70.1%, approximately 2294 km2. The natural entity of Datian County is “Nine mountains, half water and half field”. As the source of the Minjiang, Jiulong and Jinjiang rivers, the land is fertile with abundant rain and rich in forest resources. However, in recent years, the proliferation of PWD has led to the death of pine trees over large areas.
Data acquisition and marking
The onset of PWD is an important starting point for selecting satellite images. In Datian County, the high incidence period of pine wilt disease occurs from mid-July to late October. In addition, degree of cloud content of satellite images determines the amount of useful information. This study assumes that the cloud content was inversely proportional to useful information. According to the above criteria, BJ-2 satellite images collected on September 7 and October 10, 2020 were selected as the experimental data (Fig. 3).
Data preprocessing
The original data from BJ-2 was stored in a raster file (.pix) format with 4 channels. The original image data cannot be directly used as the input of the classification model. Therefore, it is necessary to carry out a preprocessing operation. The specific pretreatment steps are divided into the following steps, and the flow chart is shown in Fig. 4.
-
Original image clipping
Cut the 4-channel 16-bit raster file (image size: 38,947 × 37,263 × 4) into 3-channel 16-bit tagged image file (.tiff), named \(Img 16\) (image size: 1024 × 1024 × 3).
-
Sample making
Clip \(Img 16\) to picture \(Img Input\) of size 31 × 31 × 3. The marking process of positive and negative samples is described above. 4540 images of uninfected trees and 6000 images of infected trees were taken from all \(Img Input\) as training data.
-
Image mapping
In view of \(Img Input\) can only be processed by GIS (Geographic Information system) software such as ArcMap (Gorshkov and Novikova 2018), therefore, it was transformed to 8 bits image named \(Img 8\) by linear mapping to make the calculation results easier to view. The mapping function is as follows:
$$Img_{8} = \frac{Img\,Input}{{Img_{max} }} \times 255$$(1)where \({Img}_{max}\) is the maximum value of RGB in 3 channels. Figures 5 and 6 are processed samples of positive and negative classes, respectively.
Improved LeNet neural network structure
In 1994, Lecun et al. (1998) proposed the LeNet neural network, which was the earliest convolution neural network and named “LeNet-5” “Gradient-Based Learning Applied to Document Recognition” (5 meaning having 5 layers) (Fig. 7). It was first used to solve the visual task of handwritten digit recognition. Since then, the most basic architecture of convolution neural network (CNN) has been established: convolution layer, down sampling layer, and full connection layer. The purpose of this experiment is to detect infected individual trees, so the output results of the model should include healthy and infected individual trees. This is different from the task of handwritten digit recognition. Therefore, it is necessary to modify the original model. Specifically, a full connection layer with two outputs is added to the original model. In addition, the RELU activation function (Yarotsky 2017) is used to replace the activation function of the original model to obtain better training effect (Fig. 8).
The parameters of each layer are as follows:
-
Input layer
The function of the input layer is to receive the information of the picture. The number of neurons is equal to the size of the picture 31 × 31 × 3.
-
Convolution layer
The function of the convolution layer is to convolute the input image. With the size of 3 × 3 × 3, 32 channels’ convolution kernels, characteristic images of 29 × 29 × 32 can be obtained. The number of neurons in the convolution layer is 29 × 29 × 32, i.e., 896 parameters. The output of the convolution layer goes through the RELU activation function as input to the subsequent model. The activation function can improve the nonlinear fitting ability of the model so that the model can approach any nonlinear function.
-
Convolution layer
Using the convolution operation to further attend to the feature image, the feature map can be obtained with the size of 27 × 27 × 32. The operation uses 32 convolution cores of size 3 × 3 × 32. Therefore, there are a total of 9248 parameters. Similarly, the ReLU activation function is used to activate the output.
-
Maximum pooling layer
The function of the pooling layer is to under sample the feature image. The window size of the maximum pooling layer is 2 × 2. Therefore, an output of size 13 × 13 × 32 can be obtained.
-
Flattening layer
The flattening layer (Shin et al. 2016) is used to “flatten” the input, i.e., to convert the multi-dimensional input into one-dimensional output. It is used in the transition from convolution layer to fully connected layer. In this model, the flattening layer flattens the 13 × 13 × 32 feature image into 5408 one-dimensional vectors.
-
Fully connected layer
The fully connected layer consists of 100 neurons. The output result is used as the input of the classification layer by the RELU function.
-
Output layer-full connection layer
There are two nodes in this layer and the softmax function is used to map the output of the neuron to the interval (0, 1). The closer the output result is to 0, the greater the probability that this region is uninfected. In contrast, the closer the output result is to 1, the greater the probability that this region is infected. This makes the process of target recognition model training.
The function of the output layer is to classify according to the feature vector. It includes two neurons and Softmax function (Chen et al. 2018). Softmax function is used to map the output of the neuron to the interval (0, 1). The closer the output result is to 0, the greater the probability that the individual tree is uninfected. In contrast, it indicates that the individual tree may be infected.
Selection of meta parameters.
The Adam optimizer (Kingma and Ba 2014) was selected in this study, with a 128 batch size and a 0.0001 learning rate for 30 training iterations. The positive and negative samples of the data set are 4540 and 6000, respectively. The data set was divided into training and verification at a proportion of 8:2, with an additional 1521 samples as test negative samples and 1501 samples as test positive samples. The problem in this experiment is two-classification problem, so the accuracy evaluation index chooses “Binary_accuracy”. It evaluates the model by checking whether the index corresponding to the maximum value of the picture label value encoded by one-hot is equal to the index corresponding to the maximum value of each output of the model.
It is an accuracy that evaluates the model by checking whether the index corresponding to the maximum value of the picture label value encoded by one-hot is equal to the index corresponding to the maximum value of each output of the model.
Results
Experimental environment
The software for this experiment was Python3.6.7. The model was built by calling the Kera library (Hung et al. 2020). The training method of the model was GPU. The computer was configured with Intl Xeon E5-2660 v2 2.2 GHz 4-core CPU with 32 GB memory. The graphics card was two NVIDIA P102-100 s, and the memory 20 GB.
Identification of pine wilt disease
Figure 9 shows that the training accuracy and verification accuracy of the model obtained by each iteration in the training process were ˃ 98%. Figure 10 shows the model training loss and verification loss values obtained during each iteration of the training process are controlled within 0.05. With the increase of iterations, the training accuracy of the model gradually improved. At the same time, with the increase in the number of iterations, the loss value decreased gradually, and there was no over-fitting phenomenon.
Figure 11 shows the detection results of an \(Img16\) image. The detection process includes the following steps: first, the picture is clipped into a plurality of \(Img Input\) pictures. It is then used as the input of the model, and the classification results are obtained. If the individual tree in \(Img Input\) was infected, the coordinates of \(Img Input\) in \(Img16\) were recorded. After judging all the \(Img Input\), the coordinates of the infected trees in \(Img16\) were obtained. According to these coordinates, using bounding box (Rajchl et al. 2017) to mark it in \(Img16\), the individual infected tree can be detected.
Discussion
Satellite remote sensing is important technology. This approach is widely used to collect various data of the earth (Ma et al. 2014). The satellite image data obtained have four characteristics: high spatial resolution, high temporal resolution, high spectral resolution and high radiation resolution. After the basic processing of remote sensing image processing platforms, such as information extraction, radiation correction and geometric correction, remote sensing images can provide more comprehensive and accurate information for national defense, national economy and even forest pest monitoring. The emergence of new technologies such as big data and artificial intelligence provides more innovative processing methods for processing remote sensing images. In the field of forestry, satellite data images have the advantages of wide area, speed, simultaneity, and economic feasibility.
At present, the common monitoring methods of pine wilt disease in China are manual examination and UAV remote sensing. Manual examination is time-consuming and laborious, with a small review area. It is difficult to monitor the disease on a large scale. This method is often used in the verification of areas initially diagnosed as infected. The UAV remote sensing method can achieve high precision PWD detection in the designated area it is expensive. Compared with satellite images, the monitoring range is limited. In addition, the number of drone flights per unit time is limited. PWD has a short onset time, and UAV technology often misses the best time to prevent or control the disease. Satellite remote sensing technology is very suitable for PWD detection missions, and can cover most areas of the country, resulting in large-scale surveillance of the pine wood nematode disease. The use of satellite remote sensing technology to locate the incidence area and the accurate establishment of diseased plants by manual examination or UAV remote sensing is an effective method for the prevention and control of the disease.
Conclusion
In this study, the image recognition of damaged pine wood was completed with satellite image-based data and the deep learning method. The results show that the training and verification accuracy reached 98.3 and 95.9%, respectively. Therefore, the detection of PWD on this model is feasible, and the method of locating the infected trees is achieved.
This approach can rapidly identify pine trees infected by PWD and is suitable for large-scale identification. In addition, this method can continuously track the development of PWD in areas, and provide timely and accurate information for forest managers and forest protectors. At the same time, it also provides an objective basis for the disaster loss assessment of PWD and the formulation of the disease control standards by pine forest management departments.
References
Chen DP (2014) Analysis and research on soil and water conservation of main afforestation tree species in Datian County. Forestry Prospect Des 2:90–95 (in Chinese with English abstract)
Chen LF, Zhou MT, Su WJ, Wu M, She JH, Hirota K (2018) Softmax regression based deep sparse autoencoder network for facial emotion recognition in human-robot interaction. Inf Sci 428:49–61
Falk T, Mai D, Bensch R et al (2019) U-Net: deep learning for cell counting, detection, and morphometry. Nat Methods 16:67–70. https://doi.org/10.1038/s41592-018-0261-2
Fan YL, Lou DB, Zhang CQ, Wei YJ, Jia FD (2021) Research on iron tailings information extraction based on object oriented method -- a case study of remote sensing image of Beijing-2 in Qianxi Area. Remote Sensing for Land and Resources 1–10. Available at: http://kns.cnki.net/kcms/detail/11.2514.P.20210812.1325.002.html (in Chinese with English abstract).
Fang WQ, Li H, Li LL, Chen XY (2021) Recognition of pine wood nematode disease based on deep learning. J Forestry Eng, 1–6. https://doi.org/10.13360/j.issn.2096-1359.202012041 (in Chinese with English abstract)
Gorshkov A, Novikova O (2018) Estimating the validity of the recognition results of earthquake-prone areas using the ArcMap. Acta Geophys 66(4):843–853
Hung J, Goodman A, Ravel D, Lopes SCP, Rangel GW, Nery OA, Malleret B, Nosten F, Lacerda MVG, Ferreira MU, Renia L, Duraisingh MT, Costa FTM, Marti M, Carpenter AE (2020) Keras R-CNN: library for cell detection in biological images using deep neural networks. BMC Bioinformatics. 21(1). https://doi.org/10.1186/s12859-020-03635-x
Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint https://arxiv.org/abs/1412.6980
Kong FC, Liu HJ, Yu ZY, Meng XT, Han Y, Zhang XL, Song SZ, Li H (2020) Identification of japonica rice panicle blast in alpine region by UAV hyperspectral remote sensing. Trans Chin Soc Agricultural Eng 36(22):68–75 (in Chinese with English abstract)
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791
Li H, Xu HH, Zheng HY, Chen XY (2020) Research on pine wood nematode surveillance technology based on unmanned aerial vehicle remote sensing image. J Chin Agric Mechanization 41(9):170–175 (in Chinese with English abstract)
Ma ZW, Hu XF, Huang L, Bi J, Liu Y (2014) Estimating Ground-Level PM2.5 in China using satellite remote sensing. Environ Sci Technol 48(13):7436–7444
Qin J, Wang B, Wu YL, Lu Q, Zhu HC (2021) Identifying pine wood nematode disease using UAV images and deep learning algorithms. Remote Sensing 13(2):14
Rajchl M, Lee MCH, Oktay O, Kamnitsas K, Passerat-Palmbach J, Bai W, Damodaram M, Rutherford MA, Hajnal JV, Kainz B, Rueckert D (2017) DeepCut: object segmentation from bounding box annotations using convolutional neural networks. IEEE Trans Med Imaging 36(2):674–683
Shin HC, Roth HR, Gao MC, Lu L, Xu ZY, Nogues I, Yao JH, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298
Wan SH, Sotirios G (2020) Faster R-CNN for multi-class fruit detection using a robotic vision system. Comput Netw 168:107036. https://doi.org/10.1016/j.comnet.2019.107036
Wu ZF, Shen CH, AntonVD HA (2019) Wider or deeper: revisiting the resnet model for visual recognition. Pattern Recogn 90:119–133
Xu XM, Xie MS, Miao PQ, Qu W, Xiao WP, Zhang HD, Liu XT, Wong T (2021) Perceptual-aware sketch simplification based on integrated VGG Layers. IEEE Trans Visual Comput Graphics 27(1):178–189. https://doi.org/10.1109/TVCG.2019.2930512
Yang X (2010) Detection and identification method and result analysis of pine dead wood in Fujian Province. Wuyi Sci J 26(1):38–46 (in Chinese with English abstract)
Yarotsky D (2017) Error bounds for approximations with deep ReLU networks. Neural Netw 94:103–114
Ye JR (2019) Epidemic status of pine wilt disease in china and its prevention and control techniques and counter measures. Scientia Silvae Sinicae 55(9):1–10. https://doi.org/10.11707/j.1001-7488.20190901(inChinesewithEnglishabstract)
Yin XJ, Ning C, Zhang YC (2015) Hyperspectral remote sensing identification of processing tomato early blight based on GA and SVM. Remote Sensing Information 30(2):94–98 (in Chinese with English abstract)
Zhan YS (2014) Status Quo of Bursaphelenchus xylophilus (Sterner et Buhrer) nickle occurrence in tai’ning and its prevention and control measures. Biological Disaster Sci 37(1):91–94 (in Chinese with English abstract)
Zhang Z, Zeng HB, Yang KJ (2018) Application and evaluation of BJ-2 satellite in remote sensing monitoring of land use change survey. Land Resources Herald 15(04):44–47 (in Chinese with English abstract)
Zhang R, Xia L, Chen L, Xie C, Chen M, Wang W (2020) Recognition of wilt wood caused by pine wilt nematode based on U-Net network and unmanned aerial vehicle images [J]. Trans Chin Soc Agric Eng (Transactions of the CSAE) 36(12):61–68 (in Chinese with English abstract)
Zhao JJ, Huang JX, Yan J, Fang GF (2020) Economic loss of pine wood nematode disease in Mainland China from 1998 to 2017. Forests 11(10):1042. https://doi.org/10.3390/f11101042
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Project funding: The work was supported by the National Science and Technology Major Project of China’s High Resolution Earth Observation System (21-Y30B02-9001-19/22), and the Heilongjiang Provincial Natural Science Foundation of China (YQ2020C018).
The online version is available at http://www.springerlink.com.
Corresponding editor: Yu Lei.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zhou, H., Yuan, X., Zhou, H. et al. Surveillance of pine wilt disease by high resolution satellite. J. For. Res. 33, 1401–1408 (2022). https://doi.org/10.1007/s11676-021-01423-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11676-021-01423-8