Introduction

Rice is one of the three staple crops in the world with a long history of cultivation and is widely distributed in Asia, Africa and North America. China is the largest rice producer and consumer in the world, ranking first in terms of total production, with more than 60% of Chinese people taking rice products as their staple food (Qiu et al., 2021). Nitrogen is a nutrition element with the highest demand compared to the other 16 elements such as phosphorus, potassium and silicon (Lu et al., 2020) and plays an important role in rice growth, while the leaf nitrogen content in rice is an important indicator to characterize its nitrogen nutrition status (Fu et al., 2020). Proper application of nitrogen can enhance photosynthesis and improve quality. Nitrogen deficiency will cause the leaves to dry up and turn yellow, while excessive nitrogen will cause rice lodging, waste of resources and environmental pollution (Sun et al., 2016). Therefore, timely, quickly and precise monitoring of rice LNC is of great significance for guiding the decision-making of precise fertilization in rice fields.

Traditional approaches for rice nitrogen monitoring usually use portable instruments to measure key growth indicators, such as the GreenSeeker (2.0, Ntech Inc. Oklahoma, USA) or SPAD (502, Minolta Inc. Tokyo, Japan), which are fast and convenient but obtain point information from the crop. When in practical application, the crop information for the whole field is usually replaced by point information, which is difficult to apply on large-scale fields. On the other hand, UAV remote sensing has developed rapidly in rice nitrogen monitoring due to its flexibility, timely data acquisition and high spatial resolution of imaging. (Wang et al., 2021) used optimized artificial neural network (ANN) and support vector machine (SVM) algorithms to build a model for monitoring rice nitrogen nutrition from UAV hyperspectral imagery. Although the use of UAV digital imagery with high spatial resolution alone (Ge et al., 2021) or UAV multispectral imagery with high spectral resolution (Colorado et al., 2020) combined with vegetation indices to monitor rice nitrogen has been proven to be effective, most of these studies have used single sensor data and did not complement the advantages of UAV remote sensing platforms and multiple sensors. Although (Zheng et al., 2018) simultaneously used UAV digital, multispectral and color infrared cameras and evaluated the ability of these sensors to monitor nitrogen in rice leaves, it was only a mutual comparison and analysis of the monitoring effects based on different single sensor data, without considering the application of UAV multi-sensor and multi-modal fusion data in crop nitrogen nutrition monitoring.

Image fusion is an image processing technique based on resampling the same features from different types of images, resulting in an image that absorbs the characteristics of the image before fusion. The Gram-Schmidt pan sharpening (GS) fusion method (Mahour et al., 2015), on the other hand, is not limited by the wavelength of the band, improves the disadvantages of traditional methods such as Principal Component Analysis, where the information is too concentrated, and is specifically designed for high spatial resolution images. A single RGB image contains high spatial resolution but lacks sufficient spectral information, while a single multispectral image contains high spectral resolution but lacks spatial texture information. Therefore, the fusion image obtained based on the RGB image and multispectral image has both high spatial resolution and high spectral resolution, which well retains the spatial texture information and spectral feature information of the image, and can fully utilize the complementary advantages of multi-type sensors of the UAV remote sensing platform.

The feature variables used to characterize the data greatly influence the performance of the inversion model. Among them, the individual spectral bands (bands) and various vegetation indices (VIs) made up of their linear or non-linear combinations allow a comprehensive characterization of the physiological and biochemical status of the crop. It has been shown that the sensitive feature wavelength of vegetation nitrogen is located in the short-wave infrared region, but this wavelength range is susceptible to moisture absorption and is therefore easily masked for fresh crops containing high levels of water (Khaled et al., 2020; Shicheng et al., 2021). Also, nitrogen is an important component of chlorophyll and there is a close relationship between them, so the chlorophyll correlation VIs have been chosen for the analysis of nitrogen in this paper (Feng et al., 2008; Hansen & Schjoerring, 2003). Texture features form gray level co-occurrence matrices (GLCMs) are visual features that reflect homogeneous phenomena in an image, they reflect the properties of the surface structure of an object with slowly or periodically changing organization. In this paper, five common GLCMs were selected to complement the VIs. The mean (ME) reflects how the image texture varies, the variance (VA) reflects the degree of deviation of each pixel point, the entropy (EN) reflects the distribution of the image texture, the second moment (SE) matrix reflects the coeval matrix variability, which gives the degree of image texture distribution, and the dissimilarity (DI) is a representation of the difference in the grey values of the image elements. Recent research has shown that the combination of different types of features has the potential to improve crop nutrient estimation in agricultural applications over using the functionality of individual sensors (Maimaitijiang et al., 2017). It was reported that (Li et al., 2015) significantly improved the ability of the model to estimate crop biomass by combining canopy structural features with spectral features, while (Bendig et al., 2015; Jakob et al., 2014) made progress in grain yield estimation. In addition, combining texture information with spectral features, for example, has been shown to perform well in predicting soybean yields (Maimaitijiang et al., 2019). However, less is known on the contribution of combined canopy spectral bands, vegetation indices and texture feature information to rice nitrogen prediction in the context of UAV image fusion and precision agriculture.

In addition, related studies usually use conventional correlation analysis to select spectral feature variables to construct remote sensing models for monitoring crop status. However, with spectral feature variables selected by conventional correlation analysis methods, it is often difficult to eliminate the redundancy and collinearity between variables, thus reducing the performance of the prediction model (Shicheng et al., 2021). In recent years, the optimal spectral feature algorithms such as principal component analysis (PCA), successive projection algorithm (SPA), and minimum redundancy maximum correlation (mRMR) have been widely used in ground-crop monitoring or satellite remote sensing research. These algorithms can reduce the risk of overfitting and improve the accuracy, stability and generalization of the model (Khaled et al., 2020; Samsudin et al., 2015). The Pearson correlation algorithm only performs correlation analysis on the selected feature variables to measure the closeness between two variables, and cannot find and eliminate invalid feature variables, which can be theoretically expressed as the quotient of the covariance and standard deviation between two variables. The mRMR algorithm, on the other hand, uses the mutual information obtained by the calculus of probability densities as a benchmark, and combines the relationship between feature subsets and classes to obtain the combination of feature variables with the least redundancy and the greatest correlation, which is more accurate and efficient than Pearson algorithm.

Similarly, most of the related research are based on traditional regression algorithms like Partial Least Squares (PLSR) and Support Vector Machines (SVM) to estimate target parameters, but these algorithms often lack regularity (Wu et al., 2020). As Lasso Regression (LR) and Gaussian Process Regression (GPR) perform well in fitting problems, (Piepho, 2009) used LR to build a model for the height index in crop canopy based on lidar and achieved an R2 of 0.81, (Ogutu et al., 2012) used GPR to optimize the PLSR algorithm for estimating the chlorophyll content of japonica rice in Northeast, which improved the accuracy of the prediction model. However, there are few reports on the remote sensing monitoring for LNC in rice canopy based on the information fusion from UAV multiple sensors using the LR and GPR methods, combined with the optimal spectral feature algorithms.

The focus of this research is to explore the potential of combining multiple types of feature variables with optimal feature methods and machine learning algorithms in developing the models for monitoring rice LNC, based on UAV image fusion techniques. The objectives of this study were: (i) to explore the potential of combinations of different feature variables in estimating rice LNC; (ii) to explore the performance of GS image fusion methods for estimating rice LNC; (iii) to explore the improvement of the optimal feature algorithms such as mRMR over the traditional Pearson methods and its potential to be combined with machine learning techniques such as LR and GPR for rice LNC estimation.

Materials and methods

Study area and experimental design

The study area is located in the high-quality agricultural products development and demonstration center in Ninghe District, Tianjin, China (39° 26′ 34″ N, 117° 33′ 13″ E), bordering the North China Plain to the west and Bohai Bay to the southeast, and belonging to a typical warm temperate monsoonal continental climate zone. It is a traditional rice growing area with an average annual temperature of 11.1 °C and an average annual sunshine duration of 2801.7 h. The geographical location of the study area and the UAV sampling area are shown in Fig. 1. The sampling area consisted of 12 plots, each plot was 82 m × 56 m, the variety was Jinyuan 89, and the ratio of nitrogen, phosphorus and potassium was 23–13-6 type mixed fertilizer for double-repeated and six-gradient fertilization treatment. The nitrogen fertilizer application rates were 600 kg/ha (N1), 540 kg/ha (N2), 480 kg/ha (N3), 420 kg/ha (N4), 360 kg/ha (N5) and 300 kg/ha (N6), and the water treatment was rainfed.

Fig. 1
figure 1

Overview of the study area

Full size image

Ground data acquisition and processing

In this study, representative plants in a uniformly growing area of each experimental plot were selected as samples at three key growth stages in 2021, namely, the jointing stage (5 July), the booting stage (30 July) and the filling stage (27 August), and placed in sealed paper bags and brought back to the laboratory. In the laboratory, the stems and leaves of the rice samples were first separated, killed at 105 °C for 0.5 h, and then all the leaf samples were dried at 80 °C for more than 48 h until the mass balance. Afterwards, all dry leaf samples were weighed and ground, and the LNC was determined using the Kjeldahl method.

As shown in Table 1, the LNC were significantly different at corresponding growth stages and nitrogen application conditions. In three growth stages, the LNC content increased with the increase of nitrogen application. The descriptive statistics of the rice LNC show that the coefficient of variation is between 4.36 and 5.43%, far less than 15%, meaning low variability, which also makes the use of UAV remote sensing data for rice LNC estimation possible.

Table 1 Descriptive statistics for the rice LNC across different growth stages
Full size table

UAV data acquisition and processing

The flight platform uses a quadrotor UAV (P4M, DJI Inc. Shenzhen, China) includes a take-off weight of 1.487 kg, a maximum flight altitude of 6000 m, and a maximum horizontal flight speed of 58 km/h, as shown in Fig. 2. The built-in image sensor of P4M includes one color sensor for visible light imaging and five monochrome sensors for multispectral imaging. The effective pixels of each sensor are 2.08 million. The waveband information of the multispectral camera is shown in Table 2. The UAV imagery was carried out under stable conditions at the jointing stage (July 5), booting stage (July 30), and filling stage (August 27) respectively. Before each experiment, a calibrated reflective whiteboard was set up to obtain accurate reflectance data. The flight altitude was set to 50 m, the heading overlap rate was 80%, and the lateral overlap rate was 70%.

Fig. 2
figure 2

UAV system: DJI P4M UAV, Image sensor and remote controller

Full size image
Table 2 Band parameters of the multispectral sensor for P4M
Full size table

After acquiring the image data, DJI Terra was used to stitch the imagery, and then Envi was used to perform the preprocessing such as geometric correction and radiometric calibration (Yang et al., 2007), as shown in Fig. 3. Specifically, geometric correction can eliminate distortion due to the geometric deformation and reduce the impact of errors. The UAV digital images were geometrically corrected by selecting the GNSS coordinates obtained from the actual measurements on the ground, and the selected control points were evenly distributed within the study area. The multispectral images were aligned with the corresponding calibrated digital images, taking into account the five wavebands they contain. Moreover, objects have specific absorption and reflection effects on sunlight, and the radiometric calibration is a process to establish the relationship between sensor response and spectral reflectance of a feature. The value of the pixel point in the UAV image is called Digital Number (DN), and by using the radiometric calibration plate and standard reflectance information that comes with the sensor, the multispectral image DN value and spectral reflectance can be linked. Before the experiment, the calibration plate was photographed, and the average DN value of the calibration plate in different wavebands captured by the multispectral camera was calculated, and then the target reflectance could be obtained by combining the target DN value and the calibration plate reflectance, thus completing the radiometric calibration of the UAV multispectral image.

Fig. 3
figure 3

Process flow of UAV image

Full size image

Since the UAV fusion images are from both digital and multispectral images at the same time, the geometric deformations and radiometric distortions that occur in the digital and multispectral images can significantly affect the fusion images obtained through fusion transfer. Moreover, the GS fusion method requires that the two images (digital and multispectral) overlap at the pixel level, so an accurate geometric correction is essential. The method of obtaining the target reflectance by radiometric calibration, on the other hand, is identical for both the original multispectral image and the fusion image.

UAV multi-sensor image fusion

Image fusion is an image processing technology that resamples digital images with high spatial resolution and multispectral images with high spectral resolution to generate a fusion image with both high spatial resolution and high spectral resolution (Li et al., 2021), which can fully utilize the complementarity advantages of the multi-type sensors of the UAV remote sensing platform.

The Gram-Schmidt pan sharpening (GS) fusion method selected in this study is not limited by the band, but can preserve the spatial texture information well, especially the spectral feature information. It also improves the defect of traditional methods like PCA which have too much concentrated information, and is especially designed for the latest high spatial resolution images (Sarp., 2014). Using the GS method in the image processing program ENVI to perform the pixel-level fusion of the preprocessed multispectral and digital images, it can take the advantages of high spatial resolution of digital images and high spectral resolution of multispectral images, as shown in Fig. 4.

Fig. 4
figure 4

Image fusion of GS

Full size image

Processing of spectral features

Selection of spectral feature variables

The feature variables used to characterize the data are critical to the application of machine learning and greatly influence the inverse performance of the model. A large number of researches have shown that morphological or physiological changes will cause changes in crop spectral information, while spectral feature variables are linear or non-linear combinations of various spectral bands to comprehensively characterize the physiological and biochemical conditions of the crop. In order to represent the data features to the greatest extent, based on the previous literature, this study preliminarily selected five band feature parameters, ten typical spectral VIs and five texture feature parameters as candidate spectral feature variables for estimating LNC.

Among them, the spectral VIs mainly includes typical nitrogen-related indices, soil-regulated indices and more chlorophyll-related indices, as shown in Table 3. It has been shown that the sensitive feature wavelength of vegetation nitrogen is located in the short-wave infrared region, but this wavelength range is susceptible to the spectral features of moisture absorption, while for fresh crops that grow naturally, they usually contain more moisture and the spectral information is easy to be masked (Elvidge, 1990; Fourty et al., 1996). Nitrogen is an important component of chlorophyll and there is a close relationship between nitrogen and chlorophyll (Feng et al., 2008; Perry et al., 2012). Therefore, in this paper, the chlorophyll correlation indices were chosen for the analysis of nitrogen. The texture feature parameters were extracted using grey level coevolution matrix (GLCM) (Chris et al., 2008), as shown in Table 3.

Table 3 Spectral feature variables used in this study
Full size table

Optimization of spectral feature variables

Because the feature variables often have certain correlation and collinearity, which results in data redundancy and increases the computational burden, this study re-optimized the primary feature variables to obtain the optimal spectral feature variables reflecting LNC in rice. Compared with the traditional Pearson correlation analysis and successive projection algorithm (SPA) method, this study used the minimum redundancy maximum relevance (mRMR) method. The mRMR is a feature variable selection method using mutual information as a benchmark, and its basic idea is to find a subset S of features that has the maximum relevance between the optimal features and the target variable, while minimizing the redundancy between the features (Mundra & Rajapakse, 2010).The advantage of mRMR is that it can select the combination of variables with the least redundant information and the highest correlation. The brief steps are as follows:

Denote the random variables as x and y, and their corresponding probability density functions are \(p\left( x \right)\)、\(p\left( y \right)\), \(p\left( {x,y} \right)\) is the joint probability density function of x and y.

The mutual information between variables is denoted as \(I\left( {x,y} \right)\):

$$\begin{array}{*{20}c} {I\left( {x,y} \right) = \iint {p\left( {x,y} \right)}\lg \frac{{p\left( {x,y} \right)}}{p\left( x \right)p\left( y \right)}dxdy} \\ \end{array} .$$
(1)

The correlation between the feature subset S and the response variable z is denoted as \(D\left( {S,z} \right)\), which is defined by the average of all mutual information values between each feature variable x and the response variable z.

$$\begin{array}{*{20}c} {D\left( {S,z} \right) = \frac{1}{\left| S \right|}\mathop \sum \limits_{x \in S} I\left( {x,z} \right)} \\ \end{array} .$$
(2)

The redundancy of all features in the feature subset S is denoted as \(R\left( S \right)\), which is defined by the average of all mutual information values between the feature variables x and y.

$$\begin{array}{*{20}c} {R\left( S \right) = \frac{1}{{\left| S \right|^{2} }}\mathop \sum \limits_{x,y \in S} I\left( {x,y} \right)} \\ \end{array} .$$
(3)

Combining formulas (2) and (3), the final mRMR-based objective function is:

$$\begin{array}{*{20}c} {\max \left( {D - R} \right)} \\ \end{array}$$
(4)

Modeling methods

Based on the Jupyter Notebook platform (Lösch & Schmidhalter, 2023), two algorithms, Lasso Regression and Gaussian Process Regression, were used to construct the remote sensing monitoring models for rice LNC before and after GS fusion method, and the results were analyzed and compared.

Lasso regression

Lasso regression (LR) was first proposed by Tibshirani (1993). Its characteristic is to compress some regression coefficients with small absolute values to zero by adding penalty terms, thus allowing simultaneous variable selection by ignoring certain feature variables, this property is called "L1 regularization" (Shafiee et al., 2021). The LR-CV model used in this study has a generalized cross-validation, which can adaptively adjust the hyper-parameter alpha to obtain the optimal model. Its core equations are as follows:

Denote the dependent variable as \(y = \left( {y_{1} , \cdots ,y_{n} } \right)^{T}\), the independent variable as \(X = \left( {X_{1j} , \cdots ,X_{nj} } \right)^{T}\), \(j = 1, \cdots p\), \(\beta = \left( {\beta_{1} , \cdots ,\beta_{n} } \right)^{T}\) is the vector coefficient, the underlying linear model is:

$$\begin{array}{*{20}c} {y = \beta X + \varepsilon } \\ \end{array}$$
(5)

Denote the variable selection and parameter estimation of LR as \(\hat{\beta }\left( {LR} \right)\), where λ is the regularization parameter:

$$\begin{array}{*{20}c} {\hat{\beta }\left( {LR} \right) = \arg \min \mathop \sum \limits_{i = 1}^{n} \left( {y_{i} - \mathop \sum \limits_{j = 1}^{p} \beta_{j} X_{ij} } \right)^{2} + \lambda \mathop \sum \limits_{j = 1}^{p} \left| {\beta_{j} } \right|} \\ \end{array}$$
(6)

Gaussian process regression

The Gaussian process regression (GPR) algorithm is a non-parametric regression algorithm that learns relationships between variables by fitting probabilistic Bayesian models, and is suitable for dealing with regression problems with small samples, high latitudes and non-linear complex relationships (Quinonero-Candela & Rasmussen, 2005). The core equations are as follows:

Given n sets of learning samples \(X = \left\{ {X_{1} ,X_{2} \cdots ,X_{n} } \right\}\), \(y = \left\{ {y_{1} ,y_{2} \cdots ,y_{n} } \right\}\), \(\varepsilon\) represents Gaussian white noise with mean value of zero and variance \(\sigma_{n}^{2}\), the underlying linear model is:

$$\begin{array}{*{20}c} {y = f\left( X \right) + \varepsilon , \varepsilon \sim N\left( {0,\sigma_{n}^{2} } \right)} \\ \end{array}$$
(7)

Taking the prior of the function space as a Gaussian process, denoted as GP, \(f\left( X \right)\) represents a Gaussian process with mean value of zero, where \(k\left( {X,X^{\prime}} \right)\) is the kernel function,

$$\begin{array}{*{20}c} {f\left( X \right)\sim GP\left[ {0,k\left( {X,X^{\prime}} \right)} \right]} \\ \end{array}$$
(8)

Evaluation indicators

The LR and GPR-based remote sensing monitoring model for rice LNC used in this study was constructed using Python on the Jupyter Notebook IDE. Compared with the traditional Hold-out method for data division, this study adapted K-Fold cross validation method to divide the overall data into K parts on average according to the characteristics of the data. One subset of data was taken as the test set, and the remaining K − 1 subsets of data were used as the training set, which was repeated K times and the results were weighted averaged to reduce the chance of training results and improve the utilization of the data (Wiens et al., 2008).

To assess the precision and accuracy of the model, three metrics were selected: coefficient of determination (R2), root mean square error (RMSE) and normalized root mean square error (NRMSE). The larger the model R2 is, the better the model fits, and the smaller the model RMSE and NRMSE are, the more accurate the model is. The calculation equations are as follows:

$$\begin{array}{*{20}c} {R^{2} = 1 - \frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i} - y_{i} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i} - \overline{x}} \right)^{2} }}} \\ \end{array}$$
(9)
$$\begin{array}{*{20}c} {RMSE = \sqrt {\frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {y_{i} - x_{i} } \right)^{2} }}{n}} } \\ \end{array}$$
(10)
$$\begin{array}{*{20}c} {NRMSE = \frac{RMSE}{{\overline{X}}}} \\ \end{array}$$
(11)

where \(x_{i}\) denotes the measured value of rice LNC, \(\overline{x}\) denotes the mean of the measured value, \(y_{i}\) denotes the predicted value of rice LNC, and n denotes the number of samples in the model.

Results

Correlation analysis of spectral feature variables

Based on the UAV image after radiation correction, the GNSS coordinates of the ground survey points were combined with a representative area of uniform growth. 24 plots were selected at each growth stage, each with an area of 900 pixels at 30 × 30. The size was expressed as the canopy area of one single rice plant from the UAV imagery. Then the reflectance of five bands were extracted including BLUE, GREEN, RED, REG and NIR in the pixel points, and the average value was taken to construct the spectral feature variable.

UAV multispectral images were extracted as the original images for three growth stages and the GS method was applied to obtain fusion images. The Pearson correlation analysis was calculated between the spectral feature variables constructed from the two images at three stages and the measured LNC data. The results for each growth stage are shown in Fig. 5. It can be seen from the figure that the spectral feature variables extracted from the fusion images have significantly higher relative sensitivity to nitrogen content than those of the original multispectral images, indicating that: the multispectral images fused with the digital image information should have a better potential for rice nitrogen monitoring compared to the original multispectral information.

Fig. 5
figure 5

Correlation coefficient between spectral characteristic variables with LNC based on processed images across different stages

Full size image

Extraction of optimal spectral feature variables

The traditional Pearson correlation analysis can only take into account the correlation between variables, but lacks in removing the redundancy and collinearity between variables. Therefore, this study used the mRMR algorithm for optimal feature variable selection based on the Pearson correlation analysis, setting the number of feature variables to be two to fifteen, and determining the final number of extracted variables by the minimum value of RMSE. The whole process was implemented on Jupyter Notebook via Python.

It can be seen from Fig. 6 that with the increase of the number of feature variables, the RMSE shows a decreasing trend. When there are four input variables, its value is the smallest, and then with increasing value of the variables, the RMSE increases gradually. When the input variable reaches fifteen, it gradually tends to be stable. That is due to the fact that too many variables will increase the mutual collinearity and redundancy, reducing the accuracy of the model. For example, at the jointing stage, the algorithm finally selected four feature variables, the sequences of which were 7,10,12,13 corresponding to SAVI, TCARI, DI and TVI.

Modeling analysis

Prediction results of combining multiple types of feature variables

To investigate the relationship between LNC and band feature parameters (Bands), spectral vegetation indices (VIs) and texture feature parameters (GLCMs), Lasso regression (LR), Gaussian process regression (GPR) and Gaussian process regression combined with the minimum redundancy maximum correlation algorithm (GPR-mRMR) were simultaneously used to construct the LNC remote sensing monitoring model. The results are shown in Table 4.

Table 4 Predictive performance of rice LNC using a combination of multiple types of features
Full size table

Among them, because the LR algorithm has the characteristics of dimensionality reduction, the full corresponding feature variables were used in modeling, and the LR algorithm was selected by itself. The GPR algorithm was based on the Pearson correlation analysis above, and the feature variables with the best correlation were selected for modeling. The GPR-mRMR is based on the mRMR for optimal feature variable selection, combined with GPR for modeling.

As can be seen from Table 4, the accuracy of the models is low when using a single feature as the input variable, with the worst results using single Bands features. The combination of different feature variables shows a large improvement in prediction accuracy for each model, with the highest accuracy for each model when Bands + VIs + GLCMs are used as input variables. The LR is able to improve the R2 by an average of 11%, RMSE by 2.11% and NRMSE by 0.48% for the input of Bands + VIs + GLCMs compared with the two-variable combination models, and the R2 of the GPR can be improved by 11% on average, the RMSE can be reduced by 2.02%, and the NRMSE is reduced by 0.46% when Bands + VIs + GLCMs are input into the model. When Bands + VIs + GLCMs are input to the model, the GPR-mRMR can improve the R2 by 13% on average, reduce the RMSE by 2.42%, and reduce the NRMSE by 0.55%.

As shown in Fig. 7, the prediction accuracy of the models based on GLCMs + VIs is better than the single VIs, which means texture information have obvious advantages in constructing the monitoring model. Among the LNC monitoring models based on multi-sensor imagery with multiple feature combinations, the GPR model is slightly worse, and the accuracy of the LR model is better than that of the GPR model, but the GPR-mRMR model optimized by mRMR has the best effect.

Fig. 6
figure 6

mRMR Variable Extraction process. a Variation of RMSE in the mRMR; b the optimal variables selected using mRMR

Full size image

Prediction results of GS image fusion

The corresponding fusion images were obtained by the GS method for the jointing, booting, and filling stage. Lasso regression (LR) and Gaussian process regression (GPR) were used to construct a monitoring model for rice LNC based on the original multispectral images and fusion images respectively, and the results are shown in Table 5. The GPR algorithm was based on the Pearson correlation analysis above, and the five feature variables with the best correlation in each case were selected, because the LR algorithm has the characteristics of dimensionality reduction, the full corresponding feature variables were used in modeling, and the LR algorithm selected by itself.

Table 5 Predictive performance of Rice LNC using LR and GPR algorithms for both original and fusion images
Full size table

It can be seen from Table 5 that the model accuracy based on the fusion images has been improved at all growth stages. Compared with the model for the original multispectral image, the R2 of the model for the fusion image can be improved by 7% on average, and under the same conditions, the effect of the LR algorithm is better than that of the GPR algorithm. The model predictions for each growth stage are shown in Fig. 8.

Fig. 7
figure 7

Predictive performance of rice LNC with different combinations of input features

Full size image

Prediction results of optimal feature variables

The monitoring model for rice LNC was constructed based on the original multispectral images and fusion images at the jointing, booting and filling stage, while considering the three conditions of LR, GPR-P and GPR-mRMR, and the results are shown in Table 6. Among them, because the LR algorithm has the characteristics of dimensionality reduction, the full corresponding feature variables were used in modeling, and the LR algorithm selected the optimal feature variable by itself. The GPR-P was modeled based on the Pearson correlation analysis above, and the five feature variables with the optimal correlation were selected in combination with GPR. The GPR-mRMR was based on the mRMR algorithm for optimal feature variable selection and combined with GPR modeling.

Table 6 Predictive performance of Rice LNC using LR, GPR-P and GPR-mRMR algorithms for both original and fusion images
Full size table

It can be seen from Table 6 that the accuracy of the model optimized based on the mRMR algorithm has been improved at all growth stages. In particular, the R2 of the GPR model optimized by mRMR is able to improve by 8% on average compared to the traditional Pearson correlation analysis based on the original multispectral images and fusion images. Under the same conditions, the GPR-mRMR model outperformed the LR model, and the prediction results for each growth stage are shown in Fig. 9.

Fig. 8
figure 8

Predictive performance for each stage. (I), (II) and (III) represent the rice jointing, booting and filling stage respectively; a predictive performance of rice LNC using GPR for original multispectral images; b predictive performance of rice LNC using LR for original multispectral images; c predictive performance of rice LNC using GPR for fusion images; d predictive performance of rice LNC using LR for fusion images

Full size image

Construction of spatial distribution map of LNC

By comparing the LNC estimation models of different processing and modeling methods, it was found that the model using GPR regression combined with the mRMR algorithm achieved the best estimation results. Therefore, the model was used to construct the LNC spatial distribution map in rice fields, and the results are shown in Fig. 10.

Fig. 9
figure 9

Predictive performance for each stage. (I), (II) and (III) represent the rice jointing, booting and filling stage respectively; a predictive performance of rice LNC using GPR-P for original multispectral images; b predictive performance of rice LNC using GPR-mRMR for original multispectral images; c predictive performance of rice LNC using GPR-P for fusion images; d predictive performance of rice LNC using GPR-mRMR for fusion images

Full size image
Fig. 10
figure 10

Spatial distribution of LNC in the rice canopy based on the combined GPR-mRMR method

Full size image

It can be seen from Fig. 10 that the estimated value of LNC gradually decreased from the jointing stage to the filling stage. Due to the different nitrogen fertilizer application conditions between different plots, there were differences in rice growth among different plots. N2–N5 plots had small differences in LNC due to little difference in fertilizer application, but were significantly larger than N1 and N6 plots. It can be speculated that nitrogen fertilizer application in the N1 plot was too high, while the N6 plot was too low. At the jointing stage, the LNC was the highest, ranging from 4.04 to 4.65%, at the booting stage the LNC values ranged from 3.42 to 4.03%, and at the filling stage the LNC content was the lowest, ranging from 2.80 to 3.41%, which may be related to the excessive nitrogen absorbed by the growing of spike.

Discussion

In this paper, based on the GS method, the UAV digital and multispectral imagery data were fused and reprocessed, and the LR and GPR algorithms were used to predict the LNC based on the original image and the fusion image respectively. Based on the combination of multiple types of feature variables, combined with the mRMR, the feature variables were screened, and the estimation accuracy of the conventional Pearson correlation analysis was compared and analyzed.

Evaluation of UAV image fusion

In this study, to take full advantage of the UAV remote sensing platform and multiple types of sensors, the GS fusion method was used to synthesize UAV digital and multispectral images to obtain fusion image data with both high spatial resolution and high spectral resolution. It was found that the LNC estimation model of rice based on fusion images outperformed the original multispectral images from the jointing stage to the filling stage, and the accuracy of the model could be improved by 7% on average, with the best model based on the LR for fusion images at the jointing stage (R2 of 0.66, RMSE of 11.96% and NRMSE of 2.76%). This may be related to the fact that the fusion images absorbed features from different sensors and contained rich data, which is consistent with the findings of (Xu et al., 2021). Therefore, the consideration of information fusion methods is important for the remote sensing monitoring of rice LNC based on UAV, and may also provide a reference for research on dryland crops such as wheat and corn.

Evaluation of modeling analysis

In this study, in order to explore the relationship between rice LNC and Bands, VIs and GLCMs, a combination of multiple types of features was used and it was found that the highest accuracy of each model was achieved when Bands + VIs + GLCMs were used as the input variables. On this basis, in order to fully consider the redundancy and collinearity between the feature variables, a selection method for feature variable, mRMR, was chosen based on Pearson correlation analysis. In this study, the optimal variables obtained using Pearson correlation analysis were MCARI, DI, SE, NDVI and MNLI, and the optimal variables obtained using the mRMR algorithm were SAVI, TCARI, DI and TVI. It was found that from the jointing to the filling stage, the accuracy of the models based on the feature variables extracted by the mRMR algorithm was all higher than that of the traditional Pearson correlation analysis method, which indicated that the mRMR algorithm had solved the redundancy and collinearity problems, improving the accuracy and generalization ability of the model, which is consistent with the research of (Mundra & Rajapakse, 2010).

This study is based on two machine learning regression algorithms: LR and GPR, where the LR algorithm has the characteristics of dimensionality reduction, which is advantageous when applied alone, while the GPR algorithm is able to prevent overfitting of the model. After combining the GPR algorithm with the mRMR algorithm, it was found that the fusion image at jointing stage based on the GPR-mRMR algorithm had the best results (R2 of 0.68, RMSE of 11.45% and NRMSE of 2.64%). Therefore, the modeling methods such as GPR-mRMR that combine feature variable selection methods with regression algorithms are important for remote sensing monitoring of rice nitrogen content based on UAV imagery.

In addition, the results of this paper are slightly insufficient at the booting stage and filling stage, and the accuracy of the model decreases slightly, which may be related to the growth and development of the ears. This paper focuses on rice leaves, and will try to remove the effects of the rice ears at the booting and filling stages in the future to further improve the prediction performance of the model.

Conclusions

This study was based on UAV digital and multispectral imagery for estimating rice LNC. The UAV imagery was reprocessed using GS fusion methods. The aim is to explore the potential of combining a feature variable selection method, mRMR, with machine learning techniques such as LR and GPR in the remote sensing and monitoring of rice LNC under a combination of multiple types of features. The following conclusions were drawn:

  1. 1.

    The combination of multiple types of feature variables improved the accuracy of the prediction models, with the highest accuracy of each model when Bands + VIs + GLCMs were used as the input. The GPR-mRMR model was the most effective, being able to improve the R2 by 13% on average, reduce RMSE by 2.42% and NRMSE by 0.55% compared to the models combined two variables.

  2. 2.

    The fusion images obtained by the GS method have both high spatial resolution and high spectral resolution, and the accuracy based on the fusion images was higher than that of the original multispectral images, which can be improved by 7% on average, with the best model based on the LR algorithm for the fusion images at the jointing stage (R2 of 0.66, RMSE of 11.96% and NRMSE of 2.76%).

  3. 3.

    The feature variable optimization method mRMR was able to reduce the redundancy and collinearity between variables based on Pearson correlation analysis, which was better than the traditional Pearson correlation algorithm. After combining with the GPR algorithm, the accuracy of the model had been improved again, and the GPR-mRMR had the best results for the model of fusion images during the jointing stage (R2 of 0.68, RMSE of 11.45% and NRMSE of 2.64%), with an average of 8% improvement in accuracy compared to the conventional GPR-P model using Pearson correlation analysis for variable selection.

Overall, the results of this study showed that the information fusion of multi-sensor imagery from UAV combined with variable optimization methods can be used for accurate estimation of rice LNC, and was important for guiding the decision-making of precise fertilization in rice fields.