Keywords

1 Introduction

Technology advancement must be utilized to provide farmers with rapid soil analysis in making farming more sustainable. Modern sensing technologies in agriculture have been given an important role for the improvement of agricultural productions, e.g., crop yield, animal production, aquaculture generation, and sustainable farming, to keep up food security. It has been realized that up-to-date agricultural innovation can economically enhance agricultural production. As agriculture is a major contributor to the increasing environmental damage, change toward a more sustainable way of farming is needed. Up-to-date information on key soil properties is imperative for sustainable agricultural and environmental management. Conventional soil sampling and laboratory analyses cannot efficiently give this information since they are slow and expensive and sometimes incorporate environmentally damaging chemicals. Its high costs have hindered the introduction of precision agriculture in several parts of the world [1]. The cost for conventional soil characterization is about US$2,500 per pedon and requires 6–12 months to be done [2,3,4]. So, the traditional soil analysis appears to be irrelevant to many users and does not possess “a market with land administrators and policy makers” [5]. Thus, an accelerated and environmentally friendly method is expected to analyze soils rapidly. One of the cost-effective methods in soil analysis is the reflectance spectroscopy. A growing demand for high-resolution spatial soil information for precision farming leads us to an exploration question: Can spectroscopy replace the outdated methods of soil analyses?

Soil reflectance spectroscopy is a well-known technique to assess soil properties quickly and quantitatively [6]. Reflectance spectroscopy is relatively less costly and quicker than traditional wet chemical measurements [7]. Visible and near-infrared and shortwave-infrared (VNIR-SWIR) reflectance spectroscopy is a promising tool for the productive identification and monitoring of soil properties in the spectral range 350–2,500 nm, “because of the occurrence of robust spectral features inferable to soil constituents in this region” [8,9,10]. However, the relevant information for soil characteristic prediction should be mathematically extracted from the spectra so it can be linked to soil attributes. Chemometrics technique and multivariate statistical methods [11, 12] must be utilized to examine the relationships between reflectance spectra and soil properties. Partial least squares regression (PLSR) is viewed foremost method, which has been developed to relate the soil spectrum to soil attributes. The PLSR performance is usually high and clarifies more than 80% of the soil variability [13, 14]. PLSR has been frequently successful than other models [15,16,17] as it deals with serious collinearity, it is suitable for the sample number less than the variable number, and it considers both the independent and dependent variable. However, PLSR is not able to remove the noise band or the irrelevant variables. Thus, useless information may influence the model to uncover a more reasonable relationship. Therefore, it is useful to utilize a PLSR-based band selection and transformation procedure [6].

The choice of the spectral pre-processing method [18] was observed to be crucial for the performance of multivariate calibration. Albeit numerous transformation methods have been proposed, the decision of which pre-processing to utilize is minimally known. Since the right method can improve the predictive capability of the models [19, 20], this research aims to advance the use of VNIR-SWIR spectroscopy for soil property prediction of Bahr El Baqar region, Egypt. The present study is geared toward building up a rapid spectroscopic method for soil property determination. The main objective is to test the capability of PLSR for predicting soil organic matter (OM), clay content, and salinity using VNIR-SWIR spectroscopy. To achieve the main objective, the following specific questions are examined in this study:

  1. 1.

    Is there variation within the spectra of soil samples that allow differentiation between different soil properties?

  2. 2.

    What impact do different transformations have on the soil sample signature?

  3. 3.

    Can PLSR be used to model the key soil properties from soil spectra and reference chemical measurements in the area, where ground data is not accessible?

2 VNIR-SWIR Spectra of Soil Properties

Many soils and agricultural applications do not always be precisely quantified, but only require a soil classification with respect to a critical test value for key soil properties (e.g., clay content, salinity, and OM). Shepherd and Walsh [21] utilized the laboratory VNIR analysis for the soil discrimination, falling above or below specific cutoff values for most key soil properties, which influence soil fertility and productivity. VNIR-SWIR spectroscopy could be a promising technique for the prediction of soil (physical, chemical, and biological) properties [19, 22,23,24]. The results of the VNIR-SWIR prediction models of soil properties are summarized in Table 1. Different modeling techniques, as well as various pre-processing methods, were employed in the development of these models.

Table 1 Summary result of soil property prediction via VNIR-SWIR spectroscopy
Full size table

Regarding soil chemical properties, various authors have reported accurate VNIR-SWIR spectroscopy predictions of soil total C and N [40] and pH [21]. This is reliable, considering that numerous bonds between C and O, N or H absorb light in the near-infrared (NIR) region, while pH prediction has been attributed to O–H groups [41]. Good predictions have also been achieved for K [21] and for soil P and N [21, 41,42,43]. Contradictory predictions have been reported for electrical conductivity [41], organic carbon [44, 45], and salinity [1, 13, 14, 35,36,37,38]. The OM has distinctive spectra, particularly in the NIR due to the formation of covalent bonds with a variety of molecules [46]. Some studies have indicated that the prediction of soil OM was better in the visible range at 410, 570, and 660 nm [24, 47] where a few scientists had announced a substantial relationship between OM in soil and spectral reflectance in the range 545–830 nm [48].

Reflectance spectroscopy has been applied to predict soil physical properties, such as texture and clay content [11, 49]. VNIR has yielded good results for soil particle size distribution [50], soil moisture [51], water holding capacity [52], infiltration of crusted soils [53], maximum temperatures reached by burned soils [52, 54], and total clay content [7, 55]. The spectral properties of clays are most dominant in the NIR area where unique absorption bands can be utilized to give quantifiable data on clay mineralogy [56]. Figure 1 displays the main characteristics of spectral signatures and corresponding soil attributes and also “shows certain main spectral regions for some soil characteristics” [6].

Fig. 1
figure 1

Reflectance spectra of soil sample indicating the spectral features of the most important constitutes

Full size image

3 Prediction of Soil Properties Using Spectroscopy Technique

3.1 Soil Chemical and Spectral Analysis

The study area, Bahr El Baqar region, is located in northern Egypt between 31° 50′–32° 20′ longitude and 30° 40′–31° 10′ latitude (Fig. 2). Soils, which are neighboring to El Manzala Lake, are defined as heavy saline-alkali low-lying clay, which is lacustrine deposits. The land surface is level, slightly sloping to the north, and ranges in elevation from below sea level to 4 m (a.s.l) in the highest point [57]. A reconnaissance field trip was achieved in the study area to be familiar with different landscape features: land-use and land cover patterns. The field surveys were directed with a global positioning system (GPS) receiver.

Fig. 2
figure 2

Footprints and Landsat 8 image for selected sample locations in the study area

Full size image

Soils were categorized by their main physicochemical characteristics. A total of 35 soil samples were collected from Bahr El Baqar, Egypt (Fig. 2). Soil samples were collected randomly from selected agricultural fields from the upper horizon, which was air-dried and sieved. Each soil sample was equally divided into two parts, one for chemical analysis by a typical method and the other part for spectral measurements. Soil reflectance spectra are collected using a portable spectroradiometer (FieldSpec Pro, Analytical Spectral Devices), which measures reflectance over a range from 350 to 2,500 nm with a resolution of approximately 10 nm and a sampling interval of 3 nm [6]. For each sample, two spectral measurements were averaged. Outliers were examined on all these data, and one sample was removed. Thirty-four samples remained for soil property modeling and analysis.

One examined soil physical property (clay content) and two soil chemical properties (salinity and organic matter) were quantified. “Particle size distribution was measured by the pipette method” [58]. The electrical conductivity (EC) of soils was determined using 1:2 soil to solution (H2O) ratio. Soil pH was determined in deionized water (pHw) and in 0.01 M CaCl2 (pH CaCl2) (in 1:2.5 suspensions). Organic carbon (OC) was analyzed by the wet digestion [59].

3.2 Steps Used to Predict Soil Properties

The application of VNIR-SWIR spectroscopy for soil property prediction does not involve any (hazardous) chemicals. It is a nondestructive technique well suited for analyses of some of the essential constituents of the soil [60]. The sample only takes a few seconds, and several soil properties can be estimated from a single scan. Moreover, it allows for flexible measurement configurations and in situ as well as laboratory-based measurements. Thus, for the purpose of this research, Fig. 3 proposes the methodology used.

Fig. 3
figure 3

The overall steps used in this research to predict key soil properties using spectroscopy and chemometrics technique

Full size image

3.3 Pre-processing Techniques and Transformations

Different pre-processing techniques (e.g., averaging, smoothing, standardization, and normalization) were utilized to expel any inappropriate information, which cannot be controlled correctly by the modeling procedures. Spectral pre-processing algorithms were tested to reduce noise or enhance the spectra or both.

Three factors gave the impetus for data reduction and averaging. First, the spectra were all smoothly changing, with no unexpected transitions such as sharp peaks or valleys. It was anticipated that no substantial spectral information would be lost using an averaging procedure. Second, correlation data for the complete set of wavelengths indicated high correlations between reflectance values at widely contrasting wavelengths. These correlations were greatly distinguished between nearly adjacent wavelengths. Thus, it was also expected that substantial predictive power would not be lost because of averaging. Finally, computational demands for choice methods increase exponentially with the number of initial independent variables. Selecting a subset from 2,151 variables would be extremely time intensive. These reflections lead to a conclusion to utilize averaged spectral data. Spectral averaging comprised of a 10 nm boxcar strategy at wavelength intervals of 10 nm from 350 to 2,500 nm and was achieved using spreadsheet software. Thus, the new spectrum comprised of reflectance values at 350, 360, 370,… 2,500 nm, inclusive. This technique leads to a new dataset, with reflectance values quantified for 214 wavelengths for all samples, that was approximately 1/10th the original data volume while maintaining sufficient spectral information [6].

A multiplicative scatter correction (MSC) is a standout among the most generally utilized normalization procedures. In MSC, the light scattering is assessed for each sample with respect to an ideal sample acquired by averaging the complete wavelength range of the dataset. Each spectrum is then corrected such that all samples seem to have a similar scatter level as the reference spectrum [61]. Spectroscopic measurements were achieved in the transmission mode, which can be computed to absorbance using Beer’s Law [62]. The measured reflectance (R) was converted to apparent absorbance as log10(1/R). The Kubelka-Munk transformation is similar to absorbance, but accounts for scattering [62].

Continuum removal (CR) was figured using a convex hull [63], which is fitted to the spectrum, and the spectrum is then divided at each wavelength by the hull. The CR value can be obtained by dividing the corresponding reflectance value of the reflectance level of the convex hull at the corresponding wavelength, with the constraint that its maximum value could not be greater than 1. With the CR technique, the reflectance spectra are standardized permitting correlation between absorption features, for example, depth, depth position, the area of the absorption, etc. These features can be figured from a continuum-removed spectrum to improve absorption features in the recorded soil reflectance spectra. The CR will limit any brightness differences and emphasize the spectral absorption bands.

Derivatives were often used to remove baseline shifts and superposed peaks. The 1st derivative of spectra removes additive baseline (“offset”) effects that could be ascribed to the roughness of soil samples. Second derivative spectra can correct for both additive and multiplicative effects (like MSC). They are generally computed by Savitzky-Golay algorithm [64], which yielded the best transformations in pre-processing [65]. A second-order polynomial was utilized. Utilizing derivatives of the spectra may enable pertinent information to be separated from the near-infrared range [66]. The parameters of the calculation (interval width, polynomial order) should be carefully selected to avoid enlargement of spectral noise [62].

3.4 Development of Calibration Models for Soil Property Prediction

Modeling refers to relate a set of spectral parameters that are derived from the spectral information (after the pre-processing treatment) to the real soil properties. Various algorithms can be utilized to calibrate soil VNIR-SWIR spectra to predict soil properties. They incorporate multiple linear regression (MLR), principal component regression (PCR), and partial least squares regression (PLSR) as well as data mining techniques like artificial neural networks (ANN), multivariate adaptive regression splines (MARS), and boosted regression trees [12]. The linear ones are more straightforward and most commonly used. In the meantime, the use of data mining is increasing, particularly for vast, diverse datasets where data mining is indicated to perform slightly better than linear analyses [12].

The spectral measurements were compared with their equivalent chemical measurements. Then multivariate models were constructed and tested to predict each soil property. PLSR is a technique widely used in chemometrics (e.g., [6, 34, 38]) when there are many predictor variables, highly collinear. The main advantage of multivariate analysis using PLSR calibration is its ability in the fast determination of the components in mixtures, especially with signal overlapping. PLSR is based on the latent variable decomposition of two sets of variables: the predictor X, which is the spectral reflectance bands (independent data), and the responses Y that are the soil property (dependent) data. PLSR selects orthogonal factors that maximize the covariance between the independent (X) and the dependent variables (Y), where the limited number of PLSR factors selected explains most of the variation in both predictors and responses. The PLSR analysis was carried out using Unscrambler 10.4 software [67].

The PLSR analysis was applied to the raw reflectance and to the different transformation methods. PLSR analysis between one attribute (clay content, salinity, or OM) and the spectral data (average spectra in the range from 350 to 2,500 nm) was conducted using spectroscopy. Most of the variation can be captured within the first few latent variables/factors, while the remaining latent variables describe random noise or linear dependencies between the wavelengths/predictors. This process amplifies the “peakedness,” which is considered advantageous for noisy and scattered data. The soil analysis data determined by conventional chemical analysis techniques were added to the resultant spectral files using the WinISI II (Ver. 1.04) (FOSS NIRSystem/TECATOR, 1999). Matlab (version 8.1.347) and PLS-Toolbox 4.2 were used to fit partial least squares regression (PLSR) calibration with leave-one-out cross-validation.

3.5 Prediction Accuracy

The samples were randomly divided into two sets to develop the statistical model of each soil property: a training set (25 samples = 75% of the total database) for developing the prediction model and a validation set (9 samples = 25% of the total database) to test the accuracy of the model. As few as 25 samples can result in better predictions at the farm or field scale [6, 56]. One hundred to two hundred samples might be at the lower limit for a model planned to cover a large area with various soil types [6, 60]. The choice of the ideal number of PLSR components is a key step to getting a calibration model with good predictive power, which is done via cross-validation of calibration samples. The cross-validation was performed using the leave-one-out method [68]. To quantify the goodness of prediction models, the coefficient of determination (R 2) and root mean square error (RMSE) was used. The aptitude of the reflectance procedure to predict a soil property was estimated by the lowest standard error of prediction (SEP), the lowest root mean square error of prediction (RMSEP), and the highest coefficient of correlation.

4 Rapid Soil Analysis Using Spectroscopy

4.1 Physicochemical Properties of the Bahr El Baqar Soils

Table 2 shows a summary descriptive statistics of representative physicochemical properties of the Bahr El Baqar soils. The mean, standard deviation, and range for each soil property are presented in Table 2. The pH of the Bahr El Baqar soil solution maintained at the alkaline condition with average 8.00. It could be attributed to the presence of calcareous parent material. The relatively high content of OM (average 1.6%) is mainly related to the high OM flux to the soil due to direct discharge of domestic and industrial wastewaters.

Table 2 Summary descriptive statistics of representative physicochemical properties of the Bahr El Baqar soils
Full size table

The mean value of sand, silt, and clay in the soil is 43.00, 30.38, and 26.61%, respectively. The range of the minimum and maximum values is 93.12, 55.85, and 76.76%, respectively, which is large. There are great variations and high skewness for clay (1.01). CV percentage values reflect the mean variation of each sampling site in the population. The order of the CV % for each element from high to low was sand, clay, and silt. To determine the relationship among key soil parameters, Pearson correlation coefficients were calculated (Table 3). The results revealed that the OM is good correlated with clay and moderately correlated with silt (r = 0.630 and 0.565, respectively). Furthermore, there is a good correlation between salinity and silt (r = 0.548).

Table 3 Pearson (parametric) rank order correlations
Full size table

4.2 Rapid Spectral-Based Models to Predict Key Soil Properties

4.2.1 Overall Soil Spectral Analyses

Figure 4 illustrates the spectra of all soil samples measured in the laboratory. In each region of the wavelength domain, soil spectra plotted to display the typical soil reflectance shape. Reflectance is generally lower in the visible range (350–650 nm) and higher in the near infrared with specific absorbance bands around 1,400, 1,900, and 2,200 nm.

Fig. 4
figure 4

(a) Reflectance curves obtained from the measured reflectance of soil samples, (b) soils with different clay content, (c) soils with different organic matter, and (d) soils with different salinity levels

Full size image

4.2.2 Calibration Models for Key Soil Properties

Table 4 summarizes the cross-validation results of the PLSR models developed with raw and various pre-processing transformed VNIR-SWIR spectra against each soil property. For raw spectra, PLSR models produced poor prediction accuracy with R 2 of 0.22, 0.38, and 0.46 and RPD of 1.73, 0.93, and 0.94 for soil clay, salinity, and OM, respectively. However, with appropriate spectral pre-processing algorithms, model performance was improved to a certain degree. For example, PLSR model developed for clay after CR transformed spectra resulted in R 2 of 0.52 and RPD of 3.75. By using the second derivative pre-processing technique, prediction of soil salinity was improved with R 2 of 0.62 and RPD of 3.44. The best calibration model for OM (R 2 of 0.44 and RPD of 3.75) was obtained when the spectra were transformed by MSC followed by a de-trending technique. It is worth noting that these optimized PLSR models need less latent variables (3–5) than those for raw spectra. In general, the fewer the latent variables used, the better is the model developed. For clay content, PLSR calibration with raw and various transformed spectra failed to produce useful models.

Table 4 VNIR-SWIR cross-validation of PLSR for predicting key soil properties
Full size table

4.2.3 Modeling Transformed Spectra (TS) of Soil Clay Content

The absorption feature of clay content in the VNIR-SWIR region can be better analyzed using CR transformed spectra (TScr) (Fig. 5 and Table 4). The dominant absorption features near 1,400 nm (TScr1400) and 1,900 nm (Scr1900) should be related to the presence of both hydration and crystallization water. Moreover, since TScr1400 should also be related to the –OH vibration of water in silicates, it could be used to indirectly explain the percentage of the sand fraction that is constituted by these minerals. The absorption peak at 2,200 nm should be ascribed to the –OH-Al bending modes, as observed in various clay soils. Therefore, TScr2200 can be directly used to explain clay fraction of spectral data. The CR values at 1,400 (TScr1400) and 2,200 nm (TScr2200) extracted from continuum-removed spectral dataset were linearly correlated with measured clay and sand fractions (%) in the prediction of soil texture:

Fig. 5
figure 5

Continuum-removed spectra and reflectance curves. (a) Soils with different clay content and measured vs. predicted soil texture using the CR approach (b)

Full size image
$$ \mathrm{Clay}\ \left(\%\right)={a}_1+{b}_1\ {\mathrm{TS}}_{\mathrm{cr}2200}=28.15+\left(-0.092\times {\mathrm{TS}}_{\mathrm{cr}2200}\right)\ \left({R}^2=0.57\right) $$
(1)
$$ \mathrm{Sand}\ \left(\%\right)={a}_2+{b}_2\ {\mathrm{TS}}_{\mathrm{cr}1400}=37.91+0.289\times {\mathrm{TS}}_{\mathrm{cr}1400}\ \left({R}^2=0.55\right) $$
(2)

where a and b are coefficients obtained from the linear regression analysis. Since no specific absorption feature can be directly related to the silt fraction, the latter was calculated subtracting the sum of the other two from 100. The difference and the corresponding estimates were compared to the measured values.

Figure 5 shows the continuum-removed spectra and reflectance curves (Fig. 5a) of soils with different clay content. There are differences in shapes of spectral curves, slopes, overall reflectance intensity, and position of absorption bands. The performance of the CR approach to predict soil texture is summarized in (Fig. 5b) for validation of the datasets. As can be observed, the CR approach allows obtaining a moderate-level performance only for clay prediction (R 2 = 0.57), characterized by RMSE values of 26.51 and 24.50 for the calibration and validation dataset, respectively.

4.2.4 Modeling Transformed Spectra (TS) of Soil Salinity

Soil salinity prediction in the VNIR-SWIR region can be better analyzed using second derivatives transformed spectra (TScr) (Fig. 6 and Table 4). The 34 soil sample spectra were divided into five salinity classes. In each class, an average spectrum was calculated to represent each soil salinity class (Fig. 6a,b). The plot shows that the reflectance curves display two deep absorption regions at 1,415 and 1,915 nm and several weak absorption regions near 494, 673, 1,748, 2,207, and 2,385 nm. The absorption region depth varies with the level of soil salinity. These features suggest that with increasing salinization, the soil moisture content increases.

Fig. 6
figure 6

Both the average spectra (a) and the CR (b) represented each soil salinity classes and measured vs. predicted soil salinity using the CR approach (c)

Full size image

Figure 6 demonstrates that the overall reflectance decreases between 589 and 803 nm as samples become more salinized due to an increase in the evaporate mineral content within the soil. The reflectance curves show a shoulder at around 803 nm and are similar to one another with nine distinct absorption features at 487, 671, 905, 1,144, 1,416–1,447, 1,800, 1,911–1,945, 2,203, and 2,345 nm. However, the depth of the absorption feature varies with the degree of soil salinization. As discussed in the literature, the absorptions at 671, 905, 1,144, and 1,800 nm were due to hydroxyl ions and water within the lattice of various hydrated evaporate minerals. Compared with wavelengths above, broader absorption features are found around 1,144 and 1,800 nm. The increases in absorption intensity at 671, 905, 1,144, 1,416–1,447, and 1,911–1,945 nm become more pronounced as salinity level increases.

The deeper water absorption features at around 1,416–1,447 and 1,911–1,945 nm are probably the combined results of O–H stretches and H–O–H bending fundamentals and overtones, and the features become more extended and asymmetrical as salt contents increase. The position of the two absorption features at 1,416 and 1,911 nm moved to long wavelengths from 1,416 to 1,447 nm and from 1,911 to 1,945 nm with increasing salinization. Since, in this area, the salt composition was mainly highly hygroscopic salts like MgCl2 salts, hygroscopic salts in air-dried soil samples could absorb water vapor from the environment. This leads to an increase in soil moisture. These results are consistent with Farifteh et al. [13].

As shown in Fig. 6, the low and nonsaline soils produced weak absorption features at 671, 905, and 1,144 nm. This phenomenon is reversed at around 1,800, 2,203, and 2,345 nm as spectral absorptions increase as soil salt contents decrease at these longer wavelength positions. The less salinized soil samples possess a well-defined, narrow, and deep hydroxyl feature at 2,203 nm. This phenomenon suggests the presence of well-crystallized clay minerals in soils [14]. The absorption intensity at 2,203 nm was reduced in the more salinized soils, which may be a result of the loss of crystallinity in the clay minerals. The absorption feature at 2,345 nm was weak, but distinct in asymmetrical form as the soil salt content decreases:

$$ \mathrm{Salinity}\ \left({\mathrm{dSm}}^{-1}\right)=72.66+135.05\times {\mathrm{TS}}_{\mathrm{CR}\ 1431/1928}\ \left({R}^2=0.62\right) $$
(3)

The performance of the CR approach to predict soil salinity is outlined in Fig. 6c for validation of the datasets. As can be observed, the CR approach allows obtaining a high-level performance for salinity prediction (R 2 = 0.62), characterized by RMSE values of 4.60 and 4.81 for the calibration and validation dataset, respectively. The ratio index (for the CR at 1,431 and 1,928 nm) compared with the difference and normalized indices might be the optimal index for soil salinity prediction because the bands at 1,431 and 1,928 nm are found in the NIR reflectance region, where the reflectance is primarily affected by the water content. The NIR band appears to be a good indicator of soil salinity. This result is in agreement with the findings of Farifteh et al. [13], Weng et al. [14], and Mashimbye et al. [20].

4.2.5 Modeling Spectra of Soil Organic Matter

The direct relation between reflectance values and OM content was investigated for each wavelength individually. The general reflectance pattern was investigated visually (Fig. 7) for the presence of absorption features in the reflectance from 1,600–1,800 nm (related to lignin) and 2,000–2,300 nm (related to cellulose, starch, and lignin).

Fig. 7
figure 7

Spectral signature in the range 1,600–1,800 nm and 2,000–2,300 nm part of the spectrum of three soils with a large range of organic matter content (a) and measured vs. predicted soil organic matter using the CR approach (b)

Full size image

Based on the reflectance pattern for wavelengths presented in Table 4, the highest correlation (R 2 = 0.44) between OM and reflectance is found using MSC followed by de-trending (Table 4), which is low. Also, highest correlation (R 2 = 0.75) between OM and reflectance is found in the visible range (640 and 690 nm), with a maximum around 600 nm and around the water absorption features (1,400 and 1,900 nm), and in the SWIR region (2,212 nm). This spectral region corresponds with the results reported by Bartholomeus et al. [32].

To select the most sensitive spectral bands to OM, the correlation coefficients between soil spectral reflectance and OM contents were calculated. The correlation was higher in the range of 500–700 nm. Thus, the OM content was estimated using a selection of spectra in the range of 500–700 nm.

The deviation of the arch (DOA)-based regression of the spectra at 600 nm is defined as the difference between the average reflectance value at 550 and 650 nm and the reflectance at 600 nm of each spectral curve [69]. The DOA reflects the concave or convex feature of the spectral curve in these bands. The variations in the DOA with OM are shown in Fig. 8. As the OM content increased, the DOA decreased; i.e., the soil spectral curve between 550 and 650 nm flattened [69]. Because the soil spectral curve between 550 and 650 nm was flattened, the estimation of OM contents was performed using a linear model.

Fig. 8
figure 8

(a) Coefficient of correlation between organic matter and soil spectral reflectance and (b) three distinct spectral curves at 500–700 nm showing the changes in the deviation of the arch (DOA) at 600 nm with soil organic matter (OM)

Full size image

The best calibration and validation results were obtained using the linear model, which yielded the highest R 2 (0.75) and the smallest RMSE (0.53):

$$ \mathrm{Soil}\ \mathrm{organic}\ \mathrm{matter}\ \left(\%\right)=a\times {\mathrm{TS}}_{\mathrm{DOA}600}+b=1.344+0.0197\times {\mathrm{TS}}_{\mathrm{DOA}600}\ \left({R}^2=0.53\right) $$
(4)

where TSDOA600 is the deviation of the arch of the spectra at 600 nm and a and b are the model coefficients.

5 Overall Discussion

Sensing technology must be utilized to provide farmers with low-cost and rapid soil analysis in making farming more sustainable. Modern sensing technologies in agriculture have been given an important role for the improvement of agricultural productions, e.g., sustainable agriculture, to maintain food security. As agriculture is a major contributor to the increasing environmental destruction, change toward a more sustainable way of agriculture is urgent. Up-to-date information on soil properties is imperative for sustainable agricultural and environmental management. One of the cost-effective methods in soil analysis is the reflectance spectroscopy.

The use of VNIR-SWIR spectroscopy permitted the analysis of spectral properties of soils, which related to chemical, physical, or biological soil properties that might not have been revealed by conventional analyses and which had an influence on the parameter to be predicted. The choice of soil variables that were the focus of this study was partly guided by the goal of obtaining an integrative picture of key soil properties. It is worth noting that the visible range (350–760 nm) associated with soil color shows the huge influence on model accuracy, which is in line with other studies [11, 24]. The absorption feature in the range 350–1,000 nm might be due to the Fe oxides in the soil, mainly hematite and goethite [12]. Influential wavelengths located between 1,000 and 2,500 nm can be attributed to water, clay minerals, and organic matter [12].

Correlation analysis between the reflectance and OM is critical for deciding the soil spectral bands mostly affected by OM. The spectral reflectance displayed the highest correlation with the OM content in the range of 500–700 nm. Zhang et al. [69] compared the correlations between the reflectance and OM of various soils. The results indicated that the reflectance in the range of 500–700 nm could be utilized to determine OM in many types of soils. Peng et al. [70] confirmed that the reflectance in the range of 570–630 nm is affected by OM, and the maximum correlation coefficient between them appears at 600 nm. Therefore, the deviation of the arch (DOA) of the spectra at 600 nm contains the majority of information regarding OM and can be used to estimate OM. As the OM content increased, the DOA gradually decreased. The intercept of the linear model of the DOA was 19.76, indicating that the DOA would be under 0 when the OM content was higher than 19.76 g kg−1; i.e., the reflectance would decrease significantly because of the strong absorption by OM. Therefore, spectral features of other soil properties would be weakened due to the significantly decreased reflectance [70]. When OM content exceeded 19.76 g kg−1, the OM played a dominant role in producing soil spectral features. Whereas when the OM content was less than 19.76 g kg−1, it became less effective in masking out the spectral features of other soil properties.

6 Conclusions and Foresight for Decision Making

Concern about agricultural sustainability requires the development of modern technologies that do not have adverse effects on the environment, are effective for soil analyses, and lead to improvements in food productivity. This chapter has demonstrated that reflectance spectra contain much information and variation related to key soil parameters. Utilizing laboratory VNIR-SWIR spectroscopy good predictions can be achieved for some chemical and physical properties. The method allows the assessment of primary properties with direct spectral responses, which are directly affected by combinations and overtones of fundamental vibrations for organic functional groups and water, particle size, and surface properties of organic matter (OM), salinity, and clay minerals to have a direct spectral response in the VNIR-SWIR spectroscopy. In this context, it is possible to detect clay content, salinity, and OM with high accuracy. The pre-processing technique has been improved the prediction accuracy of a wide range of key soil properties. In the case of the clay content, the prediction capacity of the method proved to be high (R 2 = 0.57) using CR. The CR spectra yielded the best calibration models with respect to estimates of the soil salinity, which generated R 2 values of 0.62. The visible band is important for OM estimation using spectroscopy. The DOA at 600 nm is indicative of the convex and concave features of the spectral curves generated by OM and can be used for estimating OM. When the OM content exceeded 19.76 g kg−1, the decreased reflectance due to the strong absorption by OM could mask the spectral features of other soil properties to a certain degree. VNIR-SWIR spectroscopy is a promising method for the fast prediction of key soil properties. Laboratory VNIR spectroscopy offers a low-cost solution for soil parameters monitoring, which could allow an increase in their spatial coverage and an increase in their sampling frequency. However, a critical research need in Egypt is the advancement of soil spectral libraries that will enhance the predictive ability of VNIR spectroscopy for soil quality attributes whatever the soil type.

The road ahead calls for using a point or image thermal spectrometer, which will enable obtaining more information that is spectral with better accuracy [71,72,73]. Decision makers are increasingly demanding fast, low-cost, easy-to-use, and nondestructive methods for monitoring changes in the physical and chemical characteristics of soil, water, and plants from the early stages of crop development until harvest. Therefore, the development of tools that farmers can use to evaluate efficiency at the farm level will open a new era for agricultural sustainability. Remote and proximal sensor tools have been used to monitor different aspects of agricultural production (e.g., fertilization, crop diseases, etc.). Most of these tools are characterized as nondestructive and noninvasive, and most of them are based on near-infrared (NIR) spectroscopy. Advances in NIR spectroscopy and chemometrics provide another possibility to traditional methods of conducting soil, water, and plant analyses [71,72,73].

Adapting and using modern technologies is a promising way to efficiently and reliably improve management farming practices, as well as to move toward the application of best management practices in the agricultural processes. Different studies have shown the important role of proximal sensors based on NIR spectroscopy in the analysis of crops. The use of NIR spectroscopy on agricultural harvesters has shown the potential to reduce the labor and expenditure required for the determination of relevant properties in different crops. Moreover, the availability of hyperspectral cameras has provided exciting new possibilities for online defect detection, which were not achievable with the use of sensors based only on the visible range of the electromagnetic spectrum (e.g., detection of defects in fruits). In addition, spectral imaging technologies, which acquire single or multiple images at selected wavelengths, might be used to detect specific quality attributes (pigments, sugar, starch, water, protein, etc.) in a wide range of crops and horticultural products. Finally, one of the potential advantages of using NIR spectroscopy for in-field crop monitoring is the analysis of fresh plant materials (e.g., leaf, whole plant) without the need for drying, grinding, or sending the sample to the lab. Various spectroscopic and imaging techniques have been evaluated for the detection of symptomatic and asymptomatic plant diseases.