Skip to main content

Advertisement

SpringerLink
Log in

Improving the accuracy of soil organic carbon content prediction based on visible and near-infrared spectroscopy and machine learning

  • Original Article
  • Published:
Environmental Earth Sciences Aims and scope Submit manuscript

Abstract

Choosing appropriate multivariate calibration and preprocessing transformation techniques is important in the determination of soil organic carbon (SOC) content based on visible and near-infrared (Vis–NIR) spectroscopy. The performance levels of partial least-squares regression (PLSR), support vector machine regression (SVMR), and wavelet neural network (WNN) calibration methods coupled with different preprocessing approaches were compared using three kinds of criteria, including the coefficient of determination (R2), root mean square error (RMSE), and residual prediction deviation (RPD). A total of 328 soil samples collected from the south bank of Hangzhou Bay were used as the dataset for the calibration–validation procedure and SOC content inversion. The effects of spectra preprocessing transformation methods were evaluated for raw spectra, Savitzky–Golay smoothing with the first derivatives of reflectance (FDR) and Savitzky–Golay smoothing with logarithm of reciprocal of the reflectance (log R−1). The results indicate that the SVMR is superior to the PLSR, and WNN models for SOC content prediction. The combination of the SVMR model with FDR provided the best prediction results for the SOC content, with R2p = 0.92, RPDP = 2.82, RMSEP = 0.36%, and a kappa correlation coefficient of interpolation as high as 0.97. The FDR of Vis–NIR spectroscopy combined with the SVMR model is recommended over the PLSR and WNN modeling techniques for the high-accuracy determination of the SOC content.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abdi D, Tremblay GF, Ziadi N et al (2012) Predicting soil phosphorus-related properties using near-infrared reflectance spectroscopy [J]. Soil Sci Soc Am J 76:2318–2326

    Article  Google Scholar 

  • Annea NJP, Abd-Elrahmana AH, Lewis DB et al (2014) Modeling soil parameters using hyperspectral image reflectance in subtropical coastal wetlands [J]. Int J Appl Earth Obs Geoinf 33:47–56

    Google Scholar 

  • Askari MS, Cui JF, O’Rourke SM et al (2015) Evaluation of soil structural quality using VIS-NIR spectra [J]. Soil Till Res 146:108–117

    Article  Google Scholar 

  • Bartholomeus HM, Schaepman ME, Kooistra L et al (2008) Spectral reflectance based indices for soil organic carbon quantification[J]. Geoderma 145(1–2):28–36

    Article  Google Scholar 

  • Ben-Dor E, Inbar Y, Chen Y (1997) The reflectance spectra of organic matter in the visible near-infrared and short wave infrared region (400–2500) during a controlled decomposition process[J]. Remote Sense Environ 61(1):1–15

    Article  Google Scholar 

  • Brown DJ, Bricklemyer RS, Miller PR (2005) Validation requirement for diffuse reflectance soil characterization models with a case study of VNIR soil C prediction in Montana [J]. Geoderma 129(3–4):215–267

    Google Scholar 

  • Cambou A, Cardinael R, Kouakoua E et al (2016) Prediction of soil organic carbon stock using visible and near infrared reflectance spectroscopy (VNIRS) in the field [J]. Geoderma 261(2):151–159

    Article  Google Scholar 

  • Cisty M, Bajtek Z, Bezak J (2011) Support vector machine based model for water content in soil interpolation [J]. Geophys Res Abstr 13:1–2

    Google Scholar 

  • Clairotte M, Grinand C, Kouakoua E et al (2016) National calibration of soil organic carbon concentration using diffuse infrared reflectance spectroscopy [J]. Geoderma 276:41–52

    Article  Google Scholar 

  • Erzin Y, Rao BH, Singh DN (2008) Artificial neural network models for predicting soil thermal resistivity [J]. Int J Therm Sci 47(10):1347–1358

    Article  Google Scholar 

  • Guo L, Zhang HT, Shi TZ et al (2019) Prediction of soil organic carbon sock by laboratory spectral data and airborne hyperspectral images [J]. Geoderma 337:32–41

    Article  Google Scholar 

  • Hazama K, Kano M (2015) Covariance-based locally weighted partial least squares for high performance adaptive modeling [J]. Chemometr Intell Lab Sys 146:55–62

    Article  Google Scholar 

  • Heinze S, Vohland M, Joergensen RG et al (2013) Usefulness of near-infrared spectroscopy for the prediction of chemical and biological soil properties in different long-term experiments[J]. J Plant Nutr Soil Sci 176(4):520–528

    Article  Google Scholar 

  • Hong YS, Liu YL, Chen YY et al (2019) Application of fractional-order derivative in the quantitative estimation of soil organic matter content through visible and near-infrared spectroscopy [J]. Geoderma 337:758–769

    Article  Google Scholar 

  • Jafarzadeh AA, Pal M, Servati M et al (2016) Comparative analysis of support vactor machine and artificial nenural network models for soil cation exchange capacity predicition [J]. Int J Environ Sci Technol 13(1):87–96

    Article  Google Scholar 

  • Janik LJ, Cozzolino D, Dambergs R et al (2007) The prediction of total anthocyanin concentration in red-grape homogenates using vis–NIR spectroscopy and artificial neural networks. Anal Chim Acta 594(1):107–118

    Article  Google Scholar 

  • Jiang QH, Li QX, Wang XG et al (2017) Estimation of soil organic carbon and total nitrogen in different soil layers using VNIR spectroscopy: effects of spiking on model applicability [J]. Geoderma 293:54–63

    Article  Google Scholar 

  • Kennard RW, Stone LA (1969) Computer aided design of experiments [J]. Technometrics 11(1):137–148

    Article  Google Scholar 

  • Knox NM, Grunwald S (2018) Total soil carbon assessment: linking field, lab, and landscape through VNIR modeling [J]. Landsc Ecol 33:2137–2152

    Article  Google Scholar 

  • Kuang B, Mouazen AM (2011) Calibration of visible and near infrared spectroscopy for soil analysis at the field scale on three European farms[J]. Eur J Soil Sci 62(4):629–636

    Article  Google Scholar 

  • Kuang BY, Tekin Y, Mouazen AM (2015) Comparison between artificial neural network and partial least squares for on-line visible and near infrared spectroscopy measurement of soil organic carbon, pH and clay content [J]. Soil Till Res 146:243–252

    Article  Google Scholar 

  • Kusumo BH, Hedley MJ, Hedley CB et al (2010) Predicting pasture root density from soil spectral reflectance: field measurement [J]. Eur J Soil Sci 61:1–13

    Article  Google Scholar 

  • Kweon G, Maxton C (2013) Soil organic matter sensing with an on-the-go optical sensor [J]. Biosys Eng 115(1):66–81

    Article  Google Scholar 

  • Li Y, DemetriadesShash TH, Kanemasu ET et al (1993) Use of Second derivation canopy reflectance for monitoring prairie vegetation over different soil backgrounds [J]. Remote Sens Environ 44:81–87

    Article  Google Scholar 

  • Li S, Shi Z, Chen S et al (2015a) In situ measurements of organic carbon in soil profiles using vis-NIR spectroscopy on the Qinghai-Tibet plateau [J]. Environ Sci Technol 49:4980–4987

    Article  Google Scholar 

  • Li YL, Pan C, Meng X et al (2015b) Haar wavelet based implementation method of the non–integer order differentiation and its application to signal enhancement [J]. Meas Sci Rev 15(3):101–106

    Article  Google Scholar 

  • Lucà F, Conforti M, Castrignanò A et al (2017) Effect of calibration set size on prediction at local scale of soil carbon by Vis-NIR spectroscopy [J]. Geoderma 288:175–183

    Article  Google Scholar 

  • Malley DF, Williams PC (1997) Use of near-infrared reflectance spectroscopy in prediction of heavy metals in freshwater sediment by their association with organic matter[J]. Environ Sci Technol 31:3461–3467

    Article  Google Scholar 

  • Martens H, Næs T (1989) Multivariate calibration [M]. Wiley, New York

    Google Scholar 

  • Mcdowell ML, Bruland GL, Deenik JL et al (2012)  Soil total carbon analysis in Hawaiian soils with visible, near-infrared and mid-infrared diffuse reflectance spectroscopy [J]. Geoderma 189–190(4):312–320

  • Mouazen AM, Kuang B, Baerdemaeker JD et al (2010) Comparison among principal component, partial least squares and back propagation neural network analyses for accuracy of measurement of selected soil properties with visible and near infrared spectroscopy [J]. Geoderma 158(1–2):23–31

    Article  Google Scholar 

  • Mutanga O, Skidmore AK, Prins HHT (2004) Predicting in situ pasture quality in the Kruger National Park, South Africa, using continuum removed absorption features [J]. Remote Sens Environ 89:393–408

    Article  Google Scholar 

  • Nelson DW, Sommers L (1974) A rapid and accurate procedure for estimation of organic carbon in soils[J]. In: Proceedings of the Indiana academy of sciences, pp 456–462

  • Palacios-Orueta A, Ustin SL (1998) Remote sensing of soil properties in the Santa Monica mountains I. spectral analysis [J]. Remote Sens Environ 65:170–183

    Article  Google Scholar 

  • Peng XT, Shi TZ, Song A et al (2014) Estimating soil organic carbon using Vis/NIR spectroscopy with SVMR and SPA Methods[J]. Remote Sens 6(4):2699–2717

    Article  Google Scholar 

  • Rozenstein O, Paz-Kagan T, Salbach C et al (2015) Comparing the effect of preprocessing transformations on methods of land-use classification derived from spectral soil measurements [J]. IEEE J Sel Top Appl Earth Obs Remote Sens 8:2393–2404

    Article  Google Scholar 

  • Samadianfard S, Asadi E, Jarhan S et al (2018) Wavelet neural networks and gene expression programming models to predict short-term soil temperature at different depths [J]. Soil Till Res 175:37–50

    Article  Google Scholar 

  • Sarkhot DV, Grunwald S, Ge Y et al (2011) Comparison and detection of total and available soil carbon fractions using visible/near infrared diffuse reflectance spectroscopy [J]. Geoderma 164(1):22–32

    Article  Google Scholar 

  • Schlerf M, Atzberger C, Joachim H et al (2010) Retrieval of chlorophyll and nitrogen in Norway spruce (Piceaabies L. Karst.) using imaging spectroscopy[J]. Int J Appl Earth Observ Geoinf 12:17–26

    Google Scholar 

  • Schlesinger WH, Andrews JA (2000) Soil respiration and the global carbon cycle[J]. Biogeochemistry 48:7–20

    Article  Google Scholar 

  • Shao YN, He Y (2011) Nitrogen, phosphorus, and potassium prediction in soils, using infrared spectroscopy[J]. Soil Res 49:166–172

    Article  Google Scholar 

  • Shi TZ, Cui LJ, Wang JJ et al (2013) Comparison of multivariate methods for estimating soil total nitrogen with visible/near-infrared spectroscopy [J]. Plant Soil 366(1–2):363–375

    Article  Google Scholar 

  • Stenberg B, Viscarra Rossel RA, Mouazen AM et al (2010) Visible and near infrared spectroscopy in soil science [J]. Adv Agron 107:163–215

    Article  Google Scholar 

  • Stevens A, Udelhoven T, Denis A et al (2010) Measuring soil organic carbon in croplands at regional scale using airborne imaging spectroscopy[J]. Geoderma 158:32–45

    Article  Google Scholar 

  • Stockmann U, Adams MA, Crawford JW et al (2013) The knowns, known unknowns and unknowns of sequestration of soil organic carbon. Agr Ecosyst Environ 164:80–99

    Article  Google Scholar 

  • Thissen U, Pepers M, Üstün B et al (2004) Comparing support vector machines to PLS for spectral regression applications[J]. Chemometr Intell Lab Syst 73(2):169–179

    Article  Google Scholar 

  • Viscarra Rossel RA, Behrens T (2010) Using data mining to model and interpret soil diffuse reflectance spectra [J]. Geoderma 158:46–54

    Article  Google Scholar 

  • Viscarra Rossel RA, Hicks WS (2015) Soil organic carbon and its fractions estimated by visible-near infrared transfer functions [J]. Eur J Soil Sci 66(3):438–450

    Article  Google Scholar 

  • Viscarra Rossel RA, Lark RM (2009) Improved analysis and modelling of soil diffuse reflectance spectra using wavelets [J]. Eur J Soil Sci 60:453–464

    Article  Google Scholar 

  • Viscarra Rossel RA, Mcglynn RN, Mcbratney AB (2006a) Determining the composition of mineral-organic mixes using UV-vis-NIR diffuse reflectance spectroscopy [J]. Geoderma 137(1):70–82

    Article  Google Scholar 

  • Viscarra Rossel RA, Walvoort DJJ, Mcbratney AB et al (2006b) Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties[J]. Geoderma 131(1–2):59–75

    Article  Google Scholar 

  • Viscarra Rossel RA, Behrens T, Ben-Dor E et al (2016) A global spectral library to characterize the world’s soil [J]. Earth Sci Rev 155:198–230

    Article  Google Scholar 

  • Vohland M, Besold J, Hill J (2011) Comparing different multivariate calibration methods for the determination of soil organic carbon pools with visible to near infrared spectroscopy [J]. Geoderma 166:198–205

    Article  Google Scholar 

  • Wang WS, Ding J (2003) Wavelet network model and its application to the prediction of hydrology [J]. Nat Sci 1:67–71

    Google Scholar 

  • Wilding LP (1985) Spatial variability: it’s documentation, accommodation and implication to soil surveys [M]. In: Nielsen DR, Bouma J (eds) Soil spatial variability. Pudoc, Wageningen, pp 166–194

    Google Scholar 

  • Xu SX, Zhao YC, Wang MY et al (2018) Comparison of multivariate methods for estimating selected soil properties from intact soil cores of paddy fields by Vis–NIR spectroscopy [J]. Geoderma 310:29–43

    Article  Google Scholar 

  • Yang H, Kuang B, Mouazen AM (2012) Quantitative analysis of soil nitrogen and carbon at a farm scale using visible and near infrared spectroscopy coupled with wavelength reduction [J]. Eur J Soil Sci 63:410–420

    Article  Google Scholar 

  • Yu Y, Liu Q, Wang YB et al (2016) Evaluation of MLSR and PLSR for estimating soil element contents using visible/near-infrared spectroscopy in apple orchards on the Jiaodong peninsula [J]. CATENA 137:340–349

    Article  Google Scholar 

  • Zou P, Yang JS, Fu JR et al (2010) Artificial neural network and time series models for predicting soil salt and water content [J]. Agric Water Manag 97(12):2009–2019

    Article  Google Scholar 

Download references

Acknowledgements

This research was funded by geological survey project of China Geological Survey (Grant numbers 12120115048501 DD20160320), Zhejiang Geological Exploration Fund Project (Grant number 201312), Key project of Zhejiang Gongshang University (Grant number X13-03), and Science and Technology Project of Department of Natural Resources of Zhejiang Province (Grant number 2020-33). The spectral reflectance of soil sample was measured by the Institute of Agricultural Remote Sensing and Information Technology of Zhejiang University, P. R. China. We sincerely appreciated the editors and the reviewers for their constructive suggestions and insightful comments which helped us greatly to improve this manuscript.

Author information

Authors and Affiliations

  1. Zhejiang Institute of Geological Survey, Hangzhou, 311203, China

    Mingxing Xu, Xianyao Chu & Yesi Fu

  2. Engineering Technology Innovation Center of Agricultural Land Ecological Assessment and Rehabilitation in Plain Area of Ministry of Natural Resources, Hangzhou, 311203, China

    Mingxing Xu, Xianyao Chu & Yesi Fu

  3. School of Tourism and Urban-rural Planning, Zhejiang Gongshang University, Hangzhou, 310018, China

    Mingxing Xu

  4. 11Th Geologic Team of Zhejiang Province, Wenzhou, 325006, China

    Changjiang Wang

  5. Institute of Land and Urban Rural Development, Zhejiang University of Finance and Economics, Hangzhou, 310018, China

    Shaohua Wu

Authors
  1. Mingxing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianyao Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yesi Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changjiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaohua Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingxing Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, M., Chu, X., Fu, Y. et al. Improving the accuracy of soil organic carbon content prediction based on visible and near-infrared spectroscopy and machine learning. Environ Earth Sci 80, 326 (2021). https://doi.org/10.1007/s12665-021-09582-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12665-021-09582-x

Keywords

Search

Navigation