1 Introduction

Emissions of carbon dioxide (CO2) as a main greenhouse gases (GHGs) have been significantly increased up to 40% since pre-industrial time attributed to anthropogenic activities and fossil fuel combustion (Mbow et al. 2017). Extensive farming activities like overgrazing, intensive tillage, and improper use of chemical fertilizers and manure lead to emissions of GHGs (Ray et al. 2020; Zaman et al. 2021). Forest or grasslands conversions to arable land remained one the biggest cause of soil carbon release in atmosphere since late seventeenth century. Organic matter decomposition and reduction in plant biomass reduced the soil carbon content and released 1.2—4.6 Gt C/year in past decades (Mitchell et al. 2019; Sauerbeck 2001). Enteric fermentation, pasture manure, biomass burning, and synthetic fertilizers has major contribution of CO2 emissions from agriculture perspective (Campbell et al. 2014; Sainju et al. 2008; Tubiello et al. 2013). Various climatic factors, such as temperature, rainfall and soil moisture also alter the soil microbial activities and increasing root respiration and resulting in elevated soil CO2 emissions (Bååth 2018; Mirzai et al. 2022; Ray et al. 2020; Zechmeister-Boltenstern et al. 2018). Hence, soil respiration is the largest contributor to carbon (C) emissions from terrestrial ecosystems (Almagro et al. 2013). Depending on the soil management (SM) and the practices implemented, soil can acts both as either a sink or source of CO2 emissions (Shakoor et al. 2021).

Different soil cultivation methods, including tillage or non-tillage, have varying impacts on CO2 emissions. For instance, reduced or no tillage practices contribute to lowering soil temperature and restricting CO2 emissions (Li et al. 2022; Mancinelli et al. 2023).

On contrary, conventional tillage accelerates the soil microbial activity and decomposition of organic matter that is more relatable with increased CO2 emissions along with other GHGs (Bhattacharyya et al. 2022; Devkota et al. 2022). While conservation tillage is noted for carbon sequestration and preventing CO2 emissions by leaving sufficient crop residues covering the subsurface soil (Devkota et al. 2022; Fageria and Moreira 2011; Mitchell et al. 2019). Hence, dynamics of GHGs emissions from agriculture soil are vital to monitor for efficient and strategic mitigating practices and achieving climate smart agriculture (Devkota et al. 2022).

Experimental methods for quantifying of CO2 emissions faces several constraints including the long-term monitoring, labor requirements, and extensive analysis (Yadav and Wang 2017).Simulating GHGs emissions incorporating various factors like fertilization, climate, soil cultivation practices, and many other factors have been done in numerous research (Li et al. 2010b, 2016; Lloyd et al. 2019; Yu et al. 2013). Previously, several biophysical models such as Root zone water quality model (RZWQM) (Jiang et al. 2018), DayCent (Necpálová et al. 2015), CERES-EGO (Durandeau et al. 2010) DNDC (Li et al. 2010a; Yadav and Wang 2017) had been employed to model the GHGs emissions in different scenarios bounded with some limitations. Main constraints in utilizing biophysical or process based models included large data input, high agro-environmental expertise, complex procedures of model calibration and validation (Smith et al. 2010).

Several Machine and deep learning (ML and DL) methods provide a smart alternative to biophysical methods for various environmental phenomenon such as drought and flood prediction (Hu et al. 2019; Mohammed et al. 2022a), predicting ground water level and agricultural water quality (Mohammed et al. 2024; Singh et al. 2024). Classical ML models such as random forest (RF), support vector machine (SVM), gradient boosting regression (GBR), and least absolute shrinkage and selection operator (LASSO) have been used for prediction modelling of GHGs in many regions (Table 1) (Abbasi et al. 2021; Hamrani et al. 2020; Saha et al. 2021; Yan et al. 2020; Zhou et al. 2022).

Table 1 Review of prediction of some GHGs emissions studies using machine and deep learning algorithms in recent years
Full size table

Deep learning also involves utilization of more effective algorithms such as artificial neural networks (ANN) and convolution neural networks used for (CNN) for complex modelling (LeCun et al. 2015; Reichstein et al. 2019) and predicting non-linear complex relationships (Altikat 2021; Madu et al. 2017). Exceptional capabilities of NN like one input–output communication, quick response to data noise, more flexibility and adaptability, error tolerance and robust non-linear predictions make them advantageous of classical ML algorithms (Shabani et al. 2021). Wide applications of NN in various environmental studies such as soil erosion (Gholami et al. 2018), nutrients runoff (Kim and Gilley 2008), heavy metals in soil (Naderi et al. 2017), groundwater levels (Iqbal et al. 2020), and hydrological modelling (Pradhan et al. 2020) make it a good choice to be used for modelling GHGs emissions from various sectors (Ganzenmüller et al. 2019). It is also being employed for prediction modelling of CO2 emission from soil and land use land cover in many countries such as Iran (Altikat 2021; Shabani et al. 2021), Brazil (Freitas et al. 2018; Vitória et al. 2022), Turkey (Yılmaz and Bilgili 2018) and Bangladesh (Fattah et al. 2021). Noticeably, the analysis of the available research published in the Web of Science database revealed that machine and deep learning (green line) is associated with published research related to CO2 emissions (Fig. 1). However, previous studies on CO2 emissions did not account for multiple factors in different input combinations or scenarios in varying climatic conditions. In addition, applicability of machine and deep learning algorithms for predicting CO2 emissions also needs to be tested in different scenarios and environmental settings for a wider applicability. Hence, keeping in view the significant application of advanced machine learning for GHGs predictions, our study aimed to evaluate the ability of two machine learning algorithms namely GBR, SVR, and two deep learning algorithms namely FNN and CNN. This evaluation is conducted across three distinct scenarios of input combinations to predict CO2 emissions from Maize fields in two agroclimatic regions, i.e., continental (Debrecen-Hungary) and semi-arid (Karaj-Iran).

Fig. 1
figure 1

Bibliometric analysis of research published in the Web of Science database related to machine learning and CO2 emissions, using the search equation: (“CO2 emissions") AND ("artificial intelligence" OR "deep learning") AND (soil) AND (climate OR "climate change"). This analysis revealed 550 published research articles and identified 5 clusters

Full size image

2 Method

2.1 Study Area and Data Collection (Observation Stage)

CO2 gas samples were collected from maize (Zea mays L.) fields in two different agroclimatic regions. The first one was at the Agriculture Research Station of the College of Agriculture and Natural Resources, University of Tehran, Karaj (Iran) (50°58′ E, 35°48′ N), representing a semi-arid climate. The second was a continental climate located at the Látókép research station at the University of Debrecen (Hungary) (47°33′ E, 21°26′ N). The experimental design included two different soil management practices: (1) conventional tillage and (2) non-tillage. In Karaj, gas samples were collected using the static closed chamber methodology, one of the most used and cost-effective techniques for measuring GHG emissions from agroecosystems. The CO2 concentration was measured using gas chromatography (Teif Gostar Faraz, TG 2552), equipped with a thermal conductivity detector (TCD).

In Debrecen, CO2 emission was measured using the digital meter Testo 535 (TESTO; 0560 5350) which measures CO2 concentration via infrared absorption with 1 ppm of CO2 (Törő et al. 2019). The average CO2 emission from Debrecen was 138.78 ± 72.04 ppm (n = 36), while the average from Karaj was 478.98 ± 174.22 ppm (n = 36) (Table 2). More detailed information about the experimental design and the interaction between different environmental variables (i.e., soil temperature (Δ), and soil moisture (θ)) and CO2 emission could be retrieved from Mohammed et al. (2022b). Observed CO2 emissions from both experimental sites were collectively taken as a response or input variable to be predicted in ANN modelling architectures.

Table 2 Summary of descriptive statistics for observed CO2 emissions in two agroclimatic regions (i.e., Karaj and Debrecen) (02.08.2018–13.09.2018)
Full size table

2.2 Machine Learning Modelling

The study employed two classical machine learning models, GBR and SVR, alongside two deep learning models, namely FNN and CNN. This analysis was conducted under three different scenarios, incorporating meteorological and soil management factors (Table 3).

Table 3 Scenarios used for predicting CO2 emissions (CO2-Prd) in all algorithms
Full size table

The general principle of machine learning modelling is based on a test size split of 0.25 (75% training and 25% testing). The hyperparameters of machine learning models are tuned using Randomized Search CV function from scikit-learn with 10 iterations and five-fold cross validation to identify the optimal model. The optimal selection of deep learning models is determined through manual adjustments of learning rates, neurons, and activation function. All models are run in python 3, leveraging the computational capabilities of T4GPU in 32 GB RAM for enhanced training and prediction. The hyperparameters details of all models are presented in Table 4.

Table 4 Hyperparameters selected in machine and deep learning architectures
Full size table

2.2.1 Gradient Boosting Regression

The gradient boosting regression (GBR) is based on the boosting algorithm encompassed by gradient descent, where subsequent trees are fitted to the residuals of the preceding model, gradually reducing the overall prediction error, and boosting functions widely applied in regressions and classifications problems (Jerome 2001). The iterative process of GBR enhances the predictions and improves the performance of the model. Unlike random forest, which trains multiple decision trees independently and averages their predictions, GBR constructs a series of models sequentially, with each subsequent model focusing on the errors made by its predecessors (Arshad et al. 2023b). This sequential learning process enables Gradient Boosting to correct earlier mistakes effectively, leading to superior predictive accuracy (Otchere et al. 2022). Additionally, it inherently deals with heterogeneous data types and automatically selects relevant features during the process of model training. This ability of selecting appropriate features not only simplifies the modeling process but also enhances the interpretability of model achieving accurate predictions (Wen et al. 2021). Currently, GBR is initialized on standardized features (meteorological and soil management) using randomized search hyperparameter tuning with five-fold cross validation (Table 4). The best fit model is trained and implemented to achieve high prediction accuracy.

2.2.2 Support Vector Regression

Support vector regression (SVR) is a supervised machine learning method that is based on statistical learning approach derived from support vector machines (SVM), which aims to identify the hyperplane that best fits the train data with maximizing the margin between hyperplane and nearest data points, called support vectors (Zhang and O'Donnell 2020).

SVR develops an optimization problem to acquire a regression function that connects input predictor variables to observed response values. Nevertheless, SVR offers a robust and adaptable approach to regression tasks, particularly in situations characterized by non-linear relationships and a tolerance for forecast errors (Arshad et al. 2023a; Awad and Khanna 2015). Currently, hyperparameter tuning of SVR is performed using Randomized Search to explore the best regularization parameter (c), kernel type (linear, and rbf), and kernel coefficient for the radial basis function (gamma) (Table 4).

2.2.3 Feedforward Neural Network (FNN)

Feedforward architecture of ANN consists of three layers: [input layer, hidden layer (s), and output layer], where each layer has several neurons that are characterized by different biases and weights (Widiasari and Nugroho 2017). The data flows from the input layer to the output one through the hidden layer (s), process also called as forward pass (Harun et al. 2010). Currently, FNN for CO2 prediction is employed using the Keras library with a TensorFlow backend in python. An FNN is a foundational architecture in deep learning, comprising layers of interconnected nodes, where information flows unidirectionally from the input layer through hidden layers to the output layer (Khadem et al. 2022).

$${Y}_{j}=\left(f\right)*\sum_{i=1}^{n}({X}_{i}.{W}_{ij })+ {b}_{j},$$
(1)

where \({Y}_{j}\) is the weighted sum of inputs for neuron j, \({X}_{i}\) is the ith neuron in previous layer, \({W}_{ij}\) is the weight associated between the neurons, and \({b}_{j}\) is the introduced bias term in jth neuron, \(f\) denotes activation function. The FNN architecture of the current study is defined as a sequential model with an input layer, two hidden layers employing rectified linear unit (ReLU) activation functions, and an output layer. Features are standardized using the scikit-learn StandardScaler. The Adam optimizer is employed with a specified learning rate, and the mean squared error is chosen as the loss function. The model is then trained on standardized training data for 50 epochs, with a batch size of 32 (Table 4). The code also allowed for experimenting with different learning rates, providing flexibility for hyperparameter tuning to optimize the neural network's performance.

2.2.4 Convolutional Neural Network (CNN)

Convolutional neural network is a kind of deep and feedforward artificial neural network that is particularly designed for image processing, classification, and object detection. The basic architecture of CNN comprises of convolutional layer to apply convolutional operation to the input data, and pooling layer to downscale the spatial dimensions (Cui and Fearn 2018). Convolutional layers use filters or kernels for feature extraction and capturing spatial hierarchies. The down sampling process by pooling layer helps to improve the computational complexity. Fully connected layers, traditionally located at the network's end, establish connections between each neuron, and every neuron present in the previous and subsequent layers, thereby facilitating high-level reasoning (Manaswi 2018). During forward propagation, the input of each feature signal in layer l is the accumulation of the final output of the previous feature signal (l − 1) convolved with proper filters and passed through a nonlinear activation function (Malek et al. 2018). Feedforward architecture of 1D CNN is presented as

$${a}_{i}^{l}= {b}_{i}^{l}+ {\sum }_{j=1}^{{m}^{l-1}}conv1D ({W}_{i,j}^{l} {S}_{j}^{l-1}) \left(i=1\dots .,{m}^{l}\right),$$
(2)

where \({a}_{i}^{l}\) presents ith feature signal input, \({b}_{i}^{l}\) is the bias term, \({W}_{i,j}^{l}\) is the kernel (filter) weights, \({S}_{j}^{l-1}\) is the jth feature output on previous (l − 1) layer. The currently employed CNN architecture is formed by layers, each having a distinct function in the learning process. The input layer has been customized to accommodate the shape of our preprocessed data. A 1D convolutional layer that consists of 128 filters and a kernel size of 3 is employed which is particularly responsible for learning local patterns and features within the sequential data. The rectified linear unit (ReLU) activation function has been utilized to introduce non-linearity. After the convolutional layer, a flattening layer was introduced to alter the multi-dimensional output into a one-dimensional array. The architecture applied incorporated two dense (fully connected) layers. The first dense layer consisted of 128 units with a ReLU activation function, thereby facilitating the extraction of high-level features. The final dense layer, which consisted of a single unit, served as the output layer for CO2 prediction. Like FNN, the Adam optimizer is employed with a specified learning rate (0.01), with mean squared error as the loss function. The model was trained for 50 epochs with a batch size of 32.

The input layer included multi factors including soil management techniques (SM), and meteorological factors includes soil temperature (Δ), and soil moisture (θ) with predicted CO2 as an output layer. (Table 3).

2.2.5 Machine Learning Models Performance

All models’ evaluations were performed using four indicators, namely: (1) coefficient of determination (R2), Root mean squared error (RMSE), degree of agreement (d), and the Nash–Sutcliffe efficiency (NSE) presented in Table 5. Moreover, Taylor diagram (Taylor 2001) was implemented to exhibit the performance of all models in different scenarios of CO2 predictions. It provides a concrete overview of the model’s performances based on the correlation and standard deviation between observed and predicted values (Zhu et al. 2020).

Table 5 Performance evaluation metrics for all models for predicting CO2 emissions
Full size table

3 Results

3.1 Efficiency of Machine and Deep Learning Models in Predicting CO2 Emissions

3.1.1 Predicting CO2 Emissions at Training Stage

Three scenarios were considered to reach the optimal combination between input data and predicted CO2 emissions. For the first scenario (SC1), all variables (SM + Δ + θ) were treated as input for both machine and deep learning algorithms. Performance evaluation results of train set revealed that high prediction accuracy is achieved by GBR in SC1 (SM + Δ + θ) with the highest R2 = 0.8778, RMSE = 72.05, d = 0.96, and NSE = 0.870 followed by GBR in SC3 (SM + θ) with the R2 = 0.854, RMSE = 78.73, d = 0.950, and NSE = 0.850 (Figs. 2, 3).

Fig. 2
figure 2

Scatterplots and box plots distribution of observed (actual) and predicted CO2 (ppm) emissions in all machine and deep learning models in train set. a scatterplot of actual and predicted CO2 (ppm) emissions from all models in SC1 (SM + Δ + θ). b Boxplots showing the mean and range of actual and predicted CO2 (ppm) emissions from all models in SC1 (SM + Δ + θ). c Scatterplot of actual and predicted CO2 (ppm) emissions from all models in SC2 (SM + Δ). d Boxplots showing the mean and range of actual and predicted CO2 (ppm) emissions from all models in SC2 (SM + Δ). e scatterplot of actual and predicted CO2 (ppm) emissions from all models in SC3 (SM + θ). f Boxplots showing the mean and range of actual and predicted CO2 (ppm) emissions from all models in SC3 (SM + θ)

Full size image
Fig. 3
figure 3

Performance evaluation metrics (RMSE, R2, d, and NSE) of all machine and deep learning model in train set

Full size image

Following the sequence of training accuracy, FNN outperformed in SC1 (SM + Δ + θ) with the R2 = 0.791, RMSE = 94.08, d = 0.93, and NSE = 0.790, and in SC3 (SM + θ) R2 = 0.757, RMSE = 101.43, d = 0.920, NSE = 0.750. Furthermore, SVR is found to perform least with R2 = 0.660, RMSE = 120.06, d = 0.860, NSE = 0.660 in SC1 (SM + Δ + θ), and R2 = 0.735, RMSE = 105.9, d = 0.900, NSE = 0.730 in SC3 (SM + θ). SC2 (SM + Δ) provided with least accurate CO2 predictions with R2 = 0.715 for GBR, followed by R2 = 0.653 for FNN, R2 = 0.642 for CNN, and R2 = 0.530 for SVR (Table 6, Figs. 2, 4).

Table 6 Performance evaluation of machine and deep learning algorithms in predicting CO2 emission in train and test sets
Full size table
Fig. 4
figure 4

Tylor diagram of machine learning performance: a SC1 training stage, b SC2 training stage, c SC3 training stage; d SC1 testing stage, e SC2 testing stage, f SC3 testing stage

Full size image

Overall, the performance of machine and deep learning algorithms for high prediction accuracy also presented in Tylor diagram (Fig. 4) is ranked as GBR > FNN > CNN > SVR.

3.1.2 Predicting CO2 Emissions at Testing Stage

In testing stage, CO2 predictions obtained on the unseen test data revealed the highest prediction accuracy is achieved by FNN in SC1 (SM + Δ + θ) with R2 = 0.918, RMSE = 67.75, d = 0.97, NSE = 0.910 followed by FNN in SC3 (SM + θ) with R2 = 0.907, RMSE = 72.22, d = 0.97, and NSE = 0.90, and GBR in SC1 (SM + Δ + θ) with R2 = 0.887, RMSE = 79.881, d = 0.960, and NSE = 0.880 (Table 6, Fig. 5).

Fig. 5
figure 5

Scatterplots and box plots distribution of observed (actual) and predicted CO2 (ppm) emissions in all machine and deep learning models in test set. a scatterplot of actual and predicted CO2 (ppm) emissions from all models in SC1 (SM + Δ + θ). b Boxplots showing the mean and range of actual and predicted CO2 (ppm) emissions from all models in SC1 (SM + Δ + θ). c scatterplot of actual and predicted CO2 (ppm) emissions from all models in SC2 (SM + Δ). d Boxplots showing the mean and range of actual and predicted CO2 (ppm) emissions from all models in SC2 (SM + Δ). e scatterplot of actual and predicted CO2 (ppm) emissions from all models in SC3 (SM + θ). f Boxplots showing the mean and range of actual and predicted CO2 (ppm) emissions from all models in SC3 (SM + θ)

Full size image

Afterwards, based on performance evaluation, CNN in SC1 (SM + Δ + θ) performed with R2 = 0.886, RMSE = 80.11, d = 0.96, NSE = 0.880, and GBR in SC3 (SM + θ) with R2 = 0.859, RMSE = 88.96, d = 0.950, and NSE = 0.860. Like training stage, SVR is found to perform less compared to other algorithms with the highest achieved R2 = 0.809, RMSE = 103.8, d = 0.92, NSE = 0.80 in the SC3 (SM + θ) followed by R2 = 0.690, RMSE = 132.2, d = 0.860, NSE = 0.690 in SC1(SM + Δ + θ), and R2 = 0.515, RMSE = 165.3 d = 0.760, NSE = 0.510 in SC2 (SM + Δ) (Table 6, Figs. 5, 6).

Fig. 6
figure 6

Performance evaluation metrics (RMSE, R2, d, and NSE) of all machine and deep learning model in test set

Full size image

During the prediction of CO2 emissions, machine learning tends to either overpredict or underpredict emissions compared with the original data. To capture the error, the observed data were compared with the predicted data, and the differences were displayed as a box plot, as seen in Fig. 7. In the training stage, the smallest differences were captured in GBR-SC1 and GBR-SC3, while some cases of overprediction were recorded in CNN-SC3 and NN-MLP-SC2 (Fig. 7A). In the testing stage, the analysis of errors revealed that the smallest errors in predicting CO2 emissions were observed in FNN-SC1 and GRB-SC2 (Fig. 7B), while overprediction of some values was noted in SVR. In this context, a ridgeline chart confirms this outcome, comparing the distribution of actual data (black line) with other prediction data in each scenario to reveal some differences (Fig. 8). In the training stage, GBR (red color) was found to have the closest distribution to the original. However, SVR exhibited a less similar distribution compared with the original. In the testing stage, FNN (blue) and GRB (red) had the closest distribution compared with the actual data.

Fig. 7
figure 7

Box plot displaying the errors in prediction compared with observed data for different scenarios: A training, B testing

Full size image
Fig. 8
figure 8

Ridgeline chart comparing the distribution of actual data with machine learning predicted data for different scenarios: A SC1 (SM + Δ + θ, training), B SC2 (SM + Δ, training), C SC3 (SM + θ, training), D SC1 (SM + Δ + θ, testing), E SC2 (SM + Δ, testing), F SC3 (SM + θ, testing)

Full size image

Overall, based on the performance evaluation for the prediction accuracy, algorithms are ranked as FNN > GRB > CNN > SVR as also presented in the Tylor diagram (Fig. 4). Moreover, three different scenarios provided a better exploration for identifying the best input combinations. It revealed that SC1 with (SM + Δ + θ) provided the best input combination for achieving high prediction accuracy during model training with optimized hyperparameters and five-fold CV followed by SC3 (SM + θ) and SC2 (SM + Δ). Scatterplot of observed and predicted CO2 emissions from the four applied algorithms also presents more closeness towards fit line by FNN and GBR. Furthermore, box plots distribution also revealed that FNN provided a more similar range of data distribution with the actual ones followed by GBR and CNN specifically in SC1 and SC3 (Fig. 5). The low performance of SVR is also clearly depicted in the three scenarios. Overall, both deep learning architectures (FNN and CNN) proved a better performance for accurate CO2 predictions compared to SVR.

4 Discussion

4.1 Comparative Performance of Machine and Deep Learning Models for CO2 Prediction

Currently, implementation of machine and deep learning algorithms for CO2 emissions revealed the outperformance of GBR in train and FNN in test datasets. The outperformance of GBR for accurate predictions is already proven in various studies in unseen test data (Adjuik and Davis 2022; Romeiko et al. 2019; Wang et al. 2024). The sequential learning process of GBR enables a gradual reduction of errors, with a focus on areas of poor model performance. The repeated process enables the model to capture the hidden non-linear relationship unless highly accurate prediction is achieved. The outperformance of GBR in SC1 (SM + Δ + θ) with highest R2 = 0.877 on train and 0.887 on test data (Figs. 3, 6) aligned with the findings of Kaur Dhaliwal et al. (2022) for accurate crop yield prediction from soil management practices. The high performance might be attributed to less sensitivity of GBR towards outliers. This allowed minimizing their influence for accurate CO2 predictions and helped achieve high accuracy in model training as explained by Ibrahim (2023). Another reason for high prediction accuracy is automatic feature selection during model training that helped in prioritizing best features with minimized prediction errors (Otchere et al. 2022). Furthermore, mixed data handling ability of GBR also explored by Cha et al. (2021) without extensive preprocessing also lead to capture the best relationship between soil management practices and CO2 emissions. Support vector regression (SVR) differs from classical regression models as it not only aims to minimize prediction errors but also incorporates a margin of tolerance. This margin allows for deviations from the hyperplane within a specified epsilon-insensitive tube. This inherent flexibility empowers SVR to capture of the overall trend of the data while accommodating minor fluctuations or outliers (Cai and Ma 2019). The effectiveness of SVR lies in its ability to handle non-linear relationships through the utilization of kernel functions, which implicitly map the input features onto a higher-dimensional space where linear separation becomes feasible (Wang and Bi 2023). This procedural approach empowers SVR to comprehend intricate patterns and correlations within the data without explicitly defining the transformation. For instance, Singh and Kumar (2022) also reported the superior performance of SVR with polynomial and RBF kernels for predicting agricultural GHG emissions. However, currently, SVR is observed to perform with a minimum R2 = 0.690 in best performed SC1 (Figs. 3, 6). The performance of SVR is highly dependent on the appropriate selection of hyperparameters, including the choice of kernel function, regularization parameter (C), and epsilon (Yan et al. 2019).

Furthermore, neural networks (NN) are applied for a comparative assessment and its wide applicability due to their inherent capability of designing qualitative input–output (Küçüktopcu and Cemek 2021). Increasing the depth of NNs and effectively tuning the network may contribute to better prediction performance (Chowdhury et al. 2021). For instance, our findings demonstrated the outperformance of FNN over all algorithms in SC1 (SM + Δ + θ) with highest R2 = 0.918, d = 0.97, and NSE = 0.91 (Figs. 5, 6). Several distinct capabilities of FNN including scalability, approximating complex functions, and automatic feature learning make them a better choice over a range of classical ML algorithms (Kosamkar and Kulkarni 2021a). Hence, FNNs are more flexible to capture the temporal dependencies and intricate patterns of the data with limited need of manual feature engineering (LeCun et al. 2015). Moreover, parallel processing capabilities due to GPU hardware architecture accelerates the optimization process and reduces the model training timings. Currently used FNN architecture of two hidden layers with 128 neurons each proved to better optimize the model performance achieving highest prediction accuracy.

The unique architecture of CNN has revolutionized image processing due to its exceptional capability of capturing local patterns through filters. Currently, 1D convolutions employed in temporal dimensions of CO2 were able to extract the local patterns within the time series, identifying the intricate relationships between the soil management practices and CO2 emissions. Overall, the performance of CNN is ranked 3rd for accurate predictions of CO2 emissions on test dataset with R2 = 0.886 in best performed SC1 with a close competition to GBR with R2 = 0.887 (Figs. 3, 5, 6). The current high performance of deep learning algorithms is aligned with the previous studies of Altikat (2021) and Küçüktopcu and Cemek (2021) for CO2 predictions in different environmental settings. Specifically, the high performance of CNN is also attributed to its ability of hierarchical representation learning, where lower layers capture simple temporal patterns and deep layers learn more complex relationships (Jing et al. 2017). Moreover, CNNs do not apply parameter sharing in convolutional filters to decrease the number of trainable parameters when compared to fully connected networks. This efficiency in parameters makes CNNs a good option for modeling time series data, particularly with large datasets. However, the limited global context of the model may reduce its capability to capture long range dependencies in the time series affecting their predictive performance. Hence, in some CO2 predictions studies, hybrid of CNN-LSTM is reported to perform better (Amarpuri et al. 2019; Han et al. 2023). Overall, FNN has proven to be the most robust and effective method for accurate CO2 prediction on unseen test dataset attributed to it high flexibility towards capturing complex relationship. Hence, accurate prediction modelling using advanced machine and deep learning approaches can facilitate for an informed decision-making process for adopting effective soil management practices. Moreover, despite good hyperparameter adjustments, each of the algorithms is bounded with some limitations addressed further in limitations.

4.2 Limitations

In this research, only soil parameters were considered, specifically soil moisture and soil temperature. These data were collected in the field during measurements of CO2 emissions. Other factors, such as climatic variables (temperature, relative humidity, and precipitation), were not considered due to the absence of a climatic monitoring station in the field. Incorporating these factors may enhance the performance of the studied algorithms and highlight the significant importance of these variables. Therefore, future studies will focus on the role of climatic factors in emissions.

Despite significant high predictive performance of GBR and FNN in train and test sets for CO2 emissions, the algorithms are bound to some limitations. Hyperparameters sensitivity of GBR is one of the main challenge when dealing with complex relationships or large datasets which may also cause overfitting (Malinin et al. 2020). Similarly, in contrast to the advantageous capabilities of deeply learned NNs, high dimensional data structures also cause model overfitting issues. Manual hyperparameter tuning including no. of layers and neurons, learning rates, and regularization parameters can also enhance the sensitivity towards prediction accuracies (Ying 2019). Due to black box nature of machine and deep learning models, interpretability of the features is challenging, especially in complex architectures with many hidden layers (Gawlikowski et al. 2021). Hence, the addition of more covariates facilitated by feature selection method can be used for better interpretation of models.

Another significant factor affecting the prediction accuracies of deep learning models is the length of dataset which can be facilitated by data augmentation methods to prevent overfitting problems. Furthermore, SVR might present computational hurdles, particularly for extensive datasets, owing to the quadratic optimization issue it deals with (Awad and Khanna 2015). Overall, deep learning provided reliable and accurate findings with better performance of FNN which can be further improved by increasing the sampled observations. Furthermore, incorporation of soil physiochemical and biological properties can also increase the prediction accuracy of CO2 emission from agricultural soil (Ebrahimi et al. 2019).

5 Conclusion

Recently, the use of machine learning (ML) in environmental research has accelerated. This research utilized two machine learning models, namely GBR and support vector regression (SVR), along with two deep learning architectures, FNN and CNN, to predict CO2 emissions from two agroclimatic regions. The results revealed that FNN outperformed the others, achieving the highest accuracy in predicting CO2 emissions, followed by GBR and CNN. SVR’s performance was the least accurate. Additionally, this research recommended using a combination, SC1 (SM + Δ + θ), which was found to provide highly accurate predictions in both training and testing datasets, reflecting the significance of all factors—soil management and meteorological—in predicting CO2 emissions across different environmental settings. Interestingly, the input combination of soil management practices with soil temperature yielded less accurate predictions, highlighting the more significant impact of soil moisture on CO2 emissions. Predicting CO2 emissions from agricultural soil from varied agroclimatic regions poses significant implications for agricultural sustainability and addressing climate change. The outcome of this research will serve as a tool to support decision-makers, researchers, and stakeholders in assessing the capability of ML for accurate CO2 prediction. Predictive modeling that identifies the most influential factors affecting CO2 emissions can assist in developing strategies to increase soil carbon storage and decrease greenhouse gas emissions from agriculture.