Addressing uncertainty and bias in land use, land use change, and forestry greenhouse gas inventories

Abstract

National greenhouse gas inventories (NGHGIs) will play an increasingly important role in tracking country progress against United Nations (UN) Paris Agreement commitments. Yet uncertainty in land use, land use change, and forestry (LULUCF) NGHGHI estimates may undermine international confidence in emission reduction claims, particularly for countries that expect forests and agriculture to contribute large near-term GHG reductions. In this paper, we propose an analytical framework for implementing the uncertainty provisions of the UN Paris Agreement Enhanced Transparency Framework, with a view to identifying the largest sources of LULUCF NGHGI uncertainty and prioritizing methodological improvements. Using the USA as a case study, we identify and attribute uncertainty across all US NGHGI LULUCF “uncertainty elements” (inputs, parameters, models, and instances of plot-based sampling) and provide GHG flux estimates for omitted inventory categories. The largest sources of uncertainty are distributed across LULUCF inventory categories, underlining the importance of sector-wide analysis: forestry (tree biomass sampling error; tree volume and specific gravity allometric parameters; soil carbon model), cropland and grassland (DayCent model structure and inputs), and settlement (urban tree gross to net carbon sequestration ratio) elements contribute over 90% of uncertainty. Net emissions of 123 MMT CO₂e could be omitted from the US NGHGI, including Alaskan grassland and wetland soil carbon stock change (90.4 MMT CO₂), urban mineral soil carbon stock change (34.7 MMT CO₂), and federal cropland and grassland N₂O (21.8 MMT CO₂e). We explain how these findings and other ongoing research can support improved LULUCF monitoring and transparency.

Introduction

National greenhouse gas (GHG) inventories (NGHGIs) are the primary tool for tracking anthropogenic (human-induced) GHG emissions at the country, sector, and source category level. Over the coming decade and beyond, NGHGIs will support setting and measuring progress against each country’s “nationally determined contributions” (NDCs) for reducing GHG emissions under the United Nations (UN) Paris Agreement while also supporting domestic climate policy development and evaluation (UN Framework Convention on Climate Change (UNFCCC) 2019a; UNFCCC 2019b; Andersson et al. 2008). In particular, NGHGI accounting for land use, land use change, and forestry (LULUCF) is a priority for many countries: the first round of NDCs indicates that LULUCF will provide 25% of planned GHG reductions leading to 2030 (Grassi et al. 2017). Global integrated assessment models project that enhancing natural land-based sinks, avoided deforestation, and bioenergy could provide 30% of GHG abatement required to keep temperature increase below 1.5 C by 2050 (Roe et al. 2019).

Yet LULUCF is a large source of uncertainty in estimating anthropogenic GHG emissions (Friedlingstein et al. 2020; Pulles 2017; Jonas et al. 2014; National Research Council 2011). To ensure international confidence in national GHG reporting, significant improvements in LULUCF NGHGI estimation methods and transparency will be required. In this paper, we demonstrate an analytical framework for identifying, quantifying, and reporting on sources of LULUCF uncertainty and bias in NGHGI inventories at the level of individual datasets, models, submodels, and other inputs (“uncertainty elements”). Using the USA as a case study, we suggest countries can use this analytical framework to comply with UN Paris Agreement guidelines in two ways:

• (1) Transparently reporting on LULUCF NGHGI uncertainty estimation methods, including clarifying which uncertainty elements are accounted for and how LULUCF uncertainty is calculated

• (2) Identifying the largest uncertainty elements as a first step in prioritizing inventory improvements

In our framework, we identify and attribute uncertainty across all US LULUCF GHG source and sink (collectively, flux) estimates and provide initial GHG flux estimates for omitted inventory categories.¹ We make three contributions: (1) we propose and demonstrate an analytical framework that countries can use to fulfill UN Paris Agreement transparency provisions, (2) we advance the large literature concerning NGHGI uncertainty by focusing on so-called individual “uncertainty elements,” which allows for better targeting data and research needs, and (3) we demonstrate a set of uncertainty attribution methods that can be applied across inventory categories with varying methodological complexity, including the most sophisticated (Tier 3) methods.²

Evidence of global and national LULUCF uncertainty

LULUCF estimation uncertainty results from a combination of structural and conceptual challenges, including (1) large heterogeneity in fluxes across time and space, driven by complex biological, geochemical, and physical processes combined with variable anthropogenic and natural disturbances; (2) the inability to continuously observe fluxes over time and over large areas; and (3) differences in definitions and accounting methods across countries and studies (Rypdal and Winiwarter 2001; Grassi et al. 2018). These dynamics drive higher proportional and absolute uncertainty when compared to GHG sources for which census data is available, underlying processes are better understood, and available GHG accounting guidance is more prescriptive (Pulles 2017).

NGHGIs play a useful role in tracking anthropogenic LULUCF GHG emissions. Alternative methods (global land use change bookkeeping models and dynamic global vegetation models (DGVMs)) exhibit large multi-model uncertainty for total atmosphere-to-land CO₂ fluxes, with a standard deviation equal to 10% of annual global anthropogenic GHG emissions (4.0 gigatonnes (Gt) CO₂ year⁻¹ on average, 2010–2019) (Friedlingstein et al. 2020). The disagreement is driven in part by conflicting definitions of anthropogenic LULUCF fluxes. Combining global bookkeeping models and DGVMs to align with the definition used by NGHGIs (all LULUCF fluxes on managed land) achieves results consistent with aggregate NGHGI estimates (within 0.8 Gt CO₂ year⁻¹) (Grassi et al. 2018).³ As such, NGHGIs appear to be able to collectively validate the LULUCF estimates of global models and vice versa.

However, individual NGHGIs vary widely in quality and precision, which creates challenges in tracking country-level emission trends and therefore NDC progress. The NGHGIs of major emitting countries reviewed in Table 1 cover 50% of global LULUCF fluxes (in absolute value, see Supplementary Material (SM) Sect. 1). Reviewed countries report proportional LULUCF uncertainty ranging from 12% (Colombia) to 102% (Cambodia). Of the 5 major emitting countries with the largest LULUCF fluxes, we find that four (China, USA, Russia, India) exhibit sufficiently large uncertainty that the LULUCF emission reductions proposed in their first NDCs are at risk of failing statistical significance at the 95% confidence level (Jonas et al. 2010, see SM Sect. 1 for further discussion).

Table 1 LULUCF NGHGI uncertainty for 20 major emitting countries

Furthermore, there is significant heterogeneity in uncertainty estimation methods, making it difficult to compare precision across NGHGIs and to know how well uncertainty values reflect true variance of the flux point estimate. Challenges include not reporting LULUCF uncertainty at all (India, South Korea), not reporting uncertainty for inventory categories (China, Brazil), and, most commonly, providing insufficient information on how uncertainties were calculated (no reporting on uncertainty measures for emission factors or activity data; no information on how uncertainty measures were estimated).

The large majority of LULUCF fluxes reported in NGHGIs are calculated using lower order (Tiers 1 and 2) methods, which likely limit accuracy (Ogle et al. 2003; Ogle et al. 2006). As countries look to improve LULUCF monitoring methods, uncertainty estimation will become more complex. Indeed, uncertainty estimates may increase to more closely approximate true variability, particularly as more sources of uncertainty are accounted for. Therefore, it will be important for countries to simultaneously improve NGHGI methods, transparently report uncertainty, and identify opportunities for increasing precision to ensure NDC emission reduction claims are well-supported.

To date, non-Annex I (developing) countries have lacked mandates and resources to report NGHGIs in a format comparable to Annex I countries, which has driven large heterogeneity in non-Annex I NGHGIs.⁴ Going forward, however, Parties to the UN Paris Agreement have agreed to implement an Enhanced Transparency Framework, under which both Annex I and non-Annex I countries will regularly submit NGHGIs using 2006 Intergovernmental Panel on Climate Change (IPCC) Guidelines for National Greenhouse Gas Inventories and the 2019 Refinement (IPCC 2006, IPCC 2019; UNFCCC 2015, 2019a, 2019b). All Parties are required to estimate uncertainty for all inventory categories and inventory totals and to report on uncertainty estimation methods and underlying assumptions (UNFCCC 2019a, Decision 18/CMA.1). Developing countries are given some flexibility to qualitatively discuss uncertainty for key inventory categories, where data are unavailable.

To support the Enhanced Transparency Framework, countries can use the methods demonstrated in this paper to both transparently report on NGHGI uncertainty methods and to identify the largest sources of LULUCF uncertainty as a way to prioritize inventory improvements. We use the USA as a case study due to the scale of US LULUCF fluxes (the largest of all Annex I countries, Muyskens et al. (2021), CAIT (2021), the proportion of LULUCF fluxes calculated using Tier 3 methods (97%, Table 1), and the degree of transparency in the US NGHGI. The methods and data underlying the US LULUCF inventory are based on over 130 peer-reviewed articles and government reports and improvements made over 25 NGHGI reports since 1996 (US NGHGI 2021). The USA encompasses a large variety of land uses and climatic regions, making it a useful basis for studying GHG estimation methods across LULUCF inventory categories. The USA is also active in LULUCF carbon credit markets, generating over 25% of LULUCF credits issued globally under existing voluntary and compliance carbon crediting mechanisms (see SM Sect. 1, Table 1-3).

Defining and quantifying NGHGI uncertainty

We are interested in a quantitative measure of the potential difference between an NGHGI flux estimate and the true value of the flux being estimated, referred to as model outcome uncertainty or prediction error (Walker et al. 2003; Harmon et al. 2015). Our analysis focuses on two ways model outcome uncertainty can manifest: (1) random error around the true flux and (2) bias or systematic error between the estimate and the true flux.

As recommended by IPCC (2006, 2019) guidelines, NGHGI uncertainty assessments often assume flux estimates are unbiased, that is, the true GHG flux can be recovered in expectation (Magnussen et al. 2014). Using this assumption and standard statistical inference methods, one can calculate a 95% confidence interval (CI) for each estimate, a measure of random error which indicates the bounds within with the true flux will be located 95% of the time, assuming data could be randomly sampled many times over relevant populations.

Previous work has relaxed the unbiasedness assumption by comparing independent calculations for the same inventory category (Petrescu et al. 2020; Erb et al. 2013; Shvidenko et al. 2010; Smith et al. 2008). Even if unbiasedness holds for individual flux estimates, NGHGIs as a whole can be biased by omitting inventory categories due to lack of knowledge, data, or technical capacity. Inventory-wide bias has been estimated by comparing aggregate NGHGI flux estimates across historical inventory recalculations (Hamal 2010), a method which captures bias from changes in inventory methods and inventory category omissions, but this approach will not be useful for identifying potential inventory improvements.

Many studies have assessed uncertainty across entire NGHGIs (e.g., Bun et al. 2010; Winiwarter and Muik 2010; Lieberman et al. 2007) and with a focus on agricultural and forestry inventory sectors (e.g., Petrescu et al. 2020; Shvidenko et al. 2010; Leip 2010; Nilsson et al. 2007; Monni et al. 2007a; Monni et al. 2007b), yet uncertainty estimates are limited to the level of sector, gas (CO₂, CH₄, N₂O), or flux. Few studies have performed more detailed uncertainty attribution for agriculture and forestry sectors, and where this analysis occurs, it is limited to Tier 2 inventory methods (Monni et al. 2007a; Winiwarter and Rypdal 2001; Winiwarter and Muik 2010). Studies that assess uncertainty for individual inventory categories provide useful context and inputs for our analysis (Peltoniemi et al. 2006; Ogle et al. 2010).

We look to build on these literature strands in two ways: (1) identifying individual sources of uncertainty which we term “uncertainty elements,” for each NGHGI flux estimate, with a goal of resolving uncertainty attribution at a level that is meaningful for setting programmatic, research, and budgetary priorities, and (2) attributing uncertainty across all elements as consistently as possible for the entire LULUCF sector. While for most fluxes we are unable to account for bias, we suggest a measure of NGHGI bias by providing initial estimates of omitted GHG fluxes.

Methods

Our analytical scope aligns with the IPCC (2006, 2019) definition of LULUCF fluxes, encompassing all GHG sources and sinks from US managed lands. We also broaden LULUCF to include N₂O and CH₄ emissions from agricultural soil management and rice methane for two reasons: (1) the USA uses a single model, DayCent, to jointly calculate carbon stock change and non-CO₂ fluxes on agricultural soils, and (2) previous studies identified agricultural soil N₂O emissions as the largest source of economy-wide NGHGI uncertainty (Ramírez et al. 2008; Winiwarter and Muik 2010; Petrescu et al. 2020), so including these inventory categories would likely impact our analysis.

We describe here the two components of our analysis:

• Uncertainty attribution: We quantify the contribution of each uncertainty element to the 95% CIs of all relevant LULUCF inventory categories.

• Omitted flux estimation: We provide initial estimates of known omitted fluxes, using literature review, expert input, and Tier 1 and 2 methods.

Uncertainty attribution

To identify sources of NGHGI uncertainty, we must first justify an uncertainty taxonomy tailored to the LULUCF NGHGI context. Based on the literature review described in SM Sect. 2, chapter 1.2, we define an uncertainty element as an individual input, parameter, model or submodel, and any instance of design-based sampling error. We refer to input, parameter, and model structure uncertainty collectively as model uncertainty, as distinct from sampling error. In some cases, we aggregate uncertainty elements into a group of inputs or parameters for ease of analysis and interpretation.

Given this taxonomy, we review methods for each LULUCF inventory category and identify all uncertainty elements. For inventory categories where it was possible to recalculate the central flux estimate given available data, we attribute uncertainty to each element using the contribution index method (Eq. 1).

Equation 1: Contribution index

$$Index\left(i,k\right)= \frac{Range\left(full,k\right)-Range(i,k)}{\sum_{j=1}^{J}Range\left(full,k\right)-Range(j,k)} \times 100$$

(1)

where

i = 1,…,J: refers to uncertainty element i

Range(full,k): is inventory category k 95% CI magnitude (97.5th quantile minus 2.5th quantile)

Range(i,k): is inventory category k 95% CI magnitude holding element i at its mean or point estimate

Index(i,k): is percentage contribution of element i to Range(full,k)

Other methods for uncertainty attribution have been utilized in the literature, including sensitivity analysis (McRoberts et al. 2016; Rypdal and Flugsrud 2001), uncertainty importance elasticities (Smith and Heath 2001; Winiwarter and Muik 2010), regression correlation coefficients (Peltoniemi et al. 2006; Winiwarter and Muik 2010), and Gaussian error propagation (Harmon et al. 2007; Phillips et al. 2000). We chose the contribution index method for its ability to account for full probability distributions, to allow for non-linear relationships between elements and model outputs and dependencies among uncertainty elements, and because we would be able to use previously published analyses for some inventory categories (Smith and Heath 2001; Ogle et al. 2003; Skog et al. 2004).

Where flux estimate recalculation was not possible, due to lack of access to data or methods, we use published uncertainty attribution results or, in the case of Tier 3 cropland and grassland fluxes, expert elicitation. US EPA recognizes expert elicitation as one method for NGHGI quality assurance and uncertainty analysis (US EPA 2002). We tailored US EPA (2002) NGHGI expert elicitation guidance to the objectives of our study (methods described in more detail below).

Uncertainty elements that we identified but were not able to quantify are listed in SM Table 2-1. Table 2 summarizes the uncertainty attribution methods used for each LULUCF inventory category.

Table 2 Uncertainty attribution methods for each GHG flux category

Omitted GHG flux estimation

Most of the omitted fluxes identified in this paper are already recognized in the US LULUCF GHG inventory as planned improvements. We identified additional omitted fluxes by reviewing IPCC (2006, 2019) guidance, by including prompts to identify omitted GHG fluxes in the cropland and grassland expert elicitation survey, and by prompting US LULUCF NGHGI inventory compilers to identify omitted GHG fluxes through direct communication.

For each identified omitted flux, we reviewed the literature to identify activity data and emission factors. The resulting omitted GHG flux estimates are meant to be useful only for purposes of prioritizing future work.

Methods by land use and inventory categories

We briefly summarize the methods used for each LULUCF inventory category here, with further details provided in the SM. Our analysis is based on the 2018 US NGHGI report, which covers inventory years 1990 to 2016 and which was the most complete inventory report available while the majority of our analytical work was completed. In the SM, we note any significant methodological updates in more recent US NGHGI reports, none of which meaningfully influences our findings.

Forests

Above- and belowground biomass in living and standing dead trees (SM Sect. 2, chapter 2.1): We recalculate the carbon stock change flux and 95% CI for above- and belowground tree biomass and standing dead trees (hereafter, tree biomass), accounting for uncertainty in nine groups of allometric model parameters (Table 2-2) as well as sampling error. We use Forest Inventory and Analysis (FIA) data and allometric models specific to eastern Texas as the basis for analysis to reduce Monte Carlo computational burden. Eastern Texas was chosen as a representative region for national forest carbon fluxes, encompassing both shrub species common in the western USA and hardwood and softwood species present in higher precipitation regions. We find that eastern Texas tree biomass exhibits similar proportional uncertainty to national uncertainty reported in the US NGHGI (see SM Sect. 2, chapter 2.1 for more detail).

Litter and soil (SM Sect. 2, chapters 2.2 and 2.3): Using literature estimates of mean litter carbon stocks by forest type (Domke et al. 2016), and the reported model prediction uncertainty for litter carbon stocks (US NGHGI 2018), we use Monte Carlo simulation to estimate the national 95% CI for litter carbon stock change. Similar methods were used for soils, accounting for model prediction uncertainty from estimating soil carbon stocks to 100 cm depth at a subset of FIA plots as well as the random forest model used to extrapolate soil carbon stock estimates to all FIA plots (Domke et al. 2017). A significant shortcoming of our approach for both litter and soil carbon pools is that it requires assuming covariance of carbon stocks between two time periods, because the US NGHGI does not report 95% CIs by forest carbon pool. For this reason, we provide sensitivity analysis for different levels of intertemporal covariance.

Non-CO₂ from forest fires (SM Sect. 2, chapter 2.4): We recalculate the CH₄ and N₂O emissions from forest fires and their respective 95% CIs, using Monte Carlo simulation to account for uncertainty from four input variables (burned area, fuel availability, combustion factor, and emission factor) using standard deviations reported in the US NGHGI (2018) and IPCC (2006).

Harvested wood products (SM Sect. 2, chapter 2.5): We modify contribution index results from Skog et al. (2004) to focus on inputs and parameters used in Skog (2008), which most closely aligns with US NGHGI (2018) methods.

Cropland and grassland

The US NGHGI uses consistent methods across many cropland and grassland inventory categories, so we collapse analysis across the two land uses. The US NGHGI uses Tier 3 methods on 78% of managed cropland and grassland soils, and Tiers 1 and 2 on organic soils, federal grasslands, shaley and gravelly soils, and minor crop types.

Carbon stock change, N₂O, and rice CH₄ on Tier 3 soils (SM Sect. 2, chapter 3.1): It was not possible to recalculate Tier 3 fluxes, due to National Resources Inventory (NRI) dataset confidentiality. Therefore, we use expert elicitation to identify the largest sources of uncertainty stemming from inputs and structure of the biogeochemical model DayCent as well as scaling NRI plot estimates to population area. The expert elicitation included prompts to identify primary research, model development and intermodel comparison, and data priorities for reducing cropland and grassland Tier 3 flux uncertainty. Participation in uncertainty attribution sections of the survey required knowledge of Century, DayCent, or similar biogeochemical soil models and IPCC GHG accounting guidance. Respondents were asked to confirm that they possessed this knowledge before completing the survey. Respondent expertise was concentrated in soil science (87%), biogeochemistry (67%), and the carbon cycle (67%); 53% worked in academia, 33% in government, and the remainder in NGO or private sectors. Details on the expert elicitation protocol and results are provided in the SM Sect. 2, chapter 3.1, and the full expert elicitation survey is available in SM Sect. 3.

Carbon stock change and N₂O in Tier 1 and 2 soils (SM Sect. 2, chapter 3.2): We apply contribution index results from Ogle et al. (2003) to 95% CIs reported in the US NGHGI (2018).

Non-CO₂ from grassland fires (SM Sect. 2, chapter 3.3): We recalculate 2014 CH₄ and N₂O emissions, the most recent year for which burned grassland area estimates are available, and follow methods similar to the forest fire inventory category.

Omitted cropland and grassland GHG fluxes (SM Sect. 2, chapter 3.4): We use IPCC (2006) default equations and literature emission factors to estimate carbon stock change in woody biomass and litter (USDA 2012; Udawatta and Jose 2011); non-CO₂ emissions from woody biomass in grassland fires (US NGHGI 2018; IPCC 2006); soil microbial CH₄ sink (Dutaur and Verchot 2007; Del Grosso et al. 2000); and select GHG sinks and sources on federal cropland and grassland (US NGHGI 2018).

Settlements

Carbon stock change in urban trees (SM Sect. 2, chapter 4.1): We recalculate the CO₂ flux and 95% CI attributable to carbon stock change in urban trees (Nowak et al. 2008; Nowak et al. 2013). We attribute uncertainty to all inputs (Table 2-30) using error propagation and contribution index methods.

Carbon stock change in yard trimmings and food scraps (SM Sect. 2, chapter 4.2): We recalculate CO₂ fluxes and 95% CIs attributable to yard trimmings and food scraps discarded in landfills (US NGHGI 2018; De la Cruz and Barlaz 2010), accounting for uncertainty from all inputs.

Omitted settlement GHG fluxes (SM Sect. 2, chapter 4.5): We estimate CO₂ emissions resulting from US settlement mineral soils, which is omitted from the US NGHGI due to lack of data, consistent with IPCC (2006) guidelines. Using Tier 1 methods and IPCC (2006) default values, we provide an initial estimate of this flux.

Wetlands

The US NGHGI (2018) indicates that there are 43 million hectares of wetlands in the USA, yet GHG fluxes are calculated for only 2.9 million hectares of wetlands. The omission is due to lack of data that would allow for designating non-coastal wetlands as managed (that is, wetlands directly created by human activity or areas where the water level has been artificially altered) (US NGHGI 2018). Due to this data gap, we were not able to estimate omitted wetland fluxes (SM Sect. 2, chapter 5).

Alaska, Hawaii, and US territories

Alaska, Hawaii, and US territories comprise nearly 20% of the total US land base (nearly all of this in Alaska), but they are not completely accounted for in the US NGHGI. The 2019 US NGHGI included forest carbon stock changes in interior Alaska for the first time, an area covering 24.5 million acres (9% of US managed forest area). We provide estimates for omitted fluxes in Alaska, Hawaii, and Puerto Rico (the largest US territory), based on IPCC (2006) guidance, emission data derived from the US NGHGI (2018, 2019), and literature review (SM Sect. 2, chapter 6).

Results

Uncertainty contribution results are reported as the uncertainty element’s contribution index value (%) multiplied by its respective inventory category 95% CI range (MMT CO₂e). We present the 10 largest sources of uncertainty for each land use category and then collectively show omitted GHG flux results. Complete results for all inventory categories and uncertainty elements are available in the SM.

Forests

The largest source of forest GHG flux uncertainty is design-based sampling error in estimating tree biomass carbon stock change (434.3 MMT CO₂e) (Table 3). Two groups of allometric parameters are the largest sources of uncertainty in estimating individual tree biomass (together, 131.9 MMT CO₂e), which govern the conversion of tree diameter and height to gross bole volume (volume coefficients) and the conversion of bole volume to biomass (wood and bark specific gravities).

Table 3 Forest GHG flux uncertainty elements

While we find that allometric volume coefficients are a large source of forest carbon stock change uncertainty, we were not able to find an empirical estimate of volume coefficient uncertainty. Sensitivity analysis of the coefficient of variation (5%, 10% (base case), and 20%) found that this assumption has large impacts on both the tree biomass 95% CI and the uncertainty contribution ranking of allometric parameter groups (SM Table 2-4).

Model uncertainty for soil and litter carbon stock change are substantial (together, 91.7 to 288.9 MMT CO₂e); we report a range for these pools to reflect sensitivity to carbon stock intertemporal covariance (SM Sect. 2, chapters 2.2 and 2.3).

Cropland and grassland

The DayCent model accounts for the vast majority of cropland and grassland soil carbon stock change, agricultural N₂O, and rice methane uncertainty (Table 4). DayCent model structure and parameters (including organic matter formation and decomposition; nitrification and denitrification; leaching, runoff, and volatilization) collectively contribute 117.2 MMT CO₂e, while DayCent inputs (including tillage, fertilization management, and manure and organic fertilizer application) contribute 222.0 MMT CO₂e. Input uncertainty is primarily driven by randomly assigning management activities to NRI plots consistent with county-level statistics (Ogle et al. 2010).

Table 4 Cropland and grassland GHG flux uncertainty elements

Settlements

Urban tree gross to net sequestration ratio contribution is an order of magnitude larger than any other settlement uncertainty element (Table 5). This uncertainty arises due to a majority of states lacking data on net urban tree growth rates, requiring use of a national average (Nowak et al. 2013).

Table 5 Settlement GHG flux uncertainty elements

Yard trimmings and food scraps carbon stock change inputs account for less than 12% of settlement GHG flux uncertainty, with negligible contributions from remaining fluxes (carbon stock change on drained organic soils and N₂O emissions from soil N additions).

Uncertainty attribution synthesis

Our findings suggest higher LULUCF uncertainty in the US NGHGI than is currently reported. While our recalculated uncertainty estimates generally align with reported values, two notable exceptions are forest carbon stock change and cropland and grassland Tier 3 fluxes, where we found 5–27% (with sensitivity to litter and soil carbon stock change uncertainty) and 94% larger CI ranges, respectively. Total LULUCF CI magnitude could be 18–35% higher than US NGHGI (2018) reported values (Fig. 1).

Fig. 1

Reported and recalculated confidence intervals (CI) by inventory category. Magnitude of one-direction CI as percentage of the point estimate is shown at the end of each bar. US NGHGI (2018) values for “LULUCF” reflect only inventory categories assessed in this paper and so is inconsistent with US NGHGI (2018) Table 1-5; “non-LULUCF CO₂” results are as listed in Table 1.5. “Forest carbon pools” (which includes tree biomass, soil, and litter) CI estimates are aggregated using error propagation to allow for comparison with NGHGI (2018) reported values. “Forest carbon pools” and “LULUCF” results show sensitivity to soil, litter, and tree biomass volume coefficients uncertainty attribution (all uncertainty contribution values in MMT CO₂e: soil carbon stock change (CSC) = *255.7, **81.2, ***81.2; litter CSC = *33.2, **10.5, ***10.5; tree biomass volume coefficient = *77.7, **77.7, ***16.9)

Higher cropland and grassland Tier 3 uncertainty can be directly attributed to the expert elicitation, which directed respondents to identify the uncertainty contribution from elements not currently accounted for in reported US NGHGI CIs, which ultimately included the two largest DayCent uncertainty elements (soil properties; leaching, runoff, and volatilization) (US NGHGI 2018).

It is less clear whether higher forest carbon stock change uncertainty can be attributed to our choice of analytical region (eastern Texas), including a larger number of uncertainty elements in our analysis, or other assumptions made in our analysis (e.g., intertemporal covariance for litter and soil carbon pools). Accounting for sensitivity to uncertainty contributions from soil and litter carbon stock change and tree biomass volume coefficient assumptions, our high (low) end uncertainty estimates for these elements result in 27% (5%) higher forest carbon pool CI compared to US NGHGI (2018) reported values.

A meaningful reduction in US LULUCF uncertainty would require addressing many of the largest elements simultaneously. No single element or element group would reduce the LULUCF CI by more than 10% except for tree biomass sampling error (Fig. 2a). A 50% reduction in LULUCF CI magnitude would require reducing tree biomass sampling error by at least 15%, and reducing contributions of all other uncertainty elements by at least 50% (Fig. 2b). The optimal uncertainty reduction approach depends on availability and costs of alternative methods, but this exercise illustrates the inevitable need to focus on forest sampling error, soil carbon modeling, and urban tree methods.

Fig. 2

Inventory uncertainty reduction potential. Percent reduction in LULUCF NGHGI 95% confidence interval (CI) magnitude (97.5% upper bound – 2.5% lower bound) given reduction in uncertainty contribution for each uncertainty element or element group. a LULUCF uncertainty reduction for each uncertainty element, holding all other element contributions constant. b Cumulative LULUCF NGHGI uncertainty reduction if element uncertainty contributions are sequentially reduced by 50%. “ < 10% cum. CI reduction” refers to uncertainty elements that, in aggregate, reduce LULUCF NGHGI CI magnitude by less than 10% if known with complete certainty. Forest soil model contribution is 255.7 MMT CO₂e

Omitted fluxes

In total, we find that net emissions of 123 MMT CO₂e could be omitted from the US NGHGI, with the majority occurring on croplands and grasslands (Fig. 3). The largest omissions are due to data gaps in Alaska, where grassland soil carbon stock changes (31 MMT CO₂e) and wetland soil carbon and methane emissions (41 MMT CO₂e) are not currently estimated.

Fig. 3

US NGHGI omitted GHG fluxes. “CSC” = carbon stock change. “Alaska” fluxes labeled as (A) wetland soil CH₄, (B) wetland soil CSC, (C) grassland soil CSC, and (D) agricultural soil management N₂O. “Omitted flux as % of NGHGI reported fluxes” is calculated by summing absolute values of all omitted fluxes by land use category and dividing result by sum of absolute values of all fluxes for that land use category as reported in US NGHGI (2018)

Emissions from settlement mineral soils are not included in the US NGHGI due to a lack of activity data and emission factors, a challenge that the IPCC acknowledges in allowing this omission as a Tier 1 method (IPCC 2006, 2019). We find that settlement mineral soils could emit 35 MMT CO₂e, assuming they are managed similarly to low input cropland (IPCC 2006, 2019).⁵ While the low input cropland emission factor may reasonably reflect dynamics in undisturbed lawns and parks, settlement soils undergo intensive disturbance at irregular intervals, driven by landscaping and land grading, building development, and impervious surface cover, which are unlikely to be captured by cropland emission factors. However, an emission factor based on Boston mineral soil emissions suggests that the omitted flux value could be much higher (Decina et al. 2016).

The US NGHGI does not currently account for indirect and direct N₂O emissions from federally owned croplands and grasslands with the exception of pasture, range, and paddock (PRP) sources. Assuming that federal croplands and grasslands emit direct and indirect N₂O at the same per-area rates as non-federal lands, net of PRP N₂O emissions, we find that this omission could reach 22 MMT CO₂e.

The largest omitted sink category is microbial methane sequestration in cropland, grassland, and forest soils (− 25 MMT CO₂e). However, we note that the soil methane sink is directly tied to methane’s atmospheric lifetime and is likely already incorporated to some extent in methane global warming potential (GWP) values. The IPCC (2006, 2019) does not yet provide guidance on these issues. If countries decide to include the soil microbial methane sink in NGHGIs, new methods may be needed to align inventory reporting with methane GWP estimates.

We do not provide error bars for these estimates to avoid suggesting precision—as described above, these values are generated using highly simplified assumptions about average GHG fluxes over large areas. Our estimates are meant only to provide a basis for prioritizing research and data collection.

Discussion

Comparison to other studies

Our results compare well with US NGHGI Approach 2 key category analysis, which ranks source and sink categories, as defined by UNFCCC common reporting format (CRF) guidelines, by their one-direction 95% CI magnitude (IPCC 2006, 2019). The top five LULUCF key categories as identified using Approach 2 encompass the largest uncertainty elements identified in Fig. 2 (US NGHGI 2018).

However, our analysis provides important additional detail. For example, “Net CO₂ Emissions from Settlements Remaining Settlements” is the second ranked key category, while our analysis finds that addressing DayCent model uncertainty would have a larger impact than focusing on urban trees. This inconsistency is due to the fact that the DayCent model is used across nine different CRF key categories. Thus, uncertainty attribution analysis can usefully focus on highly ranked CRF key categories, as long as cross-cutting uncertainty elements are recognized.

It is difficult to compare uncertainty attribution results across studies, since they vary widely in scope and structure. However, our findings are consistent with studies that suggest design-based sampling error outweighs allometric model uncertainty (Breidenbach et al. 2014; Ståhl et al. 2014; McRoberts et al. 2016), that forest soils are a large source of uncertainty (Peltoniemi et al. 2006; Monni et al. 2007b), and that N₂O emissions drive uncertainty in croplands and grasslands (Winiwarter and Muik 2010; Ramírez et al. 2008; Monni et al. 2007a; Petrescu et al. 2020).

Opportunities for inventory improvements

Countries looking to improve LULUCF GHG estimation methods can take advantage of existing research, data gathering, and model development targeting the largest uncertainty elements identified above.

Forest sampling error

Increasing the sampling rate or number of plots in existing forest inventories is a costly option for reducing sampling error. Rather, research has increasingly focused on using remote sensing data like LiDAR or radar to generate wall-to-wall forest biomass estimates (e.g., Blackard et al. 2008; McRoberts et al. 2016; Ma et al. 2021). Model-assisted estimators that utilize LiDAR and plot data have increased aboveground forest biomass precision by 2.5–6 times compared to plot-based simple random sample or post-stratified estimators (McRoberts et al. 2013; McRoberts et al. 2016; Gregoire et al. 2016). Historically, the necessary LiDAR and radar data has been costly to collect and only intermittently available over space and time, but new and planned global LiDAR and radar missions, including GEDI, ICE-Sat2, and NISAR, have the potential to greatly improve LULUCF monitoring precision and to help align aboveground biomass monitoring methods across countries (Duncanson et al. 2020; Babcock et al. 2018). Ongoing availability of LiDAR or radar data will be critical to ensure countries can sustain new LULUCF monitoring methods.

Care must be taken in comparing precision of plot-based and remote sensing-based methods. Countries with national forest inventories tend to use design-based or probability-based statistical inference to estimate forest carbon fluxes, assuming that uncertainty is a function of the probability of selecting a given sample (observations are considered constant). When using remote sensing-based models, analysts may instead choose model-based inference, assuming that uncertainty is driven by a population probability distribution (observations are realizations of a random variable) (McRoberts 2010). It is not valid to rank precision across the two methods due to different assumptions about the source of randomness (McRoberts et al. 2013). Inventory compilers are therefore encouraged to clarify inference frameworks used to ensure uncertainty reporting transparency.

Annually updated remote sensing data products can help address concerns that land cover and land use changes are not reflected in LULUCF flux estimates, a source of uncertainty that we were not able to evaluate in this paper due to data constraints. For example, the 2018 US NGHGI uses the 2011 National Land Cover Database (NLCD) to stratify eastern Texas forest by canopy cover. Though individual plots could capture disturbance after 2011, spatial weights would reflect only area disturbed prior to 2011. As a result, large changes in US forest GHG fluxes would not be reflected in the inventory for up to five years under current stratification methods. To address this issue, the USA has begun generating annual NLCD updates to more closely monitor land use change (LCMAP 2021, LCMS 2021).

Tree-level biomass estimation

We find a higher contribution from allometric model uncertainty compared to other studies (e.g., McRoberts et al. 2014; Breidenbach et al. 2014; Ståhl et al. 2014), possibly due to our assumption that allometric parameters are assigned by tree species or species group for each Monte Carlo iterate rather than to individual trees. This approach was chosen for its computational efficiency and mimics a high degree of positive covariance between individual trees of the same species or species group, but results in higher variance of forest carbon stocks across Monte Carlo iterates than studies that assume independence at tree-level.

Tree-level biomass estimates are an important input to remote sensing models and so will be key to inventory methods even as remote sensing data is increasingly utilized. Challenges to allometric model improvements include inconsistent methods in biomass measurement field studies (Weiskittel et al. 2015); a dearth of data and models for estimating belowground biomass (Russell et al. 2015); a lack of accounting for impacts of climatic variables on tree density and other allometric parameters over time (Clough et al. 2017); and a lack of species-specific or region-specific data and incomplete or non-random samples across studies (Jenkins et al. 2003).

In an effort to address some of these challenges, the US Forest Service has compiled the Legacy Tree Data platform, which contains over 15,000 individual tree biomass measurements (Radtke et al. 2015). However, to address the climatic dependency of tree variables and to fully address the data limitations described above, ongoing data collection and targeted research programs are required.

Cropland and grassland fluxes

Our expert elicitation survey asked respondents to rank research, modeling, and data priorities, as identified in the literature, for reducing uncertainty in cropland and grassland Tier 3 GHG flux estimates (SM Table 2–18).

Survey respondents noted that they were keen to have more empirical data in order to improve and validate existing soil models (Schmidt et al. 2011; Spencer et al. 2011). They acknowledged the difficulties in modeling such a complex system but noted that more data is the primary way to help reduce both input and structural uncertainty. For example, the NRI plot system, which provides key inputs to DayCent, could form the basis of a national soil carbon monitoring network, similar to FIA plots for forests. The US NGHGI notes that the US Department of Agriculture (USDA) is developing a national soil monitoring network (US NGHGI 2018), but it is unclear the extent to which this framework will address limitations identified in this study—particularly, the input uncertainty driven by lacking model output (GHG fluxes) and model input observations at the same plots.

Survey respondents also indicated that increased collaboration among model developers would help refine soil carbon flux predictions (Paustian et al. 2016; Schmidt et al. 2011). Increased intermodel comparison, model validation, and collaboration were highly ranked as opportunities to reduce uncertainty (Brevik et al. 2015; Stockmann et al. 2013).

Application to other countries

Other countries with similar land cover and NGHGI methods can use US-based uncertainty attribution to inform priorities for further analysis. For example, most of the world’s forest area is now covered by strategic forest inventories, with many countries utilizing statistical sampling methods similar to the USA (McRoberts et al. 2010). Large forested countries continue to develop systems to increase precision and accuracy of forest carbon stock estimates, particularly in response to REDD + financing programs (Brazil NC4 2020; Tewari et al. 2020; Zeng et al. 2015). For example, as part of the Estimativa de biomassa na Amazonia (EBA) program, Brazilian researchers are working to integrate forest plot data, allometric models, and remote sensing (both LiDAR and Landsat) data to estimate landscape-scale aboveground forest biomass (INPE 2021). Many of the same uncertainty elements described above are relevant to countries developing such systems.

There is more international heterogeneity in non-forest flux estimation methods, with many non-Annex I countries omitting these inventory categories entirely (Smith et al. 2020). Other countries may use results from this paper to inform priorities for expanding inventory coverage. Several of the omitted fluxes identified here will be relevant for all other countries, given current IPCC (2006, 2019) inventory guidance, including the soil microbial CH₄ sink and settlement mineral soil fluxes.

Conclusion

Many countries have deprioritized NGHGI uncertainty estimation and reporting due to lack of data and programmatic resources, as well as the complexity of uncertainty methods. As Brazil indicated in their Third National Communication (2016), “Quantifying uncertainty for individual data items is as or more difficult to assess as the actual information sought.” Countries are likely to prioritize improvements in LULUCF accuracy by increasing the use of Tier 3 methods and updating Tier 1 and 2 methods with the most recent science (Yona et al. 2020). However, investments in uncertainty estimation and transparency will also be required as more complex methods are adopted.

NGHGI LULUCF uncertainty is a challenge for many major emitting countries and, for some, including the USA, is large enough that planned LULUCF emission reductions fall within the margin of estimation error. The analytical framework suggested here is one approach that governments can use to both transparently report uncertainty estimation methods and to identify opportunities for improving NGHGI accuracy and precision, with a view to increasing international confidence in NDC emission reduction progress.

Using the USA as a case study, we detail the contribution of over 90 LULUCF uncertainty elements and omitted fluxes to uncertainty and bias in the US NGHGI. Most inventory uncertainty is driven by a small set of elements distributed across forestry, cropland and grassland, and settlement land use categories. Omitted fluxes could account for up to 13% of the current LULUCF inventory on an absolute value basis, primarily driven by CO₂ and CH₄ emissions in Alaska and urban mineral soils. Other countries can use these results to inform initial priorities for further analysis, particularly those using similar NGHGI methods or those that plan to take up similar methods in the future.