1 Introduction

Normalization is an optional step of a life cycle assessment (LCA), supporting the interpretation of the results of the characterization in terms of relative environmental relevance of the impacts. In fact, normalization allows the LCA practitioner expressing results after characterization using a common reference impact (Laurent at al. 2011), and it may be particularly of help if results need to be communicated to decision makers in business and policy contexts. Normalization factors (NF) are calculated as results of regional/global inventories of emission and resources characterized through impact assessment methods. Several modelling choices and assumptions are needed for building the underlying inventory and this may imply high uncertainties of the final figures.

Uncertainties evaluation and sensitivity analysis in life cycle assessment, from inventory to impact assessment, are increasingly discussed in literature (e.g. Hong et al. 2010; Clavreul et al. 2012, 2013; Imbeault‐Tétreault et al. 2013; Wei et al. 2015). Moreover, review articles on uncertainty have been published by various authors (Heijungs and Huijbregts 2004; Heijungs and Lenzen 2014; Lloyd and Ries 2007).

Inaccurate normalized data have been recognized as a possible source of uncertainty (Huijbregts et al. 1998; Ciroth et al. 2004). As discussed by Heijungs et al. (2007), both normalization factors and process inventories generally suffer from incompleteness due to a lack of emission data and/or characterization factors. This leads to biases which, in turn, might lead to over or underestimation of the normalized results for some impact categories, with severe problems in using normalized scores (Heijungs et al. 2007). Indeed, moving from inventory to characterization, up to normalization and weighting, may increase uncertainty as the number of sensitivity coefficients which have to be taken into account increases as well (Hejiungs 2010). Nevertheless, the uncertainties possibly related to normalization factors are rarely discussed and tested ((e.g. Hung and Ma 2009) which included normalization in the uncertainty assessment) not even in papers focusing on normalization only (e.g. Lautier et al. 2010). To date, a quantitative uncertainty and variability assessment due to the normalization step has not yet been performed.

Sala et al. (2015) presented a methodology for an extended inventory underpinning normalization factors for Europe in 2010 where key choices on sources and flow mapping as well as methodological assumptions for building proxy indicators have been detailed. From the results, few flows dominate the results of many impact categories (e.g. NOx) and the qualitative assessment of the robustness of some impact category is low (e.g. water depletion). The present work aims at quantifying the influence of different methodological choices on the final normalization factors by conducting uncertainty and sensitivity analysis on them.

The paper is organized as follows: after presenting an overview of the sources of uncertainties, the methodological section illustrates the sensitivity analysis conducted on the inventory used for the calculation of the normalization factors for EU27 (e.g. the total of emissions and resource extractions in the year 2010), whereas in the results and discussion section the outcome and the implications on the characterized factors (i.e. normalization factors) are presented.

2 Sources of uncertainties in the calculation of normalization factors

Uncertainties in the calculation of the normalization factors in LCA may be related to different sources; these are listed in Table 1 alongside of the LCA steps.

Table 1 Possible sources of uncertainties in the normalization factors for each step of the LCA-based calculation and factors included in this analysis
Full size table

The specific sources of uncertainty, i.e. factors of uncertainty, which were analyzed in this study are reported in Table 1 and listed below by groups:

  • F1: the selection for the sources of data amongst statistical database for NOx, SOx, NH3, CO, PM2.5/PM10 and water withdrawals (from F1.1 to F1.6);

  • F2: the classification of environmental statistics as ILCD elementary flows for NOx, and SOx (F2.1 and F2.2);

  • F3: the classification of PM2.5 and PM10;

  • F4: the assumptions made on the typology and height of emission sources for NOx, SOx, NH3, CO and PM2.5/PM10 (F4.1, F4.2, F4.3, F4.4, F4.5);

  • F5: the use of regionalized characterization factors (CFs) for water depletion only.

The uncertainty factors above refer to specific methodological choices which are performed when calculating normalization factors, and were selected because of their expectedly high effect on the calculation of the normalization factors performed by Sala et al. (2015). This is either because of their effect on the elementary flows contributing the most to the normalization factors developed by Sala et al. 2015 or because of the low robustness of the underlying impact assessment models (e.g. water depletion).

For each of the factors listed above there is at least an alternative approach which can be adopted for calculating the normalization factors, in addition to the one adopted by Sala et al. (2015), being still coherent with the ILCD recommended impact assessment models. The existence of such alternatives generates what could be defined as a methodological uncertainty.

The impact categories which are affected by the uncertainty factors introduced above are the following: acidification (ACID); eutrophication terrestrial (ET); eutrophication marine (EM); photochemical ozone formation (POF); respiratory inorganics/particulate matter (RIPM); and water depletion (WD). Not all sources of uncertainty which affect the normalization factors calculated by Sala et al. (2015) were assessed in this study. We did not assess some of the general uncertainties associated with normalization factors identified in Table 1: (i) the definition of the system boundaries; (ii) the estimation of elementary flows not included international statistics (see Sala et al. 2014) and the techniques applied for filling data gaps; and (iii) the underpinning impact assessment models and related characterization factors. In order to include the latter point, separate uncertainty and sensitivity analysis should be conducted by models’ developers on the underpinning models of estimation, such as those used for toxicity-related impact categories (multimedia models for characterization, emission estimating model for heavy metals and pesticide, etc.) and made available to users. However, this is not the case for the majority of impact assessment models used, and therefore it is beyond the aim of the present study. The sources of uncertainties which are included in this analysis are further detailed in the following sections. Details of calculations and underpinning data are reported as Electronic Supplementary Material and references in the following text (e.g. Table S1, Electronic Supplementary Material).

2.1 Sources of uncertainty in the inventory

2.1.1 Selection of the data sources (F1)

Sala et al. (2015) proposed a hierarchical approach to the selection of data sources. The order of preference was the following: (i) officially reported data provided by EU and international bodies (e.g. Eurostat, FAO, OECD etc.), based on agreed models, methods and standards, with documented metadata and periodical quality checks. 2. Activity-based estimations, derived as ‘activity data times emission factor’; (ii) activity data were from officially reported data, emission factors were based on scientific literature, grey literature (e.g. sectorial reports) and available life cycle inventories (LCIs); (iii) statistical proxies (time, flows) when the correlation is statistically significant; (iv) speculative assumption(s), based on cause-effect models, not statistically tested.

For what concerns NOx, NH3, SOx and CO the priority in selecting the data sources has been set as follows: UNFCCC (2013)> EMEP_modeled (EMEP/CEIP 2013b) > EMEP_reported (EMEP/CEIP 2013a) > EDGARv4.2 (EC—JRC & PBL 2011). This is coherent to decisions by a team of experts from EC-JRC, Netherlands Environmental Assessment Agency (PBL), United Nations Framework Convention on Climate Change (UNFCCC), European Monitoring and Evaluation Programme (EMEP), as reported in EC-JRC (2011b) on the basis of ECE (2010). The data sources for those substances are then: UNFCCC for CO and NOx (reported as NO2) and the EMEP/CEIP database for NH3, and SOx (reported as SO2) (EMEP/CEIP 2013b).

In Tables S1 and S2 (Electronic Supplementary Material) are reported, the data retrieved from different sources for the elementary flows NOx, NH3, SOx, CO, PM2.5 and PM10 and their relative difference between what was selected by Sala et al. (2015). In Table S3 (Electronic Supplementary Material) the statistics on water withdrawal retrieved from different sources (Sala et al. 2015 and Vandecasteele et al. 2014) are provided. Within the uncertainty and sensitivity analysis carried out in this paper, all of the data sources reported in Tables S1, S2, S3 (Electronic Supplementary Material) were given the same likelihood.

2.1.2 Classification of data as LCI elementary flows (F2)—NOx and SOx

Both nitrogen and sulphur oxides are pollutants which contribute to several impact categories of those recommended by the ILCD; SOx contributes to acidification, photochemical ozone formation and respiratory inorganics, and NOx contributes to terrestrial and marine eutrophication in addition to the ones listed for SOx. Nevertheless, the ILCD list of recommended characterization factors does not provide factors which are specific for these two groups of substances. Therefore, the attribution of a characterization factor to these groups of substances is a methodological choice to which the impact categories listed above are expected to be sensitive to. This is due to the fact that NOx and SOx are major contributors to the total EU27 figures (Sala et al. 2015) as well as to the fact that currently available statistics do not differentiate between NO2 and NO, (or sulphur oxides) (only NOx totals expressed as NO2 equivalents in mass is available), in combination to the fact that the characterization factors might differ sensibly amongst different oxides within the same group (e.g. the CF for NO can be twice as the one for NO2).

Sala et al. (2015) mapped the group of substances SOx and NOx into NO2 and SO2 for the calculation of the normalization factors. It is important to assess how sensitive is the result to the mapping of the groups of substances NOx and SOx into a specific oxide. In order to assess the uncertainty associated to this methodological choice, we assumed that the mapping of the groups of substances into a specific elementary flow would be equally possible. An additional source of uncertainty which is not included in the current assessment, is represented by the fact that for several impact categories the CFs are generally calculated and made available by models developers only for NO2 and SO2, and that the factors for NO, SO and SO3 were approximated by the EC-JRC (2011a) on the basis of molecular weight and charge. Similarly, the breakdown of these groups of substances into individual substances and the disaggregation of reported emissions into e.g. NO and NO2 are beyond the purpose of this paper.

2.2 Sources of uncertainty in the characterization

2.2.1 Classification of substances (F3)—PM2.5 and PM10

The model recommended by the ILCD (EC-JRC 2011a) for assessing impacts associated to respiratory inorganic substances and particulate matter (Humbert 2009) characterizes both PM2.5 and PM10. However, the characterization factor associated to PM2.5 is estimated by the model developer by assuming that all impacts originating from the exposure to PM10 are due to PM2.5 (Humbert 2009). This implies that only one of the two inventoried group of substances should then be characterized, as PM10 contains PM2.5, being the latter a fraction of the former. The selection of which of the two elementary flows between PM10 and PM2.5 to account and characterize further is therefore a clear source of methodological uncertainty. In this assessment, we assumed that both options are equally viable.

2.2.2 Specification of the emission compartments (F4)

The lack of detail regarding the specific emission compartment into which pollutants are emitted represents another source of uncertainty which could potentially affect the normalization factors.

Some of the ILCD recommended impact assessment models provide different CFs for substances emitted into air, depending on to the height of the emission source. In fact, emissions to air can be further specified into different flows: ‘emissions to lower stratosphere and upper troposphere’, ‘emissions to non-urban air or from high stacks’, ‘emissions to urban air close to ground’ and ‘emissions to air, unspecified’. The detail regarding the height of emission sources is generally not found in country-scale statistics; therefore, Sala et al. (2015) applied generic CFs (i.e. ‘emissions to air, unspecified’) in order to characterize the inventory of EU27 total emissions. For the impact categories ACID, ET, EM, and POF analyzed in this paper, such detail is irrelevant as it does not lead to differences in the CFs for any of the pollutants included in the analysis, whereas this aspect is expected to be of relevance for RIPM (see Tables S4 to S8, Electronic Supplementary Material). In fact, PM2.5 emitted to air has different CFs ranging between 3.3, 1 and 0.35 kg PM2.5eq/kg, respectively, for emissions close to ground in urban areas, generic emissions and emissions to non-urban air or from high stacks. With the purpose of the assessment of uncertainty of the normalization factors, all the three options listed above were considered to be equally viable, given the fact that the distribution of emissions of particulate matter is unknown.

2.2.3 Spatial differentiation of characterization factors (F5)

Increasingly, life cycle impact assessment models provide spatially resolved characterization factors. This implies that the selection of generic factors vs spatially resolved ones may lead to uncertainties associated to neglecting the spatial variability. We tested these aspects for the impact category of water depletion. According to the ILCD recommendations (EC-JRC 2011a), water depletion can be quantified by either applying a generic characterization factor or by applying spatially resolved factors at country level. In fact, the method recommended in the ILCD (Frischknecht et al. 2008) provides spatially resolved CFs for 29 OECD countries. In Sala et al. 2015 the ‘OECD average scarcity’ factor was used for characterizing impacts associated with water scarcity. This choice was made in order to guarantee consistency with the other impact assessment methods, for which only a limited set of regionalized flows is available.

However, as reported in Table S3 (Electronic Supplementary Material), the CFs for water stress present a wide variability across the EU countries. Characterization factors are not available for a number of countries (see Table S3, Electronic Supplementary Material) and for this reason two alternative extrapolations were adopted: (i) attributing the characterization factor ‘OECD average scarcity’ to the missing factors and (ii) attributing characterization factors by analogy of climatic conditions. Both choices have been estimated in this analysis in order to assess the relevance thereof on the final result.

3 Uncertainty and sensitivity analysis

Uncertainty and sensitivity analysis (SA) have been conducted testing different modelling choices applied for calculating the normalization factors developed by Sala et al. 2015.

As defined by Campolongo et al. (2011 p. 978), ‘uncertainty and sensitivity analyses study how the uncertainties in the model inputs (X1, X2, . . ., Xk) affect the model’s response Y, Y = Y (X1, X2, . . ., Xk). Uncertainty analysis quantifies the output variability while Sensitivity Analysis describes the relative importance of each input in determining this variability’. As reported by Pannell (1997), SA can be useful for a range of purposes, such as testing the robustness of the results of a model or system, reducing uncertainty and enhancing communication from modellers to decision makers. This is possible because SA helps in increasing the understanding of the relationships between input and output variables in a system or model and has been increasingly demonstrated in scientific arenas and recognized by international institutions (Campolongo et al. 2011). SA methods can be generally classified into local and global methods (Campolongo et al. 2011; Wei et al. 2015). While local sensitivity measures, also called one-at-a-time (OAT) methods, assess how uncertainty in one factor affects the model output keeping the other factors fixed to a nominal value (Campolongo et al. 2011), global sensitivity measures (GSM) assess the effects of uncertainty factors while being changed simultaneously. The advantage of global sensitivity analysis is that it is able to take into account interactions amongst factors, whereas OAT methods cannot (Campolongo et al. 2011; Wei et al. 2015). As reported by Wei and co-workers (Wei et al. 2015), in LCA, most case studies are done with OAT methods (Bala et al. 2010; Heijungs et al. 1994), whereas global methods are rarely applied (see Padey et al. 2013; Wei et al. 2015).

In order to assess the effect of the methodological choices described in Table 1 on the normalization factors, the SA has been performed by means of the Sobol global sensitivity indices (Sobol 1993, 2001). According to Saltelli et al. (2010 p.159), variance-based methods such as the Sobol indices, ‘are the computer experiment equivalents of the experimental design’s analysis of the variance of an experimental outcome (Archer et al. 1997). Unlike experimental design, where the effects of factors are estimated over levels, variance based methods look at the entire factors distribution, using customarily Monte Carlo methods’.

The analysis of uncertainty and sensitivity through variance based method implies the following steps:

  1. 1.

    Definition of the set of uncertainty factors which are in input to the model, including their probability distribution;

  2. 2.

    Generation of a sample set of combinations of uncertainty factors, accordingly to a specific design (e.g. random, quasi-random, etc.);

  3. 3.

    Evaluation of the model’s outputs for the generated sample;

  4. 4.

    Calculation of Sobol’s sensitivity indices consistently with the sample design via Monte Carlo-based estimators.

The following sections details how the steps above were developed within the current analysis.

3.1 Step 1: Definition of the uncertainty factors and their probability distribution

All the possible alternatives configurations for the sources included in Table 1 were assumed to have equal probability, simulating a situation of ‘complete ignorance’ relative to the plausibility of probability distribution of the input factors (e.g. although it is unlikely that all the emissions of PM2.5 reported in statistical datasets are occurring in urban areas, this has been assumed to be equally probable to the other alternatives listed in Table 1). Therefore, the probability distribution for each of the factors is a discrete distribution, with a number of points equal to the number of alternatives for that factor (see Table 1 and Electronic Supplementary Material).

3.2 Step 2: Generation of a sample set of combinations (input)

The sample set of combinations has been generated within the software environment SimLab v2.2 (Tarantola 2005), selecting the Sobol’s sampling method (Sobol 1993, 2001). The Sobol’s sampling method was selected as it maximizes the computational efficiency of the exploration of the space of input factors. A sample of 32,768 combinations was generated; it was assumed to be of a sufficient size considering the number of factors included in the analysis (i.e. 15). The sample set is reported in Table S10 (Electronic Supplementary Material). Each of the factors can assume a value between 1 and the number of its alternative states. For instance, factor F11 can assume values from 1 to 5, as NOx can assume 5 alternative values according to the data source selected. This information is thus translated in the next step, ‘evaluation of the model’s outputs’, so that the configuration of factor F11 is translated into the corresponding value for emissions of NOx (see Table S1, Electronic Supplementary Material) e.g. if F1.1 = 1 therefore the value of NOx which is reported by the first data source listed in Table S1 (Electronic Supplementary Material) (i.e. Sala et al. 2015), whereas if F1 = 5 the fifth data source (i.e. EDGARv4.2) is selected. The same procedure applies to all factors.

3.3 Step 3: Evaluation of the model’s outputs

The model’s outputs (i.e. the normalization factors for each of the considered impact categories) were evaluated for each of the 32,768 samples generated in the previous step. This is done by converting the value of the factors into the corresponding quantities (both inventory values and characterization factors) which can be used for the calculation of the normalization factors. The calculation of the normalization factor consists of the sum of the inventory flows multiplied by the respective characterization factors. This operation is performed for each of the combinations of factors generated in the previous step (see Table S11, Electronic Supplementary Material).

3.4 Step 4: Calculation of the Sobol’s sensitivity indices

The calculation of the Sobol’s sensitivity indices was performed within the software SimLabv2.2. The software allows importing the outputs of the model’s calculation and supports the estimation of the Sobol’s sensitivity indices. In this paper, the first order and total order sensitivity indices were calculated (see Saltelli et al. 2010; Wei et al. 2015). The first order index (Si) for factor i-th provides the estimation of the variance explained by the i-th without considering its interactions with other factors, whereas the total order index (STi) for factor i-th provide the estimation of the variance explained by factor i-th and its interaction with other factors.

4 Results and discussion

4.1 Analysis of the model’s outputs

A set of statistics (arithmetic average—AVG, median—MED, minima and maxima—MIN and MAX, absolute and relative standard deviation—STD and RSTD, mean absolute deviation from the median—MAD and its relative value—RMAD) was calculated on the probability distributions of the normalization factor of each impact category. The results were compared to the normalization factors calculated by Sala et al. (2015) (see Table 2). The model’s output obtained after step 3 was plotted as box plot (Tukey 1977) in the Matlab Mathworks® environment, so to compare their spread over a central value (Fig. 1). In order to compare different impact categories together in Fig. 1, each of the model’s outputs was divided by the average value of the normalization factor of the respective impact category. Therefore, the value 1 in the y-axes of Fig. 1 represents the arithmetic average for each of the impact categories reported in the chart. Additionally, the median, average and the values Sala et al. (2015) were plotted in Fig. 1 as red lines, blue crosses and green asterisks, respectively. In Fig. 1, the central mark is the median, the edges of the box are the 25th and 75th percentiles and the whiskers extend to the most extreme data points not considered outliers. Points are drawn as outliers in the box plot and marked as red crosses if they are larger than q3 + w(q3–q1) or smaller than q1–w(q3–q1), where q1 and q3 are the 25th and 75th percentiles, respectively, and w is set by default as 1.5.

Table 2 Normalization factors (NFs) from Sala et al. 2015, measures of central tendency (arithmetic average and median), measures of dispersion (maximum and minimum, standard deviation, mean distance from the median, relative standard deviation and relative mean distance from the median), ratios between arithmetic average and NFs from Sala et al. (2015) and between median and NFs from Sala et al. (2015)
Full size table
Fig. 1
figure 1

Box plot of the normalization factors calculated for impact categories: ACID acidification, ET eutrophication terrestrial, EM eutrophication marine, POF photochemical ozone formation, RIPM respiratory inorganics, WD water depletion. The following statistics are shown in the chart: arithmetic average value (blue cross), median value (red line) and confidence boundaries (black lines). The green asterisks are the normalization factors calculated in Sala et al. (2015)

Full size image

4.2 Analysis of the sensitivity indices

The sensitivity indices of first and total order were calculated for the uncertainty factors included in the analysis, for each of the impact categories. The detailed results are reported in Table S12 (Electronic Supplementary Material). The value of the interaction of the i-th factor with the others has been calculated by subtracting the value of the total Sobol index for factor i-th by the value of the Sobol index of first order for the same factor.

As the Sobol’s indices express the share of variance explained by each of the factor, the resulting values numbers can be read also as relative percentage of variance explained. When models are non-linear, the total order indices of the factors sum up to values higher than 1 and the difference between the variance explained by first order sensitivity indices and the total order indices quantify the interaction. In Table 1 are reported the shares of variance explained by the different factors, including the overall interaction. The SimLab v2.2 software has some limitations to the accuracy of the estimation of the Sobol’s indices, which, according to the authors of this paper can reach up to the value 0.05; therefore such errors should be taken into account when making comparisons.

4.3 Analysis by impact category

4.3.1 Acidification (ACID)

The uncertainty associated with the normalization factor for the impact category ACID can be considered medium to low as the maximum and minimum values observed for the distribution are roughly +30 and −20 % of the arithmetic average, respectively (see Table 2). Moreover, the arithmetic average and the median are very close one to each other and the size of the box is small i.e. the 25th and the 75th percentiles of the distribution are not too distant from the average and median values (see Fig. 1 and Table 2). The measures of dispersion relative standard deviation (RSTD) and relative mean difference from the median (RMAD) are relatively low, being 10 and 8 %, respectively (see Table 2). The normalization factor calculated by Sala et al. (2015) reported in Table 2 is roughly 12 % lower than the arithmetic average and the median (see Table 2). As reported on Fig. 2 and Table 1, uncertainty factors F1—‘selection of the sources of data’ and F2—‘classification of data as LCI elementary flows’ explain, respectively, 52 and 48 % of the variance associated with the normalization factor of the impact category acidification. In particular, the selection of the elementary flow to which to map the group of substances NOx i.e. NO or NO2 (factor F 2.1), explains the highest share of variability (43 %), whereas the selection of the data source on NH3 (F1.3) accounts for 35 % of the variance (see Fig. 3). Sala et al. (2015) defined a hierarchy for selecting data sources according to some rules, leading to the selection of the most robust and, therefore, most likely dataset values. Instead, in this analysis all data sources are assumed to be equally probable and this has the effect of inflating the uncertainty related to the normalization factor for acidification, as it is most likely that the real value of emissions is closer to the inventory value used by Sala et al. (2015). Instead, no information is available regarding the level of detail about the speciation between NO and NO2, therefore, this can be considered a source of uncertainty not currently assessed in the calculation performed by Sala and co-workers. Because of the reasons explained above, the normalization factor calculated by Sala et al. (2015) for the impact category acidification can be considered as relatively robust.

Fig. 2
figure 2

Share of the variance explained by each uncertainty factor (F1: F5) and by the interactions amongst them

Full size image
Fig. 3
figure 3

Share of the variance explained by each of the specific uncertainty factors and by the interaction amongst them

Full size image

The use of regionalized characterization factors for this impact category was not investigated; therefore, the uncertainty affecting normalization factors for acidification is likely to be underestimated. Further refinements of this analysis should look at this aspect as well.

4.3.2 Eutrophication—terrestrial (ET)

The normalization factor for the impact category terrestrial eutrophication calculated in this paper is characterized by a relatively low dispersion around a central value. This can be observed in Fig. 1 as the size of the box is relatively small, and therefore, the majority of the values is not extremely dispersed. This is confirmed by the measures of dispersion RSTD and RMAD are relatively low for this distribution, being equal to 13 % of the average and 10 % of the median, respectively. The majority of the uncertainty observed for this impact category (66 %) is explained by the classification of data on NOx as elementary flows (NO or NO2), while the remaining 32 and 2 % are due the choice of the data source for NH3 and NOx, respectively. The normalization factor calculated by Sala et al. (2015) for this impact category is around 18 % lower than the arithmetic average (see Table 2), meaning that Sala and co-workers might have underestimated actual impact on terrestrial ecosystems due to eutrophication. However, this holds true only if all data sources are assumed to be equally likelihood to the real value. Instead, Sala et al. (2015) documented that a proper hierarchy of data sources can be established and therefore, the main uncertainty factor which remains to be addressed in Sala et al. (2015) is the translation of NOx emissions into NO and NO2. Although the difference in absolute terms is not very high, still a better refined inventory for NOx emissions would allow for better estimation of the NF.

The use of regionalized characterization factors for this impact category was not investigated; therefore the uncertainty affecting normalization factors for terrestrial eutrophication is likely to be underestimated. Further refinements of this analysis should look at this aspect as well.

4.3.3 Eutrophication—marine (EM)

Similarly to what discussed for ACID and ET, the normalization factor for marine eutrophication is sensitive to factor F2.1. In fact, the majority of the uncertainty observed for this impact category (97 %) is explained by the classification of data on NOx as elementary flows (NO or NO2), while the remaining 3 % by the choice of the data source for NOx emissions to air. The dispersion of the distribution of normalization factor values is relatively low, as RSTD and RMAD score, respectively, 10 and 11 %. The normalization factor calculated for this impact category by Sala and co-workers is very similar the arithmetic average and the median value of the distribution, differing by 12 and 5 % (see Table 2) from the former and the latter, respectively.

4.3.4 Photochemical ozone formation (POF)

According to Fig. 1, POF presents the lowest uncertainty associated to its average normalization factor (RSTD = 4 %, RMAD = 2 %, see Table 2). However, not all of the relevant uncertainty sources were analyzed for the normalization factor of POF, as the only uncertainty factors input data on CO, NOx and SOx which led to some variability of the outputs (67, 32 and 2 %, respectively—see Table 1), whereas it is well known and documented in Sala et al. (2015) that non-methane volatile organic compounds significantly contribute to this impact category as well. No uncertainty is associated with the mapping of group of substances (factors F2.1 and F2.2) when using the ILCD characterization factors as the CFs for NOx and SOx are the same within the two groups of substances for this impact category for the ILCD recommended impact assessment model. However, this might vary significantly according to other models, such as in the case of Jenkin and Hayman (1999) and Derwent et al. (1998), for which NO and NO2 have an antagonistic role.

4.3.5 Respiratory inorganics/particulate matter (RIPM)

As it is possible to note from Fig. 1 and Table 2, RIPM is the impact category showing the highest uncertainty. In fact, the values of normalization factors span from a maximum 3.15 times higher than the arithmetic average, to a minimum equal to 0.36 times the arithmetic average. The median value is 67 % of the arithmetic average and the measures of dispersion are very high, 77 % for RSTD and 71 % for RMAD. Nevertheless, the normalization factor calculated by Sala et al. (2015) for this impact category is very close to the arithmetic average calculate in this analysis (see Table 2). The variability which characterizes the normalization factor of this impact category is almost entirely explained by the classification of PM2.5 or PM10 (F3) (23 %), in combination with the height of the emission source for PM (F4.5) (36 %), as well as their interaction (40 %), whereas only 1 % of the overall uncertainty is explained by the selection of the data source (see Tables 1 and S12). Other sources of uncertainty are estimated as negligible for this impact category. As described in the respective section, PM2.5 was selected by Sala et al. (2015) for calculating the normalization factors, as this choice is consistent with the assumptions underlying the impact assessment model (Humbert 2009). Additionally, such approach was recommended by the method developer himself (Humbert 2014). But no information on the height at which the emissions of the pollutants contributing to RIPM could be accessed and the characterization factor ‘emission to air, unspecified’ was used by Sala and co-workers. Therefore, better inventories specifying emission sources and location should be then developed in order to improve the precision and the robustness of normalization factor for respiratory inorganics. This is of paramount importance given the significant correlation between residential exposure and health impacts (Raaschou-Nielsen et al. 2013).

4.3.6 Water depletion (WD)

Water depletion shows high variability of results, with a maximum 1.4 times higher than the arithmetic average and the minimum equal to 0.25 times the average, and RSTD and RMAD being equal to 52 and 29 %, respectively (Table 2). The use of the generic factors ‘OECD average scarcity’ for characterizing water withdrawals calculating normalization factors as done by Sala et al. (2015) results in a normalization factor 5.4 times lower than the one obtained by applying regionally differentiated characterization factors (Table 2). The normalization factor calculated by Sala et al. (2015) for water depletion is proven to be very sensitive to the choice of regional characterization factors instead of average ones, as almost 100 % of the observed uncertainty is due to this choice (see Table 1). Therefore, better normalization factors could be calculated by making use of regionalized CFs, no matter whether the underlying statistics on water withdrawals normalization inventory is based on the data collected by Sala et al. (2014, 2015) or by Vandecasteele et al. (2014).

4.3.7 Additional sources of uncertainty not covered in the analysis

Some of the uncertainty factors listed in Table 1 were not tested within this work, e.g. uncertainty associated with the impact assessment models, although they might play a significant role. For instance:

  1. (i)

    Estimation of missing characterization factors: if the values for the specific molecules composing e.g. NOx or SOx are not provided by models’ developers, the factors for NO, NO2, SO, and SO2 SO3, should be approximated as done in EC-JRC (2011a), simply through molecular weight and charge. Indeed this might entail some errors. To judge whether this approximation is good enough is outside the scope of this paper, anyhow this aspect should be taken into account, together with the limitations regarding the robustness of the impact categories.

  2. (ii)

    Consistency between models and inventories’ system boundaries. The models underlying the characterization factors might have system boundaries which are not necessarily consistent with the data statistics collected for the inventory.

  3. (iii)

    Use of marginal or average characterization factors. ILCD recommended LCIA methods might refer to marginal characterization factors or average ones (Huijbregts et al. 2011). In case marginal factors are used for characterizing the inventory values, this might lead to cross-scale inconsistencies and to the wrong quantification of the normalization factors (see e.g. for water Pfister and Bayer 2014; Benini et al. 2015).

It is important to consider that the ranges of uncertainties which are reported above are specific for the combination of the inventory values (e.g. the total of emissions and extractions in the year 2010 for EU27) and the ILCD recommended impact assessment methods which lead to the ILCD normalization factors. If other models were tested they would have probably lead to different results. For instance, the effect of NO and NO2 is modelled to be the same according to the model recommended for the impact category photochemical ozone formation within the ILCD (Van Zelm et al. 2008), having the same characterization factor. Instead, their effect is antagonistic according to e.g. Jenkin and Hayman (1999) and Derwent et al. (1998). Therefore, the conclusions derived in this work should be correctly interpreted and not extrapolated to different contexts than the one of the ILCD recommended impact categories.

Beyond the quantitative uncertainty sources estimated in this paper, a set of quality issues (e.g. robustness and completeness of the inventory and underlying estimation techniques) can potentially undermine the robustness of the results and questioning its applicability in decision making. Hence, qualifying the robustness of the output is essential for the use and communication of the results, especially in the science-policy interface (Maxim and van der Sluijs 2011). In fact, as suggested by Functowicz and Ravetz (1990), both estimations of uncertainty (qualitative and quantitative) should be jointly considered when using a model output. In fact, quantitative assessment of uncertainty and sensitivity cannot substitute ‘sensitivity auditing’ (Saltelli and Funtowicz 2014) as the former refer to probabilistic assessments which account only for factors’ uncertainties for those factors already included within a model, whereas the latter points towards systematic biases of the analysis, such as the lack of the inclusion of some relevant flows or mechanisms, or the limited robustness and quality of impact assessment models. Therefore, Sala et al. (2015) applied a qualitative assessment of the normalization factors to this purpose. This allows making explicit and systematically reflecting upon various dimensions of uncertainty (van der Sluijs et al. 2005).

5 Conclusions

Various methodological uncertainties may affect estimated normalization factors. The results of this paper show that for some impact categories methodological uncertainties are high, although they can be reduced by further refinements at the level of the inventory data and at the level of impact assessment models. The uncertainty associated with the impact categories RIPM and WD, is the highest and certainly represents a priority for future improvement of the ILCD-compliant normalization factors. The other impact categories, ACID, ET and EM, are characterized by relatively low uncertainty. The only exception is POF, which is characterized by very low uncertainty values, most probably because of the fact that only a part of the uncertainty factors which characterize its calculation is taken into account.

A number of improvements in the inventory would be needed in order to reduce methodological uncertainty. Improving the resolution of the inventory by adding information on the emission sources (i.e. breakdown into specific substances, specific height and geographical location of the emission or withdrawal) would allow for using more specific factors for calculating ILCD-recommended normalization factors therefore improving their robustness. From an operational perspective, it is recommendable that the uncertainty figures calculated in this paper are taken into account by practitioners using the normalization factors calculated by Sala et al. (2015), especially when interpreting the results of their LCA studies. Therefore, as a complement to the normalization factors provided by Sala et al. (2015) we recommend to test, for all impact categories assessed in this paper, at least the median value and the arithmetic average + 1 standard deviation (see Table 2, in bold). As a conclusive remark, we warmly recommend the joint use of quantitative uncertainty estimates when applying the normalization factors calculated by Sala et al. (2015) and qualitative assessments of the robustness of the factors estimated by Sala et al. (2015) along the lines of the sensitivity auditing proposed by Saltelli and Funtowicz (2014).