1 Introduction

Since the late 1980s, the emergence of the sustainable development concept has sparked interest from academia, government agencies, business organizations, and civic communities in developing a tool to help decision-making towards sustainability, called sustainability assessment (SA). SA refers to “a methodology that can help decision-makers and policy-makers decide what actions they should take in an attempt to make society more sustainable” ([15], p. 9). The main aim of SA is to ensure that plans and activities make an optimal contribution to sustainable development [82]. SA has increasingly become a common practice in various areas, such as product, policy, and institutional appraisals [70], as well as in project evaluations [8]. As a concept, sustainability generally denotes a balance of economic, social, and environmental goals with a long-term (intergenerational) concern [34, 70].

In transportation projects, SA is applied to evaluate whether a project “contributes to favor economic development and fulfill the transportation needs of the society in a manner consistent with ecological and human values” ([8], p. 642). SA is an advanced methodology to ensure that decision-making is comprehensive and inclusive, meaning that it covers all three dimensions/pillars of sustainable development (i.e., environmental, social, and economic dimensions), including the indirect effects [37, 70]. Political ambition can play a huge part in the planning of road projects. Such projects have vital roles in enhancing regional growth and economic competitiveness, especially in developing countries [17]. However, environmental aspects are relatively neglected and frequently only incorporated later on. Traditional impact assessment tools are often solely concerned with the environmental dimension, while the social and economic dimensions are less often considered (see Fischer [23] for strategic environmental assessment of transportation projects). This paper focuses on road infrastructure projects because of their impacts on the environment and society [30, 50, 83]. These projects are often key drivers of landscape transformations, habitat fragmentation, and societal change on both global and local scales [26, 67], with impacts lasting for long periods (e.g., [22, 72]) and producing intergenerational consequences (e.g., [32, 52]). Therefore, a better inclusion of sustainability dimensions is needed.

Bueno et al. [8] categorized methods and tools for the SA of transportation projects into three distinct approaches: (1) project appraisal methods for decision-making, (2) techniques for impact assessment, and (3) sustainability assessment methodologies. These approaches often adopt generic indicators that allow for uniform application in different situations. The purpose of these indicators is to identify trends, predict problems, set targets, evaluate solutions, and measure progress [51]. The indicators also serve as a compass for desirable development paths and communicate knowledge through the use of specific variables. The investigation of indicators in the SA of road infrastructure projects can provide general insights into whether a project and its components are contributing to sustainability. First, these approaches differ in their application of the indicators with regard to focus, number of attributes, and the methodological concepts (and frameworks) used [34]. Second, the interpretation of indicators varies concerning what sustainability means and which indicators to include [5].

After years of deliberation and experimentation, “it is not difficult to discern a limited number of common themes and broadly accepted general positions” ([29], p. 95) to interpret sustainability. Gibson et al. [29] developed eight basic requirements to attain greater sustainability that highlight the main criteria/aspects in SA. Based on these criteria/ aspects, this study examined the indicators for the SA of road projects in academic papers. Bond and Morrison-Saunders [5] suggest that, at present, SA seems prone to manipulation to suit particular discourses. This paper therefore provides a starting point for the development of an inclusive and balanced use of indicators. Such an effort can avoid the tendency to promote a specific frame of outcomes (such as economic growth instead of societal wellbeing) in the SA of road infrastructure projects. The primary research question (RQ) was: To what extent have sustainability criteria/aspects been incorporated as indicators to assess road infrastructure projects? Three sub-RQs were also formulated:

  1. (1)

    Which sustainability criteria have the papers already included as indicators to assess road infrastructure projects? Is there a robust assessment approach based on the inclusion of these criteria?

  2. (2)

    Which sustainability criteria are sufficiently or insufficiently covered as indicators in the examined papers?

  3. (3)

    How can an integrated indicator set be developed and be further implemented to assess the sustainability of road infrastructure projects?

In order to answer the RQs, both quantitative and qualitative research methods were employed. A systematic literature search was conducted in two databases, Scopus and Web of Sciences. The following section outlines the theoretical framework. This is followed by the research methods.

1.1 Theoretical framework

1.1.1 Approaches to the sustainability assessment of road infrastructure projects

Scholars distinguish SA approaches differently. Sala et al. [70] categorize them according to the level of integrated-ness, ranging from a general method for decision support (such as multicriteria analysis and fuzzy analysis) to a more integrated tool/method (such as a genuine progress indicator or lifecycle sustainability assessment). De Ridder et al. [13] divide the approaches based on their potential role in the assessment phases: (i) participatory tools, (ii) scenario tools, (iii) multicriteria tools, and (iv) accounting and model tools. Bueno et al. [8] classify the methods and tools in the SA of transportation projects into three distinct approaches: (i) project appraisal methods for decision-making, (ii) techniques for assessing impacts, and (iii) sustainability assessment methodologies. This study adopted Bueno’s classifications to investigate how SA is applied to guide decision-making in different project stages (i.e., planning, construction, usage) in order to capture various sustainability elements of road infrastructure throughout its lifecycle. The first approach has been extensively used by decision-makers to plan road projects. Some of the tools in the third approach, such as the rating system tool, have now become popular [33].

Project appraisal methods for decision-making

This approach is employed as an ex-ante evaluation to compare and select alternatives once it has been decided to implement a road project. Two methods are included in this approach. First, cost-benefit analysis (CBA), which supports sustainability by providing a “tangible and rational” judgment of the benefits and costs associated with alternative versions of a project [12]. CBA is based on the monetary values of user benefits (e.g., travel time savings) and other “negative” effects (e.g., energy consumption, resource use, and CO2 emissions). The second method is multi-criteria analysis (MCA). By using this method, several criteria – including those that are difficult to monetize and quantify – can be considered simultaneously [4]. MCA can cover project impacts comprehensively (i.e., the environmental, social, and economic impacts) and enable the involvement of stakeholders through the inclusion of their subjective judgments [64].

Techniques for assessing environmental/social impacts

The second approach is aimed at quantifying the environmental efficiency of road projects [76]. Life-cycle assessment (LCA) is a technique used to assess the environmental impacts of a product, activity, or process (mostly in construction and operation stages). It is deployed to evaluate the sustainability performance of the whole project cycle, from cradle to grave (material extraction, manufacturing, transportation and distribution, utilization and maintenance, energy consumption, and waste handling) [73]. Second, social LCA (SLCA) was developed to incorporate social impacts in LCA [41]. This method – which is often called a social impact assessment – quantifies the social and distributional effects of projects throughout their lifecycles. Bueno et al. [8] argue that SLCA uses a broad definition of social impacts, but still lacks a specific framework to guide implementation.

Sustainability assessment methodologies

The sustainability assessment methodologies approach is an ex-post project evaluation, aimed at assessing full accounts of project effects based on best practices. Bueno et al. [8] elaborate it into (i) rating systems and certification, and (ii) frameworks, models, and guidelines. First, the rating system and certification contain a collection of best practices to incorporate sustainability into road projects [49]. This tool is associated with a standard metric of points or credits that are used to evaluate and compare the sustainability elements of projects (e.g., pollutant loading in stormwater runoff, pavement design life, recycled material uses, pedestrian access). The rating system often comprises a self-evaluation mechanism developed for civil infrastructure projects, e.g., Greenroads, GreenLites, I-LAST, INVEST, and BE2ST-In-Highways. The second category has a much broader scope and includes software tools for modelling and forecasting. Some of the tools have already been extensively applied, such as the UK Department of Transport Analysis (Web TAG) and Scottish transport appraisal guidance (STAG). These tools are deployed to (i) to represent best practices, (ii) provide expert advice for transportation projects, and (iii) establish criteria for assessing options. Therefore, tools in this approach use criteria to provide information about best practices and procedures related to ideal road projects and to improve road sustainability performance based on the assigned criteria [8].

1.1.2 Core sustainability criteria for evaluating indicators

Numerous sustainability criteria can be extracted from literature to examine indicators in SA. The literature provides criteria for extensive areas of practice, including agricultural undertakings [2], urban development [14], nature conservation [35], and spatial planning [63]. However, none of the criteria found is explicitly used for assessing road infrastructure projects. The criteria of Gibson et al. [29] are used here to develop what they refer to as “a minimal set of core [sustainability] requirements” (p. 95) and “key changes needed for progress towards sustainability” (p. 115). The criteria are elaborated into (i) socio-ecological system integrity, (ii) livelihood sufficiency and opportunity, (iii) intragenerational equity, (iv) intergenerational equity, (v) resource maintenance and efficiency, (vi) socio-ecological civility and democratic governance, (vii) precaution and adaptation, and (viii) immediate and long-term integration. Based on these criteria, this paper gauges whether approaches and indicators in the SA of road infrastructure projects have already considered sustainability in implementation.

Gudmundsson et al. [34] distinguish three aspects of indicators for transportation development, namely (i) dimension, (ii) comprehension, and (iii) staging/position. The “dimension” refers to the movement of the indicators (in space and time) to illustrate the importance of contexts in SA. The comprehension of indicators conveys an inclusion of information about what needs to be measured, for example, sustainability pillars (i.e., economic, social, and environmental). The staging presents activities at different stages (i.e., design, planning, construction, usage) that the indicators support to achieve the sustainability of projects. The sustainability criteria and aspects of indicators are listed in Table 1. The criterion immediate and long-term integration is omitted from the list because it includes cross-cutting criteria that should be evaluated at once.

Table 1 Core sustainability criteria (a1–g4) and indicator aspects (h–j) to evaluate the papers (based on [28, 29, 34])
Full size table

Table 1 was also used to extract detailed indicators from the reviewed papers. The following section explains the research methods used to investigate the approaches, the criteria, and the indicators.

2 Methods

2.1 Selection and categorization of papers

A systematic literature search was conducted by using two academic databases – Scopus and the Web of Sciences – on June 24 and 25, 2019. The search strategy was initiated by identifying diverse terms that may refer to SA in the databases, such as “sustainability appraisal,” “sustainability impact assessment,” “sustainability evaluation,” and “integrated assessment” (e.g., [28, 37, 63]) (Table 2). Both the singular and the plural form of these terms were searched for.

Table 2 Synonyms and replacement words as key search terms
Full size table

In this first selection, 490 papers were extracted by using the key search terms in Table 2. Papers representing assessments of other infrastructure projects, such as waterways, energy, and railways, were excluded from our selection. We also excluded papers on small or fragmented elements of road infrastructure (e.g., pavements, roadside facilities) and technological assessments (e.g., innovative construction materials, intelligent systems) to concentrate on the road project scope. Next, we filtered out papers identified as similar reports. Finally, a dataset consisting of 31 papers was analyzed (Fig. 1). The papers in the dataset originated from the disciplines of engineering, ecology, environmental sciences, geography, and social sciences. Most (15) papers concern European countries, namely Germany, UK, Spain, France, Denmark, Croatia, Poland, and Hungary. North American countries constituted cases in eight papers. Seven papers originated from Asian countries and one from an African country.

Fig. 1
figure 1

The search process

Full size image

We extracted all indicators found in the examined papers. We categorized the indicators into core sustainability criteria and elaborated on the criteria based on the descriptions given in Table 1. The number of criteria applied was also noted.

2.2 Analysis methods

Both a quantitative and a qualitative method were used to examine the paper set. To answer the first sub-RQ, we grouped papers by using a cluster analysis, based on the coded description from “a” to “j” in Table 1. For the second sub-RQ (Which criteria are sufficiently or insufficiently covered as indicators in the examined papers?), we counted the number of papers using the criteria in indicators. The third sub-RQ was based on qualitative content analysis.

2.2.1 Quantitative content analysis

The clusters were formed using a complete-linkage technique, namely an agglomerative hierarchical clustering technique that is appropriate for the analysis of a relatively small sample size [59]. Papers with similar characteristics were combined into a cluster [58]. The application of this technique has more flexibility because no predefined number of clusters should be set. It allows a more intuitive way to define the number [75] by exploring the similarity of the characteristics of the dataset in detail based on the criteria included. The cluster set was represented in a tree diagram (a “dendrogram”). There is no exact rule about defining the sample size [59]. Dolnicar [18] observes this size ranging from 10 to 20,000 elements and, by using Pearson’s and Spearman’s correlation, concludes that “even very small sample sizes are used for clustering in very high dimensional attribute space” (p. 2). The size may be less relevant to consider since the analysis works with an unknown structure (see [19]).

We used the descriptions in Table 1 to establish the coverage of the criteria. A descriptive statistic was applied to represent mode and the percentage of the criteria. Next, categorical principal component analysis (CatPCA) was performed to evaluate the correlation between the criteria. We used the Varimax rotation method to examine the correlation between the criteria and visually present their proximity so that they could be grouped into smaller criteria. The method maximizes the sum of the variances of the squared loadings (or squared correlations) within fewer dimensions [56]. The result was a bi-plot informing the dimensions of correlated criteria.

2.2.2 Qualitative content analysis

We started the analysis by extracting all indicators found in the examined papers. All indicators were grouped by using a configurative method [31] to develop an integrated set. If an indicator did not match a specific group, a new group was added as complementary to the set. To avoid redundancies, we also investigated whether the extracted indicators addressed specific criteria. Lastly, we compared the findings with the result of the quantitative content analysis.

3 Results

3.1 Results 1: sustainability criteria in the SA of road infrastructure projects

Figure 2 presents the outcome of the cluster analysis. Four major clusters were identified. Cluster 1 contains the largest number of papers (n = 21) with no more than seven criteria adopted in each paper. This cluster can be divided into two smaller groups (sub-clusters 1a and 1b). Sub-cluster 1a contains all papers using the criteria socio-ecological system integrity (a1) and livelihood security and opportunity (b). Sub-cluster 1b contains papers that include indicators that apply the criteria socio-ecological system integrity (a1), resource maintenance and efficiency (e3), and comprehension of pillars (i).

Fig. 2
figure 2

The resulting clusters and grouping of the examined papers [1, 7, 9,10,11, 16, 21, 24, 25, 36, 38,39,40, 42,43,44,45, 47, 48, 53, 55, 57, 60, 61, 66, 69, 71, 78, 79, 81, 84]

Full size image

Only three papers were found in cluster 2, and only two in cluster 3. These clusters included indicators with more exhaustive and diverse criteria than the other clusters. Finally, cluster 4 comprises five papers with indicators that adopt three similar criteria: socio-ecological civility and democratic governance (f1 and f3) and comprehension of pillars (i).

In Fig. 2, the clusters represent the diverse approaches deployed. Cluster 1 contains papers applying all three approaches. One sub-sub-cluster (cluster 1b.1) mainly comprises papers deploying “techniques for impact assessment.” All papers in sub-cluster 2 and cluster 3 apply “project appraisal methods.” In cluster 4, all papers deploy “sustainability assessment methodologies.” Considering that clusters 2 and 3 adopt more criteria as indicators, the approach deployed can be considered more comprehensive than the others. Papers in clusters 2 and 4 successfully adopt the criterion comprehension of pillars (i).

The bar plot in Fig. 3 shows the number of papers that adopt the criteria in Table 1 as indicators. Criteria a1 and b are the most used criteria, adopted in 29 of the 31 papers (93.5% of the papers). The criterion socio-ecological system integrity (a1 and a2) is used in 28 and 18 papers, respectively. The least adopted criteria are precaution and adaptation (g3) and intergenerational equity (d) (each appear in only one paper). On average, seven criteria are adopted as indicators in the examined papers.

Fig. 3
figure 3

Bar plot showing the sustainability criteria addressed as indicators in the papers reviewed

Full size image

Figure 4 depicts two principal components (PCs) that position the proximity between the sustainability criteria/aspects and the approaches. The line direction (vector) visualizes the correlation of the criteria/aspects with the PCs. A strong correlation is shown by the vector proximity that corresponds to the PCs. PC1 and PC2 represent 19.6% and 16.2% of the total variance, respectively. Four criteria strongly correlate with PC1, namely: socio-ecological system integrity (a2), resource maintenance and efficiency (e2), comprehension of pillars (i), and dimension (j). This implies that the criteria/aspects can be grouped into fewer criteria. However, these criteria are in a negative correlation with “techniques for impact assessment,” meaning that they are hardly included as indicators in this approach.

Fig. 4
figure 4

The bi-plot of CatPCA derived from the coded descriptions in Table 1

Full size image

Figure 4 shows that the criteria intergenerational equity (f2) and precaution and adaptation (g4) and the “sustainability assessment methodologies” approach strongly correlate with PC2. Both the criteria and the approach are in negative correlation, meaning that the criteria are less adopted as indicators in the approach. The other criteria are more independent than previously mentioned, so they are grouped into a much smaller number of criteria. The figure also shows that the “project appraisal methods” approach has a similar direction to the criterion ‘precaution and adaptation (g4), implying that the approach consistently adopts the criterion. Another finding is that the “techniques for impact assessment approach” has a closer relationship with the aspect of complete staging (h).

3.2 Results 2: sustainability indicators extracted from the examined literature

The qualitative content analysis revealed 10 major groups of indicators in the examined papers (see Appendix 1 for details). These groups categorized the assessment indicators into: (1) Mitigation of species habitat fragmentation and land use management, (2) Mobility and accessibility improvement, (3) Pollution (soil, water, air, light, noise) prevention, (4) Climate change adaptation and resilient infrastructure, (5) Community livability improvement, (6) Resource efficiency, (7) Societal wellbeing and equity (both intrageneration and intergeneration), (8) Integrative planning and decision-making, (9) Technological utilization for impact mitigation, and (10) Context-sensitive development.

The findings show that the indicators adopted are not limited to environmental protection aspects (mitigation of habitat fragmentation, land use management, pollution prevention, and resource efficiency), but also cover socioeconomic aspects (community livability, societal wellbeing, and equity) – thus revealing the importance of integrative decision-making to achieve sustainability goals. Two distinct groups of indicators were found concerning the utilization of technology for impact mitigation and context-sensitive development. The finding implies that both process and context are vital in the SA of road projects. The results show that road projects are assessed against various indicators and that some indicators are used more often than others. Without considering the adoption of the sustainability criteria in Table 1, the SA of road infrastructure projects may serve specific discourses, such as the mitigation of ecological impacts. The following section discusses this matter.

4 Discussion

Based on the results, this section discusses i) the robust SA approach to road infrastructure projects, ii) the criteria sufficiently or insufficiently covered, and iii) the development and operationalization of an integrated indicator set.

4.1 Finding a robust approach to assess road infrastructure projects

This paper shows that although considerable efforts have been made to include sustainability criteria in SA approaches to road infrastructure projects, none of the approaches includes all criteria/aspects. This finding substantiates the conclusion drawn by Bueno et al. [8] that “none of the [existing] methods and tools can be used to carry out a holistic appraisal.” Fig. 2 shows that two clusters (clusters 2 and 3) use a more exhaustive set of criteria than the others. Both clusters contain papers applying “project appraisal methods” that consistently adopt more than eight criteria. MCA, in particular, identifies criteria, evaluates alternatives, assigns weighting coefficients to the criteria, and finally evaluates sustainability criteria by ranking the alternatives [4]. The method allows decision-makers to account for complex problems within biophysical and socioeconomic systems through the inclusion of multiple elements using the criteria (see [46]). Pope and Morrison-Saunders [64] also argue that MCA allows many considerations to be incorporated into the decisions and enables diverse stakeholder perspectives to consider transparently. The “project appraisal methods” approach therefore has the potential to enhance project performance, as the chance of incorporating sustainability improves in the early part of the project lifecycle [68].

Both the “project appraisal methods” and the “sustainability assessment methodologies” approach have become useful to incorporate all pillars of sustainability, as found in clusters 1b and 3. The rating system tool is mostly applied to “rank and score projects against sustainability performance by putting economic, environmental, and social aspects together” ([8], p. 632). The “techniques for impact assessment” approach tends to include the criterion complete staging (h). The bi-plot result (Fig. 4) indicates that the approach and the criterion are closely correlated. This finding substantiates that LCA is better deployed to assess project sustainability performance with regards to the efficient use of material and energy throughout the lifecycle (reuse, recycling, recovery, and final waste handling). As few papers apply it, this finding is just a weak indication.

The cluster analysis also shows some problems with the deployment of the approaches. First is the lack of coherence use of criteria to develop indicators in the assessments. The selection of these indicators tends to be arbitrary. Gibson [28] suggests a sustainability test by using the core criteria set to distinguish whether the assessments are genuinely aimed at achieving sustainability. Second, none of the approaches can successfully include indicators based on the criteria/aspects in Table 1. The realistic way to include all criteria is to combine diverse approaches/methods, such as the combination of LCA and CBA. LCA can better assess the inter-temporal aggregation of impacts (intergenerational equity), while CBA covers thoroughly the sustainability pillars as the basis for identifying the project effects in monetary terms (e.g., [54]).

4.2 Sustainability criteria fully covered/uncovered as indicators

This paper demonstrates that the sustainability criteria have been varyingly incorporated as indicators. The two most frequently used criteria are socio-ecological system integrity and livelihood security and opportunity. The criterion socio-ecological system integrity is often used to develop indicators that refer to project effects across scales from climate change and ozone layer depletion at a global scale [11, 24, 48, 55, 71], to soil and local water quality at a fine spatial scale [45, 60, 81]. The criterion socio-ecological system integrity is associated with the indicators concerning the mitigation of species habitat fragmentation, land use management, and pollution prevention. The criterion livelihood security and opportunity is adopted to construct indicators related to the socioeconomic effects of projects. These indicators can be grouped into mobility and accessibility improvement, community livability [9, 16, 24, 45, 48, 55, 57, 84], and societal wellbeing and equity [10, 21, 36, 42, 71]. Several indicators concern intergenerational equity (e.g., direct and indirect effects on employment), and transportation costs are also derived from the criterion [42, 71].

Two criteria are the least covered as indicators in the examined papers. One is precaution and adaptation, which is aimed at evaluating whether irreversible damage and risks to people and the environment have been taken into account in projects (UNCED, 1992). A group of indicators reflects this criterion: resilient infrastructure and climate change adaptation. By using the criterion, indicators are developed to assess the ability of road infrastructure to withstand shocks and unpredicted events (e.g., climate disaster, earthquakes) [42]. Gibson et al. [29] identify the possible barriers to their incorporation: (i) unawareness of the assessor, (ii) cognitive uncertainty regarding the condition being assessed, and (iii) methodological difficulties. Salling and Pryn [71] suggest a certainty analysis in CBA to estimate future costs and possible changes in the value of benefit and cost ratios. Bueno and Magro [7] also recommend the application of sensitivity analysis in MCA to identify to what extent the geographical context of the projects has varied, resulting in different risks (and uncertainty) to consider in the assessments.

The second least adopted criterion is intergenerational equity. The criterion is used to evaluate the cross-generational effects of projects through indicators concerning societal wellbeing and intergenerational equity (e.g., long-term employment opportunities). The inherent methodological limitation is often blamed for the lack of inclusion. Gasparatos et al. [27] argue that most SA methods/tools focus only on economic efficiency, and not on equity. Bueno et al. [8] state that the “traditional” assessment methods/tools only identify impacts for limited time-horizons, most of which are intangible. Joumard and Nicolas [42] express the criticism that the typical linear accounting method (such as CBA) imposes a much lower present impact valuation, which is critical for future generations. Therefore, the components of the discount rate need to be reframed in such a way that the intergenerational inequity concerns of the projects can be included and evaluated, such as concerns about agricultural land losses and community disruptions. These findings show that pragmatism might play a role in the inclusion of the indicators. Therefore, a robust SA approach to road infrastructure projects based on the criteria included is still a long way off.

4.3 Developing an integrated indicator set

This study categorized assessment indicators in the examined papers into 10 main groups. These groups show that sustainable road infrastructure projects are reflected not only in the mitigation of environmental impacts, but also in the improvement of societal wellbeing and community livability. Some papers included indicators about processes to ensure that sustainability is achieved. Consequently, a group of indicators concerning integrative planning and decision-making was added to the set.

Two criteria – namely intergenerational equity and precaution and adaptation – were identified in one cluster, and the two are closely correlated (see Fig. 2). However, both are infrequently adopted as indicators, but can be incorporated in the SA of road projects by applying scenarios, adaptive management plans, and socio-environmental risk estimations [42]. The criteria are further elaborated in two groups of indicators, that is, “resilient infrastructure and climate change adaptation” and “technological utilization for impact mitigation.”

Three criteria – resource maintenance and efficiency, socio-ecological civility and democratic governance, and comprehension of pillars – were identified in a similar dimension and are highly correlated in the bi-plot (see Fig. 4). On the one hand, administrative and market arrangements (standards, regulations, and carbon markets) can enforce efficient uses of energy and materials in road construction and operation. On the other hand, efficiency can be achieved if these arrangements are available and used to guide decision-making if no conflicts are found between the arrangements and the actual implementation [24]. However, Bond and Morrison-Saunders [6] doubt that on their own, the arrangements will ensure effective implementation.

Better inclusion of the aspect of comprehension of the pillars can be made possible if inclusive decision-making is carried out [60]. This finding underlines that sustainability is not only about outcomes, but also about processes, such as stakeholder involvement, the coordination of responsible agencies, and sustainable funding mechanisms [34, 65]. Therefore, integrative planning and decision-making are included as one distinct group of indicators.

Sustainability needs to take into account the aspect of dimensions (space and time) so that the assessment can differ according to the place and the social conditions [8]. The qualitative content analysis explored a group indicator that includes options and actions to harmonize road development with the surroundings; for example, roads are designed to suit local contexts (e.g., safe streets for school zones) and to meet local regulations and standards. In the examined papers, road infrastructure projects already take into account aesthetic, environmental, and art/culture/community values [51, 60].

4.4 Operationalizing the indicator set

The integrated indicator set provides a guideline on which indicators should be included in the SA of road projects or whether sustainability has already been considered. The full application of the set may be difficult because resource availability (e.g., money, funding, and data) and the complexity of the decision-making process can act as barriers. How should the indicators be chosen in actual assessments?

Some scholars suggest that a framework is needed as a constraining factor when choosing the appropriate indicators [20, 34]. This framework maintains the link between the sustainability objective and the indicators applied to monitor progress. Svarstad et al. [77] show that frameworks tend to favor the particular discourses of the organizations that construct them. For example, the DSPIR (driver–state–pressure–impact–response) framework tends to focus on the pressure indicators (e.g., mobility improvement in congested regions) rather than the state or impact indicators (e.g., species habitat fragmentation and community disruption) [80]. Bell and Morse [3] suggest that the participation of affected stakeholders can obviate the selection bias and increase opportunities to incorporate multiple discourses in the indicators.

Still, the sustainability outcomes of road projects will depend on the tested alternatives and the baseline against which the individual indicators are applied. For example, if the aim of a proposed road passing through a protected forest is to connect isolated communities, an alternative policy may entail the construction of the road away from the forest, but lead to much longer travel times. Another alternative is to adopt indicators with regard to the mitigation of species habitat fragmentation. But this option may not be so beneficial to people’s mobility and areal accessibility, or to intra-generational equity and societal wellbeing (improved access of community members to public services). Irrespective of the indicators chosen, the choice often depends on the decision makers offering contextually sensitive solutions that respect the local environmental and community values, and applying technologies that make the project less harmful to the surrounding area.

This study suggests that the SA of road infrastructure projects should prioritize the inclusion of indicators that can secure natural capital and manage its long-term changing state. Most of the examined papers acknowledge that negative impacts are inevitable and use indicators to illustrate these impacts (e.g., pollution prevention and technological utilization for mitigation). But the assessments are applied without testing whether any critical natural capital is lost or secured (see [80]). As a consequence, the criteria precaution and adaptation and intergenerational equity – both of which are less considered in the examined papers (Fig. 3) – need to be incorporated as indicators. By integrating these criteria, SA can identify those who are affected by the change of critical resource/capital and in what ways road infrastructure projects cause less damage to the environment.

5 Conclusion

This paper examined the extent to which the assessment of road infrastructure projects has considered sustainability through the inclusion of indicators closely associated with sustainability criteria in the literature. Some criteria appear to have become mainstream indicators, while others deserve attention. None of the reviewed papers considers all criteria, probably for feasibility reasons, but also sometimes out of pragmatism. Special attention should be paid to the criteria precaution and adaptation and intergenerational equity. Both criteria are either tricky or inconvenient to elaborate as indicators. We therefore suggests that these criteria should be included as indicators more often in future applications. The safest choice is to follow the “methodological pluralism” argument (i.e., the combination of multiple methods/tools) [27] for an exhaustive criteria inclusion. Without considering the core sustainability criteria in Table 1, the development and implementation of indicators can become arbitrary and tend to serve particular discourses of outcomes [5]. The integrated indicator set presented here provides the full account of the discourses.

The advantage of using a systematic review is evidential with regard to transparency [62]. However, there are also drawbacks. First, in our case, relatively few papers were evaluated, raising the question whether the studied sample was sufficiently representative. Only a small selection of instances of the SA of road infrastructure projects are published in peer-reviewed scientific journals, and we have to bear in mind that these somehow deviate from the majority, which are published in the grey literature. For a paper to be accepted in a scientific journal, it needs to contain some innovative elements, such as the use of an innovative method or a new set of indicators. If that is indeed the bias of our sample, it suggests that the broader body of the literature is likely to be more “on the beaten track” than the papers evaluated here. This issue means that specific indicators are probably even more pronounced in the grey literature.

Future research should be able to elaborate further on the integrated indicator set. The set needs to be completed so that all sustainability criteria can be fully incorporated. The criteria intergenerational equity and precaution and adaptation require further elaboration, as do the ways in which frameworks can be constructed to better incorporate the criteria. Another research avenue is the investigation of distinct perspectives on sustainable development, namely the comprehensive and the sectoral view [34], which may influence the selection of these indicators. The use of the indicators also differs according to the scale of the assessments in which they are applied (e.g., global, regional, local, or neighborhood level). Context-specificity may determine how the indicators are selected. The present study shows that the SA of road infrastructure projects is not only a matter of technical deployment of the approaches, but also an integrated decision-making process [74]. Therefore, to improve effectiveness, not only must the approaches be advanced, but also the process and contextual barriers must be identified.