1 Introduction

Despite sustainability in transport has become a crucial element in urban planning, it has often received poor attention in developing neighbourhoods [1, 2]. Yet the city is sustainable if its parts meet sustainability criteria, which appear quite feasible to be included in neighbourhood design [3, 4]. Moreover, pertaining sustainable mobility, a well connected and designed district level does appear as the most effective [5], not only for planning transport and land-use [6], but even for fixing policies and investments [7, 8], and for enhancing people participation [9]. Moreover, there is some evidence that at the local level, people should mainly concern the common good and have positive attitude towards collective actions [10].

Evaluation (ex-post and ex-ante) of transportation plans and projects have been carried out in the past, using a variety of methodological frameworks. The methods can be grouped in two major categories: the single criterion method (monetary approach), and the multicrieria methods (non-monetary approach). The Cost Benefit Analysis (hereinafter CBA) belongs to the first category, the Multicriteria Analysis (hereinafter MCA) to the second one.

Within this context, the present paper focuses on some methodological problems in the field of ex-ante assessment of Sustainable Mobility (“SM” in the following) policies at the neighbourhood scale. Also in this field the two different evaluation approaches (MCA and CBA) are considered. If MCA is a well acknowledged technique for the assessment of sustainability at neighbourhood level, CBA is mainly used for infrastructure projects (where public expenses are expected), and policies.

Both methodologies present some prickly points. MCA, for instance, is not considering public expense efficiency at all, but only its effectiveness in achieving the SM goal. On the other hand, critics often consider CBA not appropriate in managing intangible impacts, which are more relevant to policies and strategies at this scale for achieving SM, and urban quality of life.

The joint use of the two methodologies could, thus, overcome the limitations, providing a coherent methodology for assessing both efficiency and effectiveness of SM policies and projects, and taking coherent and shared decisions on policy actions.

Within this context, the aim of the paper is twofold: (i) highlight strenghts and weaknesses of the two techniques - MCA and CBA - especially when assessing SM at the neighbourhood scale; (ii) investigate the applicability of MCA and CBA to evaluate some relevant SM strategies and policies at neghbourhood scale.

The paper is structured as follows. After the Introduction, in section 2 a literature review on the MCA and CBA methodologies is conducted. Section 3 focuses on the application of the two techniques to assess SM policies, trying to identify the specificities associated to the scale and to the peculiar object. A discussion of the strengths and weaknesses of MCA and CBA is provided in Section 4. Section 5 is dedicated to conclusions and future research questions.

2 Multicriteria analysis and Cost Benefit Analysis: definition and literature review

2.1 Multicriteria analysis

Usually, a decision-making problem has more than one goal to reach, and there is always a trade-off between the different goals, advocated by different interest groups or stakeholders. Within this context, the MCA is seen by some authors as the most appropriate tool to adopt [11–15]. The MCA is a tool for selecting alternative projects, which have significant social, economic, environmental impact, that allows to take into account several criteria and the stakeholders’ opinions. Therefore, the inclusion of multiple stakeholders in the decision making process is widely acknowledged and it is often a crucial factor for the successful implementation of the measure or project under consideration in the transport sector [15].

Within the MCA, the objectives to reach must be specified and corresponding attributes or indicators must be identified. The actual measurement of indicators need not to be in monetary terms, but it is often based on scoring, ranking and weighting of a wide range of qualitative impact categories and criteria.

The MCA is mainly organised into the following phases:

  • Phase 1. Definition of the projects or actions to be judged.

  • Phase 2. Definition of judgment criteria.

  • Phase 3. Analysis of the impacts of the actions.

  • Phase 4. Judgment of the effects of the actions in terms of each of the selected criteria.

  • Phase 5. Aggregation of judgments.

Multi-criteria methods can be classified according to the nature of the decision problem restrictions (implicit or explicit), and the nature of the results (deterministic or random) [14].

There are several methods that might be applied to transport evaluation, the most suitable are: (i) Analytic Hierarchy Process (AHP), (ii) Analytic Network Process (ANP), (iii) REGIME, (iv) ELECTRE family, (v) Multi Attribute Utility approach, (vi) ADAM type (for a detailed review, see [16, 17]). In the present work particular attention is placed on the AHP, ANP and REGIME, being the AHP and the ANP two of the most used and well-known multi-criteria techniques, and the REGIME adopted to assess sustainability at the neighbourhood level.

The AHP, developed by Saaty [18, 19], is a three-stage method: (i) building the hierarchy; (ii) weighting the indicators by a pair-wise comparison, and (iii) calculating the final value for the alternatives. The AHP, indeed, consists of decomposing a complex decision making process into a hierarchical structure, with the ultimate goal at the top of the hierarchy, the primary criteria in the second level, and the subcriteria in the third level. In the following level there might be additional subcriteria, while at the bottom level of this “probability tree” [14] there are the discrete options under consideration (for a review see, among the others, [20]). The elements of the hierarchy can relate to any aspect of the decision problem—tangible or intangible, carefully measured or roughly estimated, well- or poorly-understood—anything at all that applies to the decision at hand.

The stakeholders, public and private, might take part to the construction of this hierarchy, and “especial care must be taken when building up the hierarchy such that pernicious double counting of attributes is avoided” [14]. Once the criteria and sub-criteria have been settled, a set of weights is required. These weights represent the relative importance of the criteria, subcriteria and attributes belonging to a specific nest in the hierarchy. According to the original procedure developed by Saaty, these weights are obtained from pairwise comparison matrices, for each nest in the hierarchy. Once weights are available, the hierarchical structure is collapsed, following a folding back procedure. For each option under study, there will be a final weight. These final weights are used to rank the options.

Another procedure that is widely adopted is the ANP. The ANP provides a general framework to deal with decisions without making assumptions about the independence of higher level elements from lower level elements and about the independence of the elements within a level. In fact the ANP uses a network without the need to specify levels as in a hierarchy. Influence is a central concept in the ANP. The ANP is a useful tool for prediction and for representing a variety of competitors with their surmised interactions and their relative strengths to wield influence in making a decision (for a review see, among the others, [21]).

The ANP is a coupling of two parts. The first consists of a control hierarchy or network of criteria and subcriteria that check the interactions. The second is a network of influences among the elements and clusters. The network varies from criterion to criterion and a different supermatrix of limiting influence is computed for each control criterion. Finally, each of these supermatrices is weighted by the priority of its control criterion and the results are synthesized through addition for all the control criteria.

With the ANP a problem is often studied through a control hierarchy or control system of benefits, a second for costs, a third for opportunities, and a fourth for risks, and each is represented in the controlling system. The synthesized results of the four control systems are combined by taking the quotient of the benefits times the opportunities, to the costs times the risks for each alternative, then normalizing the results over all the alternatives to determine the best outcome.

The REGIME method uses pairwise comparison (for a detailed review see [22, 23]) on the basis of which a synthetic index is calculated. The index defines a ranking among alternative options: the higher is the index, the most preferable is the option. In this case, the synthetic index expresses the level of performance of the different selected indicators as respect to the criterion referring to each analyzed experiences, and make them explicit in the ‘information matrix’. In the REGIME analysis the main focus is the sign of differences between impacts of alternatives. In general terms, an evaluation table is given and composed by scores of a number ‘n’ of alternative options with respect to ‘m’ criteria. In the case of ordinal information, the weight can be represented by means of rank orders in a weight vector: the higher the value of the weight, the better the correspondent criterion. The alternative options will be compared pairwise for all criteria and for two alternative choice options, the difference of the criterion scores is assessed.Footnote 1

2.2 Cost benefit analysis

Cost Benefit Analysis is the most used evaluation technique for assessing infrastructural investments. In the transport field, it is the basic tool in the majority of countries in Europe ([24]; OECD, ECMT [25]) and in the rest of the world ([26]; EVA TREN [27]), and it is also widely adopted by all the international bodies ([28, 29]).

A number of official guidelines exist.Footnote 2 They, despite some differences in how a CBA must be actually performed, always refer to one single common theoretical framework. It is not the aim of the present paper to go in deep with the well known CBA theory, however, some key aspects can be pointed out.

Firstly, the CBA is based on monetisation and inter-temporal discount. Money is the measure unit used as a common numerary to translate all costs and benefits associated to an investment or a policy. Apart direct monetary costs in perfect markets (e.g. untaxed cost of energy) whose monetisation is trivial, also non-market goods and goods traded in an imperfect market are quantified. The first ones (e.g. time or environmental costs) are translated into the common numerary by means of the willingness to pay or by deriving prices from substitute markets (hedonic prices method). The second ones are translated into their opportunity cost by subtracting taxes (e.g. fuel prices) and by looking at the direct effect only (e.g. shadow price of labour cost).

Once all relevant effects of an investment are quantified, the concept of inter-temporal discount is used to translate future costs and benefits to present day by means of a social discount rate. In this way, the future can be compared with present.

The core of CBA technique is the social surplus, sum of users’ surplus, producers’ surplus, and, if the case, non-users’ and Government surplus. Surplus is the difference between the willingness to pay/sell of users/producers for a good (which is the combined effect of perceived utility and income distribution), and the effort needed to obtain such good (the monetary cost or any other kind of effort). A scheme generates a variation of surplus, between the situation “with” and “without” it. Following this concept, CBA essentially compares among trade-offs: total benefits must exceed the total opportunity cost of consumed resources (labour, time, monetary costs, etc.) to make a project feasible. Otherwise, social cost exceeds social benefits and the scheme should be rejected.

In order to have a significant result, two hypotheses must be fulfilled. The first is represented by the Kaldor-Hicks criterion [30], which states that a resources allocation change is efficient if the surplus obtained by some actors exceeds the surplus losses paid by other actors, i.e. if the benefit for a person can be compared with the cost of another one. The second hypothesis assumes that a scheme is marginal, i.e. does not change upstream and downstream markets.

From a practical viewpoint, transport CBA usually quantifies the investment plus running cost of a scheme and compare it with direct benefits, that usually are represented by time, running costs and environmental cost savings. Recently, CBA can also include wider benefits, i.e. macroeconomic benefits that are not subsumed in the direct benefits (agglomeration effects, labour pooling, efficiency; see 3.2).

Moving to the topic of SM policies, several quantitative studies do exist. For example, Farrell et al. [31] and Beria [32] assess quantitatively the effect of different mobility policies in terms of avoided emissions, but only in terms of policies’ effectiveness. They, however, do not evaluate the costs associated to such policies, at least in strict microeconomic terms, and thus they do not perform a socio-economic assessment.

In contrast with the importance of CBA for infrastructural investments, in fact, the use of such tool for softer measures, typically those associated with sustainable mobility, is less common. The above cited examples of CBA guidelines actually refer to infrastructural investments. Motivations that lay behind to the less common application of CBA to policies assessment are clear and will be discussed in a while. However, some examples of CBA applied to policy analysis exist in scientific literature, together with some applications.

From a general perspective, Farrow and Toman [33] state that CBA can be used to improve environmental regulation. Its limits must be known, but it is a necessary tool to cope with scarcity of resources. They describe an evaluation process that reflects the same flow used in physical investments: the definition of a base scenario in which the state of the world goes on with already decided actions, the definition of a complete set of policy alternatives, the identification of the changes to the costs and benefits due to the policies, the assessment and finally a sensitivity/risk analysis in order to—to evaluate robustness.

A warm debate is open on the limits the use of CBA in the environmental field policies. Among many, Heinzerling and Ackerman [34], Hahn [35], and Turner [36] analyse, from radically different viewpoint, pros and cons of the approach. While the first heavily criticise the approach, claiming that it is trying to price priceless things, the other two conclude that, despite the well-known limitations of the technique, it still plays a role also in the environmental policy appraisal. In particular, Turner [36] revises the relevant literature on the topic, underlining that CBA usefulness is particularly true if moving from a prescriptive role to that of information and decision support and when scarcity of resources exists. Only in the policies or actions involving “values” (poverty, cultural aspects, beauty, etc.) the role of CBA must be only partial. In general, CBA is seen as part of a multiple-criteria policy analysis process.

Another specific field of policy assessment in which the debate on CBA is particularly active is that of safety in transport. For example, Elvik [37] analyses the conditions under which applying CBA in safety policies is justified or not, showing that it is appropriate if used as a tool to find “the most cost-effective measures to reduce the number of accidents and injuries”. Wijnen et al. [38] go more into the practice and present the method to assess safety effects (by the estimation of the value of a statistical life).

However, apart from specific ethical and theoretical reflexions on the applicability of CBA to policy analysis, fewer contributions exist explaining “how” to perform a CBA of a generic policy instead of an infrastructure. In fact, all manuals and theories usually refer to the “physical” projects, involving an investment cost and some future benefits. Policies, instead, represent a much broader world, where infrastructures are only one of the possible options, aside to technological investments, education, pricing, etc. to obtain a chosen goal.

One of the policies more studied in literature is road user charging. Rich and Nielsen [39], Transport for London [40], Eliasson [41], and Rotaris et al. [42] perform a CBA to road user charging schemes in Copenhagen, London, Stockholm and Milan, respectively. A similar measure, but applied at a country scale is described in Glaister and Graham [43]. In this kind of policy, however, main benefits and costs are similar to those related to infrastructures: investment and fixed running costs, time savings or costs, revenues, reduction of congestion and possibly of pollution.

Another typical sustainable mobility policy is car sharing. Fellows and Pitfield [44] use the standard British methodology (the so called “COBA”) to analyse a soft measure like car sharing. Methodologically, however, they perform it “exactly in the same way as new road schemes”, quantifying benefits from reduced vehicle kilometres, increased average speeds and savings in fuel, accidents and emissions.

Moving to measures less and less dominated by an investment cost, Sælensminde [45] performs a CBA of walking and cycling networks, taking into account all the relevant aspects, some of which are usually not considered: travel time, insecurity, accidents, savings for school buses, health effects, parking costs, environmental external costs. On the same topic - cycling - Börjesson and Eliasson [46] demonstrate that, in a CBA of cycling support policies, health effects on cyclists, which are usually considered as external benefits, are actually internalized. In fact, the generalized cost of a cyclist seems to include also the health benefit of cycling: he/she accepts a slower mode vs. car or public transport also because perceiving a personal benefit in terms of health. For this reason, considering health benefits as an external benefit to be added in case of an improvement to cycling network, introduces a double counting if consumers’ surplus is correctly accounted.

Finally, a practical application for environmental policies is offered by Massé [47]. The author is calculating the costs and benefits associated to the compulsory introduction of anti-particulate matter filters on cars and trucks in France. He obtains a very good B/C ratio, thanks to the large benefits in terms of saved human lives. On the same topic of pollution, Bollen et al. [48] perform a CBA to evaluate policies that reduce jointly or separately local and global pollutants.

2.3 MCA and CBA: integrated approaches

The combination of MCA and CBA has been applied in several studies. For instance, CBA has been combined with AHP or ANP in order to cope with the CBA’s weakness in reflecting stakeholders’ knowledge in the evaluation process of projects [49]; while a financial CBA plus a MCA matrix was created for an economic evaluation of urban transformations [50].

As concerns specifically transport issues, in the EU member states (for a detailed review see [51]), different evaluation techniques are adopted with a great prevalence of CBA. In most of the cases (i.e. in Sweden, Netherlands, United Kingdom), CBA is supplemented by a specific appraisal for impacts that are difficult to be monetized; in some others (i.e. Belgium, Austria and Greece), MCA is used, but it includes CBA as one of the criteria. Finally, in France CBA has recently been considered weak in stimulate stakeholders’ interactions, thus, in order to create a larger public debate, MCA tools have also been adopted [51]. Even at the urban scale of transport investment evaluation, as the outcome of CBA did not match with the one selected by MCA, a combination of the two methods was suggested [14].

As regards sustainable development, for instance, in the ExternE-Pol project [52] it was possibile to integrate multicriteria methodology into the wider structure of CBA by using a specific framework. By doing so, all the stakeholders’ preferences could be exploited, thus, deriving indirect monetary values for environmental goods and impacts .

Besides, as for the specific environmental impacts assessment from transportation projects, either for small-scale (local) or large-scale (regional/national), a combination of CBA and MCA has been developed in the Evaluation Framework of Environmental impacts and Costs of Transport (EFECT) [53]. EFECT is a methodological framework, which aims to cover all kinds of transport environmental initiatives, namely policies, plans and projects, by using an additive function.

Finally, as concerns the evaluation of SM at neighbourhood level, as it will be described in details in the following sections, a first attempt to use MCA has been done [54], while no specific evidence has been provided for the application of CBA. This is due to the fact that CBA well fits in the assessment of specific infrastructures or policies where monetary or monetizable costs and benefits prevail (investment, time, environmental benefits, etc.) (see 3.2 for a discussion), therefore when it is applied to assess soft policies at the urban scale, it can be well supplemented by a broader evaluation approach like MCA. The MCA, indeed, effectively evaluates effects like social inclusion, change in behavior of citizens, change in the use of city, quality of life, etc.

3 Assessing sustainable mobility policies at neighbourhood level

3.1 The case of MCA

Within the application fields of the MCA, transportation plays an important role. Specifically, there has been a flourishing literature investigating the assessment of SM, both ex-ante and ex-post at the urban scaleFootnote 3 and the city level, while the literature at the neighbourhood level is still poor.

The literature on the city level presents the indicators to measure SM, and adopts the MCA to obtain the indicators weighting (see, among the others, [55]). Among the many works focusing on the city level, the PROPOLIS Report [56], the TRANSPLUS Report [57] and the PROSPECT Report [58] present some successful good practices connecting measures in land use and urban transport, and develop specific strategies and indicators to achieve SM. Specifically, the PROPOLIS Report aims at finding policies that could, in an ideal case, simultaneously improve all dimensions of sustainability compared with the reference solution and, if possible, even improve the current level of sustainability. Other applications of MCA are found in the work by Campos Gouvêa et al. [55], which proposes a set of mobility indicators with the objective to integrate transport and land use planning in order to achieve an efficient sustainable urban development. The case study analyzed by the authors is Belo Horizonte, the third largest city of Brazil. Besides, Tzeng et al. [59] adopt MCA to evaluate alternative-fuel buses in Taiwan urban areas; Yedla and Shrestha [60] focus on the selection of alternative options for environmentally sustainable transport system in Delhi.

As stated above, the literature on MCA, carried out at neighbourhood level, is very scanty and usually refers to the ex-post perspective. At least to our knowledge, only the works by Cerreta [61], and Cerreta and Salzano [62] focus on the ex-post assessment of sustainability at neighbourhood level. This work originates from the “Best Practices and Local Leadership” programme developed by the UN-Habitat, which analyses, since 1996, the best urban renewal cases (for a detailed review, see Fusco Girard et al. [63]). The database contains about 1,700 ex-post evaluations, which have underlined that the best practices as concerns environmental, social and economic sustainability, have been characterised by the mutual investment in six different dimensions, belonging to the so-called Hexagon model [64]. The Hexagon model focuses on the integrated vision of sustainability, and is based on six different kind of capitals which are strongly interconnected (See Table 1).

Table 1 The Nijkamp Hexagon model
Full size table

With reference to the conceptual framework suggested by this multidimensional model, it is clear that the level of sustainability achieved by a project, a plan or an intervention, depends on how many dimensions are considered in it and on the degree of their mutual relationships [54, 61, 62, 65].

In particular, Cerreta [61] and Cerreta and Salzano [62] adopt the Hexagon model in order to assess the sustainability degree of a sample of eight “best practices” neighbourhoods in Europe. The authors have translated the data for each case study into specific indicators able to express the interventions and strategies’ performances, measured throughout a score of the performance level (from 0 = none to 5 = excellent). It was, thus, possible to develop the evaluation matrix and compare the profiles of the eight neighbourhoods according to an ex post evaluation approach. The methodological approach used the multicriteria method to define the significance of the practices as concerns sustainability. Specifically, the REGIME method was adopted in order to make explicit the satisfaction level of each criterion, and to verify the performance of each neighbourhood, thus obtaining a synthetic index representing the selected indicators.

Starting from the work by Cerreta and Salzano [62], in recent studies Maltese et al. [54], and Maltese and Mariotti [65] have focused on the SM assessment of 20 European neighbourhoods. To do so, they have selected several SM strategies, derived from: (i) a very rich literature review on SM indicators (see among the others, [66–72]Footnote 4; [73–76])Footnote 5; (ii) the Nijkamp Hexagon model and the ex-post assessment model for achieving SM by Holden [77]; (iii) data availability at the neighbourhood level.Footnote 6

Table 2 shows the selected SM strategies, which can be grouped into two categories: “Direct SM strategies”, selected in grey, and “Indirect SM strategies”. The first belong to the Nijkamp “Transport” criteria and have a direct impact on transport mobility, infrastructures and modal choice, i.e. include strategies for car reduction, parking planning, etc.Footnote 7 The second ones have fostered SM throughout other long-term strategies: land use policies (e.g. mixed use of land, primarily aiming at a more liveable place, that leads, as additional effect, to car reduction); new technologies (e.g. energy provided by renewable sources, that can also be used to enlighten bike lanes or recharge electric vehicles); and the so-called green attitudes (e.g. campaigns for sustainability and funds to promote SM impact on citizen’s lifestyle thus fostering an environmentally friendly behaviour and responsiveness)

Table 2 Indicators to assess sustainable mobility
Full size table

According to a widely accepted sustainability rating systems (see for example, GBTool, SBTool and Itaca ProtocolFootnote 8 for assessing sustainability of buildings, but even the above-mentioned [62], for sustainability of neighbourhoods), the presence of each SM strategy has then been measured by a quantitative score, which ranges from 0 to +2 (0 = deficient performance: no strategy is adopted; +1 = ordinary practice, which is the minimum acceptable performance, strategy is partially adopted; +2 = good performance, strategy is fully or strongly adopted). For example, as regards the measures for private transport degree the score is 0 if no measure regarding private transport is undertaken; 1 if some traffic calming measures, inside the neighbourhood, have been adopted; 2, if the neighbourhood is a car free area. Of course, no mention is provided about the effectiveness of measure (for instance, if it is respected or not) or its efficiency (assessing costs or benefits deriving from the specific intervention).

The study concludes that SM can only be achieved through the homogeneous involvement of direct (from transport strategies for reducing car use to bicycle and pedestrian paths) and indirect (from mixed use of land, density and urban form to community involvement) SM strategies.Footnote 9

3.2 The case of CBA

As we already stated, literature and common thinking suggest that CBA can also be used to assess policies. In fact, any action in the transport sector (an investment, a policy measure, a new restrictive norm) potentially involves public or private costs, in change of direct or external benefits. And when a social cost exists, it is appropriate to take public decisions after having verified the opportunity-cost of it: is the effort efficiently spent because giving a higher benefit, at least aggregate, and not only effective? Moreover, the distribution of costs and benefits among groups can be relevant. In other words, for any non-win-win policy it makes sense to verify if the monetary or non-monetary effort is well paid, and CBA is widely recognised as an adapt tool for such task.

Two main problems however exist in assessing SM policies:

  • “ethical” problems, related to the issue of measurability of some costs/benefits;

  • practical problems, on how to perform such analysis.

Ethical arguments are the most used to criticise the use of CBA in this field.Footnote 10 Practical problems, instead, remain unsolved or not sufficiently clarified in CBA for SM policies. This issue is amplified when focusing on the relevant sub-problem of SM at the neighbourhood level.

In fact, it is well known to practitioners that CBA tends to be more complex, the smaller the scale of the problem. While for an infrastructural problem it is “easy” and obvious to identify and quantify the main costs and benefits, and leaving to qualitative decisions other effects, such task is more difficult for policy measures, where non-monetary benefits and costs prevail.

When considering SM policies, and policies in general, the traditional categories of cost and benefit become less and less defined for the following reasons:

  • available solutions are much broader (not only infrastructures);

  • the range of impacts is broaderFootnote 11;

  • direct public investment is relatively less dominant on other positive and negative effects;

  • some policies are at zero monetary cost, but determine other kind of efforts for part of the users (e.g. car free zones introduce a cost for car users).

At the neighbourhood level this is even more evident: nearly no policies involve a relevant public cost to be assessed in term of trade-offs, but this does not mean that all policies have no social cost.

The main point to understand when and how CBA can be used to assess SM policies is to clarify which are the main categories of costs and benefits involved. In principle, all of them exist, but only some of them are actually relevant in a certain policy. For example, a car free area is nearly costless for the public subject. The main negative impact to compare with positive effects is the extra-costs inflicted to those users that are no more allowed to enter in the area and must spend longer trips with public transport or change their destination. The following Table 3. clarifies which are the main costs and benefits associated to a typical mobility policy.Footnote 12

Table 3 Main categories of cost and benefit according to stakeholder
Full size table

Columns divide the type of benefits and costs to be considered according to the degree of adherence of monetisation to reality. For example, expenditures and time costs are naturally expressed in monetary terms. In this case, CBA uses simply the shadow value of market costs. Other costs are not monetary, but can be reasonably translated in monetary terms (e.g. the environmental costs). This “translation” is not accepted by everybody, but it is commonly used in the large majority of guidelines. The third column includes all the effects that can be judged as positive or negative, but for which associating a conventional monetary cost can be too difficult, unclear or simply a nonsense. For example, it is obvious that a nice bridge is better than an ugly one, but how to measure “nice” and “ugly”, even if practically possible, will be too undefined (involving personal judgements) and of scarce economic meaning.

Table 3 is very synthetic in some parts, therefore, needs some further specifications:

  1. 1)

    Time is not traded into a market, but its monetisation is rather intuitive.

  2. 2)

    A general and important issue is that of double counting, i.e. benefits (or costs) that are accounted two times under different forms. The largest attention must be paid to it, because biasing the result and very common in SM policies, and CBA forces to pay such attention. Examples are the accessibility (see below), the way taxes are accounted, the effect on land prices, the way users’ benefits are calculated [78] and many more.

  3. 3)

    Accessibility is not listed in the table because already included in two separate terms. On one side, it is the inverse function of generalised cost, implicitly calculated in the users surplus. The other “form” of accessibility is related to the positive externality of getting nearer to markets or workforce. This effect is usually treated in CBAs as a “wider benefit”, listed in the last row. The two effects (the internal, perceived one and the external one) must not be associated also with a further measure of “accessibility” to avoid double counting in favour of projects. Similarly, “accessibility” must not be confused with the network effect, that economics treats as a positive externality [79].

  4. 4)

    That of wider benefits is a complex and important issue.Footnote 13 It is widely recognised that under some conditions the direct transport benefits (e.g. users’ surplus) do not fully represent all the benefits associated to a scheme or a policy. The main external (or wider) benefits are the macroeconomic ones, namely agglomeration economies ([80–83]), increased long term productivity [84] and labour pooling [82]. For the conditions ruling the existence of such benefits, see the same sources. In general one can say that they are relevant in case of true capacity constraints/bottlenecks/missing links and/or in developing countries. A rule of thumb for European economies says that these effects account for 0% to 30% of direct benefits, according to the type, place and size of project.

  5. 5)

    Land use associated benefits are among the most difficult to manage. Apparently, to a scheme that increases land prices should be associated a further benefit apart direct ones. Actually, this is often configuring a double counting. For example, an increase in estate prices due to a car free area can be added because representing the willingness to pay of citizens for a quieter area, but in this case noise reduction must not be accounted. To the contrary, all things equal, if the increase of prices is generated by a new metro station, all relevant benefits except externalities are already in the users surplus: the house costs more exactly because more accessible. In other words, one should always recognise when land price increase is a net benefit or is the dual of another benefit already considered. In addition, the analysis is complicated by the time dimension: direct benefits over time transform into land rent [84].

  6. 6)

    When analysing policies, the issue of distribution of impacts is more and more relevant. CBA allows to inform the decision-maker on the distribution of quantifiable impacts by means of a distributional matrix, or Kaldor-Hicks matrix (see Table 6 below and [85–87]). Such a matrix excludes non monetary aspects, that must be considered apart.

To analyse the possible SM policies in terms of CBA, we used the list provided by the KONsult studyFootnote 14 and Kelly et al. [88], limiting only to those policies applied to a small scale. Of course, other classifications are possible. For each policy, we describe the most relevant types of costs and benefits to be accounted in a CBA (Table 4).Footnote 15

Table 4 Costs and benefits to be considered in SM policies
Full size table

The last column of the table underlines a key concept: the introduction of a constraint by a policy. Some policies are additive: e.g. the introduction of a new technology into the market. Other policies are constraining the existing conditions: e.g. a new technology becomes compulsory. In the first case, the policy must be assessed only by looking at how demand responds to the new situation (more demand, less costs, etc.). The second case also introduces, at least for part of the users, a constraint that translates into a cost. For example, the adoption of a compulsory environmental standard for truck emissions causes enterprises to purchase new vehicles even if the existing ones are not yet amortised.

CBA is then capable not only to assess public expenditure, that is its “traditional” use, but also to explicitly take into consideration the private costs associated with the introduction of new constraints. The same sustainability goal can be obtained through different public decisions and actions (e.g. emissions reduction can be obtained with new technologies, public transport expansion, restrictions to cars or city areas). Some involve a public subsidy, some others are without subsidy, but involve increasing private costs. CBA can clarify losers and winners and the net balance, but can also be used to fine tune the policy. For example, car free areas are more costly in sprawled or public transport deficient areas than in compact areas already infrastructured.

4 Strengths and weaknesses of MCA and CBA in the assessment of sustainable mobility

The previous sections have described in details MCA and CBA applications to SM decisions problem; the aim of this section is to provide additional considerations about the two methodologies (Table 5) and investigate their strengths and weaknesses (Table 6).

Table 5 Comparing MCA and CBA
Full size table
Table 6 Strengths and weaknesses of MCA and CBA
Full size table

Referring to when the analysis can be conducted, CBA is more used as an ex-ante tool, while MCA is adopted for both ex-ante and ex-post assessments (Table 5).

The application of the two methodologies is then often suggested by the size of the project or of the policy to be evaluated: at a large scale, i.e. when public and private costs are consistent, CBA is necessary; MCA appears instead useful at the micro-scale, where all the stakeholders can be easily individuated and consulted and are able to express informed opinions on their priorities.

If the aim of CBA is social expenditure’s efficiency, the aim of MCA is to find the better way to get the final target, if conducted ex-ante, or the level of performance reached - that is the proper effectiveness- when the evaluation is ex-post.

Furthermore, MCA seems to be better in measuring intangibles and soft impacts than CBA; actually, it uses more than one criterion introducing qualitative aspects in the analysis. CBA, instead, translates all the effects in a common numerary and expresses them into a single aggregate measure. Finally, if the CBA is used to generate a ranking of alternatives that will help the decision maker in choosing (that may obviously differ from the CBA one), the MCA takes the personal ranking of the decision maker as an input and weights it together with other stakeholders’ ones.

It is, therefore, clear that the two approaches are not totally alternative to assess the same object. Rather, especially according to some aspects, the two look completing each other and the joint use can add value to the assessment.

Broadly speaking, CBA appears more rigorous, transparent and formal [33, 89], providing a rational framework for assessing projects and clear results easy to be communicated and, thus, shared and engaged [89]; nevertheless, the technical procedures for applying is much more complex [89], and expensive [49]; the monetization process is questionable for some intangible aspects (among the others, [15, 16, 17, 53, 91]).

On the other hand, MCA, in a more informal process, allows every stakeholder to be heard about the action to be realized [15], enlarging the “democracy” in terms of public debate [89]; furthermore, it could be useful to settle possible arguments ([15],) and to get preferences and priorities revealed (Lee et al. [49]. In this case, a thoughtful question may be posed about the way of weighting different opinions [17], and the consequent accountability, transparency and coherence of the method [90].

In the assessment of public policies, the main added value of CBA in this field, not treated in a sufficiently formalised way in other approaches, is the evaluation of the outcome in terms of public expenses. In other words, when a policy is not involving significant public expenditure (e.g. education campaign to promote bicycling and walking) or where public expenditure and costs in general are dominated by less tangible effects (e.g. a reorganisation of public spaces of a neighbourhood to promote bicycling and walking), MCA can already provide consistent answers. On the contrary, when significant public costs exist (e.g. the expansion of public transport network) or when a costless policy actually inflicts extra-costs to citizens in the name of SM (e.g. the introduction of a large car-free area), it is necessary to evaluate the outcomes not in absolute terms, but in terms of costs or efforts. CBA allows, then, to consider the opportunity cost of a policy [33], i.e. the value of alternatives use of public money spent for a certain policy. To make a simple example, if two policies are both generating a +5% patronage of public transport, but one generates a cost to the public administration or to the car users and the other obtains the same benefit without generating costs, the judgement on the two policies must be different and the second must be chosen (it is, for example, the case of constraints policies or supply-side policies vs. education-based policies).

Table  7 summarises the applicability of CBA and MCA to the typical range of SM policies that can be undertaken at the neighbourhood scale. Not all the actions, in fact, can be easily evaluated throughout the CBA like, for instance, all the actions modifying the inhabitants attitudes (indirect indicators) or the benefits of the communication and information campaigns about SM, because the benefits are second order effects, and develop in the long term.Footnote 16 By contrast, the MCA does not effectively assess some private or public costs. As an example, it hardly evaluates the social cost of the travel time increase due to restrictive regulation (i.e. car free area) in an existing neighbourhood.

Table 7 Applicability of MCA and CBA to assess SM actions
Full size table

A point is however extremely important. The two methods do not provide the same information in terms of decisions. MCA “summarises” the stakeholders’ implicit priorities, possibly weighting stakeholders in terms of importance in the decision, while, CBA is the translation into a number of the social welfare function, measured independently using physical measures, models and literature values. The output is not the decision, but one of the inputs to the public decision maker when taking the decision.

5 Conclusions and further research

In this paper we analysed the existing literature on CBA, MCA and investigated the strengths and weaknesses of the two techniques, once applied to the SM policies field. We also discussed how such evaluations could be practically developed, and underlined the following aspects:

  1. 1)

    on the micro-scale, MCA is commonly used, ex ante, and ex-post at the urban or suburban scale, while CBA is much more adopted for infrastructure projects at a bigger scale, as an ex ante approach;

  2. 2)

    on sustainability, MCA has often been adopted for evaluating ex-post the level of general sustainability [62, 64] and SM [54, 65] at lower scales;

  3. 3)

    on transport and mobility, ex-ante evaluation has been conducted primarily with CBA methods [51], even if MCA is used sometimes to evaluate transport projects ex ante (see section 2.1);

  4. 4)

    on SM many difficulties arose when measuring all relevant impacts of a transport strategy (either policy or project) in monetary terms, concerning intangible aspects, thus leading to the expansion of monetary evaluation (unique criterion) to evaluation methods using more than one criterion, e.g. multi-criteria methods ([93]).

It is clear that a traditional MCA approach has the risk to fail in evaluating the real trade offs generated by the policy. To the contrary, the traditional CBA approach is not catching all the effects involved in SM policies, where intangible aspects are not secondary, but dominating. This is particularly true when the problem is related to the peculiar case of small scale.

It can, therefore, be concluded that as concerns the specific case of SM assessment at the neighbourhood scale, MCA and CBA might be integrated to fully consider all relevant actions available. Specifically, MCA is a good tool for the indirect actions (where soft and indirect effects prevail) and to be applied at a suburban scale. CBA, on the contrary, better works for the direct strategies (where monetizable costs and benefits prevail), besides, it better takes into account the external costs, which are very important at the urban dense areas, and can no longer be neglected.

Therefore, the joint use of the methodologies might be done in two ways:

  1. 1)

    developing CBA and evaluate the soft effects with MCA;

  2. 2)

    developing a MCA for a broad screening of options and evaluate public expenditures and consumer surplus with CBA.

The joint use of the two methodologies can guarantee a deeper analysis (and knowledge) of priorities and impacts of an action, always remaining “beyond the market” [52]. However, it is worth observing that the concurrent use of the two techniques may even drive to contrasting results among the alternatives (see, for example, [14]). On the other hand, the joint use of the techniques, where the focus of each one is clear and well defined, does not lead to distortions in the decision, lowering ambiguity and misleading on the hierarchy/prioritization.

Specifically, at the neighbourhood scale, the joint MCA - CBA can be easily conducted in order to assess the SM level; furthermore, it can create a stronger and aware participation to the decision making process, spreading out knowledge, not only about the goal, but also about the alternative solutions to problems and their costs.

This conjoint assessment procedure allows a neighbourhood to choose among different SM options by referring to its own (revealed) preferences (MCA), while being aware of budget constraints and externalities (CBA). This means that, as an economic subject, the neighbourhood is showing its own “willingness to pay” to reach the SM goal.

Therefore, further research could, from one side, develop more deeply the methodological framework for selecting indicators and for developing a joint evaluation methodology; from the other side, focus on the empirical analysis, that is the implementation of the conjoint methods to scenarios, or real cases.