Performance diversity
Performance diversity refers to the differences that can be observed in the overall sustainability performance for each of the solar PV and urban mobility trajectories in the two countries. Aggregating the appraisals of all individual stakeholders, this diversity is expressed as contrasting ranking intervals compared across the different socio-technical options within each sector. Figure 2 illustrates this for the Thai urban mobility case.
In this figure, the overlaps in the ranges for different trajectories show (as is typical in MCM) the combined effect of a high degree of uncertainty, ambiguity and variability in the performance orderings of different options. This uncertainty is typically understated in other kinds of appraisal method. This said, it might cautiously be observed that non-motorised transport (such as walking and cycling) received relatively high optimistic scores on average, compared to all other trajectories. A majority of stakeholders in this case agreed that non-motorised transport is the most desirable, as they perceived that they are flexible, affordable and least harmful to the environment. One of the Thai stakeholders adds “…in addition, walking and cycling provide better and easier access to the small alleys (soi) in Thai cities.” This strong preference is reflected in a relatively high mean ranking for the non-motorised trajectories (the mean ranking being the midpoint of the thick orange bars).
The mean ranking for the alternative public transport trajectory is also quite high. Despite the carefully documented differences, stakeholders agreed that alternative public transport systems such as bus rapid transit are in general more inclusive, provide better accessibility and minimise congestion. Where a method (like MCM) avoids forcing closure in appraisal, the emergence of such convergence is correspondingly more robust. Likewise, both cycling and walking as well as alternative public transport trajectories tend to be perceived in general as more sustainable options than the alternatively fuelled vehicle such as CNG cars, electric vehicles and shared transport trajectories in Thailand. This result might be thought significant in relation to frequent patterns of emphasis in innovation for sustainable urban transport.
Interestingly, the mobility trajectories that appear most sustainable are those which are less dependent on high-tech innovations, are more reliant on behavioural shifts and are compatible with existing infrastructure. Alternatively, fuelled vehicles and electric vehicles received considerably lower ranks, as the stakeholders argued that these require high initial investment; they are non-affordable by the poor and middle-income groups (that constitutes a large section of the population) and, therefore, are non-inclusive in nature. Shared transport systems mainly received pessimistic scores due to their current non-environment friendly fuel use and a substantial role in creating congestion, air and noise pollution in the cities of Thailand.
Analysing performance diversity for solar PV trajectories in India, we observed even more pronounced uncertainties, ambiguities and variabilities in final rankings—measured through high ranges of optimistic and pessimistic scores resulting in substantial overlaps in the sustainability performances of the various trajectories. Figure 3 (below) illustrates this.
This is despite the fact that the different trajectories in this case involve much more similar kinds of technologies than in the transport sector (since all in this case involve photovoltaic cells). One finding in this respect is that decentralised solar PV options like solar lanterns, solar home systems (SHS), and rooftop solar applications seem to display somewhat higher sustainability performance in terms of highest average ranks, compared to centralised large scale solar PV applications like large grid connected solar power plants and solar cities. This picture is revealed by the qualitative discussions of participants during appraisal, where it was argued that decentralised options tend to display advantages over large-scale systems in terms of cost minimisation, easy and quick installation, and operation and maintenance facilities. An Indian stakeholder summed it up in her comments during the interview process “…these small scale individual household based applications will have maximum positive social, economic and environmental impact through maximum accessibility to remote areas deprived of electricity, minimum emission and transmission losses and everything, assuming the subsidies continue for some time; the production and disposal of the equipment are hazard-less.”
Overall, these findings urge caution over more simplistic accounts of the sustainability performance of the different innovation trajectories in either solar PV or mobility. The picture is not straightforward and depends on highly specific visions of what is meant by sustainability. In itself, this holds important implications for notions of transitions and experimentation in which sustainability or its technological implications are held to be self-evident. A key implication is that manifestly divergent informed opinions led to quite extreme ranges in scoring. This underscores the importance of uncertainties in individual perspectives, ambiguities across contrasting perspectives and variabilities across different contextual condition that can often be missed in appraisal. This understanding leads us to the next dimension of diversity, namely appraisal diversity.
Appraisal diversity
Appraisal diversity is defined as contrasts in understandings, perceptions and values as between different stakeholders participating in the appraisal process. These divergent perspectives on the meanings of ‘sustainability’ were reflected in participants’ selection of criteria, the ways in which these criteria are weighted, divergent patterns of scoring and expressions of uncertainties under individual criteria. This appraisal diversity can be captured by comparing the responses of the stakeholders either at an individual level or at a semi-aggregated level where each of a number of variously definable groupings of stakeholder perspectives can be compared with each other.
As an example of this analysis at a semi-aggregated level, we compared the weights assigned to each group of criteria (technical, social, environmental and economic) under stakeholder perspectives disaggregated across ‘consultancy’, ‘Industry’, ‘NGO’, ‘governance’ and ‘academics’). Results from the solar PV case in India are shown in Fig. 4.
It is noteworthy that the individuals identified on the basis of their affiliations as consultants assigned strikingly higher weights to social criteria in the appraisal of solar PV in India. They emphasised the importance of local skill development for decentralised maintenance and operation of the solar PV systems, need for supportive policy targeted towards the benefit of “common people”. Another interesting result in Fig. 4 is that the individuals identified as industry actors assigned almost negligible weight to the environmental aspects of sustainability. In expressing their own framings of sustainability criteria, these actors in India emphasised the more socio-economic ‘sustainability’ criteria (like policy and awareness, value of stakeholders, profitability, affordability and entrepreneurship opportunities).
The consultants also expressed significantly higher levels of uncertainty for the centralised solar PV options like power plants and solar cities. Technical consultants tended to take into account the subsequent risks of the systems being highly subsidy dependent, and the policy strategies and financial schemes being less transparent and heavily subject to corrupt practices. An illustration of this point can be quite clearly seen in Fig. 5 (below), where the green and orange bars representing solar power plant and solar city trajectories, respectively, are manifestly tallest for the consultancy perspective.
In the urban mobility cases in both countries, it was quite striking that it was the governance actors who assigned the highest importance to social sustainability of the emerging mobility trajectories (in India alongside NGOs). This is depicted in Fig. 6.
One qualitative substantiation of this result emerged when one of the participants, categorised as a governance perspective in India, explained that people will only prefer a mode of mobility if they think it matches with their status and position in the society. The criteria are thus closely linked with judgements concerning social and cultural perceptions and mind sets about the different forms of mobility. Under a criterion of community involvement, this participant also emphasised the importance of sufficient knowledge dissemination as a social criterion.
Looking carefully at the graph for the Thai Urban mobility case, (Fig. 6) it can be observed that there is a considerable difference across stakeholder groups, in the assignment of weights to what might be considered more ‘technical’ criteria in the framing of sustainability. These criteria typically included time predictability of mobility services and minimisation of travel time as well as energy efficiency, adaptive-ness and compatibility in energy systems. The governance actors and the researchers assigned very low weights to these technical issues, when compared to the consultants and NGO representatives.
As mentioned in the beginning of this section, appraisal diversity also refers to the differences in ranking patterns at an individual level of contrasting viewpoints. Figure 7 demonstrates the comparison of the appraisals by two individual stakeholders for urban mobility in the Indian case. We can interpret that the engineer at a state pollution control board (left graph) was highly uncertain about sustainability of bus rapid transit (BRT) systems and CNG vehicles in spite of being overly optimistic about the fact that both are sustainable options. He was also optimistic about trajectories like Walking and Cycling and pessimistic about vehicle parts innovation like ultra capacitor and electric vehicles with moderate degree of uncertainty. In contrast to his appraisal, however, another stakeholder from a science technology and development research institute (right graph) expressed high optimism for sustainability of walking and cycling trajectories and pessimism for BRT—all with negligible amount of uncertainty.
The qualitative information collected in this MCM analysis tells us that appraisal diversity can also be identified if we look carefully into the ways in which each stakeholder perceived the scope and potential of each trajectory. For example, although the ‘solar city’ trajectory is considered a centralised system by some stakeholders, others consider this trajectory to be a ‘collection of technologies’, or even an ‘enabling environment to experiment with different solar technologies—each of which can be managed in small units’. From this point of view, solar city is a desirable option if there is a community or household ownership of individual applications constituting a large solar city project. This ‘if’ resulted in the especially diverse extreme ranges displayed in the scoring the trajectory. Some stakeholders argued that the solar city concept has great potential to address environmental sustainability issues, thus assigning high optimistic scores to this trajectory. Others explained that policy framings of this option are currently quite opaque, resulting in less confidence in more optimistic scenarios for the performance of this trajectory.
This section has demonstrated the importance of highlighting differences in criteria and uncertainties across social groups—as well as their associated patterns of reasoning. These may easily be missed in attending only to the aggregate picture in Sect. 4.1. The next section continues with differences in sustainability across different sectors.
Sectoral diversity
Diversity can also be observed across the two sectors studied in this research (energy through solar photovoltaic and urban mobility). This is evident, for instance, in respect of criteria, definitions and uncertainties as between degrees of optimism and pessimism. One of the striking differences between the two sectors is that criteria for environmental sustainability did not seem to be as important, either in numbers or in weights for the solar PV trajectories compared to urban mobility. Figure 8 (below) shows this diversity across the two sectors in India.
Here, we can observe that the number of criteria proposed to assess environmental sustainability of solar PV systems was significantly exceeded by the criteria proposed for social, economic or technical issue (the graph to the left). In contrast to this, for urban mobility (Fig. 8, graph to the right), environmental issue contains the second highest number of criteria, which follows after the highest number of criteria proposed for social issues of sustainability. In light also of associated qualitative findings, we can conclude that concerns over environmental sustainability were notably more pronounced and nuanced in the urban mobility sector cases in both countries.
The relatively low salience of environmental aspects of sustainability in the solar PV case in both countries can be interpreted in two ways. First, stakeholders suggesting environmental criteria in this case tended to assign relatively low weights to these criteria. Associated comments suggest that many of them simply assumed solar applications to be environmentally sustainable and, therefore, felt it more important to evaluate other (social, economic or technical sustainability) aspects more relevant to distinguishing between the relative merits of these trajectories. Second, many stakeholders mentioned just one or two environmental criteria, defining these such as to incorporate several environmental concerns in a single criterion. As an example of this, one stakeholder named her criterion in this case, ‘Reduction in environmental impact’. In the description of this single criterion, she talked about local air pollution, noise pollution, global climate mitigation strategies all the same time.
From a sectoral diversity point of view, this is an interesting observation, since such integration of several concerns in one criterion can only be seen in the solar PV cases. In the urban mobility cases, by contrast, criteria were much more reflective of specific environmental aspects of sustainability. Notwithstanding this overall pattern, it is all the more striking that a few stakeholders in the solar PV appraisal did raise specific concerns about provision for battery disposal for solar home systems, and use of agricultural land for construction of power plants. It can be concluded that even if the solar PV trajectories are perceived to be using fairly similar technologies, there were some instances when concerns arose over particular environmental issues under which options performed differently.
Diversity across the two sectors is also reflected in the expression of uncertainties, as illustrated in Fig. 9 for the two sectors in Thailand.
For the solar PV case (at the top), the highest range of uncertainties was expressed for policy-related criteria (rather than environmental, economic, social and technical criteria), while for urban mobility case (at the bottom), uncertainties were most prominent for social and environmental sustainability criteria. For mobility trajectories, none of the stakeholders even mentioned a supportive policy environment to be a relevant sustainability criterion. Instead, the stakeholders were more concerned about sustainability of urban mobility trajectories in terms of providing accessibility to all areas and to all people of the society and in terms of their capability to reduce pollution, congestion, emissions, etc. This difference, observed for two sectors in the same country, is intriguing because this implies that the stakeholders perceive that sustainability of solar PV systems is more dependent on enabling policy and governmental support than is the case for urban mobility systems.
In sum, this analysis of sectoral diversity demonstrates that even within the same country, the perception of sustainability differs markedly across energy and mobility sectors. Not only are the sustainability criteria and their respective weights different across the two sectors, but the ambiguities and uncertainties about the sustainability of the various trajectories also differ. Such diversity would have been less easy to observe, in a technique involving prior definition by the analyst of what constitutes ‘sustainability’. Finally, in the next section, we will turn to the diversity exhibited across the two countries.
Geographical diversity
Geographical diversity concerns the contrasting difference in the appraisal results in the two case study countries, namely India and Thailand. One of the first observations in this regard can be presented in terms of the diversity of sustainability criteria expressed in the two countries. Affordability, for instance, is proposed as a crucial economic sustainability criterion by almost all the participants in India, while it is mentioned only once in Thailand. On the other hand, many stakeholders state safety issues as sustainability criteria in the urban mobility workshop in Thailand, but not much in India. These qualitative differences in type and frequency of the criteria proposed shows that, even while appraising the same types of trajectories-stakeholders in India and in Thailand, participants reflected upon their local and regional context and experiences and thereby set different priorities in ensuring sustainability of the systems.
Another notable geographical diversity for solar PV appraisal is that a far greater number of criteria related specifically to governmental support and policy incentives in Thailand than in India. Qualitative data in this regard justify distinction in Thailand but not in India, of a separate group of criteria under the heading of ‘policy’. In the appraisal of solar PV trajectories in Thailand, these policy criteria also received higher average weightings than did social, economic, environmental and technical sustainability criteria. This result is illustrated in Fig. 10, where the graph on the top represents the situation in Thailand, as compared to India in the bottom where participants rated social and economic issues of sustainability the highest.
Following this assignment of highest significance to policy-related criteria in Thailand (but not India), it is perhaps relevant (despite major uncertainties) that Thai participants appraising solar PV tended also to express a discernibly stronger preference towards those solar trajectories that receive governmental policy and financial support. These trajectories (namely rooftop solar and solar power plants) were considered to be more sustainable options (in terms of higher optimistic scoring) in the final ranking of the trajectories. Figure 11 presents this result.
This trust and dependency on institutional policy and financial schemes seemed to be absent in the appraisal of solar PV trajectories in India, where the stakeholders were rather pessimistic and uncertain about sustainability of solar power plants in spite of supportive policy instruments like the National Solar mission in place. Here, they raised concerns over what were expressed in qualitative statements to be huge investment costs, long implementation times, transmission and distribution losses and land allocation requirements.
One of the other striking aspects of diversity between appraisal results in the two countries is in the levels of uncertainties with which the stakeholders appraised the trajectories. Relatively high levels of uncertainty can be observed in the appraisal of all trajectories for both solar PV and urban mobility systems in Thailand. Indeed, the high levels of uncertainty here contributed to a serious difficulty in interpreting aggregated performance diversity, in that it is difficult to see any clear overall difference in the sustainability performance across different trajectories (see Fig. 11, solar PV in Thailand). In the case of India, however, contrasting patterns of optimistic and pessimistic scoring contributed to a greater degree of confidence in interpreting the differences in sustainability appraisal of the different trajectories. (see Fig. 3 on solar PV in India).
Comparing the urban mobility cases for both countries, it can be observed that in India, stakeholders from an academic perspective expressed less uncertainty than other stakeholder groups. Interestingly, the opposite is true in Thailand, where academic stakeholders expressed the highest uncertainties when compared with other stakeholder groups in Thailand. This result is displayed in Fig. 12, where relatively short blue bars in the graph at the top represent the relative uncertainty level expressed by academicians in India, while the relative tall blue bars in the graph at the bottom represent the relative high levels of uncertainty expressed by Thai academics.
As with the other parameters analysed here, it would be hazardous to generalise to an entire country, the differences in pictures interpreted here as geographical diversity. Any such analysis would need to be based on more detailed analysis of the qualitative data to substantiate the extent to which divergent cultural factors may or may not be implicated. For the purpose of simply documenting the potential salience of diversity, however, this evidence serves quite well. Given the overall similarities in the final rankings displayed by the different experimentation trajectories in the two countries, it is quite striking that the underlying perceptions of sustainability and the specific ways in which these trajectories are appraised (optimistic, pessimistic views, expression of uncertainties and ambiguities) are so contrasting between the two countries. Following results in other MCM studies (Burgess et al. 2007), this underscores the importance of not over-interpreting the practical policy implications of wide discursive differences, and not over-interpreting any similarities in practical policy implications as indicating wider contextual similarities. Either way, it appears that diversity of many kinds remains a crucial factor to analyse.