1 Introduction

A large proportion of greenhouse gas (GHG) emissions is energy-related, and GHG mitigation implies a shift in the usage of energy carriers (IPCC - Working Group III 2007). Thus, developing GHG reduction strategies usually concords with preparing energy scenarios that describe possible pathways toward a more efficient and decarbonized energy supply and consumption structures. While traditional energy scenarios focus on technological and economic issues, awareness is increasing regarding comprehensive understanding of energy transitions. This requires a socio-technical perspective that addresses the “co-evolution and multidimensional interactions between industry, technology, markets, policy, culture and civil society” (Geels 2012:472). This insight found expression in a growing number of studies combining qualitative scenarios about the societal and political context developments and quantitative model-based energy scenarios (e.g., O’Mahony et al. 2013; Trutnevyte et al. 2014; WEC 2016). The approach enabled researchers to embed their model results on energy futures into descriptions of societies that might be able to produce those energy pathways. In this article, we term the products of such exercises as “socio-technical energy scenarios.”

The concept of informing model-based analysis by qualitative storylines about the driver developments of a system, known as the “Story-and-Simulation” approach (Alcamo 2008), found its first landmark application in climate research (Nakićenović et al. 2000). Following the traditions of climate and environmental research, storylines for Story-and-Simulation applications in energy research were developed in the past by intuitive, group discussion–based scenario techniques adopted from corporate scenario practices (“Intuitive Logics” or “IL”; Huss and Honton 1987; Wilson 1998).

While this approach enabled impressive applications in climate, environmental, and, later, energy research, it also drew concern from scholars. Garb et al. (2008) expressed unease about the growing imbalance between the rather simple storyline procedures in comparison to the sophistication of the models and observed (referring to applications in environmental research): “… the complexity of the social, economic, political and other dynamics that scenarios attempt to encapsulate is as analytically demanding as that of the biophysical systems modeled.” (Garb et al. 2008:2). As a means of improvement, scholars recommended the use of formalized storyline construction methods (e.g., Kok 2009). Several authors referred to a recently developed method, “Cross-Impact Balances” (CIB) (Weimer-Jehle 2006), as a possible alternative (Girod et al. 2009; Kemp-Benedict 2012; Guivarch et al. 2017; Carlsen et al. 2017). Schweizer and Kriegler (2012) demonstrated the added value of CIB in climate research. Lloyd and Schweizer (2014) discussed the advantages of CIB compared with IL for scientific applications from an epistemological perspective, and Kosow (2015) analyzed the impact of CIB on the traceability of socio-technical scenario construction. Taken together, the advantages of CIB were observed in terms of improved traceability and reproducibility of storyline construction, better consistency of storylines and completeness of storyline sets, and provision of the option to create a large number of storylines for statistical purposes. Thus, CIB offers the potential to bring the storyline part of socio-technical scenarios more on the level of the model part in climate and energy research. Consequently, several recent studies have collated experiences in applying CIB in socio-technical energy scenario exercises (e.g., Ruth et al. 2015; Weimer-Jehle et al. 2016; Vögele et al. 2017).

Despite the growing body of practical experiences in the application of the Story-and-Simulation approach for constructing socio-technical energy scenarios, there is little comparative analysis available on this practice. Comparative analysis is key, however, for enabling learning gains and for advising researchers interested in the application of the approach on the promises, challenges, and design options they need to consider. In this article, we present two pieces of analysis on the current practice of socio-technical energy scenarios, which are intended to provide a contribution toward filling this gap. In Section 2, we describe the general state of socio-technical energy scenario development by comparing a sample of 16 studies with respect to five categories. In Section 3, we focus on one key category: methods for storyline construction. Although the use of CIB as a methodological alternative to IL for storyline construction in energy research is a relatively recent development, there is a considerable body of applications and it should be useful to collate the experiences with this method. To this aim, we collected and systemized the statements by method researchers and practitioners on the strengths and challenges of the method. In Section 4, we discuss the findings of Sections 2 and 3 and present our conclusions in Section 5.

2 Socio-technical scenarios in energy research: state of the art

Our state-of-the-art analysis is based on studies published from 2008 to 2019 describing 16 socio-technical scenario exercises (Table 1). For collecting the sample, we combined the literature compilation provided by Trutnevyte et al. (2014) and Weimer-Jehle et al. (2016), the personal information about additional relevant studies, and the results of searching the ScienceDirect database and six energy-related journals of Springer-Link, using the search terms “energy” plus “scenario” plus “storyline.” From the results, 16 studies were selected for in-depth analysis. Criteria for selection were (i) a clear case of Storyline-and-Simulation exercise in the energy field, (ii) significant efforts in developing the storylines, (iii) sufficient methodological descriptions, and (iv) adding new aspects to the sample. The analysis does not claim completeness. Rather, it aims at an overview of the diversity of approaches, methods, and purposes in this field. The broad range of topics addressed in the storylines reflects the variety of issues that were considered relevant for understanding socio-technical transition by the authors, but cannot be covered by model analysis accordingly. We found international, environmental, political, economic, social, and cultural factors addressed in the storylines. A compilation is presented in the Supplementary Materials.

Table 1 Sample of the state-of-the-art analysis
Full size table

We compared the studies with respect to five categories.

2.1 Motivations for conducting a socio-technical scenario exercise

Many authors delivered arguments for why they opted for this approach. We identified four main arguments:

  • Storylines as a means to improve analysis

Several authors stressed that models are not suitable for exploring important parts of the dynamics of social-technical transitions, and storylines are a means to complete the analysis (Ault et al. 2008; Trutnevyte et al. 2014; McDowall 2014; Weimer-Jehle et al. 2016; Vögele et al. 2017; Pregger et al. 2019). Storylines (“framing scenarios”) can be seen a means of disburdening model analysis and a way to “...eliminate the need to explicitly model the ‘rest of the world’...” (Ruth et al. 2015:33).

  • Storylines as a means to integrate stakeholder views

Stakeholder-based storylines were expected to embody broader views (Fortes et al. 2015), to contribute novel and creative ways of thinking about the future that go beyond model insights (Trutnevyte et al. 2014), to be a dialog partner for modeling, to include key issues and tacit assumptions of decision makers (McDowall 2014), and to gain acceptance for the scenarios and prospects to be used (Trutnevyte et al. 2014).

  • Storylines as a means to facilitate communication

Storylines in socio-technical scenario exercises are a part of the story that can be told more easily than other parts of the scenario, and this can be exploited for dissemination: “...Storylines are key for communicating the results of scenario exercises. Due to their qualitative nature, they are accessible and memorable to a broad range of audiences.” (Trutnevyte et al. 2014:26)

  • Storylines as a means of cooperation in multimodel analysis

Multimodel exercises are frequently hampered by the model’s diversity of areas, scales, assumptions, and nested input-output structures. Overarching storylines are expected to mitigate these difficulties (Stocker et al 2012; Trutnevyte et al. 2014).

2.2 Models used

One might assume that the approach of coupling storylines with energy models is limited to few types of energy models with favorable properties facilitating this specific endeavor. Our sample tells a different story, however, in showing a broad range of model types. The models applied in the analyzed studies include index decomposition models (O’Mahony et al. 2013), spreadsheet models (e.g., Mander et al. 2008), accounting models (Pregger et al. 2019), optimization models (e.g., Fortes et al. 2015), system dynamics (Capellán-Pérez et al. 2014), economic models (Stocker et al. 2012), and cost models (Vögele et al. 2019).

2.3 Storyline construction

The methodologies used for constructing storylines can be distinguished by the knowledge sources used, construction methods, and procedural aspects.

  • Knowledge sources

Storyline construction needs input from knowledge carriers such as scientific experts or stakeholders. Stakeholder-based storylines were developed by Ault et al. (2008), CLUES (2012), McDowall (2014), Fortes (2015), Wachsmuth (2015), and others. Experts or literature review as knowledge sources were used for instance by O’Mahony et al. (2013), Vögele et al. (2017, 2019), and Pregger et al. (2019).

In several cases, externally developed storylines were used. For example, Capellán-Pérez et al. (2014) used storylines compiled by van Vuuren et al. (2012). In other cases, storylines were imported from overarching framework projects: Trutnevyte et al. (2014) used a storyline from Foxon (2013) and Stocker et al. (2012) used storylines from Spangenberg et al. (2012).

  • Construction methods

Corresponding to the classical Story-and-Simulation approach, many studies constructed their storylines using traditions methods like "IL" or "Scenario axes", for instance CLUES (2012), Fortes (2015), and O’Mahony et al. (2013).

CIB has emerged in recent years as an alternative technique for constructing qualitative scenarios, which can be elaborated to storylines. Ruth et al. (2015), Weimer-Jehle et al. (2016), Vögele et al. (2017), Hauser and Brodecki (2019), and Pregger et al. (2019) used CIB-based storylines to inform energy models. The strengths and challenges of this method option are discussed in depth in Section 3.

As an alternative to intuitive and algorithmic storyline discovery, storyline construction can also be guided by theoretical transition frameworks. McDowall (2014) applied the multi-level perspective (Geels 2002) and the co-evolution concept (Foxon 2011) to work out storyline motifs in terms of the frameworks. Schmid et al. (2017) applied the theory of strategic action fields (Fligstein and McAdam 2012) in a storyline analysis without connected model analysis.

  • Procedural variants

Socio-technical scenario approaches are far from being standardized operations, and we found considerable procedural diversity in the analyzed studies.

  • The classical procedure adopted from Story-and-Simulation consists of storyline preparation, translation of storylines to input data sets, model runs, and—frequently recommended but rarely executed—iteration. Several studies followed this scheme (e.g., Weimer-Jehle et al. 2016; Vögele et al. 2017, 2019; Pregger et al. 2019).

  • Instead of using storylines to define model input data, storylines and model results can also be used as independent attempts to understand energy transitions, resulting in a “dialog” between both perspectives (McDowall 2014).

  • Semi-connected approaches exist, as well. Fortes et al. (2015) developed storylines combining a socio-economic part and an energy profile. The socio-economic part was used to inform an energy model, whose results were compared with the storyline’s energy profile. Divergent results were analyzed and interpreted.

2.4 Methods for translating storylines into quantitative model parameters

The pivotal and inevitable working step for connecting storylines and models is “conversion” (Alcamo 2008), i.e., the translation of qualitative storyline elements into quantitative interpretations.

The conversion approach broadly applied in our sample, occasionally with reference to Story-and-Simulation, was to assign plausible numbers as numerical equivalents to the qualitative descriptions of the storyline elements, either before or after the storyline composition. The conversion was based on expert judgments, stakeholder interviews/workshops, and/or literature review.

A variation of this dominant approach was introduced by Mander et al. (2008). These authors started with a set of quantitatively defined alternative “end point” descriptions of the energy transition, characterizing energy demand structures. From these end points, both model inputs and qualitative scenarios about “...the form and socio-economic structure of societies in which such reductions could be achieved” were derived (Mander et al. 2008:3759), implying an inverse, quantitative-to-qualitative translation step.

2.5 Levels of integration

The building of socio-technical energy scenarios implies the merger of different scenario paradigms, disciplinary perspectives, and knowledge cultures. Not uncommonly, this materializes in different work packages conducted by different teams. Kosow (2016) proposed two criteria for classifying the roles taken by storylines and model analyses and their creators in a socio-technical scenario process: (i) degree of integration between both parts (personal overlaps, by content, by role), and (ii) mutual position of both parts (chronologically, resources, lead/service function, e.g., does the project team consider the storyline to be a tool for completing the model analysis, or vice versa?).

The range of integration in our sample varied from no integration (storylines were imported from other studies, e.g., Capellán-Pérez et al 2014) to complete integration (storylines were prepared by the model team, e.g., Vögele et al. 2017). An example for medium integration is Pregger et al. (2019), where storylines and model calculations were prepared by different teams, with modelers having a significant say during storyline construction.

The model-focused traditions in energy scenario construction might tempt one to expect a dominant position of the model part in socio-technical energy scenario exercises. Surprisingly, practice does not confirm this assumption, at least not as a general rule. The procedural descriptions in the studies suggest a rather balanced prevalence of position types. An example of a model-led exercise is CLUES (2012), where storylines were designed to fit model/quantification needs. Balanced positions were identified, among others, in McDowall (2014), who aimed at an equal dialog between both scenario types. Strong positions of the storyline part were identified for instance in Ault et al. (2008) and Trutnevyte et al. (2014).

3 “Strengthening the stories”—CIB-based storyline construction

This phrase goes back to Kemp-Benedict (2012).

As discussed in Section 2 (Subsection 2.3), two methods of storyline construction were mainly used in our sample: IL is the traditional approach, while CIB recently emerged as an alternative. Its proponents expect from CIB a more systematic and comprehensive storyline generation. IL builds on the intuitive capabilities of expert groups to design meaningful images of the future after discussing the relevant facts and trends (Huss and Honton 1987; Wilson 1998; Alcamo 2008). In contrast to IL, CIB requires experts to describe and code their insights about the system interdependencies on an integer scale and then proceeds with an algorithmic construction of qualitative scenarios or storylines.

CIB analyzes a discrete configuration space defined by a set of system variables (“descriptors,” typically 10–20) and a discrete set of alternative futures for each descriptor (typically 2–4). The alternative futures can be defined qualitatively, quantitatively, or a mix of both. This “morphological box” opens a space of typically thousands to billions of configurations or more, depending on the number of descriptors and assigned alternative futures. As a database for judging the internal consistency of configurations, a “cross-impact matrix” is built, answering the question of how a certain future of one descriptor would promote or restrict the development of a certain future of another descriptor. The impact usually is rated on a 7-point integer scale running from − 3 (strongly restricting) to + 3 (strongly promoting). Sources for the ratings can be literature review or expert/stakeholder judgments. The configuration space is searched, and only few configurations satisfying a self-consistency criterion are accepted to be “consistent scenarios.” Detailed descriptions are delivered by Weimer-Jehle (2006), Schweizer and Kriegler (2012), Weimer-Jehle (2018), and others. A short demonstration of CIB is included in the Supplementary Materials.

Given the diversity of motivations and application cases we found in Section 2, no storyline construction method can claim to be an ideal choice for each case. Well-informed choices presuppose a clear picture about the strength and challenges of the options, however. IL as a partner to environmental or energy modeling has long been applied and broadly discussed in literature (cf. Section 2). Consistent with the status of CIB as a relatively new technique, few compilations of the experiences of its use in socio-technical energy scenario construction are available. Hence, in this article, we focus on the discussion of CIB in the context of Story-and-Simulation applications and beyond, and collect and systematize statements found in literature pertaining to the strengths and challenges of CIB as a storyline generator. To allow a balanced view on the experiences that the scholars gathered from method applications and methodological research, we describe seven categories of strengths (Chapter 3.1) and seven categories of challenges (Chapter 3.2).

3.1 Strengths

3.1.1 Storyline quality

Deriving storylines of higher quality is a frequent motivation for recommending or employing CIB. The most common are references to narrative coherence and consistency (e.g., Girod et al. 2009; Schweizer and Kriegler 2012; Weimer-Jehle et al. 2016), claiming that CIB has advantages in “… ensuring internal consistency [of storylines] in the face of complexity …” (Lloyd and Schweizer 2014:2064). Scholars also refer to the advantage of providing formal consistency safeguarding in a way that keeps the process comprehensible (Wachsmuth 2015) and feasible to experts and non-experts, or as Lloyd and Schweizer put it: “… the [CIB] scoring algorithm takes pressure off any expert brains contemplating specific socioeconomic futures to test consistency, and puts it on computers that effortlessly check large numbers of scenarios for the desired consistency” (Lloyd and Schweizer 2014:2066). Kemp-Benedict concluded that the “… CIB method addresses this problem [storyline consistency], and Schweizer and Kriegler’s application of the method shows that even the best narrative-writing teams can benefit from this guidance” (Kemp-Benedict 2012:1).

3.1.2 Comprehensiveness of storyline sets

Storyline sets in exploratory scenario studies are expected to give an overview on the space of possible futures. However, several researchers have expressed concerns that intuitive approaches could easily miss important parts of the space, with potentially dramatic consequences in the case of high-stake decisions, while CIB can be expected to deliver more comprehensive sets of possible futures (Schweizer 2020; Weimer-Jehle et al. 2016). Lloyd and Schweizer (2014:2064) noted: “[While IL is] under-sampling the vast space of possible futures […] CIB can explore further than Intuitive Logics by uncovering unexpected combinations of possible outcomes.” A few existing case studies comparing IL and CIB scenario sets seem to confirm that CIB regularly generates futures overseen by IL (Schweizer and Kriegler 2012; CfWI 2014; Kurniawan 2018). Comprehensiveness is supported further by the option to generate large storyline sets, if required (Schweizer and O’Neill 2014; Pregger et al. 2019).

Other researchers reported added value from the opposite side of the problem. When Vögele et al. (2019) observed that “… without a CIB analysis, the range of possible futures might be misjudged,” they referred to a case where a purely combinatorial approach for putting together storylines would have produced misleading conclusions. The role of CIB in this case was not to extend, but rather to restrict the range of analyzed storylines to a meaningful set.

3.1.3 Transparency and other scientific criteria

Scholars comparing CIB to IL from an epistemological perspective point out that constructing scenarios by CIB is more transparent, traceable, reproducible, revisable, and, overall, more objective than the alternative IL approach (Lloyd and Schweizer 2014), making CIB more suitable for scientific development and assessment of socio-economic scenarios (ibid.). Analyzing the effects of CIB on Storyline-and-Simulation exercises, Kosow (2016:327) found that “[t] he central benefit for the participating experts is that the approach supports them in better analyzing, structuring and reflecting their assumptions, knowledge and ideas on possible future developments of our interdependent societies and environments.” Transparency, however, is not only an issue for scenario constructors: “For the external recipient users of the scenarios […], the central expected benefit [of using CIB] is that the assumptions on future uncertainty and complexity underlying different qualitative and quantitative or integrated scenarios become more accessible and critiquable. This is a prerequisite for credible and usable information—and might support the potential impact of combined scenarios in policy-advice” (ibid.).

3.1.4 Reduction of storyline subjectivity

Subjectivity is a natural ingredient of storylines if they are designed to express stakeholder visions. However, it presents a bias if storylines are expected to provide scientific analysis. Consistently, reducing (but not eliminating) subjectivity and increasing objectivity in storyline construction are a recurring argument in literature for applying or recommending CIB as a construction technique for qualitative scenarios and storylines (e.g., Wachsmuth 2015). Carlsen et al. (2017:613) noted that CIB, together with scenario diversity analysis and scenario discovery, “… can help scenario builders to be more scientific and neutral …” by providing well-defined step-by-step procedures and greater transparency, and by recognizing and systematically exploring uncertainty. Lloyd and Schweizer (2014:2085) summarize: “From a purely philosophical perspective, the CIB method clearly promotes an increase of objectivity ….” One way that subjectivity can express itself in storylines is wishful thinking, and some authors point to the capacity of CIB to confine this problem (Musch and von Streit 2017; Schweizer 2020).

3.1.5 Enabling additional insights

The prospects of additional insights conveyed by Storyline-and-Simulation approaches depend on the ability of the storyline construction method to perform a “qualitative systems analysis,” including aspects beyond the horizon of the models. CIB helps to take both qualitative and quantitative factors into consideration, which directly and indirectly influence developments (Venjakob et al. 2017; Vögele et al. 2019). Consequently, in Storyline-and-Simulation applications, CIB was found to be able to demonstrate “… significant interdependencies between societal developments and the energy transition” (Pregger et al. 2019). CIB-based storylines and their analysis by energy models “… also provide [ …] indications of possible drivers and risks related to the energy transition from which societal, economic, technological and structural conditions can interact in an amplifying or inhibiting manner” (ibid.).

3.1.6 CIB as a knowledge container

Scenarios or storylines are not the only product resulting from a CIB analysis. Its database, the cross-impact matrix, is a valuable result in itself because it makes the mental models of experts explicit and promotes focus on the most relevant factors and relationships, including those never considered before (Musch and von Streit 2017). The benefits of the database can be significantly enhanced by documenting the explanations of the experts/stakeholders about their cross-impact judgments (Weimer-Jehle et al. 2016). Verbal explanations enable analysts and recipients to decide on the soundness of the expert/stakeholder judgments and to understand the logics of the storylines.

3.1.7 CIB as a facilitator of inter- and transdisciplinary discourses

Socio-technical storylines are expected to represent relevant aspects and interdependencies beyond the traditional scope of energy modeling. To this aim, they should reflect an interdisciplinary or even transdisciplinary perspective, and the construction method should be able to handle the contributions of a broad range of experts and non-experts. IL clearly delivers on this scale. However, practitioners confirm that CIB, with skillful application, can also facilitate inter- and transdisciplinary discourses. Kemp-Benedict (2015:2) judged: “The CIB-method is well-suited to a workshop setting with participants who are knowledgeable, but not necessarily skilled at using models” and Pregger et al. (2019) reported: “[CIB] allows for close interdisciplinary collaboration between societal experts and energy modellers, leading to the identification of important new methodological insights and experiences regarding integrated scenario building ….” Venjakob et al. (2017:27–28) summarized: “[CIB’s] strength is to bring together inter- and transdisciplinary actors and enable them to design a shared and consistent picture of the future despite their different perspectives and topical foci [ …] CIB workshops [ …] convey possible future developments in a vivid way and offer an easy access to application also for laypersons.”

3.2 Challenges

These advantages do not come without burdens and limitations. Method researchers and practitioners also reported challenges connected with CIB that should be considered when choosing and employing it for storyline construction.

3.2.1 Expenditure of time

Well-done CIB exercises require considerable time resources, which smaller scenario projects might find difficult to accommodate. Several authors comment on the trade-off between analytical added value and time resources: “While the CIB method is time-consuming, it provides valuable insights, and strengthens the study results” (Kemp-Benedict 2015:4), or “We may wish to say that this level of detail is actually the strength of the [CIB] approach; nevertheless, that does not change the fact that it is costly to achieve” (Lloyd and Schweizer 2014:2067).

3.2.2 Limited descriptor numbers

The typical size of CIB analyses means that analysis is performed on a highly aggregated level and system elements (“descriptors”) of secondary, yet not negligible, importance are excluded. Both issues limit the power of the descriptors to mutually explain their behavior and unskillful selection of descriptors and their alternative futures can compromise the results. Descriptor selection can also generate pressure on the expert consultation process, since the limitation of feasible descriptor numbers can lead to lengthy discussions among the experts about the best way to reduce complexity (Musch and von Streit, 2017). Furthermore, the limitation of feasible descriptor numbers means that CIB does not cure the general problem of socio-technical scenario exercises, i.e., the problem that storylines usually do not provide sufficient information to define the complete data set needed for model runs. Instead, they still require the modelers to make interpretations: Storylines, CIB-based or not, cannot determine all parameters of an energy model in a direct way. Parameters not explicitly addressed in the storylines have to be defined by the modelers before a model run can be started. Therefore, modelers “… must set such model parameters in accordance with the modeller’s interpretation of the ‘spirit’ of the context scenario” (Pregger et al. 2019). This can lead to uncertainties in the coupling of storylines and model analysis.

3.2.3 Consistency as a construction principle

CIB may exclude scenarios for good reasons because they presuppose internal inconsistent driver constellations. Yet their discussion—at least as possible visions—might have been fruitful and their exclusion may narrow down the cognitive range of the discourse about the future (Musch and von Streit 2017). To avoid this negative effect on vision discourses, participants can be encouraged to formulate visions beyond consistency considerations and CIB can be applied for identifying implementation obstacles (Weimer-Jehle et al. 2011:31–35).

3.2.4 Subjectivity not completely removed

Storyline construction, regardless of the method used, frequently makes use of data from a broad range of sources. Data may be based, in part, on objective findings or “subjective” expert assessment may be the only accessible source. In contrast to IL, CIB processes data with “… an air of rigour …” (Drakes et al. 2017:7) that can give rise to misinterpretations. A careful interpretation of the results is crucial. Kemp-Benedict (2012:2) expressed similar concerns when he wrote: “[T] he CIB method is [ …] liable to a form of specification error, in that the worldviews of the people filling in the cross-impact table influence the results. This is a problem with many foresight techniques, but it is masked by the formalism of CIB, and there is a danger it will go unnoticed.” Different worldviews may play out in diverging assessments of interdependencies, and asking an expert group for a consensus judgment may sometimes be merely a majority vote or a result of social power relationships between the group members rather than genuine knowledge integration (Musch and von Streit 2017). The additional effort of performing separated analysis of dissenting worldviews, as practiced, for instance, by Wachsmuth (2015), may be required in those cases.

3.2.5 Need to be aware of the limitations of the approach

The role of CIB in a Story-and-Simulation approach is processing qualitative information unfit for quantitative modeling. However, “… qualitative methods, like CIB, cannot substitute quantitative analysis” (Venjakob et al. 2017:28). CIB should not be used for analyses that can be performed better by models. Further, CIB builds on the analysis of interdependencies of pairs of factors. Therefore, it is challenging to include highly complex relationships in the analysis (e.g., third factors intervening in the relationship between two other factors; Pregger et al. 2019) and it needs substantial experience to properly address such interdependencies.

3.2.6 Experience and preparation matter

Generally speaking, a lack of experience with the method can give rise to quality risks in every working step of a CIB analysis. This applies not only to the researchers but also to the knowledge contributors. If experts or stakeholder participants are expected to collaborate through the scenario construction process without sufficient preparation on the purpose and workings of the method, improper assessments (Kosow 2016) or even resistance by some participants may occur (Drakes et al. 2017). A mixed reception to the method in expert groups was also reported by other researchers (Venjakob et al. 2017), who worked with two expert groups: one easily adapting to the method and one that was reluctant. Yet, overall, those experiences seem to be exceptions, with most application reports that provide a description of the expert elicitation process not mentioning such difficulties.

3.2.7 Need of combining CIB with other methods

CIB should not be misinterpreted as an “all-in-one” method organizing the complete process of storyline preparation. Rather, it is a tool for organizing the core of the process—the actual storyline construction. This implies that there is demand for supplementary methods supporting the process before and after construction, for instance, descriptor sampling (e.g., Biß et al. 2017), conversion, or diversity techniques when selecting scenarios in cases where CIB delivers a large solution set. Selecting appropriate method combinations can be key for developing full virtues of CIB. In the words of Kemp-Benedict (2012:2): “While the paper of Schweizer and Kriegler makes a compelling argument for using CIB in global scenarios, it should be used in combination with other methods. A scenario exercise has several aims, of which consistency is one.”

4 Discussion

Although our sample cannot claim to represent the complete body of studies presenting socio-technical energy scenarios, it shows a broad variety of objectives, project settings, methods, practices, procedures, and reflections on the field. We conclude that our sample is sufficient to allow insights into the current state of socio-technical energy scenario construction in general and the strengths and challenges of CIB as an option for storyline generation.

Socio-technical scenario studies are currently not the dominant pattern in energy research, although no energy modeler can escape reflecting the interface between energy systems and society. The only choice is to take on these reflections implicitly or explicitly. We found that motivations to opt for a socio-technical energy scenario approach were diverse and ranged from epistemic considerations to practical issues, e.g., communication objectives. Given this range, it is not surprising that procedural decisions and the applied method toolbox reflect this diversity and, so far, no dominant pattern of conducting a socio-technical energy scenario exercise has emerged.

Similar diversity was found regarding the types of energy models researchers combine with storylines. The diversity of model types highlights that the basic motivations for using storylines in model exercises are generic and not confined to a certain model type. The huge variety of non-energy-topics addressed in storylines seems to indicate that many researchers and stakeholders conceptualize the process of energy transitions much more broadly than the range of processes covered by traditional energy models.

We found that the selection of knowledge sources (experts vs. stakeholders) is usually correlated with the aim of the exercise; if storylines were seen as a means of integrating normative views and visions or local knowledge into the scenario analysis, stakeholder inclusion was usually preferred. In this case, storylines can serve as a tool for making the transfer of stakeholder views to model analysis more explicit and intensifying the dialog between stakeholders and researchers.

However, authors emphasize that the effects of stakeholder inclusion go beyond the stakeholders’ role as knowledge sources. Trutnevyte et al. (2014:27) observed, “[w] hen [the storylines are] developed through stakeholder engagement, they are likely to be accepted, supported and used more often” and Wachsmuth (2015:387) added: “...it is important to integrate the stakeholders in the building of the scenarios to achieve acceptance and enable learning.” However, Wachsmuth also cautioned that “...stakeholders are to propagate their interests in such a process, even if the process has a scientific background. On the other hand, they have certain rigid assumptions that they tend not to scrutinize.”

Experts usually assumed the role of knowledge sources when storyline construction was understood as part of scientific knowledge production and storylines were expected to explore the range of possible futures or to give model input data a consistent and plausible background.

Socio-technical scenarios imply a “magic” promise of reconciling the worlds of qualitative and quantitative scenarios, attempting to bring forth the “best of both worlds” (Kosow 2016). The pivotal step to fulfill this promise is the conversion procedure. A variety of methods have been proposed for supporting conversion. Mallampalli et al. (2016) reviewed conversion techniques in the field of land-use scenarios and reported Bayesian belief networks, Bayesian reasoning, fuzzy sets, fuzzy cognitive maps, the analytic hierarchy process, role-plays, and others. In contrast, we found the practice in the field of socio-technical energy scenarios to be rather uniform; none of these sophisticated conversion methods was used in our sample, which contrasted with the general procedural and methodical diversity found in our sample. Although there may be potential in more sophisticated techniques, the dominant simple conversion approach based on direct expert judgments is a widely accepted practice. As a minimum standard, this should be made explicit and the numbers should be published transparently.

Storylines were found to be constructed in separate projects, in close or loose collaboration between modelers and storyline constructors, or directly by the modelers. Based on our own experiences in the field, we are inclined to recommend a close collaboration because low integration limits the possibility of attuning the storyline and modeling exercises to each other. Conversely, importing existing storylines can be the best (or only) option for projects with limited resources available to conduct a socio-technical scenario exercise.

Considering what practitioners reported when reviewing their own work, we found four particularly notable recommendations:

  1. i)

    Storylines cannot reflect all the technical complexity of the energy economy—this is the reason why models matter. So, several authors caution about the temptation to design storylines with strictly defined quantitative instructions for the model (Trutnevyte et al. 2014; Pregger et al. 2019). Models need a certain leeway to fill their role, or as Ault et al. (2008:9) put it: “The model could have been forced (constrained) into reproducing the scenarios exactly...” However, “...the model would have been an illustrative tool, rather than an interrogative one.”

  2. ii)

    Consider budgeting sufficient time for an iteration step—it is unlikely that a single two-step procedure delivers results without inconsistencies, and iteration is a way to clean them up, at least some of them (Trutnevyte et al. 2014; Fortes et al. 2015). The effect of iteration, though demonstrated in a different research field, can be viewed in Kok et al. (2015). An alternative approach to avoid inconsistencies is to prepare both scenario parts in parallel (CLUES 2012).

  3. iii)

    Authors warned against unrealistic expectations. For example, usually, not all parts of a storyline can be represented in the models (Trutnevyte et al. 2014). Furthermore, the transformation from storyline to model cannot be expected to be unambiguous: “[e] ach model run represents just one specific instance of inputs and assumptions for that scenario; therefore, alternative model outcomes would have been possible too for each scenario.” (Ault et al. 2008:iii) Several authors stressed that divergent results of storylines and models should not be automatically interpreted as a process failure, or as a fault of the storyline (Fortes et al. 2015). Rather, it should be understood as an invitation to analyze how different perspectives, rationales, and arguments have led to divergent results.

  4. iv)

    One study stressed the difficulties of using an “imported” energy model during a socio-technical scenario exercise (CLUES 2012). They reported the necessity of “reverse engineering” the model and the limits of understanding due to tacit model developer knowledge. Ideally, the model should be developed during the project or, at least, highly experienced model experts should be accessible for support.

IL and, recently, CIB were found to be prevalent storyline construction methods in our sample, although other construction methods exist as well and might be considered as options (e.g., morphological analysis, Rhyne 1981; Johansen 2018). The arguments set out in the literature in favor of CIB partly resulted from direct comparison with IL and partly were stated without reference to other methods. Researchers mainly saw a methodological advance in using CIB-generated storylines through better storyline consistency, better chances of discovering disregarded storyline options, and making the storyline construction process more scientific by making it more transparent, traceable, and revisable.

A major challenge of CIB in the eyes of the users was the significant time resources required for preparation and processing (leading to the pragmatic necessity to limit the number of descriptors and, consequently, the comprehensiveness of the analysis). Partly, the challenge of limited descriptor numbers may be mitigated in the future by recent method developments aiming at developing a multi-level approach for CIB analysis (Schweizer and Kurniawan 2016; Vögele 2019). Furthermore, scholars mentioned concerns about the danger of overrating the rigor of the analysis and not adequately bearing in mind the limitations of the method (for instance, the fact that expert-based cross-impact judgments are not free of at least some subjectivity). Therefore, sound experience of the method is key to an adequate execution of the analysis steps as well as for the balanced interpretation of the results.

Given these strengths and challenges, it is not surprising that, in our sample, IL was frequently used in storyline processes where stakeholders were sources of knowledge and visions. In contrast, most CIB exercises in this field aimed at scientific analysis and were based on experts as knowledge sources. However, both methods can look back on both types of storyline development.

In sum, based on currently available application experiences, CIB seems to be particularly recommendable for storyline exercises attaching special importance to the issues of consistency, traceability, and reproducibility of the storylines, or aiming at a comprehensive representation of the range of possible futures. IL seems to have advantages in storyline exercises with limited time resources, when the exercise is focused on expressing visions, or when the procedures leave no or little room for instructing participants about the construction method.

5 Conclusions

Socio-technical scenarios in energy research and their potential to support an integrated perspective on energy scenario development have gained increasing interest over the last decade. Our retrospective on the practices in the field identified several lessons that might offer advice for future applications and methodological investigations.

The diversity of motivations for opting for a socio-technical scenario approach in energy research, as well as the diversity of project objectives and model types used for quantitative analysis, makes it perfectly understandable that no clear standard procedure has emerged so far, and it might be unrealistic to expect one. However, developing consensus in the community on best practices for different project types should be a reasonable and achievable goal.

A critical issue in designing the cooperation between storyline construction and model analysis is the strength of coupling. Not all projects aim to use the models for quantifying the storylines. But if so, a very strict coupling can do as much harm as a very loose coupling. Strong coupling hinders models to employ their coded system knowledge. Loose coupling forces the modelers to excessive interpretations and storyline and model results may drift apart. A sensible design of the coupling is critical to the quality of the results.

In cases where the storylines were developed using the CIB method, researchers usually confirm that the method fulfills its core promise of improved consistency, comprehensiveness, traceability, and reproducibility of storyline construction, though it is impossible to determine the extent to which publication bias plays a role in these findings. CIB comes with a certain workload, and it works best when the method facilitators are sufficiently experienced in the method, and the people serving as knowledge sources have also received some basic training tailored to their role within the process. Researchers intending to employ the method for the first time should give themselves sufficient time for preparation, for instance, by studying well-documented application examples and conducting a substantial pre-test analysis.

Regarding the future practice of socio-technical energy scenario construction, it would be desirable if more researchers gave greater attention to reporting the problems, shortcomings, and partial failures they experience in their research. The “lifecycle” of the approach is at a stage where learning is still essential and publication bias can be a major obstacle, here.

Finally, our analysis highlighted several research areas that would be useful for supporting the development of the socio-technical energy scenario approach. Applications employing advanced conversion techniques for quantifying storyline elements would be helpful for learning what (if anything) could be gained from overcoming the currently dominant simple conversion practices. Furthermore, conducting more scenario work employing both IL and CIB to the same storyline topic would help to develop clear recommendations on which type of storyline development is best suited to different types of scenario work.