Keywords

1 Introduction

In our experience policymakers often use modelling (either themselves or by interacting with modellers) to help understand the potential impact of (in)action and identify useful points of intervention. By policymakers here we refer to civil servants who use or make models to inform government policy, while policymaking is considered as the organised attempt to select goals and methods for governmental action (Stevens 2011). However, models are often misunderstood and misused, in terms of what they can do and what models are suited to answer particular questions. The three most frequently encountered myths about the use of modelling in policymaking are outlined in Box 8.1. Greater understanding of models by policymakers is required as to what questions different models are good at answering and how they can best be used to inform energy policy.

Box 8.1 The mythology of modelling and policy

No single model or modelling process is best for policymaking. Instead, the process of designing a model is a decisive factor in what contribution the model has to policymaking (Kimbell 2011). In our experience the use of models and modelling in policymaking is often shaped by three myths.

Myth 1 is that models produce objective evidence for policymaking. Policymakers often expect that models will produce straightforward and concrete answers that can be used right away and that they will justify intervention choices. The culture of policymaking is dominated by the need to produce evidence that is statistically valid as opposed to ‘policy by anecdote’. The use of evidence in policymaking is extensively criticised (van de Goor et al. 2017; Naughton 2005). What counts as evidence is itself a politically loaded discussion (Monaghan 2008). Often uncertainty in models and accompanying narratives is reduced to bullet points, diagrams, case studies, text boxes, infographics and ‘killer charts’ or removed entirely as a potential barrier to action. In the pursuit of certainty, policymakers often lose sight of emergent complexity and contradictions. Models providing inconclusive information (with multiple caveats, limitations and elaborated uncertainty) are seen as counterproductive to creating persuasive policy stories (Stevens 2011).

Myth 2 is that models produce straightforward policy solutions. That’s why policymakers have a strong preference and expectation that models are not too complex to work with or understand, that model outputs are not too abstract and that models don’t come with high levels of uncertainty (to be able to serve as evidence for recommended policy). Models are expected to simply ‘speak for themselves’ with their policy implications being immediately apparent. In fact, models do not provide answers that can be plugged into existing policy frameworks. Modelling is a socio-technical learning process (Bollinger et al. 2015), in which models and the insight they provide develop over time by designing and using them.

Myth 3 is that policymakers need more data to take action. Big data, large, software generated and machine-readable data sets, are preferred over ‘thick data’, smaller size but deeper data that might offer greater contextual insight produced through ethnographic research. Big data can really only be used with machine learning models, which are context- and knowledge-free and can only identify patterns in data without any understanding of how and why. Big data is often left to speak for itself: ‘We don’t need theory, we have data’. Investment in required software and hardware that can process big data can be substantial, in terms of cost, time and effort. Policy institutions can end up building, ‘feeding’ and investing in large models, which progressively offer less flexibility as they grow in size. Although civil servants display a high level of commitment to the use of evidence, they are rarely able to use the huge volume of evidence they are provided with (Stevens 2011). We argue that policymakers need more contextualised understanding rather than more big data. This is particularly important in the context of energy services where users’ energy needs and wants are contextually embedded and thick understanding is needed to tell the difference between an eye twitch and a wink. A clear and well-worded case study (i.e. a narrative supported by evidence) can be just as effective in shaping policy as modelled outputs.

We use the concepts of energy demand, consumption and services to illustrate the different approaches and models that we need in order to ‘see’ demand, consumption and services. The discussion is focused on the use of techno-economic models and agent-based modelling (ABM) because they are commonplace tools used by policymakers in the UK, the Netherlands and Bulgaria, where most of our experience is based. The focus on ethnographic research has been identified as particularly welcome by the modelling community and increasingly needed for understanding energy services and their implications for policy.

This chapter is based on the authors’ combined experiences in designing and applying different types of models for understanding energy systems, as an input in the policymaking process in EU member states. In addition, the authors received input from 12 critical friends based in UK and Dutch institutions, between February and April 2018, through holding discussions around specific questions and sharing experiences of involvement in policymaking through techno-economic modelling , ABM and ethnographic input and using these in research. All discussions were recorded and summarised.

In order to explore the role of models in policymaking, and their use, we will focus on the example of modelling energy demand, consumption and services. We distinguish between ‘energy demand’ as an economic abstraction, ‘energy consumption’ as an engineering abstraction and ‘energy services’ as an ethnographic abstraction. Other definitions of these terms exist, but here we treat energy demand as the amount of energy demanded of utilities, such as the energy demand from residential heating. This amount is only loosely connected with the actual behaviour of users, which can affect utilities’ predictions of future demand. Energy consumption is what energy users actually consume, including the contributions of material factors such as house size and structure and user efforts to reduce energy use, such as through improved insulation. Energy services are what users actually want or need in terms of pure energy, such a ‘heat’, and incorporate behaviours such as users spending more time in places with shared heating (e.g. outside in the sun). Understanding the differences between these abstractions and the processes involved can enable a better understanding of the efficacy of energy efficiency improvements (GEA 2012), and the emergence of conservation movements or energy sufficiency (Herring 2009; Steinberger and Roberts 2010), and hence the design of more cost-effective energy supply regimes and better demand management programmes (Skea et al. 2011).

Residential energy use is an area undergoing significant changes in terms of policy interventions and practice, in the context of climate change, energy security, technical advancements and social and institutional dynamics. It is an important area for policymakers as it is a major contributor to overall electricity consumption and contributes significantly to peak demand, particularly during winter months in Europe (Ramírez Mendiola et al. 2017).

Policy development in the EU generally involves the collection of large volumes of data on user behaviour, for example, through electricity metering (Torriti 2014), to understand what motivates users’ behaviour and what behaviours can and should be modified. Most energy policymakers are familiar with techno-economic models of energy markets that present energy demand as an aggregate function of the decisions of individual energy users, who are generally treated as fully informed and, if not fully rational, at least predictably irrational (Huntington 2011; Wilkerson et al. 2013)—assumptions that have been heavily criticised in economics, social theory and political analysis (Sawyer 2005).

The Energy Performance of Buildings Directive (2002/91/EC) requires member states to introduce energy certification in order to reduce energy consumption in buildings. Although the directive recognises measuring both the energy demand and real energy consumption, these two approaches can lead to substantially different values. Demand represents just a ‘norm consumption’ calculated from the physical characteristics of a building, while consumption depends on many different social, technical and environmental factors (Steixner et al. 2007). Informing policy decisions on models built to represent understanding of singular concepts like energy demand can leave certain contributing factors blank or hidden while shining a light on others. For instance, in the context of residential energy demand, an abnormally cold winter or non-standard building can create a vulnerable group of consumers who are hidden to policymakers because they are unable (for financial reasons) to increase their energy consumption to the level that their energy use provides the energy service of a comfortably heated home.

Environmental concerns led to the setting of an EU-wide target in the Renewable Energy Directive of 20% of all energy consumed to be provided by renewable sources by 2020. That will require unprecedented change in the energy sector. Demand side measures and behavioural changes can significantly reduce energy demand but require an understanding of user behaviour and the implications of the built and natural environment. Hence, a good understanding of energy demand is the cornerstone of the EU’s future energy system (GEA 2012).

Looking at this problem from the perspective of energy consumption, and ultimately energy services, encourages an emphasis on this behaviour, looking at the energy that is desired ‘for the services that it produces, such as space and water heating, cooling, lighting, cooking, etc.’ (Hunt and Ryan 2015). In other words, ‘useful energy’ or ‘useful work’ of energy is being put to work in a way that is distinct from the energy use itself (Sorrell and Dimitropoulos 2008, p. 20). This ultimately is the behaviour that will be affected by any decisions made by policymakers and is therefore the behaviour that needs to be captured in the models used by policymakers.

2 What Model?

The ultimate goal of modelling is to gain insight and not necessarily the production of a number to satisfy policymakers. Models are a simplifying lens through which we look at the energy system. The choice of model we use defines what we see. Models help us focus on particular relationships between variables and how changes in different parts affect the entire system. Through a techno-economic lens, residences can be seen as subsystems of the national (even continental) energy system, which comprises a multitude of installations, both on the supply and demand sides, which are interconnected through pipelines and cables. Through a Social Sciences lens, the energy system is seen as a huge network of actors, including power and heat generators, network owners and operators, energy service providers, end-users, technology providers, energy authorities and policymakers. Through the ethnographic lens, we see residents acting in their home environment, following daily routines and interacting with appliances and installations in their built environment in ways that determine how and why they use energy. These lenses are complementary and together provide a richer, more sophisticated understanding of the energy system. Hereafter we will explore the models and understanding that have emerged from these three different lenses, and discuss their usefulness in policymaking.

2.1 Techno-economic Models

Techno-economic models are being increasingly relied on to better understand what combinations of measures, over what time frames and at what costs, will be required to meet energy policy goals (Winskel et al. 2011). Many policymakers and energy companies base their policies, tariffs and projects on data collected on user load profiles (Torriti 2014). Techno-economic models use such profile information as well as data on the underlying techno-economic characteristics of the residential energy systems. They model the underlying physical structure of the system, such as building sizes and types, and the ratings and engineered characteristics of appliances (Swan and Ugursal 2009). They focus on how much energy could be consumed by different types of households and what appliances are used and when. They provide insights into levels and timing of demand as well as long-term trends that can affect these.

Techno-economic models can be used to provide insights on residential energy demand and consumption for places, using data from a subset of the population. They specify causal relationships (based on the laws of nature) and engineering heuristics (e.g. scaling rules), and may use stochastic or exploratory analysis in dealing with uncertainties. They can come as optimisation or simulation models. Optimisation models search for the ‘best’ system configuration in a given normative scenario. Simulation models are used to develop forecasts of how the system may evolve under different scenarios, especially economic conditions. Many models informing energy policymakers are bottom-up optimisation models, based on detailed technical specifications of the subsystems and their components. Often these models have a legacy of use: policymakers are familiar with them and tend to trust their outputs (Ramírez Mendiola et al. 2017).

Techno-economic models are based on rigid mathematical formulations that can be solved analytically. In the current policy system, closed-form analytical solutions are automatically accepted as true rather than looking for alternatives that best describe what the need is, even if this problem is mathematically ‘messy’. Furthermore, the level of aggregation models use cannot take individuals into consideration. In their focus on optimising the economic performance of the aggregate technical system, they generally assume homogeneous actors. As these models are based on past behaviour and lack the granularity needed to fully understand residential energy services, they are mostly incapable of anticipating or incorporating changes in user behaviour, new trends or technologies or system shocks.

Apart from numerical outputs, techno-economic models are also accompanied by narratives which explain the levels of uncertainty built in the model and the ways in which the model can be interpreted. While these supporting narratives provide boundaries within which the model makes sense, policymakers may not fully acknowledge these limitations in their decision-making. This is problematic, given the enormous effort invested in such large-scale energy system models (and their data sources), which creates a tendency for policymakers to stick with the established models and instead fit problem formulations to the capabilities of the available modelling platform.

2.2 Agent-Based Modelling (ABM)

For a deeper understanding of what drives the behaviour of energy systems, individual-level behaviours and relationships must be added to the picture of the system-level causal relationships that are captured in traditional techno-economic models. Actors in the energy system perform certain roles, defined by institutions (norms, conventions, legislation, regulation, market rules, etc.). They can pursue their own strategies within their limits (be they institutional, technological, capital, knowledge, information, etc.). Acknowledging that the continuous interactions between actors, and between actors and the physical system, shape the behaviour of the energy system implies a socio-technical perspective on the energy system. From this perspective, understanding the interactions between the social elements and between the social and technical elements and subsystems is indispensable for providing policymakers with an understanding of where and how to intervene in energy systems.

These individual behaviours can be translated into ABMs, in which actors are represented as heterogeneous software agents. Like real actors, software agents can be programmed to exhibit varying extents of bounded rationality, imperfect information access and risk aversion and be equipped with learning capabilities. ABM can thus be seen as a crossover between the Engineering and the Social Sciences, as it aims to simulate aspects of real actor behaviour, expressed as computer algorithms interacting with physical systems, in effect translating qualitative behaviour into quantitative data and processes.

In ABM, the technical components and subsystems can be modelled as they are with techno-economic models, and economic optimisation models can be included as parts of individual decision-making routines. ABM is eminently suited for bottom-up energy system simulations that explore emergent system behaviour, for example, under varying institutional regimes. For residents to be adequately modelled as agents in ABM, insights are needed into their behaviour as energy users and how this behaviour is shaped by their service needs and their specific socio-economic, cultural and physical environment. This is where ethnographic research can play an important role.

2.3 Ethnographic Approaches

Ethnography, as an approach to collecting data, entails a wide variety of methods. The artefacts, processes and relationships studied in ethnography will depend on the context of the study. Participant observation and shadowing can be thought of as ‘traditional’ methods of ethnography. While observation implies a level of detachment from what is being studied, shadowing could involve ‘doing’ in order to enhance understanding, as well as conducting interviews. These ethnographic approaches are apt for studying the relationship between human and non-human objects in the performance of everyday activities, and the meanings ascribed to various everyday activities. Unlike techno-economic models, these can investigate worldviews, sociocultural structures and the practices that shape behaviour, and help readers to immerse themselves into the world being studied. The purpose of ethnographic research is developing what Geertz (1973) calls ‘thick understanding’, ‘a stratified hierarchy of meaningful structures’ (p.6), which can help us tell a twitch from a wink, a fake wink or a parody.

This is helped by the multitude of forms of ethnographic data (some of which are far more emotive than numbers) which could include photos, videos, quotes, objects and diaries (Kimbell 2011). This characteristic of ethnographic approaches enables them to contextualise findings and analysis and to deliver situated understanding of particular issues. Ethnographic research is widely used in the academic and non-governmental realms, as a standalone and powerful means for studying and engaging with interdependent relationships occurring in everyday life. The usefulness of ethnographic research here is discussed only in the context of the contribution it can make for policymakers in understanding the limitations of models and working with them in the process of policymaking.

While ABM and techno-economic models are led by a set of rules and assumptions, ethnographic research can be hypothesis-free and exploratory, as well as more narrowly targeted. The residential focus on energy lends itself well to an analysis through ethnographic approaches, because it can incorporate different socio-technical drivers.

3 Bringing the Approaches Together

This section discusses how techno-economic and ABM models can be used together, along with ethnographic research, in a complementary way to enhance policy understanding of residential energy consumption, demand and services. Residential energy demand models have been classified into two main approaches, top-down and bottom-up (Swan and Ugursal 2009). The top-down approach makes use of historic sector-level time series data of energy consumption through an analysis of long-term trends in macroeconomic factors such as changes in GDP, employment, housing builds and climate. Bottom-up models on the other hand look to build up models of residential demand from a hierarchy of individual end-users, houses or groups of houses (Grunewald et al. 2016).

Techno-economic bottom-up models tend to provide a static representation of user behaviour and trends based on known drivers and hence are limited in their ability to expose new or unexpected behaviour or emergent trends in the system. A more dynamic understanding of user behaviour can be developed through the use of ABM and ethnographic shadowing and observation, as an additional class of bottom-up methods. Their applicability is well explained in the task of understanding the divide between energy demand, consumption and services, which is explained through the ability of each to incorporate behavioural or ethnographic information.

Through ethnographic input, informal (and otherwise hidden) activities and practices can be included in ABM (such as switching off the fridge in the winter to save on electricity). Observations and shadowing can add value to ABMs by informing the context of player’s interactions and behaviour. In turn techno-economic models can provide physical and monetary processes and inform the description and powers of actors in ABMs.

Ethnographic outputs often do not fit the idea of evidence in policymaking due to potential subjectivity bias and lower number of cases studied. However, in the context of energy poverty, the abstraction and aggregation usually employed for ABM and techno-economic models create their own biases that can make it harder to ‘see’ the point in making policy to protect the few at the expense of many (see Middlemiss et al., Chap. 2, and Aberg et al., Chap. 4, in this collection). Middlemiss and others, for example, emphasise the need to include consideration of the ‘lived experience’ for understanding energy poverty. Ethnographic approaches can help here to create harder hitting and empathetic understanding of different aspects of energy policy.

Techno-economic models are already using ethnographic research to provide improved predictive strength, to validate system processes or to better understand conditions under which models may or may not apply (Swan and Ugursal 2009; Grunewald et al. 2016). However, it could be argued that ABMs are better equipped to utilise ethnographic research, providing behavioural insights captured in ABMs through bounded rationality in agents’ behaviour, providing them with limited information and inconsistent preferences to add realism. Ghorbani et al. (2015) have adopted such a complementary approach to produce empirically grounded reasoning in ABM.

ABMs and ethnographic research are also best placed to provide a testing ground for proposed policies. They allow policies to be assessed in realistic situations, providing feedback on the efficacy of policies in differing environments, an understanding of any perverse or unforeseen outcomes or behaviours and insights into avenues for policy improvements. Ethnographic approaches can help policymakers understand how consumption changes with a new policy based in energy services, and ABM techniques can be used to scale up such behavioural responses to examine the implications of such emergent trends on the system as a whole.

Techno-economic models usually attempt to understand the more complex behaviour of individuals through increasing sample sizes in data collection. This is a limiting factor for ethnographic research due to the labour and costs involved in observation and shadowing activities. However, big data collection processes can be improved through ethnographic research to provide a better understanding of trends in the data (Strang 1996; Ladner 2012), including giving a reference baseline for detecting trends, informing the data collection processes (Wilson et al. 2015) and providing new or improved explanatory variables for analysis. Ethnographic approaches can develop thick data, which in turn can provide a contextually embedded understanding of the systems for ABMs . Equally, what someone might show or tell an interviewer might be very different to how they actually behave (Grunewald et al. 2016), and big data on actual appliance usage supplied by engineering and statistical studies can be complementary.

The value of incorporating ABMs into techno-economic energy demand modelling does not end with the improved access to ethnographic information. Techno-economic models and ABMs , both informed by ethnographic research, can thus be used in a complementary combination of quantitative and qualitative data. A complementary approach can help produce multiple perspectives and thus produce more nuanced understandings of the problems at hand, as well as possible solutions. Including ethnographic research on a par with techno-economic models and ABM can produce a richer type of data —that is, taking into account a wider range of social and environmental elements (Lockton et al. 2013)—and understanding.

The outputs produced by such a complementary approach will aid in gaining multidimensional insight that policymakers would not get otherwise, and that can open up opportunities for exploring alternative scenarios for policy intervention. It can lead to policymakers asking different questions of the models as well as contribute to the development of a multilayered narrative to accompany the results. Rather than welcoming more complexity as an output of different models and approaches, we encourage acknowledging difference and competing information and exploring such disjunctures further rather than keeping them hidden or discounting them completely.

4 Changing Attitudes and Practice Towards a Complementary Understanding: Recommendations to Policymakers

Increasingly policymakers need to understand the needs, capacities and perspectives of a variety of actors, including citizens, service users and beneficiaries, so that policies are fit for purpose and deliverable (Kimbell 2011). Therefore, energy policy needs a complementary understanding from techno-economic models, ABM and ethnographic observation and shadowing. However, this will require changing attitudes and cultures of policymakers and policymaking. Building on the three myths outlined in Box 8.1, the first three steps in this direction should involve:

  1. 1.

    Appreciation of limitations, rather than seeking evidence, as a modelling output. The policymaking process needs to develop greater awareness and appreciation of the limitations of different models available, both in terms of modelling process and outputs. This involves expanding policymakers’ understanding of what constitutes evidence (to include a wider variety of data, especially thick data), where it comes from and how evidence is developed. The latter involves getting more directly engaged with the process of modelling and allowing for a complementary use of model types and approaches. This would imply a slower and more involved evidence-gathering process.

  2. 2.

    Confronting modelling outputs rather than looking for straightforward policy solutions. Policymakers need to appreciate the value of combining and ‘confronting’ outcomes of different modelling approaches, rather than seek non-controversial answers and evidence. This can be achieved through training in modelling, using models with researchers and modellers, as well as testing findings against ethnographic research. Confronting involves a more open process of evidence gathering for policymaking, as well as developing additional steps of confronting, reconciling and embracing controversy and complexity in the policymaking process.

  3. 3.

    Building a community of interdisciplinary policymaking users rather than focusing on providing more data for policymakers. Policymakers need to be enabled to work in interdisciplinary contexts and with interdisciplinary research teams. This involves a concerted effort and investment in building a community of policymakers willing to accept and embrace interdisciplinary models and outputs. Within the UN and peacebuilding contexts, for example, managers are accountable for the data they use (or don’t) to shape their decisions. This involves asking managers if they have the right kind information for the question at hand, enough information, and whether that information is reliable (Kimbell 2011).

The three recommended steps should be approached simultaneously, as they reinforce each other. For example, investing in building a community of interdisciplinary policymaking users will help foster the skills needed for confronting modelling outputs and appreciating the limitations of modelling and approaches used, and vice versa. However, further research is needed to fully understand how these three approaches can be brought together in a complementary way and what the limitations of this complementarity may be. To start with this research agenda can explore the extent to which complementary understanding can be achieved when strongly conflicting understandings are produced through different approaches (i.e. what are the limits of confronting and reconciling differential understandings within a policy area). Such research will need to be based on more interactions and experiments between policymakers, models and complementary approaches. These could take the form of open policymaking labs, and take more informal and direct formats, such as modelling and policymaking hackathons, studio workshops and walks.

The proposed approach can be considered at odds with dominant policymaking processes, where policymakers have to research, design and propose policy interventions within limited time frames, usually months. However, most policy is anticipatory rather than responsive, and considering unanticipated policy impact if policy is not fit for purpose, the case for ‘slow policymaking’ can go a long way in managing potential risks. Furthermore, with the development of a community of interdisciplinary policymaking users with experience in confronting modelling outputs, associated costs (in terms of invested time, effort and money) can decrease.