Keywords

1 Introduction

Analysing energy demand and energy-efficient solutions in final energy sectors or in the transformation sector is a difficult and cumbersome task. However, even more challenging are projections of future energy demand in these sectors and of thousands of possible more efficient applications, particularly in industry. Research on energy demand and its possible reduction by more efficient energy use starts in the early 1970s when the report of the Club of Rome was published in 1972 and, more importantly, the first oil embargo was initiated by the OPEC-countries in late 1973 and the second oil price increase in 1979–1980. Until this time, energy policy focused on sufficient energy supply: oil, coal and nuclear energy, the new abundant and inexpensive form of primary energy.

In the 1950s and 1960s, energy demand projections were a domain of energy economists who calculate the final energy demand at a high level of aggregation by the elasticities of energy prices and demand drivers such as the number of private households, of cars or the net production of industry (Morrison and Readling 1968). Knowledge of engineering was focused on the transformation sector (i.e., power plants, refineries, coal and coke production). Foreseeable structural changes in industry, in the economy, or saturation effects in private households were not yet considered in energy demand projections (Kraus 1988). The energy future of the decades until 2000 seemed to be quite clear and straightforward.

During the five decades starting in 1970 the attention given to energy demand substantially changed with regard to the political debate and policymaking as well as to research on energy using technologies. We portray how both new political challenges and related energy policy strategies led to new energy-efficient technologies and solutions. These developments induced the need for more technological content in the energy demand projections leading to advances in methodologies of energy systems research. The new energy demand models also captured the dynamics of technical innovation, structural change of the economy, saturation effects, and patterns of decision making. This chapter describes how energy systems research developed from an area which is little known into a differentiated field of research within five decades.

This chapter is organised by decades beginning with the 1970s. There are developments covering two decades; however, they may be more important for future energy demand in their initial phase or at a later stage. One development, which can be observed throughout the five decades is the increasing attention given to energy-efficient solutions. However, the present energy policy in many countries pays lip service to catchphrases such as “Efficiency First”. This changing attention was reflected in the policy of the IEA (International Energy Agency n.d.) which focused on nuclear energy in the 1970s and on efficient energy use today (Lantzke 1980; Geller and Attali 2005; IEA 2022a).

2 The 1970s: The End of the Energy Growth Dream

The early 1970s were characterised by the two preceding decades of substantial economic growth in all OECD countries. Primary energy consumption increased until 1970 with a demand elasticity of around 1.0 in many OECD countries, and the elasticity of electricity demand often even surpassed that level (Lebert 1977; Berndt 1978; Bohnen 1982). Most energy economists of the 1970s were convinced that energy demand increases at the same rate as the gross national product. National energy policy focused predominantly on increasing energy supply, particularly in the fields of mineral oil and nuclear energy.

Energy demand projections of the 1970s until the target year 2000 expected further substantial increases in the demand for heating oil, gasoline, and diesel as well as coal and nuclear energy for electricity generation. However, three events dramatically changed the scene of energy economics research and energy policy in OECD countries:

  • In 1972, the Club of Rome published its famous analysis “Limits to Growth”, which applied newly developed system dynamics models to worldwide economic developments and their consumption of natural resources like fossil fuels and basic materials, but also to food production and environmental pollution (Meadows et al. 1972).

  • In September 1973, the West German government released its First Energy Programme (BMWi 1973), 4 weeks before OPEC decided to reduce oil production by 25% and placed an embargo on the large oil importing harbour of Rotterdam due to the Suez crisis. The oil price (fob) quadrupled to around 12 US$/ barrel. The second oil crisis in 1979 accelerated the challenge to the paradigm of ever increasing energy demand.

  • In the early 1970s, new computer systems became available that were able to calculate complex optimisation and simulation model runs which had not been possible in the 1960s.

There were numerous reactions of different governments and the research community:

  • OECD countries established the International Energy Agency (IEA), responsible for planning oil storage and the supply of crude oil and oil products in the wake of the oil crisis, but also for strategic energy forecasts and for suggesting new energy-related policies (Lantzke 1980).

  • Following the first oil crisis, energy efficiency options in buildings (Prömmel 1978), industry (Jochem et al. 1978), and transport were for the first time clearly addressed by technological-based energy research.

  • In the 1970s and at the beginning of the 1980s, this led to considerable controversy and tension among energy researchers in many OECD countries, although this is hardly communicated internationally. Instead, the international scene was dominated by the International Association of Energy Economists (IAEE) and representatives of large publicly owned energy research organisations with their focus on nuclear energy. Attempts to report energy demand and the potentials of efficient energy use in energy journals or conferences often came to nothing.

  • In West Germany, for instance, an intensive scientific discourse between three traditional energy economics institutes (EWI, RWI, ifo) and Prognos, and the “newcomers” of energy systems analysis (ASA der Großforschungseinrichtungen: Bohn et al. 1977; Nitsch et al. 1981) and Fraunhofer ISI (Bossel and Denton 1977) began to address the future impacts of energy efficiency policies, the changes to a service-oriented economy, as well as the structural changes within industry and individual sectors.

The West German government published its first amendment to the Energy Programme in October 1973 (BMWi 1973). At national and federal state level, the ministries of economics were responsible for energy issues, but in the 1970s there was not a single organisational unit that dealt with energy demand or energy efficiency.

After the first oil crisis, the traditional energy economics institutes continued to project substantial increases in primary and final energy demand until 2000. For instance, DIW et al. (1978) projected a gross electricity demand in 1985 of between 1830 and 2030 PJ, implying demand elasticities between 1.4 and 1.55; electricity demand in 2000 was projected to grow to 3000 to 3200 PJ. Actually, however, the electricity consumption of the Federal Republic of Germany amounted to only 1230 PJ in 1985 (overestimated by about 50% for a period of only 7 years) and to 1780 PJ in 2000 for a reunified Germany (overestimated by around 70%).

During the mid-1970s, the new technology-oriented energy systems analysis groups argued that increased energy prices will encourage more efficient use of energy, that economic growth is likely to slow down to a linear per capita growth and will be induced more by growth in services and low-energy branches of industry like investment goods, durables, and consumer goods. They expected a substantial decrease in the growth of energy-intensive industries—and sometimes stagnation—in future decades. This “should result in projections of demand being considerable lower than currently available estimates” (Bossel and Denton 1977; Möller and Ströbele 1978; Neu 1978).

In the 1960s and early 1970s, there were only very few research institutes specialised in the efficient use of energy in OECD countries. Taking Germany as an example, there was only one research institute at the Technical University of Karlsruhe (Mueller 1957) which examined energy efficiency. In addition, some branch-specific institutes covered the topic of energy efficiency as a “by-product” while conducting research to improve branch-specific production processes (e.g., VDEH (Verein Deutscher Eisenhüttenleute, the Association of German Steel Manufacturers), the Brick Research or Textile Research Institute, etc.). Berg (1976) characterised the situation by his introductory first sentence in the first volume of the Journal “Annual Review of Energy”: “In the winter of 1973–1974 energy conservation became a popular subject of discussion, so much so that the subject was often, and not entirely inaccurately, referred to as energy conversation”.

The first studies on the potentials of energy efficiency in energy-intensive industries were published in the late 1970s (Jochem et al. 1978; Berndt 1978). The methodology applied included interviews with energy managers, process engineers, and applied researchers as well as analytical statistics. The results of these efficiency studies were used to simulate and project industrial energy demand under higher energy prices in OECD countries. The first macroeconomic studies on a reduced oil supply with the associated impacts on growth, employment, and emissions appeared in 1976 with models by Bossel et al. (1976a). Scenario methodology in energy systems analysis was applied systematically for the first time. This approach was developed by the Batelle Institute in 1976, but has its origins in 1969 and was originally designed to consider military strategies (see Cuhls et al. 2024 in this anthology).

These first approaches of energy system modelling as a basis for energy demand projections were—compared to today—still rudimentary, among other things because national energy balances were usually very rough; energy engineering-based models had not yet become available. To start with, systems dynamics methods are taken up by natural science/engineering groups (Mesarovic, Pestel, Kortzfleisch in Germany), and other computer-based modelling is initiated by many other research groups (Bossel et al. 1976b). However, these approaches were abandoned in the 1980s, also because the parameters required for science-based modelling could not be estimated with sufficient accuracy due to the lack of empirical data.

3 The 1980s: Starting Energy Efficiency Policy—Supported by a First Generation of Technically Based Energy Models

3.1 First Energy Efficiency Policies

The second oil crisis of 1979 induced first substantial policy measures on the efficient use of energy, in particular regulations and standards for new buildings (Schipper et al. 1979; Gruber et al. 1982). In the mid-1980s, low-energy buildings were already a legally required energy standard for new buildings in Sweden and Denmark. Even at that time, it was considered to further develop the principles of low-energy housing, such as first-rate insulation, prevention of thermal bridges, airtightness, insulated glazing, and controlled ventilation. Based on these considerations, the “Passive House” was launched by Bo Adamson and W. Feist in Sweden in 1988 (Levine and Adamson 1988).

The complex interaction of the factors driving energy demand addressed by Darmstadter et al. (1977) led to a method explaining how various drivers ex post contribute to the observed changes in energy demand and intensities by energy conservation indicators—for the EU (Morovic and Schön 1987, 1989). The individual influencing factors such as inter-industrial and intra-industrial structural change, short-term structural changes in industry by the business cycle (Garnreiter et al. 1986), structural change in passenger and freight transport (Schipper et al. 1997), the influence of the heating and summer period as well as the more efficient use of energy were analysed using time series and provided indications for scenarios and projections of energy demand in the future.

These studies were flanked by detailed technological examinations of energy-intensive industries at the sectoral level and process technologies, e.g. specific analyses for the basic chemical industry regarding energy efficiency and raw material substitution as well as demand reduction. Another example is the non-metallic minerals industry with its very energy-intensive production processes and energy efficiency potentials offered by innovative technologies for cement, brick, lime, and ready-mixed concrete as a service. The textile industry has almost completely been outsourced and only finishing partly still takes place in Germany and in most OECD countries. This is an example of intra-industrial structural change in the 1970s and early 1980s.

Based on these insights, energy efficiency policy measures were increasingly developed and proposed to national governments or the European Commission by administration, researchers, and trade associations. These include, e.g. regulatory policies for energy efficiency standards for new buildings, speed limits in road transportation that are implemented, eco-design guidelines for mass-produced products, energy labels for domestic appliances, EU energy efficiency directives, energy management systems such as ISO 50001, EMAS, and lists of energy efficiency consultants. Financial incentives for new energy-efficient solutions and consultation were developed for industry, as well as accelerated depreciation, which also applied to private consumers and based on scientific findings. These include energy taxes with exemptions for energy-intensive or internationally competitive industries to help them retain their competitiveness.

Rather late on January 1, 2005, the European Union Emissions Trading System (EU ETS) was implemented in order to limit the emissions of greenhouse gases of energy-intensive industries and electricity generation. And even three years later, the first scientific journal devoted to efficient use of energy, Energy Efficiency, started its first volume.

3.2 First Generation of Technically Based Energy Models

Early bottom-up models emerged in the 1980s and quickly became more and more complex. Based on how the models consider technology choice and adoption, they can be grouped into accounting models, simulation and optimisation models (Fleiter et al. 2011). Accounting models are simply based on exogenous assumptions of, e.g. energy efficiency progress or technological change, to calculate resulting energy demands and CO2 emissions. They aim to answer “What if …?” questions. Examples include early models like MEDEE or MED-Pro (Chateau and Lapillonne 1978; Chateau and Lapillonne 1990; Lapillonne and Chateau 1981), PRIMES (Capros 1995), IKARUS in Germany (Hake et al. 1994), or the leap model framework. In the 1980s and 1990s, this led to optimisation models which were linked to input–output models and macroeconomic models, usually transferring the respective results and necessary data manually, with iterative computer runs between the different types of model (see also Figs. 1 and 2).

Fig. 1
A mapping diagram with two columns of text boxes connected by lines. The left column lists barriers, such as lack of knowledge and market transparency and financial bottlenecks. The right column lists solutions, like on-the-spot consulting, energy labeling, and financial incentives.

Development of technologies of efficient energy use and of methods projecting energy demand in final energy sectors, 1960 to 2030. Projection: ISI’s own estimate. Top: Development of the technologies for efficient energy use and of methods for projecting energy demand in final energy sectors, from 1960 to 2030; Middle: Development of historical annual average prices of crude oil from 1861 to 2021, source S&P Global Platts. Lower light grey line: historical nominal price; upper dark grey line: real price adjusted for purchasing power in 2020; Bottom: Development of the European CO2 emission allowance price (EU ETS) and the CO2 allowance price for residential buildings and transport in Germany, which is not yet subject to the EU ETS, as well as their lower and upper price projections. Source: Past: German Emissions Trading Authority DEHSt

Full size image
Fig. 2
3 images. Top. A chart with 6 columns and 2 rows from 1960 to 2030. Middle. A graph of price of C O 2 in the E T S and non-E T S sectors from 1960 to 2030, with projections for upper and lower prices. The bottom graph illustrates historical and projected oil prices from 1960 to 2030 under S T E P S, A P S, and N Z E.

Obstacles and market imperfections of energy efficiency and related policies—a scheme for policy options and integrated efficiency policies. Source: Jochem et al. 2000b

Full size image

The optimisation models at the beginning of the 1980s, initially designed and used for optimal structures of electricity generation and other energy supply, were increasingly extended to include additional parts modelling the final energy demand of buildings, electrical appliances, road, rail, ship, and air transport, as well as energy-intensive sectors and production processes. Examples include the development of simulation and optimisation programmes such as MARKAL in the USA (Abilock et al. 1979; Sweeney 1981) that were initially adopted by the major European research centres. Similar model types were developed in the EU such as MEDEE.

Over time, these different models and energy fields became increasingly electronically interconnected (Herbst et al. 2012, see also Fig. 1).

4 The 1990s: Moderate Energy Efficiency Improvements—Obstacles, but the New Driver: Climate Protection

During this decade, globally important events occurred which influenced energy perspectives and related research: the re-organisation of the countries that had belonged to the former Eastern bloc, the Conference on Environment and Development in Rio de Janeiro in 1992, and a new legislation on the liberalisation of energy markets in many OECD countries. The Framework Convention on Climate Change (FCCC) was concluded and signed at the Rio Conference on June 4, 1992 (Adede 1995). The Kyoto Protocol to the United Nations Framework Convention on Climate Change (UNFCCC) of 1997 is the most important allegiance for the future of efficient energy solutions. It contains legally binding commitments of industrialised and emerging countries for reducing anthropogenic greenhouse gas emissions. This Protocol has led to many in-depth studies including efficient energy use and energy conservation since the 1990s (IPCC 2001a).

4.1 Examining the Details of Efficient Energy Use, Options Reducing Energy Demand, and Their Social Benefits

In the 1990s, the price of oil was cut in half and there was a corresponding reduction of the interest in energy efficiency and renewable energy sources. Some energy researchers slowly became aware that the market prices for fossil and nuclear energy do not cover the external costs of energy consumption, such as soil acidification (“forest dieback”), lung diseases caused by particulates and ground-level ozone, etc. (Hohmeyer 1988). This implies that energy prices cannot be left purely up to the market, but require government intervention such as taxes on the consumption of fossil fuels or subsidies for energy-saving measures and renewable energy sources. However, there was great opposition to this idea, from circles of conservative scientists and politics (Rennings 2000).

Simultaneously, the calls to protect the climate grew louder with the 1988 Conference on the Changing Atmosphere in Toronto, the Enquete Commission’s recommendations (German Bundestag 1991), or the 1992 Earth Summit in Rio (sustainability). These events pushed energy topics higher up the agenda in science and politics. In practice, however, and due to low energy prices there was only minor progress in energy efficiency up to the end of the 1990s. Therefore, the 2nd and 3rd report of the Intergovernmental Panel on Climate Change (IPCC 2001) contain warnings about this trend, in particular about stranded investments and lock-in situations in OECD countries due to the adherence to old fossil-based energy sources.

Nevertheless, some practical progress was made. Wind power and (10 years later) photovoltaic systems spread rapidly due to the politically created favourable framework conditions in some countries. Houses with very low heating energy demand (so-called passive houses), thermal solar panels, small and more energy-efficient passenger cars, boilers with condensing technology and thermal solar support also found their way onto the market (see Panny et al. 2024 in this anthology).

In the 1990s, policy makers in the field of energy efficiency also wanted to know how effective individual policy measures were in order to learn from them for future measures. The main results for industry were (Diekmann et al. 1999): It makes sense to distinguish between electricity and the other final energy carriers. Real net production value is considered to be the suitable activity in energy intensity studies. Both inter-industrial and intra-industrial structural change (shifts in product structure, product-based services) significantly influence the development of energy intensity. Structural change caused by business cycles is also taken into account as an explanatory variable in the case varying energy intensities in industry (Morovic and Schön 1989). In this decade, it was hardly possible to specify or project the influence of (often new) energy efficiency policies, because results of measure-specific empirical evaluations were missing.

Although energy efficiency and energy intensity indicators were clearly defined by the end of the 1990s in terms of methodology and data and were recognised as very useful for understanding past energy consumption, today—20 years later—they are still hardly used and often only in a very undifferentiated form, despite the energy policy mantra of “Efficiency First”.

Only sluggish progress was made in implementing energy efficiency potentials at two levels of energy use, i.e.

  1. 1.

    Cross-Cutting Technologies—improving their efficiency in energy and exergy terms, while converting final energy into useful energy

  2. 2.

    Reducing the useful energy demand of production technologies in industry and crafts by improving and substituting processes, or by reducing final energy demand in industry by intensive use of waste heat.

Final energy demand in industry could also be diminished by reducing demand of basic products due to increased resource efficiency—today this is labelled “Circular Economy” (Jochem 1991; Angerer 1995; Radgen and Tönsing 1996):

  1. 3.

    Increased recycling and improved material efficiency of energy-intensive materials.

  2. 4.

    Substituting materials with less energy-intensive ones.

  3. 5.

    Intensifying the use of durables by sharing and leasing.

Although numerous energy efficiency improvements became highly profitable following the second oil price increase in 1979, few efficiency improvements were observed in the early 1980s in most OECD countries. High potentials of the “fifth energy resource” were overlooked by many companies, administrations, and private households or judged to be “purely theoretical” or “unfeasible”. The heterogeneity and diversity of energy consumers, the variety of energy-efficient solutions and of the related manufacturers of energy-efficient equipment contributed a low perception of the potentials offered by energy efficiency in the 1980s. Because of this variety and complexity, energy efficiency—in many cases quite profitable—had (and even still today at much higher energy prices) very little appeal for either the media or politicians (Jochem 1991). This sluggish progress in more efficient energy use despite high energy prices and despite new efficient solutions by research and development led to substantial socio-economic and psychological research on decisions of efficiency investment and behaviour of various target groups in the 1990s.

4.2 Obstacles and Market Imperfections

Obstacles and market imperfections of energy efficiency in end-use sectors have been observed and reported for more than 30 years. Although limited, the empirical quantitative research on these barriers highlighted the large diversity of individual investors (e.g., thousands of firms (capital-based, family-owned, small or large), hundreds of thousands of landlords or homeowners, and millions of consumers in a single country).

In theory, given all the benefits of energy efficiency at the business and macroeconomic levels (Hohmeyer 1991), a perfect market would optimally allocate the rewards from these energy-efficient solutions. In practice, however, researchers and consulting engineers in the 1980s and 1990s observed many obstacles and market imperfections that prevent profitable energy-efficient solutions from being fully realised. Energy policy researchers began publishing these observations in the early 1990s (Jochem and Gruber 1990; Hirst 1991; Jhirad and Mintzer 1992; Weber 1997). This led to a growing body of literature on the so-called barriers to energy efficiency. Barriers are described as “a mechanism that inhibits a decision or behaviour that appears to be both energy efficient and economically efficient” (Sorrell et al. 2004), a phenomenon that has also been described as the energy efficiency gap.

In fact, these findings in the area of energy efficiency are simply an example of market and system failure. This fact, however, has to be differentiated by several aspects, reasons, and target groups in order to set up an adequate policy design.

Although, in principle, the types of obstacles and market imperfections are universal, their importance differs among sectors, institutions, and world regions, depending on many factors including technical education and training, entrepreneurial and household traditions, the availability of capital, and existing legislation (see Fig. 2).

  • Market imperfections include many forms of subsidies and traditional legislation and rules, but also the traditions and motivations of behaviour in private households, and of decision making in companies and administrations (Sanstad and Howarth 1994).

  • The “invisibility” of energy efficiency measures (in contrast to photovoltaic or solar thermal collectors) and the problems with demonstrating and quantifying their impacts are also important factors for private households, companies, and car investments due to social prestige aspects (Sanstad and Howarth 1994).

  • Psychological reasons comprise another problem affecting energy efficiency measures. These include lack of attention, knowledge, know-how and technical skills, and unspecified transaction costs. Improved energy efficiency is brought about by new technologies or just incremental changes to a known product, process or vehicle, and by changed forms of organisation. This implies that investors and energy users are able to get to know and understand the benefits of technical efficiency improvements as well as to evaluate the possible risks against perceived benefits. This also implies that investors or users have to be prepared to realise improvements and to give themselves time to absorb new information and to evaluate the innovations (Levine et al. 1995; Sioshansi 1991; DeCanio 1998). Private households and car drivers, small and medium-sized companies, small public administrations or banks do not have enough knowledge about the possibilities and risks of energy savings or sufficient technical skills to implement them. Managers, preoccupied with daily routines and core business areas, are able to only engage in the most important and immediate tasks (Velthuijsen 1995; Ramesohl 1999). Energy efficiency, with its minor role in running a business or its potential to reduce only a small share of the energy costs of total production or household costs, was placed on the back burner.

  • Energy consumers may also face a lack of access to capital, or they may follow historically or socially formed investment patterns. Even if they acquire the knowledge they need, they often face difficulties in raising funds for energy efficiency investments. Their own capital may be limited and additional credit may be considered as too expensive. Especially if interest rates are high, small firms and private households prefer to accept higher current costs and the risk of rising energy prices instead of a later energy credit.

  • Relying on investment risk decisions and neglecting the profitability of energy-efficient investments was (and still is today) a major obstacle. Energy consumers demand payback periods of between 1 and 4 years, which are equivalent to an internal rate-of-return of about 25% to 50% (DeCanio 1998; Gruber and Brand 1991; Schröter et al. 2009). This rate-of-return expectation rules out highly profitable efficiency investments and favours investments in energy supply, resulting in an inter-sectoral disparity of profitability expectations of at least 10% to 30% distortion of energy-saving investments (Jochem and Gruber 1990).

  • Legal and administrative obstacles are observed in almost all end-use sectors. These are mostly country-specific and often date back to before 1973, when there were low and in real terms declining energy prices, and there was no awareness of the threat of global warming.

  • The investor/user dilemma describes the fact that, for rented dwellings or leased buildings, there are few incentives for tenants to invest in property they do not own. In the same way, there are also few incentives for landlords, builders, or owners because of the uncertainty of being able to recover the investment through a higher rent (Fisher and Rothkopf 1989).

For every obstacle and market imperfection discussed, there are interrelated measures of energy efficiency policy that could remove or reduce them, as illustrated by a few examples in Fig. 2. Yet, the choice of which policies to pursue has to be made with care as their effectiveness depends on many regional, cultural, political, and societal factors.

At the beginning of the 1990s, pilot tests and field test demonstrations became an additional component of research projects as demand-side management always includes the users of technologies. In addition to technical solutions, the role of economic incentives and prices became more important (Brand et al. 1988). Researchers supported and accompanied many of these first field tests, e.g. (Jochem and Gruber 1990; Hennicke et al. 1998). Economic efficiency was one of the main drivers of demand-side management approaches as was the research on demand elasticity and incentives for efficient energy use. Key outcomes identified and assessed the impacts of economic incentives and possible policy measures to improve the energy system (Zweifel et al. 1997).

The continued analyses of obstacles in industry led to further proposals for policy measures as more and more obstacles were identified that can only be overcome with bundles of policies. From the early 1990s, these were documented in the reports of the Enquete Commission for “The Protection of the Earth’s Atmosphere” (German Bundestag 1991, pp. 378–395), and also in reports at EU level and by the IPCC Group III Report (IPCC 2001 and related publications such as Jochem et al. 2000a).

5 The Noughties—Liberalised Grid-Based Energy Markets and the Takeoff of Electricity Producing Renewables: New Boundary Conditions for Energy Demand and Challenges for Energy Modelling

One important obstacle for efficient energy use, the monopolistic structure of grid-based energy industries, was widely discussed in the 1990s (Walz 1994) and gradually diminished by new legislation in all industrialised countries between the mid-1990s and the beginning of the noughties (Finon and Midttun 2004). A new business field, energy services, was widely discussed, and it was particularly to be offered by gas and electricity distribution companies. This change, however, from maximising energy sales to optimising sales and savings, was perceived as a mayor cultural change in the traditional energy supply companies. Financial incentives by governments, strict control by the antitrust authority, and stiff competition by energy service companies from investment goods industries and consulting companies gradually forced functioning liberalised energy markets in the noughties.

Early in the noughties, only a few researchers estimated the liberalised energy markets as an absolute precondition for very high market shares of renewables in the future. It is a common understanding that “more than 30 per cent of fluctuating electricity from wind and photovoltaics will not be substantially surmounted” (Nitsch 2000).

More regulation, not less, was temporarily necessary, if effective competition was to be established in grid-based energy industries. Traditional optimisation models simulating the decisions of a monopolistic energy market had to be replaced at the branch level. The liberalised markets placed new requirements on computable models: they were to provide realistic descriptions of technologies (demand, production, transport, and distribution), but also of markets and institutions. Industrial economics and computation of economic equilibrium were to help achieve this dual requirement (Smeers 1997).

The scarcity of conventional crude oil was highlighted by the buzzword peak oil resulting from an increase in the oil price to more than 100 US$ per barrel (nominal) in the period 2008 to 2014. This caused a renaissance in politics, society, and research of energy efficiency and renewable energies. Their impacts on energy demand, however, were smaller than expected due to rebound, income, and price effects.

5.1 New Policy Instruments Implementing Energy Efficiency

Expectations of further increases in energy prices (including increasing prices for emissions of energy-related CO2) and rising greenhouse gas emissions in the mid-noughties steer research away from simply analysing the obstacles to efficient energy use and more towards exploring which instruments can be used to overcome them—if possible several of them simultaneously.

A successful example of a new instrument is the “Learning Energy Efficiency Network”. This had its origins in a group of Zürich entrepreneurs in 1985 (Bürki 1999) and was further developed in the 1990s in the Swiss industry. Since 2002, the format has been adopted to the situation in Germany (Gruber and Jochem 2007). In this scheme, between 10 and 15 energy managers of companies in a region come together and agree on a joint energy efficiency target and CO2 mitigation for the network, which should be achieved within about 4 years. Prior to this, specially trained consultant engineers have analysed the energy efficiency potentials in the participating companies, and each company has to set itself a respective target. At around four meetings per year, the measures already implemented by a participating company are inspected and the experience gained is shared. In addition, external experts are invited to talk about new and interesting energy efficiency technologies and know-how is exchanged among the participants.

This approach was so successful that more than 20 associations of German industry had pledged in a voluntary agreement with the German Government in 2014 to establish and operate 350 new energy efficiency networks between 2015 and 2020 (Dütschke et al. 2016). This approach was also modified for small enterprises and local authorities and spread to Austria, Sweden (Palm and Backman 2020), France, China and, in the late 2010s, to countries in South and Central America, Asia and Africa (Durand and Damian 2019). Principally, participating in those networks substantially speeds up the implementation of energy-efficient solutions. Energy efficiency and climate protection networks can be understood as “group energy management systems”. Scope 3 emissions from upstream and downstream processes are an integral part of climate protection networks since the early 2020s (Eberle et al. 2022).

Competitive bidding for energy-efficient investments was also introduced in several countries as an incentive for industry and commerce. Energy service companies, consulting engineers, or manufacturers can compete with their efficiency investments that cannot (yet) be standardised. This instrument strengthens the competition among energy service companies and realises branch-specific efficiency potentials which are not covered by standardised incentive schemes (Pehnt and Brischke 2013).

The growth of renewable energies and the liberalisation of the grid-based energy markets brought substantial changes to the electricity and gas supply sector. The emergence of new actors and competition led to a first transformation of energy companies with the unbundling of power generation, gas production, and grid operation. High price fluctuations as well as capacity shortages, on the one hand, and missing grid infrastructure, on the other, triggered a strong push for demand-side measures. The power crisis in California in 2000 acted as a strong push for research (Faruqui et al. 2001). Major energy research questions concerned themselves with how to reduce costs through a better utilisation of assets and infrastructure, for which the demand side plays with additional flexible demand a more important role. Increasing the number of flexible demand-side participants and reducing their final energy demand induced research activities that focused on the participation of numerous and new stakeholders (Braithwait and Eakin 2002; Department of Energy 2006; FERC 2006).

5.2 Rebound, Income, and Price Effects and Technical Forecast

Despite additional energy efficiency policies in the noughties and the high oil prices between 2008 and 2014, consumption of oil products did not fall as expected. For example:

  • Car engines became more efficient per unit of power, but cars became bigger and heavier with more powerful engines.

  • Airplanes became more energy efficient, but the number of flights and passengers increased as did the distances flown.

  • Living space per capita also increased and often offset thermal insulation measures.

  • Factories using energy more efficiently became more profitable and competitive encouraging further investment and higher levels of output.

This empirically observable effect, known as the direct rebound effect (which also includes income effects and changes in preferences of private users), was intensively discussed in the noughties (Sorrell 2007; Gillingham et al. 2016). Even if demand of energy services remains unchanged, energy savings across the economy may be less than simple calculations suggest. The question remains what the saved energy budget will be spent on—whether marginal consumption or additionally possible investments induce more or less energy demand. In addition, reductions in energy demand will translate into lower energy prices which encourage increased energy consumption (energy price effects or indirect rebound effects; Sorrell 2007). Not only is this effect important for the design of policy instruments, it is also often overlooked in energy economic models and forecasts (see below).

In addition to these behavioural changes, new technological developments are also difficult to estimate. In order to provide a better scientific basis to project future developments or events, various methods of technological foresight are now combined with one another: For example, the speed at which the technological maturity of individual technologies is reached from the idea to market maturity can be mapped with the help of bibliometric and patent analyses (Jochem et al. 2009). Joint international publications on topics also reveal in which regions developments are taking place and how the players—researchers, manufacturers, and first applicants—network with each other in the early stages of the technology cycle. The number of patents in the individual world regions may indicate the time of market entry and speed of diffusion (Bradke et al. 2009).

It is possible to estimate future cost developments by analysing cumulated production and the associated production costs or market prices, and thus the budgets required and the market opportunities compared to competing technologies (Jakob and Madlener 2004). These empirical findings can be transferred to other technologies by analogy and thus enable well-founded recommendations for the need for initial financial support of new efficient technologies. In cases of mass production, they are projected to “run down their cost curve” for estimating their market diffusion in energy system models.

5.3 Demand-Side Management and Second-Generation Models

Fluctuating electricity production by renewables has started to turn around the role of supply and demand in several countries since the noughties: in the traditional electricity system, production followed the pattern of demand; in the future, however, electricity demand has to realign with fluctuating electricity production by wind, photovoltaic, and hydropower. With the rise of renewable energies and the intensified discussion about climate change, demand-side management and the related research focus on questions of system integration and the security of supply.

Especially in countries with higher shares of renewables and more liberalised markets like in Scandinavia and especially in Denmark, the demand side’s contribution to integrating renewables developed into a major topic. An intense discussion between grid operators, policy makers, and research institutions on the potential maximum shares of renewable energies in the power system was the result (see also the discussion on renewable energies in Panny et al. (2024) of this volume), as well as how to balance the power system efficiently (Moller Andersen et al. 2006; Nordel 2004; Nordel Demand Response Group 2006).

Load shifting in very electricity-intensive production processes such as manufacturing electrical steel, aluminium, and chlorine has been around for a long time, but increasing digitisation makes it economically feasible to include smaller industrial and commercial electricity consumers in load management. In several studies and empirical research activities, researchers identified and supported efficient energy use and demand-side management, not only to improve competitiveness of the power system, but also to support the integration into the electricity system and the security of supply (Klobasa 2009).

In line with these considerations, the electricity demand models became much more dynamic in terms of time to match the increasingly fluctuating electricity supply and load shifting options, particularly in countries with substantial seasonal changes of sunshine, wind, or hydropower. In addition, technically based models became more refined and considered future machines and plants, both as optimisation and in simulation models (Quiggin et al. 2012; Sensfuß 2007).

Simulation models went further and endogenised technology diffusion by simulating the investment decisions of actors (e.g., industrial companies or building owners). Although these models are a rather heterogeneous group, most of them represent the age of the technology stock and track individual age classes of, e.g. buildings, cars, or industrial facilities (Fleiter et al. 2011), which yields a more realistic modelling of system inertia (obstacles and market imperfections) and the speed of technological change. Simulation models often use a discrete choice framework that simulates technology choice as a competition among alternative investment options (Elsland et al. 2013; Fleiter et al. 2018; Palzer and Henning 2014).

For example, building sector models typically include various choices of heating systems and consumers decide which one to install based on the total costs of ownership (Stadler et al. 2007). Simulation models are, however, also more experimental and can include elements of non-rational investment behaviour. Daniels and Van Dril (2007), for instance, consider psychological energy price effects and bounded rationality. The CIMS model considers a time preference, heterogeneity of the market and a factor that integrates all other elements of non-rational investment choices (Horne et al. 2005; Murphy et al. 2007; Rivers and Jaccard 2006). In the NEMS model, investments in energy efficiency technologies are determined by payback time thresholds reflecting empirically observed simplified decision rules (Worrell and Price 2001; Energy Information Administration 2009).

Despite their many advantages and a high level of technology detail, bottom-up models also have shortcomings. These include their dependence on detailed technology data and the lacking empirical foundation for data and behaviour assumptions as well as technological optimism. Efforts have been made to estimate decision-parameters empirically (Rivers and Jaccard 2006; Rehfeldt et al. 2019; Beugin and Jaccard 2012). In addition, bottom-up models tend to look at the technological system without considering the interactions with and feedback from the economic system, which motivates researchers to develop hybrid models that draw on both engineering and economics. An important example of such a hybrid model is the Canadian CIMS model (Rivers and Jaccard 2005; Murphy et al. 2007). Other modelling teams couple different types of models in applied studies. Overall, modelling teams increasingly traverse the boundaries between individual disciplines and models incorporate the advantages of different research streams (Herbst et al. 2012; Pfenninger et al. 2018). Applications of modelling tools for policy consultancy often result in combining individual complementary models by, e.g., using bottom-up models to assess technological change in detail and top-down models to provide the overall economic frame (see Fig. 1).

6 The 2010s: Tensions—Slow Progress in Efficient Energy Use, Still Increasing Greenhouse Gas Emissions, Electrification of the Transport Sector, Sector Coupling, and Further Progress in Energy Modelling

The December 2015 Paris Climate Agreement refers to the alarming scientific evidence on global warming. The Paris outcome legitimises more climate action around the world. The question is whether this will happen quickly enough and on a sufficient scale. Certainly, it will not occur without far-reaching government intervention in energy markets and resource efficiency in the next few years, particularly in the largest polluting countries (Clémençon 2016). Energy research took up many of the challenges of the Paris Agreement prior to the 2010’s (Jochem 2004); however, the ability to speed up the transformation by politicians and citizens could scarcely be observed by the authors until the middle of 2023, when they completed this publication.

Fracking, the new method to extract additional crude oil and natural gas, was widely accepted in the USA in the 2010s, turning the USA from a net importer into a net exporter of crude oil. The surplus of available oil resulted in the price of oil dropping to around 50 US$ per barrel between 2014 and mid 2020 (see Fig. 1). On the other hand, the findings of climate research still determine energy research and energy policy to a greater extent, but with a moderate impact on the transition needed.

More recent research since around 2010 aims at embedding the barriers of efficient energy use into broader frameworks including:

  1. (a)

    The conceptualisation of the decision making on energy efficiency investments as a process,

  2. (b)

    the consideration of psychological factors as well as social dynamics,

  3. (c)

    broader analyses of the impact of energy-efficient solutions in terms of their co-benefits, but also downsides like rebound, income, and price effects.

  4. (d)

    And finally, a wider systemic look at the energy-efficient performance of the energy demand and supply side (sector coupling).

The process perspective on decision making is influenced by psychology and aims to show the different stages through which a decision in favour of an energy efficiency investment needs to pass before it is implemented. This includes identifying the need for the investment, compiling information, the actual planning and finally the decision and its implementation. However, this process can end at each of these different phases and usually other actors besides the actual investor also play a role. For example, Globisch et al. (2018) show that the expected reactions from co-workers are important for fleet managers before they invest in electric vehicles. Arning et al. (2020) also point to the crucial role of installers and crafts men in renovation decisions.

The literature on co-benefits highlights the additional effects of energy-efficient solutions like increased thermal comfort in buildings, less noise, improved illumination of production areas, constant product quality, less wastes, or increased real estate value (Reuter et al. 2020).

6.1 Sector Coupling and Integrated Energy

The increasingly ambitious climate protection targets set since the Paris Agreement at national levels require reduced greenhouse gas emissions by reducing final energy demand and by substituting fossil fuels with energy from renewable sources. On a large scale, this can mainly be achieved through electricity from wind, sun, and hydropower, which are subject to intermittency. Questions emerge on how to manage these non-controllable energy sources, how to handle excess electricity generation, and use it in an efficient way in terms of economic, ecologic, and social welfare aspects. To efficiently integrate these variable primary energies, the traditional coupling of the power sector to the residential, transport, industry, and commercial sector has to be adopted to several changing boundary conditions (Schaber et al. (2013), Schaber (2013), Richts et al. (2015)):

  • Increasing electricity demand due to substitution of fossil fuels in the transport sector, in industry and commerce (including large heat pumps for district heat systems);

  • Short-term electricity storage and longer term storage by thermal heat (including the short-term function of buildings) or hydrogen, ammonia, or methanol;

  • Integrating millions of very small photovoltaic and wind generators operated by private households (balcony collectors), small companies, and communes.

This adopted electricity system is called sector coupling (SC). Due to the strong expansion of fluctuating renewables in Central and Northern Europe as well as California since 2010, the discussion about SC started in these two regions. As the shift continues towards the energy transition, in 2017, several German ministries and international energy agencies developed detailed guidelines and information on SC (see BMWi (2016), BMUB (2016), BDEW (2017), IRENA et al. (2018)). In 2020, the European Commission presented a comprehensive EU Strategy for energy system integration (European Commission (2020). A year later, the IEA published a study highlighting the role of SC in ensuring energy security (IEA (2021).

Although the terms “sector coupling” (SC) and “integrated energy” are frequently used in the current energy policy debate, they are often not used clearly or uniformly (Scorza et al. 2018). Several different definitions can be found in Ramsebner et al. (2021). Following one of the first definitions by Wietschel et al. 2018, SC is seen as the “substitution of fossil fuels in conventional technologies with alternative primary energies (e.g., renewables including wind, solar, hydro, biomass, geothermal) in new applications or technologies”. This can be done either by directly using electricity, such as.

  • in Power-to-Heat PtH, e.g. heat pumps, electro-thermal industrial processes,

  • in Power-to-Move PtM, e.g. vehicles driven by electrical motors or by converting electricity into synthetic fuels,

  • Power-to-Gas PtG (e.g., hydrogen) as substitution of conventional fossil gases, and

  • Power-to-Liquid PtL (e.g., green ammonia, methanol, or e-fuels) as substitution of fossil fuels.

These electricity-based final energies are subsumed as Power-to-X (PtX) energies. In addition, the focus here is on the use of new or alternative technologies and less on classical power applications such as electrical motors, night storage heating, or electric trains and trams. This view of SC focuses on techno-economic issues. The broadly defined aspects of SC also encompass new standards, new business segments, IT issues (including cyber security), and legal as well as regulatory aspects.

The EU Strategy (EU 2020) concludes “that the transition to a more integrated energy system is of crucial importance for Europe. First, for recovery. It proposes a path forward that is cost-effective, promotes well-targeted investments in infrastructure, avoids stranded assets and leads to lower bills for businesses and customers. In short, it is key to accelerating the EU’s emergence from the actual economic crisis and for mobilising necessary EU funding as well as private investments. Second, for climate neutrality. Energy system integration is essential to reach increased 2030 climate targets and climate neutrality by 2050. It exploits energy efficiency potentials, enables a larger integration of renewables, the deployment of new, decarbonised fuels, and a more circular approach to energy production and transmission”. Whether energy efficiency potentials will be sufficiently exploited in this supply-oriented concept of SC will be questioned in the outlook of this publication.

6.2 Electrification of Road Transport

As already mentioned, a general long-term trend in energy systems with a large impact on society, economy, and policymaking is the gradual change towards direct use of electricity in many applications where fossil fuels are (or were) used. As electricity has been widely available for decades in OECD countries and electric motors are clearly more efficient than combustion engines, electric vehicles in road transport have been researched since the second oil crisis in 1979. Yet, the first fleet trials and vehicle demonstrations did not lead to mass-market introduction. The situation changed around 2010 due to several factors: improvements in battery technologies (lithium-ion batteries offer higher energy density and thus longer ranges), seriousness of climate change with actual policies dramatically reducing tail-pipe emissions of newly sold vehicles beyond the reach of combustion engine vehicles (in particular the 95 g CO2/km target for new vehicles in Europe).

Against this background of a changing CO2 landscape, plug-in electric vehicles (PEV) have seen strong support and research. A substantial impact on the existing electricity system is expected as the previously uncoupled sectors of electricity and road transport interact with vehicles frequently connected to the power grid (see the preceding chapter on sector coupling). Early on, the scientific debate has focused on the potential integration of intermittent renewables supplying electric cars and trucks (e.g., Dallinger et al. 2011; Dallinger and Wietschel 2012; Mwasilu et al. 2014; Wang 2021). Many simulations of PEV charging behaviour and their interaction with the grid showed that PEV represent an important additional load but offer only limited power storage capacity. Accordingly, demand-side management and smart charging are the most important aspects of PEV (Peters et al. 2012).

The impact of PEV charging on electricity grids and power generation receives considerable attention in the literature. The additional load implies that some distribution grid extensions or more controlled charging will be needed. Interestingly, although the uptake of PEV requires additional investments on the distribution grid level, the specific grid charges are reduced as the additional electricity demand increases overall grid utilisation and thus lowers its specific costs (Kühnbach et al. 2020). This effect is even higher than the increase in generation costs due to integrating flexible generation with high variable costs (Kühnbach et al. 2020).

Analyses of road transportation systems conducted in the last decade conclude that policies and decision making must be based on a thorough understanding of PEV users and the future market uptake of PEV. Many researchers make important contributions not only to the aforementioned aspects of renewable integration and grid impacts, but also to market diffusion scenarios and the characterisation of PEV early adopters. In a series of national and international publications, they improve existing methods to analyse the future market diffusion of new technologies in an empirically grounded manner (Plötz et al. 2014; Gnann et al. 2015) and help to analyse the national transition towards PEV (Plötz et al. 2013). Early adopters are described in terms of socio-demographics but more importantly for SC, they are shown to also be frequent owners of home PV systems and already use fully renewable electricity contracts (in both aspects showing much higher shares than the general German population) (Scherrer et al. 2019; Preuß et al. 2021; Lee et al. 2019). Likewise, research reveals that today’s plug-in hybrid electric vehicle (PHEV) users do not charge their vehicles as frequently as expected (Plötz et al. 2021). Interestingly, there is little research on the efficient electricity use regarding battery charging, wheel driving, storage, and recuperation (Synák et al 2021).

As the transition towards electric passenger cars is already underway, the next open research field for SC in road transport concerns heavy-duty vehicles. Initial results show that battery electric trucks can reduce well-to-wheel emissions from heavy-duty vehicles but represent an inflexible load for electric road systems (Plötz et al. 2019) and megawatt charging (Speth et al. 2022). Researchers will continue to advise policymakers and industry as well as civil society with up-to-date research on the future of electric road transport, e.g. in leading the construction of the first megawatt chargers in Europe (cf. https://www.hochleistungsladen-lkw.de/).

6.3 Energy System Modelling

Over decades of improvements in energy demand modelling, the research questions have also changed dramatically. While early bottom-up models looked at energy efficiency potentials, i.e. how much improved energy efficiency can reduce overall energy demand (e.g., Worrell et al. 2000), the focus then shifted to CO2 abatement and carbon neutrality, driven in particular by the Paris Agreement and a stronger public push for climate protection by many governments since 2016.

As a result, the models are challenged by the need to include deeper structural and technological changes and regional information to achieve greenhouse gas neutrality (Pfenninger et al. 2014). The focus shifted to topics like sector coupling, electrification, new energy carriers like hydrogen and the potential market diffusion of immature novel technologies. In addition, the simulation of policy instruments like CO2 markets, standards or subsidy schemes gained importance as policymakers demanded more guidelines on how instruments will impact on future demand and CO2 emissions (see Fig. 1).

Among other things, these developments require higher temporal and spatial resolution. Energy system models moved from considering individual generic type-weeks towards hourly resolution of the entire year to capture the effects of high wind and solar generation on the system (e.g., Sensfuß et al. 2008). As a consequence, topics such as demand response, e.g. from electric vehicles or heat pumps, have become more important (Boßmann and Staffell 2015; Boßmann et al. 2015; Boßmann and Eser 2016).

With research focusing on SC, entire teams of researchers couple specialised models of individual demand sectors with supply side or systems models with the aim of improving the resolution in the representation of decarbonisation pathways (Sensfuß et al. 2021; Del Crespo et al. 2020). Others integrate the overall system into one optimisation modelling approach that targets minimised overall system costs with perfect foresight (Pfenninger et al. 2014; Plazas-Niño 2022). This is done at the expense of losing technology information to make the optimisation problem solvable (e.g., Henning and Palzer 2014).

At the same time, spatial resolution has also increased drastically from country aggregates to, e.g., NUTS3 regions, or even individual points. A main driver for this is linking the modelling of infrastructures like electricity, gas, heat, or hydrogen transport networks. Only with high spatial resolution can such models consider structural changes in energy demand across regions. While early models calculated energy demand as annual aggregates of one region (e.g., a country), contemporary models aim at hourly resolutions of energy demand and can break down demand spatially from country aggregates to individual NUTS3 regions or even local centers of high energy demand. Some detailed sector models even went further and represented individual agents and their interactions within the simulation (see, e.g., Nägeli et al. 2020 and Steinbach 2016).

Independent of the respective research questions, the movement towards open source models has gained huge momentum over the last decade, driven by public authorities and designers of research programmes like Horizon 2020, with their greater priority for open source and transparency. This is a result of dissatisfaction with “black box” models and the associated difficulties in explaining the results and making the causal chain comprehensible. Modelling teams have reacted and many new open source models have emerged (Pfenninger et al. 2014; Hörsch et al. 2018; Brown et al. 2018), although the opening of proprietary models is still an ongoing process.

It is very likely that modelling approaches in the future will diversify even more and specialise in answering specific research and policy questions. Time will show whether open source models are a way to collaboratively build even bigger models with even more detail. Computing power will certainly continue to increase and will drive corresponding developments in energy system modelling towards greater levels of detail with higher temporal and spatial resolution. New methods like machine learning are likely to play a bigger role.

6.4 Digitalisation: Supporting the Energy System Efficiency

Already at the beginning of 2000, several research papers discussed the links between information technologies and the energy system and identified the potential benefits of closer interactions. Concepts and ideas for sustainable consumption emphasise and indicate individual solutions for different stakeholders to improve the sustainability of the energy system (Schleich et al. 2013). Knowledge about demand and generation is key to stabilise grid infrastructures, but communication links for small-scale assets are still developing. Research topics cover these aspects and several new concepts are elaborated and tested. Researchers support the development and add new dimensions focusing on participation and the acceptance of technologies and their use (Tureczek and Nielsen 2017; see also Heyen et al. (2024) of this anthology). In particular, the planned smart meter roll-out raises several questions about data security and privacy that also affect its implementation into the energy system.

With the increased number of small-scale and decentralised generation assets, the research community’s interest turns to their controllability and current status information and large-scale research programmes launched. Within the e-energy programme running from 2008 until 2013, the main goal is to optimise especially the power system using ICT technologies (BMWK 2014). The programme led to technical solutions that are still lacking suitable market solutions and regulatory frameworks. Furthermore, the inclusion of key stakeholders on the demand side and the implementation into real-life settings are identified as major gaps for fast diffusion of the solutions. This leads to new research approaches developed under the concept of smart energy showcases SINTEGFootnote 1 (Klempp et al. 2020) and real-life laboratories (REALLABOREFootnote 2), where possible solutions are developed and shown to a wider number of stakeholders in more industrial scale conditions. Five key requirements are identified that are needed in future power systems where digitalisation is seen as a key enabler: Increasing the flexibility of energy supply and demand, integrating flexibility into energy markets and grid operation, optimising and securing the control of flexibilities, testing and validating new solutions in an efficient and fast way, and finally increasing participation and acceptance of energy system users. On an international level user participation and engagement as well as market and grid integration are also identified as key areas where digitalisation can play a crucial role (CODES 2022).

With clear challenges ahead like climate change and the advancing digitalisation of the economy, research concepts are moving more in the direction of a mission-oriented approach, which defines societal goals and clear steps for how and when these should be reached. Related to smart meters and digitalisation of the energy sector the mission-oriented approach sets a clear focus on largest benefits of these technologies to reach climate goals while avoiding or minimising negative environmental impacts. Researchers support this approach with state-of-the-art concepts, including how to best use digital technologies to support energy transmission, improve system operation, and include demand-side options (Klobasa et al. 2019; Singh et al. 2021). Relevant research questions are concerned with how to adapt and change current regulatory conditions (Bekk et al. 2021) and improve demand-side participation (Kühnbach et al. 2022) on a low voltage level as well.

7 Summary and Outlook

First doubts on ever increasing energy demand were expressed in the early 1970s (Chapman et al. 1972). Increasing disputes among energy economists and energy systems analysts could be observed in the energy-related journals in the late 1970s and 1980s about the importance of energy efficiency potentials and their profitability (Hatsopoulos et al. 1978). Existing obstacles and market imperfections called for technology- and target group-specific energy efficiency policies. The importance of structural changes within the economy to a more service-oriented economy, within industry in favour of non-basic product branches, and even within individual branches to more content of services (e.g., maintenance, ready-mixed concrete), was heavily discussed. The dissent among energy economists and engineers in the 1970s and 1980s can also be understood given the expectation of very inexpensive electricity generated from nuclear power.

First recommendations for energy efficiency research programmes were made to governments, including the development of more detailed energy system modelling, based on models of operations research or simulation. However, since the early 1980s—after the second oil price increase in 1979—the topic of energy efficiency has been increasingly accepted as an “energy source” and detailed energy demand projections have received increasing acceptance. Aspects of obstacles, market imperfections, innovation, and related policies have been taken up by the scenario techniques since the late 1980s, assuming different intensities of energy efficiency policy or efforts and successes of research in efficient energy technologies (e.g., passive houses, waste heat use, electronic control and sensors (Craig et al. 2002)).

Increasing analytical details of efficient energy use, effects of structural change, and saturation on final energy demand are challenging impulses for the development of new methods such as multi-disciplinary hybrid models, complex statistical analytics, patent and bibliometric analyses, multi-criteria assessment methods (see Fig. 1). The social cost of energy use (Hohmeyer 1988), the social benefits of efficient energy investments (such as additional employment and additional exports) led to additional energy-related data and new versions of input–output models (Legler and Jochem 1977; Geller et al. 1992). Final energy demand of several EU countries dropped between the 1990s and 2020 by around 10% despite economic growth.

At the turn of the century, liberalisation of grid-based energies reduced one of the market imperfections and offered the opportunity even to energy supply companies to sell energy-efficient solutions as an energy service. This development realised the statement of some authors of the 1970s that energy efficiency should be considered as “energy source” (Lovins 1976).

Increasing market shares of renewable energy in electricity generation started to convert the role of electricity demand as a driver of electricity generation to the opposite: daily and seasonally fluctuating electricity generation from photovoltaic, wind, and waterpower increasingly determines the patterns of electricity demand. This change induces new technical and organisational innovations in areas of higher variability of electricity demand, hourly fluctuating electricity prices, electricity storage, and more intensive coupling in the energy transformation sector, particularly with heat use and storage, and hydrogen applications in the coming decades. These changes become particularly difficult in northern industrialised countries, if they have little potentials of hydropower, biomass, or geothermal energy. Thus, more efficient energy use may get more attention in the near future—supported by reduced demand of basic goods due to the upcoming circular economy (Jochem et al. 2004). Even the post-growth economy may become a game changer in the long run reducing energy demand in the future in industrialised countries.

The research activities on efficient energy use and energy demand during the last 50 years reflect the framing drivers of energy supply, energy prices, economic development, and policy changes. During the last two decades, climate change policies have given more attention to energy efficiency and conservation. This is likely to be even more important in the next few decades including more resource efficiency reducing the demand of energy-intensive basic products. The progress of knowledge, methods, and empirical data during the last five decades is substantial in understanding remaining profitable energy efficiencies, upcoming new efficiency potentials and in projecting future energy demand. However, the authors have to admit that projecting the transformation of energy systems in the next three decades is extremely difficult:

  • The speed of transformation necessary in the light of the Paris target of 1.5 °C maximum surface temperature increase may not be accepted by larger parts of the civil societies in many countries and by oil and natural gas producing countries.

  • Market entrance and acceptance of new energy-efficient technologies require a substitution of energy use by capital, a severe obstacle for many private households (moderate incomes), small companies, and organisations (e.g., sport clubs, non-profit organisations).

  • There also are high uncertainties considering the long-term performance of new energy-efficient technologies such as high temperature heat pumps, nanomembranes as low temperature separation options, nanocatalysts, electricity-based production of basic products and related interaction with the upcoming circular economy.

  • Even if technical options and target-focused policies are clarified as being feasible und accepted, the mere lack of engineers and crafts men in most countries will put a question mark on present target-oriented energy demand projections.

The progress in energy efficiency and the transition of the sectors using fossil fuels for heat generation or road transport deserve closer attention regarding two aspects.

  • The abundant options of reducing final energy use in thousands of industrial production processes and even in buildings cause the existence of numerous innovation systems (Wesche et al. 2019). This extreme variety of “energy efficiency” innovation systems leads to little lobbying power in public administration and governments, and also in the group of intermediates (i.e., the banking sector, venture capital, or standardisation; Gallagher et al. 2012). This heterogeneity of innovation systems at the energy demand side reflects the opposite of a few innovation systems of energy supply with high lobbying power (i.e., renewables, green hydrogen, green fuels, possibly also Carbon Capture and Use).

This difference in numbers of innovation systems and lobbying power reflects a high risk that the benefits of energy (and resource) efficiency are substantially underestimated. This inobservance leads to unnecessarily large and costly investments in energy supply (generation, transport, and distribution). Countries which realise and politically counterbalance this uneven situation of innovation systems will have lower energy costs in the coming decades. This will contribute to better competitiveness compared to those countries paying energy (and resource) efficiency not more than lip service (Jefferson 2016).

  • Recent social science research on clusters and related narratives that deny climate change or delay the transformation concludes that at least 20% of US inhabitants have strict reservations about the meaningfulness of climate protection (Meyer et al. 2023). This percentage is certainly not much different in the EU, Japan, or emerging countries (Dahlstrom and Rosenthal 2018).

As millions of home owners or small businesses have to make their decisions on energy-efficient use of heat, power, or other energy applications, these societal groups will not only hesitate to take timely decisions on efficient energy use, but also influence political decisions at the various levels of public institutions. Democratic countries are not well prepared to convince those societal groups that deny climate change and delay the necessary transformation. Centrally governed countries with little democratic political structures may have a substantial advantage of realising energy-efficient solutions as an important part of the transition deeper and faster compared to democratic countries. So, communication research is needed to develop methods to convince those societal groups to contribute to efficient energy use and even to lifestyles of sufficiency in high income households (Bertoldi 2022).

Regarding the tough climate protection targets of a maximum increase of between 1.5 and 2.0 °C average surface temperature in the middle of this century compared to 1880, more efficient energy use will have to be substantially supported by:

  • Reducing the demand of energy-intensive basic products by means of much more efficient use of final products, buildings, machinery, and plants,

  • improved recycling, and more services of renting and sharing (of products and vehicles),

  • restructuring global value chains along new energy sources,

  • negative CO2 emissions, and

  • the post-growth economy (including sufficiency behaviour) may become a game changer in the long run reducing energy demand in the future in industrialised countries (Vita et al. 2019).

Whether these options of efficiency and changed lifestyles in the industrial and emerging countries will lead to a primary and secondary energy supply based on renewables nuclear power that can be built in the next 25 years remains an open question. While the IEA speeds up its warnings for more efficient energy and resource use (IEA 2022b, 2023), China, India, Indonesia, and Turkey are still planning to build additional coal power stations with a total capacity of more than 100 GW in this decade (Monitor 2023). Whether this growth is justified remains an open question given a steady stream of innovations of efficient energy solutions, given saturation (supported by resource efficient policy) as well as structural effects of domestic consumption and international trade. However, experience may point to the 1970s, when high economic growth rates and nuclear energy were the “dream” of the OECD countries.