On the quality of emission reductions: observed effects of carbon pricing on investments, innovation, and operational shifts. A response to van den Bergh and Savin (2021)

To meet the Paris Agreement targets, carbon emissions from the energy system must be eliminated by mid-century, implying vast investment and systemic change challenges ahead. In an article in WIREs Climate Change, we reviewed the empirical evidence for effects of carbon pricing systems on technological change towards full decarbonisation, finding weak or no effects. In response, van den Bergh and Savin (2021) criticised our review in an article in this journal, claiming that it is “unfair”, incomplete and flawed in various ways. Here, we respond to this critique by elaborating on the conceptual roots of our argumentation based on the importance of short-term emission reductions and longer-term technological change, and by expanding the review. This verifies our original findings: existing carbon pricing schemes have sometimes reduced emissions, mainly through switching to lower-carbon fossil fuels and efficiency increases, and have triggered weak innovation increases. There is no evidence that carbon pricing systems have triggered zero-carbon investments, and scarce but consistent evidence that they have not. Our findings highlight the importance of adapting and improving climate policy assessment metrics beyond short-term emissions by also assessing the quality of emission reductions and the progress of underlying technological change.


Introduction
The temperature targets of the Paris Agreement require countries to achieve net-zero carbon emissions by mid-century, in order to stay within a finite carbon budget with minimal overshoot (IPCC 2018, Riahi et al. 2021). This contrasts with previous obligations under the Kyoto Protocol, which were for emission reductions, but not elimination. This places a new demand on national climate policies, namely that they trigger changes towards a complete transition of global energy supply to zero-carbon sources such as renewables; countries cannot rely on lower-carbon sources such as natural gas to achieve their long-term objectives. The need for transformative change implies that the primary criteria for evaluating the effects of climate policies are whether they trigger the needed technological changes. New, zero-carbon technologies must be invented and developed; these technologies deployed at scale, leading to continued innovation and cost reductions; followed by their mass diffusion, ultimately enabling the discontinuation of fossil energy sources. The effects of policies on short-term emissions are relevant to achieving the long-term system transformation only to the extent that observed emission reductions originate from these activities.
In climate policy and climate policy research, the dominant view is that pricing carbon emissions is essential for achieving the Paris Agreement. Some argue that an economywide, preferably global carbon price is the only necessary climate policy instrument, while for others, carbon pricing should be the dominant lead instrument but embedded in a wider package, especially with flanking policies to support innovation (Nordhaus 2013(Nordhaus , 2021Edenhofer et al. 2019;van den Bergh et al. 2020; van den Bergh and Botzen 2020). Beyond theoretical and modelling studies, the empirical evidence for the effectiveness of carbon pricing instruments is scarce but growing. Most of the empirical literature on carbon pricing effectiveness use short-term emissions reductions as the dependent variable. Carbon pricing review articles taking technological change -innovation and investment in zero-carbon technologies -as the dependent variable are particularly scarce. Apart from Martin et al. (2016), who included innovation in their review, there had not been a review of these studies until we published one (Lilliestam et al. 2021) (henceforth LPB2021) in WIREs Climate Change.
LPB2021 investigated several dependent variables. First, we looked at short-term effects on emissions, and found these to exist in some schemes and periods, consistent with other reviews. Second, we investigated effects on innovation activity, and found that carbon pricing instruments have triggered small increases. Finally, we looked at effects on investment in zero-carbon technologies, such as wind or solar power. We found no evidence that carbon pricing has stimulated such investment and limited but consistent evidence that it has not. Thus, we suggested in LPB2021 that other instruments, including dedicated technology support, that have been empirically shown to accelerate investment in zero-carbon energy technologies, may be of higher value for long-term climate protection.
Our review was criticised by van den Bergh and Savin (2021) (henceforth BS2021) in this journal. Their paper was devoted exclusively to critiquing our review, claiming that our analysis was "unfair", incomplete and incorrect in multiple ways. Most arguments were directed at our choice of technological change, rather than emission reductions, as the central dependent variable. BS2021 also directed attention to papers that, they believe, should have been included in our review. We were not given a chance to respond to their critique at the time it was published. We do so now and stand by our original findings. Moreover, we undertook the additional analysis that BS2021 suggested and find this to further support our original conclusion: carbon pricing has been shown to reduce emissions somewhat and trigger weak innovation effects, but there is no evidence that carbon pricing has triggered zero-carbon investments and scarce but consistent evidence that it has not. We see two criticisms as particularly important and address them in detail. First, the choice of technological change, and especially zero-carbon investment, as the central dependent variable, because this criticism questions the conceptual soundness and relevance of our analysis. We address this in Sect. 2. Second, because the claim that we omitted several important articles for our review potentially invalidates our findings, we expand our review sample as suggested by BS2021, maintaining our selection and evaluation criteria from LPB2021 (Sect. 3). We find this to strengthen our initial findings. In Sect. 4, we respond to the remaining points raised by BS2021.

Expectations on climate policy: Decarbonisation and technological change
BS2021 claim that our "benchmark" of complete decarbonisation is "unfair" in the short term, because decarbonisation will take a long time (point 2.1). We agree that transitions are slow. That is why we do not primarily evaluate carbon pricing for its emission effects so far, but whether it has triggered the necessary technological change processes. The question "does a policy instrument trigger technological change towards full decarbonisation" is not only fair, but an essential assessment criterion for climate policies.
Anthropogenic temperature rise responds, close to linearly, to the cumulative greenhouse gas emissions (IPCC 2021), meaning that carbon emissions are a very relevant metric of climate policy success. The bulk of the carbon pricing literature focuses on effects on emissions. However, a pathway to reduced emissions may be different than a pathway to eliminated emissions, so it is essential to ensure that actions today are consistent with the long-term requirement of full decarbonisation (Vogt-Schilb and Hallegatte 2014). For example, a low-carbon power system may continue using natural gas power for balancing, whereas a zero-carbon power system must balance with a combination of dispatchable renewables, demand-response, wide-area transmission, and storage (Tröndle et al. 2020) -which requires a transition to a fundamentally different system than today. Policies must be evaluated not only based on the quantity of emission reductions achieved, but also on the quality of these reductions: whether they support emission reductions consistent with a pathway to total decarbonisation (Vogt-Schilb et al. 2015).
There are three main categorical channels through which emissions may decrease ( Fig. 1): operational shifts, low-or zero-carbon investment; the latter two can be enabled and supported by innovation and technological improvement. We use "technological change" to describe innovation and deployment of new technology (Jaffe et al. 2002;del Río 2009), and thus broader than "technical change" describing innovation alone (e.g. Acemoglu et al. (2012)). In our conceptualisation, innovation and investment are related but separate activities.
First, emissions may decrease through operational shifts within an existing capital stock, such as shifting from coal to already existing gas power stations or using an existing bus network more. Such effects may rapidly reduce emissions because there are no lead times (e.g. no need for new-build), but merely a question of which units to run. Prices, including carbon prices, can be expected to have strong short-term operational effects. Indeed the coal-togas power effect has been observed, for example in the UK (Green and Staffell 2021) and Germany in 2020 (Agora Energiewende 2021). In 2021, following a rapid natural gas price increase, even the strong increase of the EU ETS carbon price could not stop a German shift back from gas to coal, highlighting the possibly transient nature of such effects: they can be quickly realised, but also quickly reversed (Agora Energiewende 2022). Operational shifts can thus reduce emissions, but are often irrelevant for the aim of eliminating emissions.
Second, emissions may decrease through investments in low-carbon technologies, such as new gas power or purchasing a diesel car to replace a gasoline vehicle, or investment in more energy-efficient assets. Such investments may, for example, be triggered by standards or price incentives (Grubb 2014). Low-carbon investments will reduce emissions if they push aside carbon-intensive assets, but cannot eliminate emissions as low-carbon assets also emit carbon (Patt et al. 2019). Low-carbon investments can be Paris-consistent if they support subsequent zero-carbon action; hence, building insulation may support subsequent heat decarbonisation by reducing the renewable energy amount needed, whereas a new diesel car may reduce emissions but has no impact on transforming the transport system.
Third, emissions may decrease through investments in zero-carbon technologies, which gradually substitute carbon-emitting ones. These options include forms of renewable energy, such as wind power, solar heat, or synthetic fuels made from renewable heat or power. They also include technologies associated with electrification, such as electric vehicles. Ultimately this is the only emission reduction channel that is potentially consistent with the long-term goals of the Paris Agreement. It is also the most demanding and slowest channel, both because the scale is very large, and especially as it often requires systemic change when new technologies conflict with existing infrastructure and institutions (Rosenbloom et al. 2020). Historically, zero-carbon technologies have been more expensive than fossil technologies. Direct support policies, such as subsidies and feed-in tariffs, have achieved rapid investment increases (Marques et al. 2019;Polzin et al. 2019;Bersalli et al. 2020;Bhandary et al. 2021) that, in turn, have triggered innovation and associated cost reductions, especially in renewable electricity technology (IRENA 2021).
Finally, innovation -improving existing technologies or inventing and developing new ones -is an essential part of technological change. It is a necessary, although in itself insufficient, condition for full decarbonisation, because many zero-carbon technologies we may require, such as solar fuels, zero-carbon cement, or dispatchable renewable power, are still immature and thus costly technologies (Lilliestam et al. 2017;Nature 2021;Schäppi et al. 2022). Different polices have been shown to support innovation, including specific R&D policies and deployment support (Johnstone et al. 2010). Because investment is a key innovation driver (del Río and Kiefer 2022), it is particularly important to assess policy instruments for their effects on investments -without investments, also the innovation component of technological change will slow down.
As BS2021 correctly state, we would raise "too high expectations" by expecting carbon pricing to have already led to zero-carbon systems. And yet movement towards such systems in the future is of critical importance under the Paris Agreement. That is why rather than looking only at emissions as the indicator for climate policy success, it is important to evaluate each of the effects separately, paying particular attention to zero-carbon investments. In the next section, and critically in Table 1, we separately review the observed effects of carbon pricing schemes on emission reductions, operational shifts, low-carbon investment, zero-carbon investment, and innovation. As in LPB2021, we see zero-carbon investments as central and draw our conclusions mainly based on observations around this variable.

Completeness of the review
The second critical claim of BS2021 is that our review was incomplete (BS2021, point 2.4) and that we ignored existing evidence for the effectiveness of carbon pricing (point 2.7). We see this claim as serious enough to warrant an expanded review. As in LPB2021, we include only peer-reviewed original ex-post analyses investigating the effects of specific, existing carbon pricing schemes on technological change (innovation and investments) or on CO 2 emissions. We do not review articles investigating effects of other instruments or policy mixes. Because the papers do not investigate it, we do not report costs or cost-effectiveness for achieving the observed effects, or the costs of "other polices" to achieve the same effects. BS2021 point to 16 specific articles that LPB2021 did not include. Of these, we include three in our expanded review: two should have been included in LPB2021 but were not identified by our search string, and one had not yet been published when we submitted our manuscript. As suggested by BS2021, we also assessed the sources in the review article Martin et al. (2016) and found three articles that fit our search criteria but were not included in LPB2021. We review these 6 articles with the same method and criteria as in LPB2021 and summarise them in Table 1. For details about the articles (not) included here and the reasons for it, see Appendix Sect. 7.1.
Although they do not investigate effects of specific carbon pricing schemes, and hence do not fit our search criteria, we discuss the findings of several papers comparing climate progress in countries with and without carbon pricing in Sect. 3.2. Finally, we discuss the findings of previous reviews and how they relate to our findings in Sect. 3.3.
All three aspects of additional evidence provided here support the conclusions from LPB2021. First, there is evidence that carbon pricing has reduced emissions, primarily through operational shifts and efficiency measures. Second, several articles observed small but statistically significant increases in innovation. Third, there is no evidence that actual carbon pricing has triggered zero-carbon investment, and some evidence that it has not.

Expanded review
All six added articles analyse the EU ETS, and all articles support the findings of LPB2021. Each paper is described in detail in Appendix Sect. 7.2 and summarised in Table 1. Table 1 also contains the summary of LPB2021 with the added column "observed effects on emissions", which was not explicitly included in the original review. We report the findings as stated by the authors, either as "no effect", if they analysed effects on a variable and found carbon pricing to have had none, or as either "weak" or "strong" if they find an effect and report the effect magnitude. We report the effect size as expressed in each article (weak, small, low; large, strong, high or similar terms) by the reviewed authors themselves, except when noted otherwise. All reviewed articles investigate at least one of our five dependent variables. Entries marked "N/A" (not applicable) indicate that the variable was not analysed. We interpret "N/A" as absence of data, but not as evidence that there was no effect. Attribution of effects to a specific instrument is difficult given the many policy instruments implemented simultaneously and a constant evolution of contextual factors, but we note that all reviewed papers have, with different methods, identified effects (or absence of effects) on at least one of the investigated variables and attributed these to carbon pricing.
In the three papers that investigate technological change effects of the EU ETS (Hoffmann 2007; Rogge and Hoffmann 2010) (power sector) and Rogge et al. (2011a) (paper industry)), the findings are in line with LPB2021. These articles find weak effects on innovation, and weak effects on low-carbon investments -particularly in higher-efficiency fossil fuel power generation, such as new coal-or gas power replacing old assets, and in efficiency-increasing retrofits. All three papers find that there were no identifiable effects on zero-carbon investment; they attribute all such investment to other policies. Rogge and Hoffmann (2010) state that the induced retrofits in coal power may have increased total emissions, because they prolonged the lifetime of these generators, compared to an early shutdown of these previously inefficient plants.
The  Bayer and Aklin (2020), find that the EU ETS reduced EU emissions by 3.8%, due to an emission reduction of 11.5% in the trading sectors in 2008-2016 compared to a world without the EU ETS.

Additional evidence for the effectiveness of carbon pricing: system-unspecific articles
BS2021 point to 3 articles not investigating any specific carbon pricing system, but comparing countries with a carbon price to such without, regarding effects on emissions (Sen and Vollebergh 2018;Best et al. 2020) or innovation (Aghion et al. 2016). We do not include them in Table 1 because they do not assess the effect of a specific scheme, but all three findings are consistent with our review: there are statistically significant effects of carbon pricing on emissions, and some effects on innovation. None of these articles investigates investment effects or reports the channel through which emissions decreased. Aghion et al. (2016) investigated innovation in the auto industry, contrasting dirty (internal combustion engine-related) and clean (electric, hybrid, and hydrogen) innovation, showing that higher prices -such as triggered by carbon pricing -trigger more clean innovation, but also that substantial price increases and long time-spans are necessary for clean innovation stock to overtake the dirty. Best et al. (2020) studied the effect of carbon pricing on CO 2 emissions growth rates over 1997-2017 across 142 countries, finding that countries with a carbon price have 2% lower emissions growth. They found that increasing the carbon price by €1/tCO 2 reduces annual emissions growth by 0.3%; Sen and Vollebergh (2018) found that a €1/tCO 2 energy/carbon tax increase would reduce fossil fuel demand by 0.73%.

Evidence from previous review papers
BS2021 criticise that we do not mention several reviews and "overview papers". Because we conduct a literature review, we do not "review reviews". The existing reviews support and do not contradict the findings of LPB2021, despite the claims of BS2021 to the contrary. The central difference between our review and existing ones is our choice of technological change as dependent variable. Hence, whereas several reviews find that carbon pricing has reduced emissions and increased innovation activity, as do we in our review, we are not aware of any review concerning investment effects.
BS2021 explicitly mention Bellas and Lange (2011), who review emissions trading schemes for various pollutants -finding mentionable positive effects -but hardly covers carbon trading, because they could not find a single peer-reviewed paper with an expost analysis. The review of Tietenberg (2013) finds that emissions in the EU ETS and RGGI (Regional Greenhouse Gas Initiative) have decreased substantially, but attributes the observed emission reduction to the economic crisis and falling natural gas prices and not to the emissions trading schemes. For carbon taxing, Tietenberg (2013) identifies emission effects in British Columbia, whereas the remaining assessed peer-reviewed articles (Bruvoll and Larsen 2004;Lin and Li 2011) find no significant effect on emissions. None of these reviews finds evidence for investment effects of carbon pricing, but some for emissions trading of other pollutants.
The other three carbon price reviews we know of Haites 2018, Green 2021) all find that theory-based predictions are abundant, but the empirical evidence is scarce and largely focused on Europe (Green 2021). This is exactly the first finding of LPB2021.
These three reviews find that carbon pricing, both taxes and emissions trading, has reduced emissions somewhat, typically by 0-2%/year (Green 2021), and possibly higher for German and French industry , mainly through fuel switching and efficiency increases in existing assets. These incremental effects, "though useful on the margins, fall well short of the societal transformations identified [as necessary] by decarbonization scholars" (Green 2021). This is the second finding of LPB2021. Haites (2018) states that although emissions under carbon pricing tend to decrease, robust attribution of that effect to carbon pricing is hardly possible.
Only Martin et al. (2016) investigates innovation effects, and finds that phase II of the EU ETS triggered a small part of the observed innovation increase, but renewable energy policy was a stronger driver. Again, this is entirely consistent with LPB2021. *, **: The reported effects refer to *gasoline or **natural gas diesel consumption. The authors do not specifically report the emission reductions. N/A: not applicable (the study did not assess this particular effect). a Authors make no judgement of strong/weak, but only report numbers. Our interpretation to make statements consistent with other articles' findings Table 1 (continued)

Point-by-point rebuttal
In the previous sections, we showed how the additional analysis suggested by BS2021 strengthens our original findings. Here, to leave no stone unturned, we present a rebuttal to the remaining five numbered criticisms of LPB2021 raised by BS2021.

Ignoring that low prices trigger low effects
BS2021 criticise (point 2.2) that we ignore that carbon prices in the assessed time periods were low, and often lower than our selection benchmark of US$25/tCO 2 . They also claim that their "own assessment of the empirical literature [shows] significant and relatively large normalized effects [on] emissions reduction [and] pure innovation effects" (conclusion) or "small but positive and significant effect[s]" (abstract). In their article, BS2021 do not include this "normalised" analysis, and present no support for these claims. As only the carbon price experienced (or expected) by each specific actor can have an effect, we disagree with BS2021's proposed "average" carbon price (of regulated and unregulated sectors combined): as no one experiences this total-economy average price, it is an irrelevant metric.
In our review, we evaluate evidence for real-world performance of existing carbon pricing schemes, with the prices that were achieved. We acknowledge, also in LPB2021, that carbon prices have been lower than proposals from most climate economists, and indeed most reviewed authors refer to "low prices" as an explanation for weak effects or absence of any effect. In contrast, Bayer and Aklin (2020), attribute emission reductions of some 1%/ year to the EU ETS, despite its low prices. However, we believe that political realities, not only theoretical expectations, should be evaluated as part of an instrument's performance: high carbon prices have proven difficult to implement. The feasibility of substantially higher carbon prices in the coming years remains to be seen.
BS2021 correctly note that carbon prices were often lower during the investigated periods than in 2019. The average carbon prices in the assessed periods were typically €15-20/tCO 2 or around US$20/tCO 2 in the EU ETS and British Columbia, and much higher in Sweden and Norway (up to US$88/tCO 2 (Sweden, average 1991-2012)). We see no clear correlation between average carbon prices and observed effects ( Table 3, Appendix Sect. 7.3). Only two articles (Bruvoll and Larsen 2004;Bohlin 1998) investigate effects on zero-carbon investments with carbon prices exceeding US$40/tCO 2 and both find that there was no effect (see Table 3); however, both investigate early periods, when zero-carbon alternatives may have been more expensive than today.

Too short time to trigger innovation
BS2021 state that whereas "carbon prices have a very short history, innovation effects take a long time, given considerable lead times and uncertain outcomes of research, development, demonstration [and] deployment" (point 2.3). We agree: new technologies often take decades from invention to the mass-deployment that would make a large difference in emissions. This is a further reason why technological change, and not achieved emission reductions to date, is the central metric for assessing progress towards full decarbonisation.
However, carbon pricing does not have a short history: the Nordic carbon taxes are currently thirty years old, and the EU ETS is over fifteen years old. There are fewer than thirty years left to 2050, the target year of most national decarbonisation plans. We do not have time to wait another thirty years to evaluate policy effectiveness because at that point, the evaluation itself will have become moot.

Downplaying positive findings
BS2021 claim that we downplayed positive findings by ignoring the weak but positive effects on innovation identified by four out of the five papers investigating innovation effects (point 2.5), and question why we included 14 articles that did not investigate effects on innovation. The conclusion on innovation from LPB2021 is that of the five articles examining this effect, "most articles identify weak effects, and one concludes that there was no effect" (LPB2021). This hardly amounts to downplaying but is exactly the interpretation of BS2021. Two of three articles included in the expanded review (Table 1) also find weak innovation effects, and one article finds that there was no effect.
We however note that BS2021 conflate technological change and innovation. Technological change, as we use it, refers to more than innovation and holds to the invention, innovation and diffusion of new technologies (Jaffe et al. 2002). In LPB2021, we state that effects on technological change have been small -despite some observed and reported effects on innovation -because carbon pricing was consistently found to not have triggered zerocarbon investment. Our conclusions are mainly based on the evidence concerning zerocarbon investment, but we also report findings concerning innovation, operational shifts, low-carbon investment and emission reductions. As innovation is not the only variable we investigated, we included many articles that do not report innovation effects (thus reporting N/A, not applicable, see Sect. 3.1).
BS2021 specifically claim that we downplay the findings of Calel and Dechezlepretre (2016). That paper indeed reports both a 9% increase in low-carbon patenting among regulated firms and that "only 2% of the post-2005 surge in low-carbon patenting can be attributed to the EU ETS" (p.174). They find this a "quite unremarkable nudge on the pace and direction of technological change" (p.189). LPB2021 correctly cites both numbers and the reasoning behind them (p.10). We cannot see that we have downplayed this or any other finding.

Not considering older evidence for effect of energy prices
BS2021 state that we do not consider older evidence for the effect of energy prices on innovation (point 2.6). This is correct: carbon pricing, rather than market price of energy, is our independent variable and the scope of our article. We agree that a review of energy price fluctuations on technological change towards full decarbonisation would be an interesting undertaking for future research, and indeed (fluctuating) energy prices may have different effects than (more permanent) carbon prices. We doubt that it would be qualitatively different that our present finding, as indicated by the articles BS2021 refer to: several of these find that higher energy prices have increased innovation, but we are not aware of articles showing effects on zero-carbon investment.
Energy prices can increase because of taxation or due to changes in market conditions often associated to geopolitical events. However, because energy is an essential good that is hard to substitute in the short term, societies have little tolerance of high energy prices.
Price increases thus often trigger societal and political forces that work to reduce prices again, which again reduces the price-driven incentives to innovate and invest in technologies less exposed to price fluctuations. The low tolerance of high energy prices was vividly demonstrated during the European gas price crisis in 2021/2022: even before the war-and sanctions-induced price hike triggered by the Russian invasion of Ukraine, several countries took measures to reduce energy prices, including lowering energy taxes and introducing price regulations -and calling for lowering carbon prices (Sgaravatti et al. 2022). This casts doubt on high prices, caused by carbon prices or other events, as the key or sole transition driver, because the social and political reality is very different than in theory-based models.

Underestimating the effect of carbon pricing
BS2021 claim that "carbon pricing will stimulate both technology innovation and adoption" (point 2.8). There is no empirical support for this claim. Our review shows that there is evidence of weak but positive effects on innovation and emissions, no evidence that carbon pricing has triggered deployment of zero-carbon technology, and limited but consistent evidence that it has not (Table 1).
BS2021 state that our review suffers from confirmation bias, seeking to confirm preexisting, negative beliefs that we held about carbon pricing. In our experience, all scientists are human and thus hold beliefs -and indeed we, too, have beliefs about the effectiveness of carbon pricing, which we have previously stated clearly (e.g. Patt and Lilliestam (2018)). In the case of LPB2021, we subjected our beliefs to falsification, using appropriate and documented methods, with the process of peer review leading to extensive manuscript revisions as a further check. We would encourage the authors of BS2021 to be as careful. BS2021, for example, was published as a research article, despite its lacking documented methods, and its entire contents' being devoted to criticising the findings and rigour of LPB2021. We note a similar example of pushing back against findings critical of carbon pricing in the case of van den Bergh and Botzen (2020), attempting to take down Rosenbloom et al. (2020) in a manner that questioned the researchers' motivations and competence. BS2021 state that "it is hard to imagine full market penetration of zero-carbon technologies and systems without getting the prices right". We do not find this difficult to imagine. For fifty years, environmental regulations have led to the complete replacement of harmful substances -e.g. DDT, leaded gasoline, ozone-depleting gases -with little-to-no contribution from market-based instruments (Kraft 2021), and without being oppressively costly. Today, numerous political jurisdictions are turning to performance standards and other forms of regulation to complete the process of zero-carbon technologies' market penetration. Within the EU, for example, there are bans on new fossil-based heating systems, and a proposed 0 gCO 2 /km emissions standard for new automobiles. History suggests to us that market-based instruments' being a necessary condition of any transition is not an observed phenomenon, but rather an artefact of theory-driven models within which prices and pricebased policies are well-represented but other important legal and social drivers of behaviour are omitted (Ellenbeck and Lilliestam 2019). Such models are valuable, if one recognises their limitations.
Prices are important, and zero-carbon technology can become relatively cheaper either by making fossil fuels more expensive (e.g. carbon pricing) or by making zero-carbon technologies cheaper (e.g. by dedicated support, triggering deployment and learning effects).
But prices are merely one barrier in a transition, and not necessarily the most critical one. For example, cars did not replace horses because countries increasingly taxed horse manure, but because innovation improved the performance and cost of cars and concomitant adaptations in infrastructure and institutions allowed the car to fully play out its range, speed and comfort advantages, leading to mass deployment (Geels 2005). A transition to a zero-carbon future is certainly imaginable without emission pricing because this is precisely how previously completed transitions worked.

Discussion
In this response, we specified the conceptual framework about possible effects of carbon pricing, expanded the review as suggested and addressed the multiple criticisms from BS2021. This has strengthened our original conclusion from LPB2021. There is evidence that carbon pricing schemes have reduced emissions, especially through fuel switching to lower-carbon fossil fuels and efficiency increases, and that they have triggered weak but positive innovation effects. Our central finding is that there is no evidence that carbon pricing has triggered zero-carbon investments but consistent evidence in all ten reviewed articles that investigated this specific effect that carbon pricing has not triggered such investments.
We believe that BS2021, rather than pointing out actual errors in our original review, have helped focus the critique of carbon pricing as the sole or dominant "lead" climate policy instrument. The carbon pricing discourse rests on the promise of reducing emissions in an efficient, least-cost way. Our review shows that carbon pricing can reduce emissions and does so by triggering incremental reorientation within existing fossil fuel-based systems. Such effects are exactly those intended to be triggered first, because they are the "lowest-hanging fruit" -and such effects would be consistent with the emission reduction requirement of the Kyoto climate policy world.
As climate policy shifted from Kyoto to Paris, the requirements on policies shifted as well. With the temperature targets, eliminating emissions became the central target of climate policy and for that, large-scale deployment of zero-carbon technology is essential. Our review shows that carbon pricing has not been shown to achieve this, and there is scarce but consistent evidence that it has not triggered zero-carbon investments. Hence, not only does carbon pricing not address the numerous non-price transition barriers (Patt and Lilliestam 2018;Rosenbloom et al. 2020), it also has no track record of triggering the zerocarbon investments that are both necessary and in principle addressed by this price-based instrument.
Our review has investigated the state of knowledge in the empirical literature on the effects of carbon pricing on a set of variables. We have not assessed the effectiveness of alternative policies and, because there is hardly any empirical literature enabling such a review, we have not assessed the cost-effectiveness of carbon pricing or other policies. These would be important questions to address in future research and in future review articles, but we note that this does not limit the usefulness or relevance of our review. There is increasing evidence (e.g. IPCC (2018)) that failing to decarbonise by the second half of the century would be potentially catastrophic with damage costs likely far exceeding mitigation costs. This means that there is a given level of effectiveness that we must demand from policy, and among the set of effective policies we should choose the instrument or mix of instruments that achieves that level of effectiveness at the least possible cost. In this work, we assessed whether carbon pricing can be observed to deliver a level of effectiveness (initiating technological change in the energy system) that one might demand. This is the precursor to the cost question.
Our review highlights the necessity of reconsidering the metrics by which we evaluate climate policy and transition progress, an issue previously raised by Hanna and Victor (2021). By focusing policy evaluation on short-term emissions only, the necessary condition of zero-carbon investment is put on equal footing with operational shifts, despite their actual different importance for full decarbonisation. Because operational shifts are often realised faster -sometimes literally by the flick of a switch -and cheaper than investments in new assets, a pure emissions focus may set wrong incentives for policymakers, who may seek to achieve quick emission reduction successes rather than build the foundation for a transition to a zero-carbon future. Therefore, it is important to assess both emissions and technological change, acknowledging that emission reductions must be the outcome of previous technological change to be true steps on the way to full decarbonisation. This changes our view of climate policy and how well the world is on a path to meet the temperature targets. For example, the world is today much better positioned to decarbonise electricity -a key also for decarbonisation of other sectors -than ten years ago, due to the technological progress of PV and wind power, triggered by dedicated support schemes. In the global perspective, despite large-scale deployment in many countries, these technologies have not mentionably bent the emission curve. Does that mean that the transition is going badly? An evaluation focused only on emissions would say yes. An analysis investigating the underlying factors of technological change would say no: it is going well, although much must still be done. We believe our analysis prompts the question of how to improve evaluation metrics to reflect both the processes underneath the surface and the magnitude of the transition challenge: without radical systemic change -a socio-technical transition in every currently carbon-emitting sector (Victor et al. 2019) -any temperature target is unattainable.
Our findings do not mean that carbon pricing is a meaningless policy, and we do not propose that existing schemes be scrapped. We show that it has not triggered the necessary effects in the past, which does not necessarily mean that it will not do so in the future. There may be other reasons than triggering investments to have carbon pricing, such as helping phase out coal power or nudging people to choose a different mode of transportation once attractive alternatives to such practices have been implemented. Carbon pricing can also raise funds for other policies triggering the necessary investments, including in infrastructure. Unlike a carbon tax, an ETS with a cap set to eventually hit net-zero, as in the EU ETS, can be a potentially important instrument, both as a signal to investors to avoid sunk carbonemitting investments and as an effective ban on net emissions. Assuming that future policymakers have the political stamina to stick to a zero cap also if it becomes painfully binding, an ETS would effectively be a ban for all but the highest-value emissions, which would instead be offset by negative emissions. Indeed, a policy mix holding instruments dedicated to technology development and deployment, infrastructure adaptation, institutional reforms, etc. and a cap eventually hitting zero likely has greater potential to be successful than a mix without such a cap.
There is thus reason to believe that carbon pricing can play a role in the policy mix for full decarbonisation. But we also believe it is of little use to wait for the political environment to possibly make higher or more global carbon prices feasible, to spend much effort figur-ing out which carbon price is needed to meet the climate targets, or the "true" social cost of carbon. Instead, we need more efforts to understand how to create and evaluate policy mixes for decarbonisation, composed of multiple instruments -one of which may be carbon pricing -each of which addressing specific transition barriers. Insisting that one instrument should be dominant over the others is not very helpful: if we are to meet the Paris targets, technological and systemic change is essential, and hence observable policy effects on these dimensions should be the first criteria for defining and evaluating the climate policy mix. 6 Appendix 6.1 7.1 Papers suggested by BS2021 and final selection for expanded review BS2021 point to 16 articles they believe we have omitted in our review, and suggest that we include the literature reviewed by Martin et al. (2016). Of the 16 suggested articles, 3 fit our search criteria and 13 do not (see Table 2); of these, two were not found by our search string although they match the selection criteria, and one was not published at the time we submitted the LPB2021 manuscript. Of the many reports and articles reviewed by , 3 clearly fit our search criteria (see bottom of Table 2) but were not identified by our search string and should have been included in LPB2021.
In our review, including the expanded review in Sect. 3.1, we include only peer-reviewed, ex-post analyses of the effectiveness of actual carbon price schemes, either regarding their effectiveness to reduce emissions or to trigger technological change (investment and innovation). We exclude papers investigating effects of energy prices on innovation (e.g. Newell et al. (1999), Popp (2002)), because they do not investigate the effects of carbon pricing; as (fluctuating) energy price changes likely have different effects than (lasting) tax increases (Sen and Vollebergh 2018). We do not review the many papers focusing on effects of carbon pricing on other factors than technological change and emissions, such as industry productivity (e.g. Commins et al. (2011)). We only review articles focusing on carbon pricing, and exclude articles investigating other pollution control policies, such as NO x and SO 2 , thus excluding a range of such papers suggested by BS2021.
BS2021 point to several articles not investigating the effects of specific carbon pricing schemes but rather comparing sets of countries with and without carbon pricing. Because they do not relate to any particular system, we do not include them in the review, but we agree with BS2021 that they do provide important empirical insight. We discuss these papers and their findings in Sect. 3.2.
Finally, BS2021 also propose that we include reviews in our review. We do not include review papers in our sample despite the recommendation of BS2021 and, following standard scientific practice, only review original research papers. To increase the robustness of our findings, we synthesise the suggested review literature -plus further critical reviews that BS2021 did not mention -in Sect. 3.3. Table 2 Articles BS2021 claim we ignored in LPB2021, the reason why we did not include them, and overview of which of these papers are included in the expanded review Suggested omissions Not included in original review, because: Included in the expanded review (reasons) Hoffmann (2007) Our search string did not find it. Yes. It fits our search criteria. Rogge and Hoffmann (2010) Our search string did not find it.
Yes. It fits our search criteria.
Bayer and Aklin (2020) It was published after initial submission of LPB2021 but should have been included during revisions.
Yes. It fits our search criteria. Sen and Vollebergh (2018) Not identified as carbon pricing (classified as an energy taxation paper); does not investigate effects of any specific carbon price scheme No, but paper discussed in Sect. 3.2. Does not investigate effects of any specific carbon pricing scheme Best et al. (2020) It was not yet published (Sept. 2020) when we submitted our revised manuscript (Aug. 2020), and does not investigate effects of any specific carbon price scheme No, but paper discussed in Sect. 3.2. Does not investigate effects of any specific carbon pricing scheme Aghion et al. (2016) It does not investigate the effects of any specific carbon price scheme, but investigates innovation effects of fuel prices.
No, but paper discussed in Sect. 3.2. Does not investigate effects of any specific carbon pricing scheme Martin et al. (2016) It is a review paper.
No. Not primary research. Discussed in Sect. 3.3. Tietenberg (2013) It is a review paper.
No. Not primary research. Discussed in Sect. 3.3. Popp (2019) It is a review paper, not focusing on carbon pricing.
No. Not primary research; out of scope (not carbon pricing). Bellas and Lange (2011) It is a review paper.
No. Not primary research, largely focusing on trading schemes for non-carbon pollutants. Popp (2002) Investigates effects of energy prices, not carbon pricing, on energy efficiency innovation.
No. Out of scope (not carbon pricing). Newell et al. (1999) Investigates effects of energy prices, not carbon pricing.
No. Out of scope (not carbon pricing). Popp et al. (2010) Not a peer-reviewed article, but a review book chapter examining effects of environmental policies on technological change processes.
No. Not original research, out of scope (no focus on carbon pricing).

Noailly and Smeets
Investigates effects of energy prices, not carbon pricing, on innovation.
No. Out of scope (not carbon pricing). Johansson (2000) It is not peer-reviewed.
No. Not peer-reviewed. Sterner and Turnheim (2009) It investigates effects of NO x control policies, but not carbon pricing.
No. Not carbon pricing. Martin et al. (2016) (all other papers cited there do not meet our criteria) Rogge et al. (2011a) Our search string did not find it. Yes. It fits our search criteria. Our search string did not find it.

Papers included from
Yes. It fits our search criteria.

Anderson and di Maria (2011)
Our search string did not find it. Yes. It fits our search criteria.

Short descriptions of each article of the expanded review
We summarise the findings of the articles included in the expanded review in Sect. 3.1 in the main text. Here, we describe the key findings of each of these 6 papers more in detail, following the same format as the single article summaries in LPB2021. In sum, the added articles confirm and strengthen each of the findings of LPB2021. Hoffmann (2007) investigated the effects of the EU ETS on investment decisions in the German electricity industry, based on interviews with firm managers during the first trading period. He highlights that there were no changes in the investment portfolio: electricity companies continued investing in gas and coal power plants. Investments in renewable power generation were not affected by the EU ETS, but were triggered by the direct incentives via the German Renewable Energy Sources Act. Concerning innovation, he found that "the EU ETS seems to be only a trigger that accelerates selected R&D activities such as carbon capture and storage, but without fundamentally changing R&D efforts". In general, Hoffmann found weak effects of the EU ETS on innovation and small-scale low-carbon investments with short amortisation times (e.g. retrofits) and no impact on zero-carbon investments, which he states were induced by other policies. Rogge and Hoffmann (2010) investigated effects of the EU ETS on the German sectoral innovation system for power generation technologies through interviews. Their findings show that the EU ETS affected "the rate and direction of technological change of power generation technologies, with the main impact occurring within the large-scale coal power generation technological regime", in which the EU ETS triggered increased R&D activity for carbon capture and storage. In addition to observed effects on modernisation of existing coal power, they also associated a temporary trend to build new, albeit more efficient, coal power with the EU ETS, which may have effectively prolonged coal power station lifetimes and increased, rather than decreased, total emissions compared to early shutdown. A central effect of the EU ETS, say Rogge and Hoffmann, is to "prepare the ground" for a future transition to low-carbon energy by putting CO 2 on managerial agendas and affecting corporate decision culture. Rogge et al. (2011a), which builds on Rogge et al. (2011b) (included in LPB2021, finding no investment effects and weak innovation effects), investigated the impact of the EU ETS (2005-09) on technological change in the German paper industry through a survey and case studies. Their results show no effects on RD&D activities or on the adoption of new technologies. They found that paper producers focus on investment costs rather than on operating costs for technology adoption and investment decisions. Thus, policies addressing operating costs, such as costs for CO 2 emissions, generate comparatively low innovation incentives. Their survey results showed that market factors have the highest relevance for RD&D decisions among the firms studied, whereas the EU ETS "is among the least relevant decision factors for technological innovations". Delarue et al. (2010) builds on and improves the model used in Delarue et al. (2008) and assesses the CO 2 abatement through fuel switching in the European power sector due to the EU ETS in 2005-06. Both articles investigate emission reductions through fuel switching from coal to gas, but not effects on innovation or investments. Delarue et al. (2010) finds a coal-to-gas switch effect of 53 MtCO 2 (cumulated 2005-06), down from the substantial 147 MtCO 2 found with the uncalibrated model of Delarue et al. (2008), triggered by the combined effect of the EU ETS and other factors, especially gas prices.
Anderson and di Maria (2011) developed a dynamic panel data model to construct credible business-as-usual (BAU) emissions in Europe without the EU ETS, thereby also assessing observed emission reductions relative to this BAU. They find that the BAU projections used for the allocation plans were "of questionable quality", but also that the first trading phase reduced emissions by 2.8% over the three years compared to a counterfactual scenario without the EU ETS, despite the considerable overallocation of certificates. The article does not provide further details about the abatement channels.
Bayer and Aklin (2020) investigated whether the EU ETS have reduced emissions, using the synthetic control method, comparing the actual real emissions trajectory with a "synthetic Europe" without the EU ETS. They found that the EU ETS reduced cumulative emissions by some 1.2 GtCO 2 , or 3.8% of total EU emissions, in 2008-2016. The sectors included in the EU ETS reduced emissions by 11.5% 2008-2016 compared to the synthetic Europe; the emissions reductions are additional to reductions caused by the financial crisis. The study found that the EU ETS has been effective despite low market prices, likely because the EU ETS signals a credible commitment to more stringent caps in the future. They did not investigate the channels through which the emission reductions happened, and made no statement on investments or innovation.

Carbon prices in the observed systems and time periods and observed effects on emissions and zero-carbon investment
BS2021 claim that we do not "normalise" effects for the low carbon prices in the observed periods (see Sect. 4.1). Because the empirical ex-post literature is very diverse, a true systematic review, drawing out and statistically analysing quantitative data from existing studies is not possible. Therefore, it is not possible to "normalise" the findings in the literature, and doing so would both distort the findings and be misleading, because that would not be an empirical study but a theoretical prediction of how carbon price systems would have performed in another world with higher carbon prices. The fact remains that carbon prices were relatively low, and that this is a characteristic of most carbon price systems to date.
As argued in Sect. 4.1, the carbon prices in the observed time periods were lower than most climate economists recommend as sufficient, but not so low as to be discarded as meaningless (we excluded all such schemes through our US$25/tCO 2 selection criterion, see LPB2021). The average carbon price in the observed periods were typically around US$20/tCO 2 (usually €13-19/tCO 2 ) in the EU ETS and the carbon tax in British Columbia, with some fluctuations upwards and downwards for specific periods, whereas the Nordic carbon taxes typically exceeded US$40/tCO 2 , and sometimes much higher, in the observed periods (Table 3). This suggests that "too low" carbon prices may not be the entire reason for the observed absence of zero-carbon investment effects.  Table 3 The average, max and min carbon prices in the reviewed articles of LPB2021 and the ones added in the extended review. Sources: World Bank (2021) Authors make no judgement of strong/weak, but only report numbers. Our interpretation to make statements consistent with other articles' findings. b Rogge & Hoffmann's interviews concerned the expected effects of the full second trading period (2008)(2009)(2010)(2011)(2012) although it was published in 2010. *, **: The reported effects refer to *gasoline or **natural gas diesel consumption. The authors do not specifically report the emission reductions. N/A: not applicable (the study did not assess this particular effect) Table 3 (continued) permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.