Long-term Equilibrium in Electricity Markets with Renewables and Energy Storage Only

In this paper, we study the optimal generation mix in power systems where only two technologies are available: variable renewable energy (VRE) and electric energy storage (EES). By using a net load duration curve approach, we formulate a least-cost optimization model in which EES is only limited by its power capacity. We solve this problem analytically and find least-cost and market equilibrium conditions that lead to the optimal capacities of VRE and EES. We show that, mathematically, an electricity price structure that depends on the period of the year (i.e. EES charging or discharging, VRE curtailment, load shedding) and on investments costs leads to cost recovery for VRE and EES. We show that when EES is the marginal technology (either charging or discharging) the price must be non-zero. More specifically, the equilibrium prices during EES charge or discharge are functions of the EES and VRE fixed costs. We confirm our analytical findings using a numerical model. We argue that, although the system we study is hypothetical and simplified, our findings provide insights and research directions for how to recover fixed costs in a future electricity market based on VRE and EES only.


Introduction
Most driving forces that apply to the power sector point towards a very large increase of variable renewable energy (VRE) sources such as wind and solar. First, unsubsidized costs of wind and solar are reaching unprecedented lows, with a 72% -respectively 90%-decrease over the last 12 years (Lazard, 2021). Then, energy policy in general pushes towards VRE: renewables shall contribute to the reduction of global emissions, one of the main objectives of the Paris Agreement: "Parties aim to reach global peaking of greenhouse gas emissions as soon as possible" (UNITED NATIONS, 2015). Finally, exacerbated concerns of security of supply related to global gas and oil trade also push towards VRE that are deemed to solve this concern. As such, VRE would contribute to meet the essential goals of delivering a secure, sustainable and affordable electricity system (Grubb, 2018) (Roques, 2017). According to IRENA (IRENA, 2021), 82% of all electricity generation capacity expansion in the world in 2021 was renewables, mostly wind and solar. In the US, the deployment of clean technologies is an essential part of the 50-52 percent target for reduction in U.S. Greenhouse Gas Pollution by 2030 (The White House, 2021). Finally, IEA considers that "A massive expansion of clean electricity is essential to giving the world a chance of achieving its net zero goals" (International energy agency (IEA), 2021).
With thousands of GW of VRE installations expected towards 2050 1 , new integration challenges are expected, because of the very nature of VRE technologies. VRE have limited dispatchability, at least upwards (with the exception of possible requirements for frequency responsive reserve or "headroom", which are not meant to solve energy issues (NERC, 2020)). For this reason, VRE increase the need for flexible resources such as storage, interconnections, demand-sidemanagement and peaker generation. In addition, surplus VRE generation lead to 0 or negative electricity wholesale market prices; an effect that increases nonlinearly with the amount of VRE (Karaduman, 2021). These challenges add up to existing concerns regarding the design of power markets (such as price caps) and lead to the conclusion that there is currently "No obvious international example of a market structure fully appropriate for renewables at scale " (Grubb, 2018).
While there is already market-design literature for the case of medium levels of VRE integration (Ketterer, 2014) (Grubb, 2018), there is less literature focusing on the case with a large VRE share. Various avenues are proposed, that range from a fully vertically integrated structure to a fully competitive system (Joskow P. L., 2021) (Roques, 2017). In between those two forms, there is the proposition in which two different market structures would handle the short-term and long-term management of power generation systems, which is summarized as "competition in the market" and "competition for the market" (Joskow P. L., 2021). In this paper however, we want to push the idea of competition and spot markets in a system beyond "high share of RES", that is with only RES and EES and therefore in the absence of price-setting conventional generators.
This work uses research carried out previously (Korpås & Botterud, 2020), to look analytically for long-term electricity market equilibrium and its implications for cost recovery, in a case where the capacities of RES and EES are optimized, rather than taken exogenously. Our aim is to analyze the market properties of this "extreme" example, even though we discuss that the concept of current markets does not apply directly to this case, and that our underlying assumptions are simplistic.
The rest of the paper is organized as follows: Section 2 is a general presentation of the analytical problem, which is solved in Section 3 for the case where load shedding is not allowed. Section 4 focuses on the situation with load shedding and corresponding scarcity prices when this occurs. Finally, section 5 concludes the paper and discusses future research directions

Section 2: Methodology
We first describe a centralized planner's problem which is to minimize the total expected cost of electricity generation considering a set of reliability constraints and physical limitations (Green, 2000). We study an "energy only" situation, meaning that only energy actually delivered is compensated: there is no pricing of either firm capacity or operating reserves. We include scarcity pricing using value of lost load -VOLL-which is "the amount that consumers would pay to avoid having supply of power interrupted during the blackout" according to (Cramton, Ockenfels, & Stoft, 2013). VOLL is a parameter that is difficult to estimate and to justify (Joskow & Tirole, 2007) (Cramton, Ockenfels, & Stoft, 2013). Outside load shedding at VOLL, no other demand sidemanagement solution is present. This is a limitation because, unlike demand-side management programs, the assumed VOLL applies to all consumers and does not make a distinction between power users (Stoft, 2002).
In the classical model with conventional generators only, the solution to the centralized generation expansion planning problem leads to time-dependent markets prices equal to the variable cost of the marginal generator, and optimal capacities as illustrated in Figure 1. Such a pricing structure is proven to lead to optimal incentives for investment in generators (fixed costs are recovered, average economic profit is zero) (Stoft, 2002), (Green, 2000). s is the optimal duration of blackouts, which is given by the fixed costs of the peaker divided by VOLL.  (Korpås & Botterud, 2020), load duration curve and optimal capacities of conventional plants.
is the optimal duration of load shedding.
Adding VRE and EES in addition to conventional generators leads to a more complicate formulation as described in (Korpås & Botterud, 2020). Their analysis shows that when VRE and/or EES, that have no direct marginal costs, are added, the general price structure based on marginal costs is still conserved (Korpås & Botterud, 2020), as long as conventional generators remain present. In other words, conventional generators continue to be price setting resources , and these resources also influence the optimal price during EES charge and discharge through the principle of opportunity cost. However, the higher the VRE capacity, the more often are market prices zero or negative (Schmalensee, 2019), (Joskow P. L., 2021), putting the profitability at risk, mostly for peak generators. What we are interested in this paper is to evaluate the results given by a similar model but this time with no conventional thermal generators, i.e. with no marginal costs. This 100% VRE system is interesting because of the rapid expansion of renewable sources, but also because it allows us to set aside the question of VRE plants possibly not being viable without support when in competition with conventional generators (Roques 2017, Karaduman 2021, Schmalensee 2016.
We are interested in the market equilibrium in a system where VRE and EES are serving a timevarying inflexible load, described by the following system of equations.
where indices s, v, e refer to load shedding, VRE and EES respectively.
The objective function in Eq. (1) is the sum of three terms. The first two terms are the annualized fixed costs for EES capacity ( ) and VRE capacity ( ), where we neglect variable costs. The third term in Eq. (1) represents the total costs associated with load shedding ( ) , where represents the shedding costs, set administratively to VOLL or some other level. Note that we assume no separate costs for energy that can be stored in the EES, that is only the MW capacity of storage is considered. In other words, the duration of the storage device is not optimized, but assumed sufficient to meet the system need. Further development of the theoretical framework to include MWh constraints of storage will be part of future research. Eq. (2) states that, at each instant t, the demand ( ) must be equal to renewable generation ( ) plus the net charging of EES, computed as − ( ) − ( ).
Charging and discharging and − are bounded by the capacity of EES , as seen from Eq. (4). The upper limit of renewable generation ( ) is described by Eq. (5). In this respect, we make two assumptions. First, we assume VRE generation can be fully reduced at no cost ("curtailment") when needed (Milligan et al. 2015). Then, ( ) ≤ ( ) • where ( ) is the availability factor at time instant . This means that ( ) applies regardless of the installed capacity, which is a simplification which in general holds better for large installed VRE capacities, unless the availability of sites with good resource starts to reduce. This linear scaling is used in other research works (Sepulveda et al. 2018;De Vita, Kielichowska, and Mandatowa 2018;Cole et al. 2016;de Sisternes, Jenkins, and Botterud 2016).
The conservation of the energy stored from one step to the next is described by Eq. (6), where is the round-trip efficiency; finally, Eq. (7) imposes that all energy stored during a year must be released within the same year, i.e. the storage content must be the same at the beginning and the end of the year.
In addition to the idealized EES representation discussed above, this set of equations comes with additional simplifications; in particular, the impacts of the transmission network are ignored and operating reserve constraints are neglected.

Section 3: the case with no load shedding
To start exploring the cost recovery implications of this problem, we make an additional preliminary simplification, which is to remove the possibility of load shedding by removing the term ( ) from the set of equations. Removing load shedding makes the model simpler to analyze and serves as a benchmark for the analysis that follows in the next section, which considers load shedding.
The situation with no load shedding focuses on the system equilibrium between VRE and EES and describes a situation where high prices (load shedding or price caps in general) are removed by a regulatory/policy decision. It is actually called a "regulatory solution" according to (Stoft, 2002).
Schematically, the dispatch follows variations in net demand as illustrated in Figure 2: extra wind energy is used to charge the storage (between and ), because it is better to use VRE directly instead of suffering losses through EES. The detailed analytical solution of the problem is presented in Appendix 2, and leads to the following optimality conditions: Af system with VRE/EES capacities / leading to durations ( , ) that fulfill conditions (8) and (9) is optimal, meaning that it minimizes total costs and meets constraints (2)- (7). From the point of view of markets, this is a neutral starting point: we have simply formulated and solved a capacity expansion problem expressed as a cost-minimization set of equations.
If we now apply the paradigm of existing spot markets (marginal cost pricing) to such a system, we must set price to 0 all the time in the absence of shedding or variable costs. In turn, it leads to the market revenue of both VRE and EES to be 0, and their profit to be negative: neither VRE nor EES does recover its investment costs through the market. In addition, the Weighted Average Price of Electricity (WAPE) is 0 all the time, with WAPE defined as follows: This is in contrast with the Average Cost of Electricity (ACE) that must reflect investment costs and is therefore nonzero and should be transferred one way or another to end-users. ACE is defined as follows: Setting the market price to 0 all the time therefore separates the investment and the operational problems, as the market clearing prices resulting from the operational problem clearly do not recover the investment costs. Now, note that Λ, defined as in equation (8), is mathematically the Lagrange multiplier associated with the supply-demand equilibrium constraint. This means that Λ represents the marginal cost of increasing the load when the net load is positive, i.e. a price for electricity that reflects the investment cost in the system, as shown in equation (8). Consequently, the corresponding marginal cost of electricity when the EES is the marginal load will be Λ times the storage efficiency. Hence, based on this approach, we can set the following time-dependent pricing structure ( ): Appendix 3 shows that using ( ) as defined above in Table 1 leads to zero profit of RES and EES, meaning that both RES and EES exactly recover their total costs. Appendix 2 also details the calculations that lead to the weighted average price of electricity to be equal to the average cost of electricity, that is: The price structure proposed in Table 1 is therefore a mathematically acceptable solution to the system planner's problem with no load shedding. Said otherwise, if RES and EES are paid such as in Table 1 for every hour of the year, they will eventually recover their costs. Note that we do not address the unicity of this solution which is left for further work. In Appendix 4, we verify that this analytical solution coincides with the multiplier found by a numerical model, with 99.8% accuracy.

Section 4: Addition of load shedding
Here, we add load shedding as a price cap; this was not considered in the previous section. Defining price caps, is a challenging trade-off between too-low and too-high. In general, prices caps below VOLL lead to the missing money problem in wholesale markets, and lead to required out-of-market actions due to the price suppression. At the same time, high prices at the VOLL level may lead to the exercise of market power (Roques, 2017) (Joskow & Tirole, 2007). Finally, caps may be changed by policy makers and regulators over time (Roques, 2017) In the situation we describe (VRE and EES only), we must first note that load shedding can happen in different ways. In the typical description of a system based on fossil generators (see section 2), load shedding happens when load exceeds the total available generation capacity, ignoring energy supply issues such as limited gas supply, forced or planned outages of power plants, or depleted hydro reservoirs; such situations are usually ignored in power market models based on the load duration curve. In the 100% RES case, load shedding can also happen because the energy content of the storage is insufficient, i.e. not only because the total EES power capacity at the time of peak net-load is insufficient. In this paper, the energy capacity of EES is not treated explicitly in the analysis. Rather, we are interested in the traditional capacity-limited load shedding, as described in Figure 3 (left) below. In Appendix 5, we provide detailed results regarding how the type of shedding is influenced with relative VRE and EES costs, and that it applies at least in the case with VRE investments costs low with respect to EES costs, in which there is no energy-limited shedding.  A more detailed derivation of the optimal dispatch is given in Appendix 1 and is summarized in Table 2.
Just like in the previous section, we obtain the value Λ given by Equation (13), the Lagrangian multiplier associated with the supply-demand equilibrium condition. We check that it is equal to the solution found by a numerical solver.
The proposed price structure is as in the Table 3 below. We observe that this set of prices, leads to WAPE=ACE and to a cost recovery for both RES and EES, as detailed in Appendix 2. It is composed of four segments: the shedding period [0; ] in which price is because load shedding happens. The discharge and charge periods [ , ] and [ , ], in which prices are Λ and Λ respectively. And finally the curtailment period in which excess generation leads to a price of 0.
The proposed structure is different to pricing electricity in a uniform way using either 0$/MWh or the ACE. On the contrary, it is time dependent, with higher values occurring for higher loads and 0 when there is VRE curtailment. As already discussed in the previous section, the main finding here is that, unlike the case with conventional generators, we can no longer separate the planning and operational problems. This is because Λ is dependent on fixed capital costs.
Still, knowing whether ( ) is an actual price poses two types of challenges. First, the price structure is dependent on the load and VRE time series chosen as an input in the optimization phase, since Λ is dependent on , (which in turn depend on optimal capacities , ). This first issue, though, also appears when optimizing generation mixes based on conventional generators. However, adding another time series subject to variation from one year to another (RES generation through ( )) increases the uncertainty. Generally, that issue could be partially solved using either a set of different weather years or superimposed capacity constraints when calculating the prices dependent on Λ. Most likely, future power markets will require new security of supply "modules", for example capacity mechanism or decarbonization module, such as introduced in (Roques 2017).
More specific to this 100% RES case is the fact that generators (either VRE or EES) should get an incentive to offer their energy at prices following Table 1 instead of offering the lowest possible prices, a situation which could eventually lead to a collapse of prices to 0 throughout the year. While it is not the objective of this paper to enter into detailed market design propositions, we argue that the supervision of prices offered by generators already exists. For example, NYISO Market Power Mitigation Measures help to avoid "unnecessary interference with competitive price signals" (New York Independent System Operator, Inc, 2022), (NY-ISO, 2012). Among the categories that may warrant mitigation are "Physical Withholding" ("not offering to sell or schedule the output of or services provided by an Electric Facility capable of serving") and Economic withholding ("submitting Bids for an Electric Facility that are unjustifiably high"). While the proposed price structure does not pose a problem of economic withholding but rather the opposite, the very existence of power mitigation measures is an interesting element.

Section 5: conclusion
In many countries, the population, and in turn governments and policy makers offer an increasing support to VRE generation. This movement will likely increase in the future, since VRE addresses three main criteria for electricity planning: affordability, security of supply, and low carbon emissions.
Currently, most RES generation is procured through calls for tenders for power purchase agreements (PPAs). In many jurisdictions, this type of procurement coexists as a matter of fact with other generators that are mostly compensated through spot markets, although other instruments such as for example capacity markets and zero emissions credits may be used to recover costs. While this approach has been successful to some extent in delivering large amounts of new generation, new procurement schemes may appear in the future in the transition towards low-carbon solutions.
In this paper, we analyze a stylized case with 100% renewable energy supported by EES, and in which all the incentives for the build-up of the power system comes from short-term prices. We show that a solution to minimizing investment and load shedding costs is given by RES and EES capacities that fulfill two equilibrium conditions, namely energy storage conservation and one given by the dual value of the supply demand equilibrium. In turn, these conditions lead to short-term prices that incorporate fixed costs, which in turn ensure that VRE and EES recover their costs. These equilibrium conditions differ from the cases with conventional generators in the mix, since in those cases the operational problem and corresponding short-term prices can be separated from the fixed costs of the generation technologies.
While we argue that our research is an interesting exploration of a competitive equilibrium between resources that have no variable costs, there are various complementary research avenues that should be explored in future work. First, the competition between a set of possible VRE generators and EES technologies instead of one of each type must be analyzed. Second, adding explicit investment costs for the storage energy capacity of the EES is an area that must be explored. Also, the fixed costs of a storage system will be influenced by the way it will be operated, which adds complexity to our analysis. Finally, adding price responsive demand (Roques 2017) will be key since inelastic demand is often recognized as a barrier to the wider deployment of RES generation and price responsive demand can influence both price formation and the need for EES in the system.

Appendix 1: analytical derivation of optimality conditions, in the case with load shedding
In this section, we provide the detailed derivation of equilibrium conditions for the case with load shedding, as discussed in section 4. The case with no shedding is simply a sub ensemble of what is presented below.
The Lagrangian of the operational problem for an arbitrary time instant is: The KKT-conditions for this problem consist of (8)-(13), in addition to the following equations: Of particular interest is the following subset of equations which is the optimal dispatch: What it shows is that the short-term price ( ) during shedding is , while it is 0 during VRE curtailment, in line with previous work (Korpås 2020). More interesting at this point of our analysis is that the short-term price is proportional to ( ) during [ , ] and [ , ] (i.e. during marginal charging or discharging of EES). During these periods ( ) is necessarily nonzero: since the storage constraint is always the limiting one, ( ) is necessarily non-zero during those periods. ( ) is therefore undetermined but non-zero.
The optimal dispatch described above contains the operational constraints solved for optimal operation over the whole duration [0,T]. We can use these operational conditions to express the total cost minimization problem as: The optimization problem can now be written in the Lagrangian form, Where Λ is the Lagrange multiplier associated with the storage conservation constraint (7). Using the Leibniz integral rule, the following equations hold for the derivatives used to derive the optimality conditions: ( , ) =− ( − ) [ , ] − ( − ) [ , ] (28) As seen from Table 1, our modeled generation mix has in the end two unknowns and which in turn determine , and or vice-versa. The two equilibrium conditions above should therefore lead to optimal capacities * and * .

The profit function of EES is
where the instantaneous charging and discharging power is given by the storage operation strategy and is generally a function of the storage capacity (power and energy) and the market price. By using the segments from the optimal dispatch, the profit function can be expressed as where is the short-term electricity price during period [ , ], when the storage is the marginal contribution to the system. From the KKT-conditions, we also know that the price during period [ , ] should be equal to (see Table 1).
By using the optimal dispatch (Table 1), we can show that is equal to: By applying the storage conservation ∫ − ( ) = + ∫ ( ) − ( − ), the profit function becomes: By setting = 0 we obtain Therefore, the short-term price during marginal discharge that allows cost recovery is equal to Λ, the Lagrange multiplier associated with the storage conservation constraint introduced in Eq. (31).
The general profit function for VRE is: Following the same arguments as above, we get: [ , ] ( − ) + [ , ] ( − ) − Cost recovery, = 0, gives the same result as above: G. Tarel, M. Korpås, A. Botterud. Long-term Equilibrium  Therefore, the short-term price during marginal charging and discharging is found to be Λ, the Lagrange multiplier for the energy storage conservation constraint.
To illustrate how the two equilibrium conditions lead to the solution of our least-cost optimization problem, we use an example with the following input costs and technology data:  (Korpås & Botterud, 2020).
Those two conditions are hardly solvable explicitly, so we shall solve it using an exploration of the ( , ) plane. This is the purpose of the graph below which uses RES capacity ( , x-axis) and EES capacity ( , y-axis). Darker color is used when condition of Eq. (8) or Eq. (9) is fulfilled. We see that there is one single intersection between the two conditions, leading to the solution.  The optimal solution, for which both conditions are valid, is in this case given by: To assess the solution found analytically, we have modelled the system as a time-sequential Linear Programming (LP) problem based on the basic formulation (1)-(7) for comparison. The LP optimization model is implemented in Matlab and is identical as the model in Julia v. 0.64 used in (Korpås & Botterud, 2020).
The input costs and technology data are summarized in Table 6. From the numerical results, we can extract the values of , and . We first verify that optimal dispatch obtained using the LP model is compatible with the dispatch described in the previous section. RES is dispatched first, as expected, at EES is used as modeled in our optimal dispatch. We can then compare the optimal RES and EES capacities found using the numerical model with those found using the equilibrium conditions. The relative error is less than a percent. In addition, we can verify that optimal values for Λ, as computed using each of the two expressions from (33) are equal; this has been tested against a large set of variations on the parameters. For example, it can be seen below in an example with varying vs. As show above, Λ is the short-term price that applies during the discharging period. It coincides with prices computed from the LP model as the dual value of the supply-demand equilibrium constraint: G. Tarel, M. Korpås, A. Botterud. Long-term Equilibrium   Regarding cost of electricity, we checked for various conditions that WAPE (Weighted Average Price of Electricity) for the costumers in the market will be equal to the Annual Cost of Electricity (ACE), as show analytically. For example, this was tested against varying investment costs for EES: