Power system flexibility: an overview of emergence to evolution

Abstract

Power systems are evolving to the networks with proliferated penetration of renewable energy resources to leverage their environmental and economic advantages. However, due to the stochastic nature of renewables, the management of the rapidly increasing uncertainty and variability in power system planning and operation is of crucial significance. This paper represents a comprehensive overview of power system flexibility as an effective way to maintain the power balance at every moment. Definitions of power system flexibility from various aspects are explained to reach the reliable and economic planning and operation of the power system. The effects of the high penetration of variable energy resources on power systems and the evolution of flexibility in response to renewables are studied. A variety of resources during the flexibility evolutionary transition are introduced and discussed. As an influential flexibility solution in current power systems integrated with renewable resources, market design improvement is widely reviewed in this paper, and required modifications in market design mechanisms are investigated pertaining to various time horizons.

Introduction

Today, there is a global tendency for electricity generation to mutate from fossil fuels toward renewable energy resources, especially wind and solar energy. To take economic and environmental advantages of renewable resources such as energy sustainability and handling climate change, an overview of renewable resource extension policy has been codified in more than 173 countries around the world [1]. For instance, Germany and Denmark are willing to supply 80% and 100% of their generation through renewable energy resources by 2050, respectively [2, 3]. While conventional energy resources are planned and operated according to electricity market mechanisms, most variable energy resources (VERs) such as wind generators and photovoltaics (PVs) are not predictable because of their intermittent and stochastic nature originating from weather conditions. PVs and wind generators have three main common features: variability, uncertainty, and location dependency [4,5,6]. Hence, power system reliability may be threatened by the growth of VERs and there thus arises the need for the typical kind of flexibility that reinforces the system with the capability of compensating for real-time generation and consumption mismatches [7, 8]. In conventional power systems, the generation reserve capacity of thermal and hydro power plants is considered the system flexibility [9]. When the uncertainty pervades to the generation side to a larger scale than demand, the types of required flexibility change simultaneously. Advances in technology have helped flexibility with new resources and services. Today, the necessity of a typical flexibility, which provides market players with the wherewithal for financial decision making and helps enhance power system reliability, is without question [10]. Thus, various aspects of power system flexibility in the presence of VERs such as flexibility of resource, system flexibility assessment, problems and barriers of electricity markets, system planning and operation requirements, and finally market design evolutions in presence of VER, have been investigated in numerous studies.

Organizing flexibility resources will come in handy to overcome the high penetration of integrated VERs in power systems. Among these resources are more flexible conventional units [11,12,13,14], demand side flexibility utilization [15,16,17,18,19], grid extension [20,21,22,23], greater control over VERs [24,25,26], market design improvement [27,28,29,30], new ancillary services [31,32,33,34,35], energy storage systems [36,37,38,39], and smart grid initiatives [40,41,42]. Such flexibility options will help control the overgeneration of VERs at low demand times, improve the electricity levelized cost in the presence of VERs, and reach the desired objectives of VER integration [43]. The significance of issues for each system depends on local requirements; although all of them intend to extend the efficient and reliable operation of a system with high penetration of VERs [44].

Various models have been suggested for system flexibility assessment. These methods contain mathematical practices, measurement-based procedures, and graphical tools. In the first type, mathematical equations are utilized to reach the power system operation margins [45], as well as maintaining the required system flexibility by flexibility envelopes which are obtained via mapping the VER uncertainty on the system generation and time axis [46]. The second model deploys specific indices such as the expected unserved demand [47], lack of ramp probability [48], insufficient ramping resource expectation (IRRE) [49], and operational flexibility index [50] for flexibility assessment. The third type proposes to exploit diagrams and graphical tools to envision system flexibility, such as a “flexibility chart” which briefly shows the capacity of individual physical flexibility sources, e.g. conventional generating units, pumped-hydro storage, and interconnections [51]. While other tools, such as renewable energy flexibility (REFLEX) model [52], or flexibility assessment (FAST) method [53], evaluate the potential flexibility of generating units through power system simulation.

Uncertainty of VERs is another challenge. One approach is stochastic resource scheduling. For instance, in [54,55,56,57,58,59], the stochastic optimization method is applied to minimize the expected values of the objective function subject to the determined system variation probabilities. In addition, [60,61,62,63,64,65,66,67,68] have exploited robust optimization to define an uncertainty set instead of a probability distribution. Finally, a chance-constrained optimization is accommodated to consider both probability and uncertainty [69,70,71,72,73,74]. The other approach is the reserve analytical model which takes the operating and contingency reserve requirements into account with deterministic models [75,76,77,78]. In some cases [79,80,81,82,83,84], the look-ahead unit commitment (UC) and economic dispatch (ED) are employed to compensate for the net load (load minus VER generation) instead of the common real-time single-interval optimization models which cover the time-coupled multi-interval constraints.

As an alternative to face both variability and uncertainty of the net load, a new service, namely flexible ramping product (FRP) has been introduced by California Independent System Operator (CAISO) and Midcontinent ISO (MISO) in the U.S. [85, 86]. The FRPs have brought about the current UC and ED equations with two new market design variables namely flexible ramp-up (FRU) and flexible ramp-down (FRD) capabilities. The profitable deployment of FRPs in the industry coincides with a raised interest of researchers. References [87, 88] reveal the advantages of a deterministic dispatch model with FRPs over a stochastic dispatch model. Furthermore, it is shown that the advantages are dependent on the FRP parameters such as net load forecast errors and up/down ramp requirements. Several strategies have been modified in order to realize the desired transitions in the system. Among them are time-coupled multi-period market-clearing models and the incorporation of ramping capacity constraints. A day-ahead scheduling model is proposed by [89] to provide the optimal FRP amount for the European Central Dispatch Systems. In [90], an FRP model is extended to control the ramp flexibility in presence of high penetrated VERs. Reference [91] has discussed the effects of FRU and FRD constraints on a risk-based ED problem while considering an extended loss of load probability as the risk index. The influence of ramp requirements on the reliable and economic operation of a system is investigated in [92]. In addition to reliability and operation costs, environmentally related aspects such as weather quality and CO2 emission on a scaled MISO system are considered in [93]. Also, there have been approaches which offer the required ramping capability by means of electric vehicles (EVs) [94] or wind generators [25, 95].

Another field of power system flexibility is dedicated to the study of mid-term and long-term flexibility requirements in energy system models. Extension of the UC problem toward a longer time horizon of several days has led to a medium-term UC which only keeps the solution for the next day in [96]. A mid-term market-based power system is developed by [97,98,99]. Reference [38] has equipped the highly penetrated VER power system with electricity storage and studied the long-term effects. In [100], a mixed-integer linear programming (MILP) solution is offered to simulate the day-ahead electricity market with integration of long-term planning and UC. Reference [101] has compared a developed model of an open source energy system (OSeMOSYS) limited by operational constraints, with a complex model which links a long-term energy system model (TIMES), a UC, and a dispatch model (PLEXOS) for the case of the Irish power system.

How market design would affect the power system flexibility has also been much researched. Different market designs in the presence of VERs are compared in [102]. The interaction of VER support schemes and market design parameters is analyzed in [103]. Reference [104] has studied the distinct market design parameters while making the most of the advantage of wind generators. Reference [105] classifies the balancing market design parameters which influence the individual control areas and cross-border cooperation. Reference [31] has declared how demand and supply of flexibility would be altered with respect to the main design parameters and short-term market functions in the Central Western European region. Reference [106] comprehensively reviews the role of current distributed energy resources as a flexibility supplier, as well as impressive factors on a market design. A thorough discussion about the barriers of a large-scale market in the presence of VERs in the European Union (EU) electricity market design is presented in [107]. A typical framework is established to find the association of market integration barriers and upcoming defections according to the EU electricity market. Reference [108] has investigated conventional and modern electricity market designs in the U.S. which are looking forward to arranging an adequate resource to recover their expenses with respect to long-term reliability. It is followed by [30] to compensate for the cost of inadequate flexibility services, or unintentional flexibility provided by resources in market designs with high penetration of VERs.

This study intends to present a broad review of power system flexibility. While several reviews on the concept of flexibility have been already published, and each one has investigated the flexibility from a distinct aspect such as individual flexibility resources [15,16,17, 32, 39,40,41,42,43,44] and market design mechanisms for enhancing the flexibility [29,30,31, 101,102,103,104,105,106,107,108], flexibility is studied more comprehensively in this paper. This paper investigates the reasons for the emergence of power system flexibility. Further, it studies how the flexibility has evolved through the development of new trends and technologies in the power system, especially among the proliferated penetration of renewable resources. Flexibility is defined from various aspects and its effects on the functionality of power system from different time horizons is introduced. Also, various types of flexibility resources and their prospects are discussed. Moreover, the existing market design and its evolution to supply the power system with flexibility are discussed.

Power system flexibility definition

The concept of flexibility has been defined recently by researchers, and it is legislated by organizations such as International Energy Agency (IEA) and North American Electric Reliability Corporation (NERC). Hence, a lot of academic and industrial researches are centred on power system planning and operational flexibility. Currently, studies can be classified into two types: long-term planning flexibility and short-term operational flexibility. However, despite the above, a universal and common definition for power system flexibility is not represented yet. Each research group has declared its own definition according to the major field of study.

Flexibility is defined as the system capability to exploit all the resources to respond to the net demand changes in [49]. In [53], it is offered as the system ability to adjust the generation and demand in response to intentional or unintentional aberrations. From the operational point of view, flexibility is considered as the potential capacity to be utilized on the specific time interval, and with response to the net demand changes [109]. Similarly, it is described as power and ramp capability in order to modify the generation in response to the net demand changes within a specific energy duration in [110]. Also, flexibility is represented as the system facility to overcome the uncertainties, in addition to the generation and demand deviations, while preserving the system reliability with minimum extraordinary cost [111]. From the technical side, flexibility is required to compensate uncertainties and generation-demand deviations. However, from the economic viewpoint, providing flexibility means extra charges. Therefore, a compromised level must be established between the amount of flexibility and its compulsory costs. Equally in [112], flexibility is cited as the system competency to respond to a set of diversions in operation which are determined by risk management criteria. They would be available through control processes, and could be supplied within a specific time at marginal cost.

According to the above definitions, four aspects could be visualized for flexibility which contain time, function, uncertainty and cost. The first three features can be considered as technical norms and the last one as indicated by economic principles. In addition to these dimensions, other attributes such as location of flexibility resources and their proximity to the transmission or distribution system are also crucial [113].

The first dimension: time

The response time interval describes how fast a system reacts to the deviations and returns to the initial state. Referring to the study objectives, the time period may vary from a few seconds to a few months [114]. Based on the chosen time interval, a system can perform at various levels of flexibility. Short time intervals concentrate on the short-term system flexibility and show the response time over a few minutes or hours. By contrast, long time intervals focus on the long-term system planning and illustrate the changes such as generation combination, legislation policies, and altering the consumption pattern over a few months. A power system might possess adequate long-term flexibility, while suffering from a lack of short-term flexibility. For instance, a system might have enough capacity for the load growth over a year, but could not afford daily demand changes. Thus, determining the time intervals to evaluate the system flexibility is essential.

The second dimension: control functions

Control functions imply a set of corrective procedures which could be taken in a described operation process over a responding time interval. The set of control functions depends on the response time period. Figure 1 depicts the common corrective functions in power systems for different time intervals. As is shown, in each time space, the operator is armed with the particular corrective processes. The more the corrective function set is extended, the more choices are drafted to the operator to handle the unwanted events. Thus, in such a condition, response cost would decrease, or a greater amount of uncertainty would be covered.

Fig. 1
figure1

Common corrective functions associated with different time intervals

The third dimension: uncertainty

In fact, uncertainty is the lack of complete information about the future state of a system. Operation and planning are always stamped by uncertainty. Conventionally, uncertainty is related to the probable outages of system components, load forecasting errors, and market price. In recent years, within the increasing penetration of renewable resources, a new type of uncertainty has been added to the system due to the inaccurate generation forecast of VERs. The system uncertainty declares how much flexibility is required to manage it. In addition, it shows how flexible the system is. The chosen interval for covering a system uncertainty is determined with respect to the objective interval which reflects the amount of preferred risk specified by the decision maker.

The fourth dimension: cost

Another aspect of flexibility is the variation and uncertainty in response cost which are functions of controlling processes. A power system planner or operator always tries to provide system flexibility, while minimizing costs. Referring to this objective function, the most economic control processes are designed to respond to the uncertainty. In addition to minimizing the flexibility cost, sometimes the marginal cost or marginal risk is considered to serve system flexibility. In such a condition, if the marginal cost is high (low risk), there would be no limitations on the controlling progresses and their related costs. However, if the marginal cost is low (high risk), some of controlling processes would be uneconomic and might be neglected.

Depending on the purpose of the study, the response time window T is specified. Then, an uncertainty function denoted by U(T) and corrective function denoted by F(T) are established for each time period. Based on the cost of control function denoted by C(F) and system marginal risk, the most economic flexibility resources will be allocated to meet the flexibility requirements. Thus, proper definition of quadruple dimensions will lead to supply and evaluation of necessary flexibility in contexts of both power system planning and operation.

Effects of increasing renewable resources penetration

In [115], it is shown that as the penetration of renewable energy resources exceeds 1 GW in the Electric Reliability Council of Texas (ERCOT), a flexibility requirement is urged. For instance, adding 14.5 GW of solar generation to the ERCOT leads to an increment in the maximum 1-hour ramp requirement to 135%, and in 3-hour ramp requirement by 30%. Reference [116] has declared that the need for flexibility highly increases at a renewable penetration greater than 30%, especially for solar generation. Also, a study of the Philippine Department of Energy estimates that about 50% of extra generation capacity required as future reserve capacity must be from mid-range and peaking generation units [117]. An analysis in [118] shows that for 100% renewable electricity supply scenarios at 2030 and 2050, a generation mix with a nominal capacity of more than twice maximum demand is required. All these studies reveal that more penetration is in parallel with the need for more flexibility. In order to diagnose and present the proper solutions to handle flexibility problems, acknowledging various effects of renewable resources on the power system is of high importance. In the following, these effects are classified and explained. According to the time sequence of flexibility, effects can be divided into short-term, mid-term, and long-term types shown in Fig. 2.

Fig. 2
figure2

Effects of penetration of renewable energy resources on flexibility requirement

Short-term effects

Increasing required reserve capacity, ramp capability, inertia and frequency response, and minimum generation output constraints are general effects of flexibility in the short-term horizon [13, 35, 85, 119, 120]. Secure and reliable power system operation require appropriate reserves allocation to compensate for the uncertainties in demand and generation availability. In the presence of sustainable resources, referring to their inaccurate generation forecast, system uncertainty will be raised. So, more capacity of conventional resources is devoted as a reserve to provide the power balance in the operational time. On the other hand, the generating unit ability to yield the reserve varies with respect to the ramp capability during the considered time interval. Hence, in addition to developing the amount of reserve, the needed ramp capability should be arranged to cover the variations and uncertainties which are caused by the presence of renewable resources. Penetration of renewable resources has led to hourly and sub-hourly increasing variations in the net demand. In such a situation, conventional generating units must be able to shift their set points proportional to the variations in the net demand. Thus, the slope of net demand changes should be smaller than the sum of ramp capability of conventional generating unit changes. Otherwise, more flexible units are called for to provide adequate ramp capability.

In the economic dispatch time horizon, generating units are planned such as to be capable of performing the load following in response to the net demand changes. In the operation with time steps of seconds to minutes, the regulation service is the only response option toward the net demand changes. The power system utilizes regulation services to compensate for the frequency deviation and/or area control errors (ACEs). Confronting a regulation service shortfall would cause deviations in the power exchange between areas, loads or VER generation loss. In such a condition, the energy price is not equal to the market price, and this may affect the long-term efficiency of the market. Thus, with respect to the increments in the renewable generation variability and uncertainty, regulation reserve and its flexibility would be influenced because of compensation in the renewable generation deviation.

Added to the increasing required reserve and ramp capability caused by proliferated penetration of renewable resources, the minimum generating power constraint of conventional units is another stumbling block. Consider a state in which the renewable generation supplies all the demands. All conventional units such as steam and nuclear units must be uncommitted. However, this might be impossible because of long turn-on and turn-off times of these units. If a drop in the renewable generation occurs, costly rapid start units would be utilized instead of cheap basic load units. This is not economical. In addition to the prolonged running up of conventional units (few hours for combined cycles, days for steam plants, and weeks for nuclear units), consecutive turn on/off would be costly from different aspects such as fuel, manpower, repair and maintenance. Also, the energy price might be converted to a negative amount which forces these types of units to pay the consumers in order to avoid repeated turn on/off.

In the presence of proliferated penetration of renewable resources, system inertia and frequency response might also be affected. In current power systems, if a disturbance occurs, conventional units will supply the required inertia and frequency response inherently. In the case of low demand and high renewable generation, conventional units would own a limited share of energy supply. On the other hand, most sustainable resources are based on power electronic components with low inertia. Hence, in the case of a striking contingency, system inertia and frequency response might be so weak as to activate under frequency load shedding relays.

Mid-term effects

As mentioned above, high penetration of renewable resources would cause conventional units to turn on/off frequently. For base-load units, the raised duty cycle could impose high costs on the components and lead to a marked up forced outage rate, and unit amortization [121, 122]. Amortization may extend the repair and maintenance costs, shrink the income because of repeated lasting black-outs, and decrease the efficiency because of low generation.

Long-term effects

In the long-term, fossil fuel and environmental constraints will shift the investments toward low carbon base-load generating units such as nuclear units, geothermal, and carbon capture and sequestration. According to the high minimum generation amount and low ramp capability, these units possess a limited flexibility which will not let them integrate with the network at any time [123]. Consequently, the rate of return for these units would be less and their attractiveness as an investment would disappear. In such a situation, fast response units with high flexibility will be more common, albeit at a higher energy price.

Referring to the mentioned effects, some indications which reveal the lack of flexibility in a power system include [124]:

  1. 1)

    Difficulties in fulfilling the power balance of generation and demand leading to frequency deviation and load loss.

  2. 2)

    A rise in renewable energy curtailment resulting from transmission constraints and over-generation.

  3. 3)

    Frequent and abrupt ACEs among different control areas caused by deviations from the scheduled power exchange. The deviation reflects the system deficiency in performing power trading and wheeling responsibility.

  4. 4)

    Negative market prices brought about by factors such as limited ramp down capability of generating units, demand inability in absorbing excessive generation, surplus of VERs, and transmission line congestion. Although, negative price emergence is possible in a system without renewable resources, renewable energy integration will intensify this issue.

  5. 5)

    Price volatility, or fluctuation among high and low prices which displays the limited transmission line capacity, fast response peak unit shortages, and absence of demand response.

In systems which portray a lack of flexibility, planners and operators are looking for solutions with most efficiency and least cost to provide the desired level of flexibility. There are various solutions to increase system flexibility which will be introduced in the coming section.

Power system flexibility resources

In general, power system flexibility can be classified into two types: physical and structural flexibilities. Physical flexibility points to the system physical competency in response to the demand and generation changes which is necessary but not sufficient for a flexible system operation. The structural flexibility is the capability to exploit physical flexibility through operational instructions or market procedures. The importance of structural flexibility should not be ignored, since reaching the proper operation is mostly possible by its means. Table 1 introduces the resources of the power system flexibility, and also their relation to the physical or structural flexibilities. If dominant flexibility provision by a resource is based on its inherent physical characteristics, it is categorized as “physical” like flexible conventional units. Structural options such as market design improvement provide flexibility mainly through market design or operational practices that are mostly unbiased towards the use of different technologies. Flexibility resources like demand side flexibility utilization require physical flexibility from the system components along with proper operational and market structures. Therefore, they are classified as both. In the following, each sort of flexibility resource is discussed.

Table 1 Resources of power system flexibility

Flexible conventional unit

System physical flexibility is promoted through fast start conventional units with high ramp capability, low minimum generation constraint, and short minimum up/down time constraints. Increasing duty cycle of fossil fuel-based power plants results in a remarkable rise in the forced outage rate of units, repairs and maintenance, and extreme depreciation and exhaustion of generating units. On the other hand, economic efficiency of conventional units is uncertain, especially for coal-fired power stations which are designed to operate with 80% of annual capacity. In addition, a noticeable hardware modification is required to make the units flexible and make it possible to operate with a capacity smaller than 40% [12]. Hence, a cost-worth analysis seems to be essential to evaluate the most economical choice among making robust, eliminating, or revising the operation procedure in order to refine system flexibility. Another choice is to exploit more flexible units such as those with a reciprocating engine or combustion turbine with low minimum generation constraint and high efficiency.

Flexibility utilization at demand side

Organizing the market to motivate and exploit demand response is a solution to boost system flexibility. Demand response needs demand side physical flexibility and its related infrastructure, as well as appropriate arrangements to motivate a favorable response. Demand side management could provide various time interval balances (on the scale of second to season) by means of energy, capacity and ancillary services bidding such as regulation, load following, contingency replacement, etc., which could be utilized to improve system flexibility [17]. Controlling the demand side with respect to system conditions, whether through price-based programs for individual consumers, or by aggregating the responsive demand, would result in flexibility increase [18, 19].

Grid interconnection and flexibility at grid side

If adequate physical equipment and infrastructure exist, interconnecting the balance areas would be the key to develop the physical and structural flexibility. An interconnected network, which is characterized by physical connections within the nearby regions, provides the increased access to a variety of demand and generation types. In addition, spreading geographic expansion would smooth out total renewable resource generation and net demand [125]. Thus, the ramping requirement of the interconnected system is diminished. This results in a discount at the required flexibility service and in costs [43, 126]. However, concerns such as regional information security and big data processing should not be neglected for an interconnected grid [127]. Moreover, transmission line congestion may prevent the system from fully leveraging the grid interconnection.

In such a case, by adjusting the physical characteristics of the power grid to control the power flow, grid side flexibility resources can provide additional operational flexibility. Grid side flexibility resources are categorized into discrete and continuous resources [128]. Transmission expansion planning [129, 130] and line switching in operation [131, 132] are typical discrete resources which change the power grid topology. Continuous resources such as high voltage direct current (HVDC) transmission and flexible AC transmission system (FACTS) devices would help the system operator monitor and manipulate the system more effectively. This leverages the greatest capability of transmission lines [133, 134].

Market design improvement

Market design improvement is a structural tool for heightening system flexibility. Being large, fast, neutral towards the players, and providing approaches based on the functions of players are four basic characteristics of a standard market design. As discussed above, grid interconnection would increase system flexibility. However, taking the most advantages of the gained flexibility is dependent on an appropriate market design. Thus, it is urgent to define proper and effective structural configurations to manage and operate the tie lines and markets in a coordinated manner. A main property of market design is having a fast and frequent energy market, commonly referred to as real-time or balancing market, which would bring about system flexibility because of near real-time operation, and a more accurate VER forecast, and better accommodation of real system conditions [12]. Note, long settlement intervals do not induce enough incentives for flexible resources. Hence, shorter intervals will lead to a better pricing paradigm. This then also satisfies the flexible resources.

Being neutral toward the players means that all the resources (regardless of their type of technology) compete fairly and correspond to their various collaborations in supplying the desired services [30]. In market design, a new ancillary service is defined regardless of the technology type and refers to the required response speed, response level, or length of the performance. In addition, payouts must be shared according to the resource operation. Two suppliers who present distinct levels of the same service should be paid relevant to their service level. More service equals higher payoff. In addition, two different suppliers who perform an identical level of service should be paid the same regardless of their type.

Greater control over VERs

The uncertainty and variability of highly penetrated VERs have led to a flexibility shortage. Hence, control over the VER generation would alleviate the situation. For instance, when the generated power exceeds the required system demand, or the system deals with a transmission line congestion, flexibility can be provided via VER generation curtailment. However, it requires structural configurations to develop the incentives and management for the lost energy, as well as physical competency for the generation curtailment. Curtailing the VER generation is the last preferred choice of owners since a portion of the income would be lost and their cost would be raised [135]. VER generation curtailment could be either executed by system operator commands or market automated proceedings. In the first method, and in a time of surplus generation, the system operator is allowed to cut off a part of renewable generation disregarding whether it is optimal or not. In a well-designed market, with on-time over-generation and low demand, negative prices will emerge. Therefore, VER owners are encouraged either to split their generation voluntarily, or invest in new flexible technologies [136, 137]. Currently in MISO and New York ISO (NYISO), VERs offer their bids to the market the same way as conventional units, although their bids are in order to reduce the generation. If bids are economic, wind generation would be shrunk to avoid over-generation [138]. New developments in the field of renewable energy resources have motivated a lot of studies of the capability of VERs in providing various ancillary services such as inertia response [139, 140], frequency regulation [32, 141], voltage support [142, 143] and operating reserves [34, 144, 145]. However, high penetration of renewable energy resources might also raise the necessity of revisions in the current mechanism and design of ancillary services, and the hidden potential for the emergence of a unique market for energy, ancillary services and VERs as discussed in [35].

New ancillary service

Regulation, spinning, and non-spinning reserve ancillary services had covered most power balance requirements either in the normal condition or in a time of contingency. However, if the growing penetration of VERs continues, these services would not be sufficient to obtain the required system flexibility as mentioned in Section 3. Thus, more desired flexibility would be resolved by three types of ancillary services in future power systems: load following, frequency response reserve, and inertia response.

Load following

Load following is a method which follows demand variations during power system operation to maintain the balance between generation and demand. Under normal conditions, a load following service is stocked by the operator through economic dispatch of conventional units within the hourly energy markets or 5 to 15 minutes real-time markets. By proliferated penetration of VERs, the required ramp capability to follow the demand might be greater than the ramp capability of conventional units participating in the real-time market. While contingency reserves are dedicated for deployment at the time of a significant contingency such as generation loss, providing a new ancillary service such as load following might be the best option. As an example of load following services, the new product named “flexi-ramp” which has been presented in MISO and CAISO markets [85, 86], is further discussed in Section 5.2.

Frequency response reserve

Currently, frequency response reserve has been counted in the system studies, although frequency response is not believed in as an ancillary service in the market yet. With an expanded presence of VERs based on power electronics devices, and inherently possessing no inertia, frequency response and inertia response as two types of ancillary services seemed to be more vital than ever [120]. There remains a potentiality to create an ancillary service in the market, namely frequency response, in the near future. On the other hand, advances in technology have led to the possibility of contributing VERs, storage, and responsive loads to the required system frequency response in addition to the conventional units [35, 146, 147]. In such a condition, the desired system frequency response will not be granted as a prerequisite for units in order to get connected to the network. Rather, it will be suggested as an ancillary service to meet the system need for an appropriate response.

Inertia response

In the case of highly proliferated VERs or equally the fading presence of conventional generation, the system would suffer from the inertia response issue [148]. The result of Western Electricity Coordinating Council (WECC) system simulations in [149] reveals that under stress condition (e.g. low demand, high renewable generation, and absence of conventional generation), a significant drop in the system frequency response will occur while the system is facing a disturbance. The intensity of frequency response drop varies with respect to wind generation amount and number of offline fossil units. In addition, new resources such as VERs, storage, and flywheels can afford an inertia response as good as or even better than the frequency response when they have no contribution [150, 151]. Meanwhile, research on supplying the inertia response by means of market, or the required conditions for the network connection will continue. Although, when there is no more empirical evidence, the desired inertia response will be obtained via the required conditions for the network connection.

Energy storage system

Deployment of energy storage in parallel with the high penetration of VERs is undoubtedly needed. While the output power of VERs is accompanied with variability, energy storage can assist the power system to absorb the surplus generation of VERs in the case of over-generation or discharge of their energy to the system to help with any case of production scarcity. Further, storage might be performed in the ternary levels of the power system, such as water in pumped-hydro storage as a primary input at the generation, bulk battery storage at the transmission, and electric vehicles at the distribution levels [44]. Although providing storage for each level imposes new operation cost to the system, their integration among the high penetration of VERs can reward the system with multiple services which will improve system flexibility [38]. On the other hand, the capital costs of storage will become more economical in the future, as storage technology develops [152].

Smart grid initiative

Smart grids are systems in which system components are connected to each other through a bidirectional communication framework and it is thought to be the best infrastructure for integration of numerous distributed energy resources among a network. Although system complexity has increased because of the transition toward smart grids, vast control over the units is gained and reliability is promoted. Also, from the market point of view, all the players involved in an intelligent connection and optimizations would lead to the satisfaction of every agent [153]. The coordination between the transmission system and distribution operators by means of smart grids provides a vast potential for both kinds of physical and structural system flexibilities [154]. Advances in electrical vehicles, small batteries, smart meters, communication links among distributed energy resources and distribution operators, controllable and interruptible loads, demand side management with either incentive or price-based scheme, and new regulations for the distribution system have all offered new flexibility options for the system [17, 106, 155, 156]. Another example for smart grid initiatives is the role of the distribution system operator in Europe, which integrates the demand response, electricity storage, and renewable energy resources at the distribution level, and further coordinates with the transmission side and offers it as a source of flexibility.

A summary of discussed flexibility resources is presented in Fig. 3.

Fig. 3
figure3

Summary of flexibility resource in power system

Sectoral integration

The integration among various types of energy hubs such as gas and electricity and their energy carriers is another source of flexibility which increases the security of energy supply. However, the system might confront different challenges. This option also allows the system to convert diverse types of energy to each other by means of storage which further increases the provided flexibility. For sectoral integration, market frameworks need to be revised in order to allow the consumer to optimize and choose the ideal type of energy resource [157,158,159,160].

Electricity market design evolution

To increase system flexibility, proper economic signals must exist, so that current sustainable resources can make a contribution to the system operation. On the investment time horizon, adequate flexibility should be extended in the long term to be utilized in the required time. By the increasing penetration of renewable resources, generation combination is shifting toward low-cost generating resources, high-cost investment, and a smaller utilization factor. This changes in the generation combination might affect the return on investment and the incentives for the contribution of flexible resources in the market.

Some of essential tools for tackling such challenges are already prepared. However, market design might not be arranged properly to bring about the primary incentives for the complete presentation of the flexible resource capabilities, sufficient deployment, and attracting enough investment. Incorrect operation of current flexible resources, or their lack of ability to provide flexibility, might result in a drop in system efficiency and reliability. This could restrict market operator access to system flexibility for making an entire response to net demand changes. Consequently, it may be hazardous to system reliability. Most often, exploiting the more expensive flexible resources instead of economic types which are not available to the market, causes a rise in the costs. In addition, when there is inadequate revenue to recover the cost of those resources which are required for supplying long-term reliability, the market would be abandoned. Therefore, it is mandatory to design the market such as to inspire the flexible resources to participate and invest in system operation. Another aspect of a well-designed market is its neutrality towards diverse technologies, and preparing the competition infrastructure for responding to system requirements. According to the tremendous potential impacts of proliferated penetration of VERs on the market, rates of income for generation resources would change, and critical products which are offered by market participants might reform priorities. These changes simply reflect the modifications in the generation sequence and desired services. However, changes might lead to unwanted consequences such as an inefficient electricity network or a nadir in reliability. Hence, common methods to face such challenges as a lack of incentives to offer flexibility in short-term operation and revenue sufficiency for the long-term reliability are introduced in the following. Also, the evolution of these approaches to improve the penetration of renewable resources in recent years is reviewed.

Market design and its primary flexibility solution

Current electricity markets are designed such that they present the necessary motivation for resources to offer a variety of services such as energy and different types of ancillary services. The existing market components which are effective in the flexibility motivation are reviewed below. Then some of the new transitions at the design stage are introduced. These heighten the system flexibility in order to handle the increasing variability and uncertainty of VERs. In current markets, there are few mechanisms which succeed in incentivizing market participants to supply flexibility during system operation. However, it is still not completely clear to what extent these mechanisms could provide the desired flexibility.

Centralized scheduling and dispatch

In this mechanism, generating units share their technical and economic data with the market operator. Then the operator meets the required system energy and ancillary services for power system security, as well as biddings and constraints of generating units [161]. If the generation scheduling responsibility is upon the generating units, the operator would be dealing with a pre-set schedule before clearing the market. If a high portion of the energy supply is identified in such a way, the operator might have to confront a shortage in availability of flexible resources to maintain the power balance. It could be that the system physical flexibility might suffice, but the market operator cannot access it.

Repeated generation scheduling and short clearing intervals

An “intraday” market with a time interval of 6-12 hours is running in the electricity market in European countries. At a much shorter time interval of 5-10 minutes, a “real-time” market is running in almost all advanced markets around the world (equally as a balancing market among the European countries) to clear the imbalances between demand and supply [162]. In these markets, the unit outputs are scheduled, and share their 6 hours (for intraday market) or 5 minutes (for real-time market) length flexible bids to the operator, and bids are updated in each interval. Therefore, a better pricing for system real-time status would be performed and more incentives are brought to bear for the participant resources in this market. As in the real-time market, energy selling is restricted by unit ramping capability, and those with higher flexibility get a bigger piece of the revenue cake.

Existing ancillary service market

Regulating and operating reserve markets are running alongside the electricity markets to provide required ancillary services. As generating unit flexibility increases, the profit within the ancillary service market would grow [163]. Thus, ancillary services might work as an incentive for generation resources to raise system flexibility to gain higher profits.

Make-whole payment guarantee

In power system operation, because of the unit contribution constraints and cost non-convexity, a special condition may appear in which market prices could not recover all the unit operation costs. So, while aiming to motivate the flexible units, all the operation costs of a generating unit which has contributed in the energy and ancillary service supply will be recovered through the mechanism of “make-whole payment” guarantee [161, 164]. A simple type of this mechanism can thus be expressed as: if the total costs of a generating unit are more than its total incomes in the market, the total payment to this unit would be equal to the mismatch of unit costs and incomes.

Day-ahead profit assurance

This mechanism is intended to halt the decline of generating unit revenue caused by energy mismatches between the scheduled day-ahead market and the real-time market [165, 166]. Since this deviation is normally compensated by flexible units, the mechanism is aimed to persuade the units to participate in the real-time market. In fact, this set-up guarantees that reducing the generation in the real-time operation will not harm the generating units, comparing to the scheduled amount of the day before.

Inter-zonal market integration

This reform in market design provides power system flexibility by widening the market through market integration and mainly has taken place in the European electricity markets. The main features and examples of this integration include the efficient pan-European market, efficient cross-border market integration, and enablement of regulatory environment. The efficient pan-European market mostly seeks to harmonize the day-ahead market and intraday market within Europe such as the market coupling project which has integrated the Nordic pool to the Central Western Europe electricity market area [167]. However, integration and coordination of separated balancing markets to each other is very complex. Hence, another priority, namely cross-border balancing cooperation, is considered widely to reduce the challenges of the previous design in which reserve capacities are dedicated to each controlling area after coordination [167]. In addition to European electricity markets, coordinated transaction scheduling between NYISO and PJM [168] interchange optimization by PJM and MISO [169], and inter-regional interchange scheduling between NYISO and ISO-New England (ISO-NE) [170] are several interregional trading projects that have recently been carried out among U.S. electricity markets. Finally, enablement of the regulatory environment aims to aggregate the flexibility services such as that of the cooperation among German transmission system operators under the supervision of the German regulatory service introduced in [171]. This optimizes the cost and coordination of the frequency restoration reserve and the tertiary control reserve.

While the above-mentioned primary solutions encourage the generating units to provide flexibility over a short time horizon, there exist other market designs which bring about incentives for long-term solutions. These solutions provide motivation to attract new reliable resources, or convince the existing resources to promote their technology for the requested flexibility. A decisive factor in attracting investment of new generation resources for reliable system operation is the revenue sufficiency concept which has been recognized since the arrangement of the very first market designs. However, whether the current designs have succeeded to recover the fixed costs of resources for the long-term reliable operation is still debatable. On the other hand, with the proliferated penetration of VERs, a holistic pattern still does not exist to properly determine system resource adequacy which is influenced by the renewables. The existing solutions for long-term market design include:

1) Scarcity pricing

This method is associated with energy only markets in which the only solution for obtaining revenue sufficiency is through high prices in the energy and reserve markets in the case of generation scarcity [161]. When the required supply or ramp capability is not available and there is no marginal generator to clinch the price, scarcity prices are employed. High scarcity prices should be settled in the system operation to hand over the desired revenue for recovering the investment cost of peak load units during their lifespan. Generating units with high fixed operation costs hope for scarcity prices to recover their costs. A power market operator might set a price cap to avoid any unusual incline in the prices at a time of scarcity. This leads to a “missing money” problem and disappoints the generating units potentially participating in the market [172]. Finally, when revenue sufficiency has vanished, power system reliability would face a gross drop since suppliers are not incentivized to enter the market. So, for investment in new generation capacity, guaranteeing return on investment based on scarcity pricing incomes would be challenging. Likewise, it is also debatable whether this price is individually enough to offer investment motivation.

On the other hand, the penetration of VERs with marginal costs of close to zero (and even negative prices) has aggravated the revenue sufficiency problem for conventional generating units. Moreover, investment on these units is also fading, since their revenue sufficiency has been accompanied with uncertainty [173]. So, the current design of the energy-only market is not responding well in the presence of VERs and scarcity pricing for renewables must be settled based on their flexibility, location on the power network, and the quality of technology. In addition, the variability and uncertainty of renewables force the system to provide more operating reserve to guarantee reliable system operation, but also the need to maintain the operational reserve for this system only makes the problems more complicated.

2) Capacity market

This method aims to prevent instabilities and fluctuations in the investment of electricity generation. Capacity markets are designed wishing to arrange a market structure to confidently reach generation resource adequacy in the desired time, and assist the generation resources to recover their investment costs [161]. In the U.S., markets such as PJM, ISO-NE, and NYISO possess capacity markets which declare the required level of capacity and establish a transparent market environment to possibly recover the fixed costs of generating units [174]. The final objective of resource adequacy and related capacity markets is to serve a level of long-term reliability such that it would be compatible with power system priorities. On the other hand, in the European capacity market, each member might have his own mechanism. For instance, France and UK run a capacity market, while Spain and Italy are planning out for capacity payment or Nordic countries such as Sweden or Finland are aiming at the strategic reserves [175]. In any event, in current capacity markets, two important prospects of generation resource functionality are neglected. First, the unit capability for a flexible operation points to features such as ramp capability, minimum allowable generation, and minimum up/down time. Second is the focus on the quality of generating unit response as a capacity resource in a critical system situation. The existing capacity market designs do not distinguish flexible or inflexible units. Thus, there is no incentive for participants to provide more flexibility. An overview of current market designs in short-term and long-term time horizons has been depicted in Fig. 4.

Fig. 4
figure4

An overview of current market designs in short-term and long-term time horizons

New solutions to promote flexibility

Recently, there have been transformations in market design to provide a better motivation for the flexibility concept:

1) Scarcity pricing developments

ERCOT as the only electricity market which runs energy-only markets in the U.S. has recently taken two steps to promote scarcity pricing policies [176]. The first is that the price cap has been elevated gradually (up to 9000 $/MWh) such that the energy price might face a remarkable rise during scarcity periods. Secondly, the operating reserve demand curves are utilized in the real-time market to preferably manifest the final value of reserve reliability in the price. According to the dependency of energy and reserve, the curve affects either the reserve or energy. The operating reserve demand curve is extracted based on a probabilistic assessment with respect to the probability of load loss for various levels of reserve, as well as the value of lost load estimation. Comparing the operating reserve demand curve with the fixed reserve demand curve reveals that the market price would be higher when the system marginal reserve is greater than the required amount in the fixed curve. Consequently, the generating units are awarded more income to recover the fixed costs. However, in a few cases (when reserve margin is smaller), the operating reserve demand curve will lead to lower prices.

2) Capacity market promotion

CAISO is the first electricity market which has considered the flexibility concept in the capacity market [177]. Due to the revision newly approved in the CAISO, demand suppliers must consider sufficient flexibility for unforecast variabilities and uncertainties, as well as adequate capacity to supply the forecast peak load. With this new law, each unit which is participating in providing the required flexibility should offer a flexible capacity in the short-term energy markets (day-ahead and real-time markets). With such a mechanism, the required flexibility is confirmed based on the maximum net demand upward ramp during three hours predicted to occur in a month. In addition, the European Commission has suggested capacity market regulations to ensure the secure supply of electricity [178]. These mechanisms include incorporation of interconnection capacity into national mechanisms in which generators in an adjacent market are permitted to offer capacity bids, single capacity market design wherein national capacity mechanisms are coordinated with a single design, and a single EU-wide capacity market that could be considered as a capacity to deal with the scarcity in each market area. However, there is another alternative instrument for the capacity market in the Europe which aims to offer generation flexibility rather than capacity.

3) Supplying flexibility with non-conventional resources

Conventionally, thermal and hydro units serve the required system flexibility in the energy and ancillary services market. Accordingly, electricity markets have been settled upon these unit conditions. However, as mentioned in Section 4, in addition to conventional flexible resources, the integration of small distributed energy resources and energy storage, as well as integration of demand side management resources might be counted as new resources of flexibility, while they can be sufficiently incentivized to provide flexibility in the retail markets. Due to the increasing need for flexibility in the presence of VERs, electricity markets are transforming to reach an appropriate level of flexibility by means of new resources. More about supplying flexibility via non-conventional units and their effects can be found in [15,16,17,18,19, 24,25,26, 32,33,34,35,36,37,38]. Another type of non-conventional resource for flexibility that is introduced in the European energy market is the privileges given to the aggregators by the European Commission to enable a decentralized type of flexibility [179].

4) Evolution of regulation reserve market

Recently, there have been changes to inspire flexible resources for more contribution in the ancillary service market. Most changes are related to the regulation reserve market since the necessary regulation reserve and its deployment have increased remarkably [180]. Historically, the payment to the units which are responsible for regulation reserve is only for supplying the regulation capacity based on the defined prices. However, recently there have been struggles to design a market mechanism which is capable of analyzing the operation of units in response to the regulation reserve more accurately, and calculating the lost opportunity cost of supplying the regulation reserve capacity [181]. With this new mechanism, the payment for the regulation reserve is identified based on the requested amount for the generation adjustment, accuracy of following the automatic generation control signals, and numbers of responses to the control signals in each time step. Therefore, more flexible units will earn a higher income, and consequently there will be a drive for generating units to promote system flexibility. More details on this concept for the MISO market are presented by [181, 182].

5) Ancillary service market for primary frequency control

As explained in Section 4.6, right now the system frequency response is administered as the required condition for integration to the network. There are almost no criteria based on reliability to determine the required system primary frequency response. Also, no incentives are embedded for the generating units to come up with the primary frequency response [183]. On the other hand, in a few markets, penalties have been established for deviations from the scheduled amounts [184]. So, not only are units not obliged to equip the system via frequency response, but also they are penalized for their contribution. By increasing penetration of VERs and substituting new resources for conventional units in the primary frequency and system inertia response, the need for incentives for the frequency response would be increased [185]. If an ancillary service market is set up for primary frequency control and enough motivation is created, both conventional and non-conventional resources would be encouraged to assist in this market. ERCOT as a pioneer ancillary service market designer for frequency control has initiated studies [186]. In [120], a market has been outlined for primary frequency control ancillary service which could be referred to.

6) Extended pricing mechanism

The European electricity market has been adapted to price zone revision, which seeks to remove congestion on transmission lines. The European Commission is planning to effectively increase the number of pricing zones, since this will result in better price signals for generation, better incentives for investment in new capacity, and finally a closer approach towards a nodal pricing mechanism [187]. On the other hand, marginal pricing theorem for energy and ancillary services in the U.S. is established upon the continuity, convexity, and increment of costs. However start-up and no-load costs distort the real system costs to a non-convex type which leads to figuring a payment for balancing the cost of resources whose no-load and start-up costs are not covered [161]. A proper example is adequate pricing for a peak load gas generating unit which turns on to supply the demand. When there is an extra mega-watt demand, and a peak load unit by minimum allowable generation of 20 MW is turned on to supply it, the cheap unit should decrease its generation by 19 MW before the gas unit. In such a condition, the marginal cost is equal to the price bidded by the cheap unit which means that in spite of deploying the more expensive unit, the energy price remains unchanged. Hence, the gas units need a guarantee for complete payment. In addition, other units have been affected since the price has not risen, and it must be compensated for [188]. So, extended locational marginal pricing as mentioned in [189, 190] might be the key. Note, because of the growth of uncertainty and intermittency caused by VERs, the possibility of employing flexible resources survives only because of technical reasons, not economic ones. So, there would be an increase in the number of generating units whose costs are not recovered because of marginal pricing. It seems that the pricing mechanism should somehow solve the non-convexity of costs in the future.

7) Flexible ramping (flexiramp)

Some of the electricity markets all around the world such as CAISO and MISO have arranged new modifications in market design. These have the aim of guaranteeing to supply the system flexibility requirements when there is high penetration of VERs [85, 86]. In these markets, an explicit ancillary service is introduced for energy flexibility. Results from the study of the CAISO market reveals that with high penetration of VERs, in addition to the reserve capacity limitations, a notable growth is recognized in the slope of net demand changes. Accordingly, CAISO first added a flexiramp constraint to the economical dispatch schedule in 2011 to supply the needed flexibility capacity for the system through the real-time market [191]. The flexiramp constraint reduces the ramp shortage in the real-time market, deployment of regulation reserve, and errors of scheduled power exchange among regions [192, 193]. The flexiramp constraint is applied for the case of increasing ramp capability. The power system operator determines the required amount. Similar to the other ancillary services, meeting the flexiramp constraint might be coupled with a lost opportunity cost for generating units. So, units which have lost a part of their generation capacity due to flexiramp constraints for participating in the energy or other ancillary services are paid relevant to the lost opportunity marginal costs. Further, units do not offer any special bids to provide the required ramp.

After proving the efficiency of the flexiramp constraint, it has been proposed as a separate ancillary service. The differences between the flexiramp service and the flexiramp constraint are in employing the flexiramp in 5 minutes real-time markets instead of 15 minutes, and considering flexiramp in the day-ahead scheduling and utilizing the flexibility curves [85]. Since in CAISO, real-time markets are settled in multi periods, the desired ramp already exists in the market. This is equal to the expected net demand changes of the current and next periods. This amount is the minimum requirement. As illustrated in Fig. 5, the system operator has considered an extra ramp to compensate for the net demand changes and uncertainties, in addition to the expected ramp. Also, to avoid imposing any extra cost to serve the required flexiramp, a flexiramp curve has been viewed. The maximum price for periods in which enough flexiramp does not exist has been set to 250 $/MW. The flexiramp bid is only considered in the day-ahead market and prices are fixed based on the lost opportunity cost in the real-time market. Identifying the configuration for clearing the flexiramp in the real-time and day-ahead market is similar to the other ancillary services.

Fig. 5
figure5

An example of required ramp capability [25]

Similar to the CAISO, MISO has considered the upward and downward flexiramp as a distinct ancillary service in market design [86]. According to the MISO studies, and contrary to the CAISO, the necessity of flexiramp is not capacity limitations, but it is about the inadequacy of ramp capability. The major contrast of this ancillary service in the mentioned markets is the flexiramp cost allocation method. While CAISO and MISO have thought of flexiramp as a solution to promote flexibility in the short-term operation, other market operators have not noticed any need to offer this ancillary service. For instance, NYISO has estimated that day-ahead profit assurance, making whole payment guarantee, and optimal operation of units on their most efficient level of generation have provided sufficient incentive and drive to provide the system required flexibility ancillary service. However, ERCOT has provided a supplementary reserve service among its new ancillary services, but introduced it as a short-term and temporary solution. ERCOT has claimed that in the long term there would be no need for this type of service [186].

When market structure has evolved to improve system flexibility, because of leveraging the available resources as well as new flexibility resources such as demand side and energy storages, the system operational cost will be lowered and energy efficiency will be enhanced.

All the above-discussed solutions intend to evolve market design via providing incentives for the more flexible resources to participate in the market, and to utilize them at maximum level in power system planning and operation in the presence of VERs. Thus, it is expected that market design offers a framework such that not only resource and flexibility adequacy are satisfied, but also required flexibility resources are paid sufficiently to be motivated to participate in the market. So, long-term, mid-term, and short-term system flexibilities will be enhanced through market design evolution. Subsequently, generation mix will also change such that highly efficient technologies such as power plants with a detachable carbon capture and storage unit along with highly flexible gas-fired power plants replace traditional fossil power plants with low operational flexibility, in order to supply the remaining demand which depends on the fluctuating generation of VERs. Demand side management can improve the flexibility on the demand side, while storage can reduce the fluctuations of load.

It is not possible to give a global answer to a question which asks about the type of market design that can exploit all flexibility resources. The answer definitely depends on network structure, its requirements, and available flexibility resources. However, we can consider the following characteristics for an electricity market which can deploy the flexibility resources as much as possible:

  1. a)

    The existing traditional generation unit should be incentivized to support the VERs.

  2. b)

    Electricity markets should provide short-term and long-term incentives for non-conventional resources to respectively ameliorate their participation in flexibility provision, and encourage them for more investment.

  3. c)

    All the available resources should be deployed to reach the full extension of demand side management as a low-cost flexibility resource.

  4. d)

    Optimal transmission and distribution grids should be determined to support investment efficiency. This will reduce the need for flexibility.

  5. e)

    The market should determine the location where flexibility is needed.

  6. f)

    Expanding markets vertically and horizontally can improve flexibility.

Conclusion

Power systems are always planned and operated such that they present an appropriate level of flexibility to establish the power balance at any time. High penetration of renewable resources and their variability, intermittency, and uncertainty have enlarged the prominent role of flexibility in modern power systems. In this paper, first, power system flexibility is described from the four dimensions of time, control functions, uncertainty, and cost, and its necessity in the presence of VERs is discussed. Then, short-term, mid-term, and long-term effects of proliferated penetration of VERs on the power system flexibility which leads to its further evolution are reviewed. For a more secure and reliable power system operation, more flexible services are needed to deal with either forecast or unforecast deviations in net demand. Thus, a variety of new resources for flexibility are studied which will result in an evolution of power system flexibility. Each resource might promote physical or structural flexibilities up to a specific point. Indeed, given power system conditions and requirements for flexibility, each resource, or probably set of resources, may help the system to develop its operation. However, to reach the proper level of flexibility, a system structure must induce enough motivation for extension of flexible resources in the long term, and also convenient economic signals for the contribution of flexible resources are offered in short-term operation. Thus, as an influential solution for enhancing the flexibility, market design improvements for different time horizons are studied. In order to execute such a structure, common approaches are outlined to confront the challenges such as revenue inadequacy for long-term reliability, and motivation of flexibility in short-term operation. Moreover, the most current modifications of market designs for the increasing penetration of VERs are investigated.

While a wide variety of solutions and new market designs are explained in this review and each solution individually is examined as to how it provides the power system with flexibility in details, future research may be focused on concepts such as how a power system should be operated and planned in the presence of a set of described flexibility options, or how the introduced evolutionary market designs must incentivize the generation units to offer flexibility with respect to the discussed challenges in this review.

References

  1. [1]

    REN21 (2017) Renewables global futures report, great debates towards 100% renewable energy. http://www.ren21.net/wp-content/uploads/2017/10/GFR-Full-Report-2017_webversion_3.pdf. Accessed 18 March 2019

  2. [2]

    IEA (2013) Energy policies of IEA countries- Denmark 2011 review. https://www.iea.org/publications/freepublications/publication/Denmark2011_unsecured.pdf. Accessed 18 March 2019

  3. [3]

    Ohlhorst D (2015) Germany’s energy transition policy between national targets and decentralized responsibilities. J Integr Environ Sci 12(4):303–322

    Google Scholar 

  4. [4]

    Hirth L (2013) The market value of variable renewables. Energy Econ 38:218–236

    Google Scholar 

  5. [5]

    Edenhofer O, Hirth L, Knopf B et al (2013) On the economics of renewable energy sources. Energy Econ 40:12–23

    Google Scholar 

  6. [6]

    Ueckerdt F, Hirth L, Luderer G et al (2013) System LCOE: What are the costs of variable renewables? Energy 63:61–75

    Google Scholar 

  7. [7]

    Brijs T, Geth F, Siddiqui S et al (2016) Price-based unit commitment electricity storage arbitrage with piecewise linear price effects. J Energy Storage 7:52–62

    Google Scholar 

  8. [8]

    Castillo A, Gayme DF (2014) Grid-scale energy storage applications in renewable energy integration: a survey. Energy Convers Manag 87:885–894

    Google Scholar 

  9. [9]

    Cochran J, Denholm P, Speer B (2015) Grid integration and the carrying capacity of the US grid to incorporate variable renewable energy. https://www.nrel.gov/docs/fy15osti/62607.pdf. Accessed 18 March 2019

  10. [10]

    FERC (2016) Settlement intervals and shortage pricing in markets operated by regional transmission organizations and independent system operators. https://www.ferc.gov/whats-new/comm-meet/2016/061616/E-2.pdf. Accessed 18 March 2019

  11. [11]

    Balling L (2011) Fast cycling and rapid start-up: new generation of plants achieves impressive results. Mod Power Syst 31:35–41

    Google Scholar 

  12. [12]

    Cochran J. Debra L, Nikhil K (2013) Flexible coal: evolution from baseload to peaking plant. https://www.nrel.gov/docs/fy14osti/60575.pdf. Accessed 18 March 2019

  13. [13]

    Kubik ML, Coker J, Barlow F (2015) Increasing thermal plant flexibility in a high renewables power system. Appl Energy 154:102–111

    Google Scholar 

  14. [14]

    Patrick E, Antriksh S, Ndaona C et al (2016) Effect of increased renewables generation on operation of thermal power plants. Appl Energy 164:723–732

    Google Scholar 

  15. [15]

    Shariatzadeh F, Mandal P, Srivastava AK (2015) Demand response for sustainable energy systems: a review, application and implementation strategy. Renew Sustain Energy Rev 45:343–350

    Google Scholar 

  16. [16]

    Bayer B (2015) Current practice and thinking with integrating demand response for power system flexibility in the electricity markets in the USA and Germany. Curr Sustain Renew Energy Rep 2(2):55–62

    Google Scholar 

  17. [17]

    Lund PD, Lindgren J, Mikkola J et al (2015) Review of energy system flexibility measures to enable high levels of variable renewable electricity. Renew Sustain Energy Rev 45:785–807

    Google Scholar 

  18. [18]

    Salpakari J, Mikkola J, Lund PD (2016) Improved flexibility with large-scale variable renewable power in cities through optimal demand side management and power-to-heat conversion. Energy Convers Manag 126:649–661

    Google Scholar 

  19. [19]

    Gottwalt S, Gärttner J, Schmeck H et al (2017) Modeling and valuation of residential demand flexibility for renewable energy integration. IEEE Trans Smart Grid 8(6):2565–2574

    Google Scholar 

  20. [20]

    Samaan NA, Makarov YV, Nguyen TB et al (2017) Balancing authority cooperation concepts to reduce variable generation integration costs in the western interconnection: consolidating balancing authorities and sharing balancing reserves. In: Integration of large-scale renewable energy into bulk power systems. Springer, Cham, pp 189–226

    Google Scholar 

  21. [21]

    Rodriguez RA, Becker S, Andresen GB et al (2014) Transmission needs across a fully renewable European power system. Renew Energy 63:467–476

    Google Scholar 

  22. [22]

    Becker S, Frew BA, Andresen GB et al (2014) Features of a fully renewable US electricity system: optimized mixes of wind and solar PV and transmission grid extensions. Energy 72:443–458

    Google Scholar 

  23. [23]

    Frew BA, Becker S, Dvorak MJ (2016) Flexibility mechanisms and pathways to a highly renewable US electricity future. Energy 101:65–78

    Google Scholar 

  24. [24]

    Olson A, Jones RA, Hart E et al (2014) Renewable curtailment as a power system flexibility resource. Electr J 27(9):49–61

    Google Scholar 

  25. [25]

    Chen R, Wang J, Botterud A et al (2017) Wind power providing flexible ramp product. IEEE Trans Power Syst 32(3):2049–2061

    Google Scholar 

  26. [26]

    Banshwar A, Sharma NK, Sood YR et al (2017) Market based procurement of energy and ancillary services from renewable energy sources in deregulated environment. Renew Energy 101:1390–1400

    Google Scholar 

  27. [27]

    Weber C (2010) Adequate intraday market design to enable the integration of wind energy into the European power systems. Energy Policy 38(7):3155–3163

    Google Scholar 

  28. [28]

    Levin T, Botterud A (2015) Electricity market design for generator revenue sufficiency with increased variable generation. Energy Policy 87:392–406

    Google Scholar 

  29. [29]

    Riesz J, Milligan M (2015) Designing electricity markets for a high penetration of variable renewables. Wiley Interdiscip Rev Energy Environ 4(3):279–289

    Google Scholar 

  30. [30]

    Ela E, Milligan M, Bloom A et al (2016) Wholesale electricity market design with increasing levels of renewable generation: Incentivizing flexibility in system operations. Electr J 29(4):51–60

    Google Scholar 

  31. [31]

    Brijs T, De Jonghe C, Hobbs BF et al (2017) Interactions between the design of short-term electricity markets in the CWE region and power system flexibility. Appl Energy 195:36–51

    Google Scholar 

  32. [32]

    Singarao VY, Rao VS (2016) Frequency responsive services by wind generation resources in United States. Renew Sustain Energy Rev 55:1097–1108

    Google Scholar 

  33. [33]

    MacDowell J, Dutta S, Richwine M et al (2015) Serving the future: advanced wind generation technology supports ancillary services. IEEE Power Energy Mag 13(6):22–30

    Google Scholar 

  34. [34]

    Nock D, Krishnan V, McCalley JD (2014) Dispatching intermittent wind resources for ancillary services via wind control and its impact on power system economics. Renew Energy 71:396–400

    Google Scholar 

  35. [35]

    Banshwar A, Sharma NK, Sood YR et al (2017) Renewable energy sources as a new participant in ancillary service markets. Energy Strategy Rev 18:106–120

    Google Scholar 

  36. [36]

    Rugolo J, Aziz MJ (2012) Electricity storage for intermittent renewable sources. Energy Environ Sci 5:7151–7160

    Google Scholar 

  37. [37]

    Steffen B, Weber C (2013) Efficient storage capacity in power systems with thermal and renewable generation. Energy Econ 36:556–567

    Google Scholar 

  38. [38]

    Després J, Mima S, Kitous A et al (2017) Storage as a flexibility option in power systems with high shares of variable renewable energy sources: a POLES-based analysis. Energy Econ 64:638–650

    Google Scholar 

  39. [39]

    Amrouche SO, Rekioua D, Rekioua T et al (2016) Overview of energy storage in renewable energy systems. Int J Hydrog Energy 41(45):20914–20927

    Google Scholar 

  40. [40]

    Schachter JA, Mancarella P (2016) A critical review of real options thinking for valuing investment flexibility in smart grids and low carbon energy systems. Renew Sustain Energy Rev 56:261–271

    Google Scholar 

  41. [41]

    Hossain MS, Madlool NA, Rahim NA et al (2016) Role of smart grid in renewable energy: an overview. Renew Sustain Energy Rev 60:1168–1184

    Google Scholar 

  42. [42]

    Martinot E (2016) Grid integration of renewable energy: flexibility, innovation, and experience. Ann Rev Environ Resour 41:223–251

    Google Scholar 

  43. [43]

    Zakeri B (2016) Integration of variable renewable energy in national and international energy systems: modelling and assessment of flexibility requirements. Dissertation, Aalto University

  44. [44]

    Papaefthymiou G, Dragoon K (2016) Towards 100% renewable energy systems: uncapping power system flexibility. Energy Policy 92:69–82

    Google Scholar 

  45. [45]

    Zhao J, Zheng T, Litvinov E (2016) A unified framework for defining and measuring flexibility in power system. IEEE Trans Power Syst 31(1):339–347

    Google Scholar 

  46. [46]

    Nosair H, Bouffard F (2015) Flexibility envelopes for power system operational planning. IEEE Trans Sustain Energy 6(3):800–809

    Google Scholar 

  47. [47]

    Menemenlis N, Huneault M, Robitaille A (2011) Thoughts on power system flexibility quantification for the short-term horizon. In: Proceedings of IEEE PES general meeting, San Diego, USA, 24–29 July 2011, p 8

  48. [48]

    Thatte AA, Xie L (2016) A metric and market construct of inter-temporal flexibility in time-coupled economic dispatch. IEEE Trans Power Syst 31(5):3437–3446

    Google Scholar 

  49. [49]

    Lannoye E, Flynn D, O’Malley M (2012) Evaluation of power system flexibility. IEEE Trans Power Syst 27(2):922–931

    Google Scholar 

  50. [50]

    Bucher MA, Chatzivasileiadis S, Andersson G (2016) Managing flexibility in multi-area power systems. IEEE Trans Power Syst 31(2):1218–1226

    Google Scholar 

  51. [51]

    Yasuda Y, Ardal AR, Carlini EM et al (2013) Flexibility chart: evaluation on diversity of flexibility in various areas. In: Proceedings of 12th wind integration workshop, London, UK, 22–24 October 2013, p 6

  52. [52]

    Hargreaves J, Hart EK, Jones R et al (2015) REFLEX: an adapted production simulation methodology for flexible capacity planning. IEEE Trans Power Syst 30(3):1306–1315

    Google Scholar 

  53. [53]

    International Energy Agency (2011) Harnessing variable renewables: a guide to the balancing challenge. https://www.iea.org/publications/freepublications/publication/Harnessing_Variable_Renewables2011.pdf. Accessed 18 March 2019

  54. [54]

    Wang Q, Wang J, Guan Y (2013) Stochastic unit commitment with uncertain demand response. IEEE Trans Power Syst 28(1):562–563

    Google Scholar 

  55. [55]

    Papavasiliou A, Oren SS, Rountree B (2015) Applying high performance computing to transmission-constrained stochastic unit commitment for renewable energy integration. IEEE Trans Power Syst 30(3):1109–1120

    Google Scholar 

  56. [56]

    Zheng QP, Wang J, Liu AL (2015) Stochastic optimization for unit commitment—a review. IEEE Trans Power Syst 30(4):1913–1924

    Google Scholar 

  57. [57]

    Zhao C, Guan Y (2016) Data-driven stochastic unit commitment for integrating wind generation. IEEE Trans Power Syst 31(4):2587–2596

    Google Scholar 

  58. [58]

    Uçkun C, Botterud A, Birge JR (2016) An improved stochastic unit commitment formulation to accommodate wind uncertainty. IEEE Trans Power Syst 31(4):2507–2517

    Google Scholar 

  59. [59]

    Khanabadi M, Fu Y, Gong L (2018) A fully parallel stochastic multiarea power system operation considering large-scale wind power integration. IEEE Trans Sustain Energy 9(1):138–147

    Google Scholar 

  60. [60]

    Bertsimas D, Litvinov E, Sun XA (2013) Adaptive robust optimization for the security constrained unit commitment problem. IEEE Trans Power Syst 28(1):52–63

    Google Scholar 

  61. [61]

    Jiang R, Wang J, Guan Y (2012) Robust unit commitment with wind power and pumped storage hydro. IEEE Trans Power Syst 27(2):800–810

    Google Scholar 

  62. [62]

    Ye H, Li Z (2016) Robust security-constrained unit commitment and dispatch with recourse cost requirement. IEEE Trans Power Syst 31(5):3527–3536

    Google Scholar 

  63. [63]

    Guan Y, Wang J (2014) Uncertainty sets for robust unit commitment. IEEE Trans Power Syst 29(3):1439–1440

    MathSciNet  Google Scholar 

  64. [64]

    Wei W, Liu F, Mei S et al (2015) Robust energy and reserve dispatch under variable renewable generation. IEEE Trans Smart Grid 6(1):369–380

    Google Scholar 

  65. [65]

    Hu B, Wu L (2016) Robust SCUC considering continuous/discrete uncertainties and quick-start units: a two-stage robust optimization with mixed-integer recourse. IEEE Trans Power Syst 31(2):1407–1419

    MathSciNet  Google Scholar 

  66. [66]

    Shao C, Wang X, Shahidehpour M et al (2017) Security-constrained unit commitment with flexible uncertainty set for variable wind power. IEEE Trans Sustain Energy 8(3):1237–1246

    Google Scholar 

  67. [67]

    Ye H, Wang J, Li Z (2017) MIP reformulation for max-min problems in two-stage robust SCUC. IEEE Trans Power Syst 32(2):1237–1247

    Google Scholar 

  68. [68]

    Ye H, Wang J, Ge Y et al (2017) Robust integration of high-level dispatchable renewables in power system operation. IEEE Trans Sustain Energy 8(2):826–835

    Google Scholar 

  69. [69]

    Wang Q, Guan Y, Wang JA (2012) Chance-constrained two-stage stochastic program for unit commitment with uncertain wind power output. IEEE Trans Power Syst 27(1):206–215

    MathSciNet  Google Scholar 

  70. [70]

    Pozo D, Contreras J (2013) A chance-constrained unit commitment with an n − K security criterion and significant wind generation. IEEE Trans Power Syst 28(3):2842–2851

    Google Scholar 

  71. [71]

    Wu Z, Zeng P, Zhang XP et al (2016) A solution to the chance-constrained two-stage stochastic program for unit commitment with wind energy integration. IEEE Trans Power Syst 31(6):4185–4196

    Google Scholar 

  72. [72]

    Wu H, Shahidehpour M, Li Z et al (2014) Chance-constrained day-ahead scheduling in stochastic power system operation. IEEE Trans Power Syst 29(4):1583–1591

    Google Scholar 

  73. [73]

    Wang Y, Zhao S, Zhou Z et al (2017) Risk adjustable day-ahead unit commitment with wind power based on chance constrained goal programming. IEEE Trans Sustain Energy 8(2):530–541

    Google Scholar 

  74. [74]

    Zhang Y, Wang J, Zeng B et al (2017) Chance-constrained two-stage unit commitment under uncertain load and wind power output using bilinear benders decomposition. IEEE Trans Power Syst 32(5):3637–3647

    Google Scholar 

  75. [75]

    Chaiyabut N, Damrongkulkumjorn P (2014) Optimal spinning reserve for wind power uncertainty by unit commitment with EENS constraint. In: Proceedings of IEEE power and innovative smart grid technologies conference, Washington, USA, 19–22 February 2014, p 5

  76. [76]

    Ahmadi-Khatir A, Conejo AJ, Cherkaoui R (2014) Multi-area unit scheduling and reserve allocation under wind power uncertainty. IEEE Trans Power Syst 29(4):1701–1710

    Google Scholar 

  77. [77]

    Li W, Tesfatsion L (2016) Market provision of flexible energy/reserve contracts: optimization formulation. In: Proceedings of IEEE PES general meeting, Boston, USA, 17–21 July 2016, p 5

  78. [78]

    Khan S, Gawlik W, Palensky P (2016) Reserve capability assessment considering correlated uncertainty in microgrid. IEEE Trans Sustain Energy 7(2):637–646

    Google Scholar 

  79. [79]

    Xie L, Gu Y, Zhu X et al (2014) Short-term spatio-temporal wind power forecast in robust look-ahead power system dispatch. IEEE Trans Smart Grid 5(1):511–520

    Google Scholar 

  80. [80]

    Gu Y, Xie L (2013) Early detection and optimal corrective measures of power system insecurity in enhanced look-ahead dispatch. IEEE Trans Power Syst 28(2):1297–1307

    Google Scholar 

  81. [81]

    Li Z, Wu W, Zhang B et al (2015) Efficient location of unsatisfiable transmission constraints in look-ahead dispatch via an enhanced Lagrangian relaxation framework. IEEE Trans Power Syst 30(3):1233–1242

    Google Scholar 

  82. [82]

    Beuchat PN, Warrington J, Summers TH et al (2016) Performance bounds for look-ahead power system dispatch using generalized multistage policies. IEEE Trans Power Syst 31(1):474–484

    Google Scholar 

  83. [83]

    Tang C, Xu J, Sun Y et al (2018) Look-ahead economic dispatch with adjustable confidence interval based on a truncated versatile distribution model for wind power. IEEE Trans Power Syst 33(2):1755–1767

    Google Scholar 

  84. [84]

    Gu Y, Xie L (2017) Stochastic look-ahead economic dispatch with variable generation resources. IEEE Trans Power Syst 32(1):17–29

    Google Scholar 

  85. [85]

    Navid N, Rosenwald G (2013) Ramp capability product design for MISO markets. https://cdn.misoenergy.org/Ramp%20Product%20Conceptual%20Design%20Whitepaper271170.pdf. Accessed 18 March 2019

  86. [86]

    CAISO (2015) Flexible ramping product: revised draft final proposal. https://www.caiso.com/Documents/RevisedDraftFinalProposal-FlexibleRampingProduct-2015.pdf. Accessed 18 March 2019

  87. [87]

    Wang B, Hobbs BF (2016) Real-time markets for flexiramp: a stochastic unit commitment-based analysis. IEEE Trans Power Syst 31(2):846–860

    Google Scholar 

  88. [88]

    Wang B, Hobbs BF (2014) A flexible ramping product: Can it help real-time dispatch markets approach the stochastic dispatch ideal? Electr Power Syst Res 109:128–140

    Google Scholar 

  89. [89]

    Marneris IG, Biskas PN, Bakirtzis EA (2016) An integrated scheduling approach to underpin flexibility in European power systems. IEEE Trans Sustain Energy 7(2):647–657

    Google Scholar 

  90. [90]

    Navid N, Rosenwald G (2012) Market solutions for managing ramp flexibility with high penetration of renewable resource. IEEE Trans Sustain Energy 3(4):784–790

    Google Scholar 

  91. [91]

    Wu C, Hug G, Kar S (2016) Risk-limiting economic dispatch for electricity markets with flexible ramping products. IEEE Trans Power Syst 31(3):1990–2003

    Google Scholar 

  92. [92]

    Wang C, Luh PB, Navid N (2017) Ramp requirement design for reliable and efficient integration of renewable energy. IEEE Trans Power Syst 32(1):562–571

    Google Scholar 

  93. [93]

    Cornelius A, Bandyopadhyay R, Patiño-Echeverri D (2018) Assessing environmental, economic, and reliability impacts of flexible ramp products in MISO’s electricity market. Renew Sustain Energy Rev 81:2291–2298

    Google Scholar 

  94. [94]

    Zhang B, Kezunovic M (2016) Impact on power system flexibility by electric vehicle participation in ramp market. IEEE Trans Smart Grid 7(3):1285–1294

    Google Scholar 

  95. [95]

    Cui M, Zhang J, Wu H et al (2016) Wind power ramping product for increasing power system flexibility. In: Proceedings of IEEE PES transmission and distribution conference and exposition, Dallas, USA, 3–5 May 2016, p 5

  96. [96]

    Andrianesis P, Liberopoulos G, Biskas P et al (2011) Medium-Term Unit Commitment in a pool market. In: Proceedings of 8th international conference on the European energy market, Zagreb, Croatia, 25–27 May 2011, p 6

  97. [97]

    Koltsaklis NE, Dagoumas AS, Kopanos GM (2014) A spatial multi-period long-term energy planning model: a case study of the Greek power system. Appl Energy 115:456–482

    Google Scholar 

  98. [98]

    Koltsaklis NE, Dagoumas AS, Georgiadis MC et al (2016) A mid-term, market-based power systems planning model. Appl Energy 179:17–35

    Google Scholar 

  99. [99]

    Koltsaklis NE, Georgiadis MC (2015) A multi-period, multi-regional generation expansion planning model incorporating unit commitment constraints. Appl Energy 158:310–331

    Google Scholar 

  100. [100]

    Koltsaklis NE, Dagoumas AS, Panapakidis IP (2017) Impact of the penetration of renewables on flexibility needs. Energy Policy 109:360–369

    Google Scholar 

  101. [101]

    Welsch M, Deane P, Howells M et al (2014) Incorporating flexibility requirements into long-term energy system models—a case study on high levels of renewable electricity penetration in Ireland. Appl Energy 135:600–615

    Google Scholar 

  102. [102]

    Henriot A, Glachant JM (2013) Melting-pots and salad bowls: The current debate on electricity market design for integration of intermittent RES. Util Policy 27:57–64

    Google Scholar 

  103. [103]

    Hiroux C, Saguan M (2010) Large-scale wind power in European electricity markets: time for revisiting support schemes and market designs? Energy Policy 38(7):3135–3145

    Google Scholar 

  104. [104]

    Vandezande L, Meeus L, Belmans R et al (2010) Well-functioning balancing markets: a prerequisite for wind power integration. Energy Policy 38(7):3146–3154

    Google Scholar 

  105. [105]

    Doorman GL, Van Der Veen R (2013) An analysis of design options for markets for cross-border balancing of electricity. Util Policy 27:39–48

    Google Scholar 

  106. [106]

    Eid C, Codani P, Perez Y et al (2016) Managing electric flexibility from distributed energy resources: a review of incentives for market design. Renew Sustain Energy Rev 64:237–247

    Google Scholar 

  107. [107]

    Hu J, Harmsen R, Crijns-Graus W et al (2018) Identifying barriers to large-scale integration of variable renewable electricity into the electricity market: a literature review of market design. Renew Sustain Energy Rev 81:2181–2195

    Google Scholar 

  108. [108]

    Milligan M, Frew BA, Bloom A et al (2016) Wholesale electricity market design with increasing levels of renewable generation: revenue sufficiency and long-term reliability. Electr J 29(2):26–38

    Google Scholar 

  109. [109]

    Bouffard F, Ortega-Vazquez M (2011) The value of operational flexibility in power systems with significant wind power generation. In: Proceedings of IEEE PES general meeting, San Diego, USA, 24–29 July 2011, p 5

  110. [110]

    Dvorkin Y, Kirschen DS, Ortega-Vazquez MA (2014) Assessing flexibility requirements in power systems. IET Gener Transm Distrib 8(11):1820–1830

    Google Scholar 

  111. [111]

    Ma J, Silva V, Belhomme R et al (2013) Evaluating and planning flexibility in sustainable power systems. IEEE Trans Sustain Energy 4(1):200–209

    Google Scholar 

  112. [112]

    Zheng T, Zhao J, Zhao F et al (2012) Operational flexibility and system dispatch. In: Proceedings of IEEE PES general meeting, San Diego, USA, 22–26 July 2012, p 3

  113. [113]

    Tabone MD, Goebel C, Callaway DS (2016) The effect of PV siting on power system flexibility needs. Sol Energy 139:776–786

    Google Scholar 

  114. [114]

    Holttinen H, Tuohy A, Milligan M et al (2013) The flexibility workout: managing variable resources and assessing the need for power system modification. IEEE Power Energy Mag 11(6):53–62

    Google Scholar 

  115. [115]

    Deetjen TA, Rhodes JD, Webber ME (2017) The impacts of wind and solar on grid flexibility requirements in the Electric Reliability Council of Texas. Energy 123:637–654

    Google Scholar 

  116. [116]

    Huber M, Dimkova D, Hamacher T (2014) Integration of wind and solar power in Europe: assessment of flexibility requirements. Energy 69:236–246

    Google Scholar 

  117. [117]

    Ackermann T, Kuwahata R (2013) A guide to operational impact analysis of variable renewables: application to the Philippines. The World Bank Group, Washington

    Google Scholar 

  118. [118]

    Australian Energy Market Operator (2013) 100 per cent renewables study: modelling outcomes. AEMO, Melbourne

    Google Scholar 

  119. [119]

    Doostizadeh M, Aminifar F, Ghasemi H et al (2016) Energy and reserve scheduling under wind power uncertainty: an adjustable interval approach. IEEE Trans Smart Grid 7(6):2943–2952

    Google Scholar 

  120. [120]

    Ela E, Gevorgian V, Tuohy A et al (2014) Market designs for the primary frequency response ancillary service—part I: motivation and design. IEEE Trans Power Syst 29(1):421–431

    Google Scholar 

  121. [121]

    Troy N, Flynn D, Milligan M et al (2012) Unit commitment with dynamic cycling costs. IEEE Trans Power Syst 27(4):2196–2205

    Google Scholar 

  122. [122]

    Van den Bergh K, Legon T, Delarue E et al (2016) Long-term cycling costs in short-term unit commitment models. In: Proceedings of 13th international conference on the European energy market, Porto, Portugal, 6–9 June 2016, p 5

  123. [123]

    Van den Bergh K, Legon T, Delarue E et al (2016) Long-term cycling costs in short-term unit commitment models. Appl Energy 156:107–128

    Google Scholar 

  124. [124]

    Cochran J, Miller M, Zinaman O (2014) Flexibility in 21st century power systems. https://www.nrel.gov/docs/fy14osti/61721.pdf. Accessed 18 March 2019

  125. [125]

    Holttinen H (2013) Expert group report on recommended practices: wind integration studies. International Energy Agency Wind Task 25, IEA, Paris

  126. [126]

    Torriti J (2014) Privatisation and cross-border electricity trade: From internal market to European Supergrid? Energy 1(77):635–640

    Google Scholar 

  127. [127]

    Doostizadeh M, Aminifar F, Lesani H et al (2016) Multi-area market clearing in wind-integrated interconnected power systems: a fast parallel decentralized method. Energy Convers Manag 113(2016):131–142

    Google Scholar 

  128. [128]

    Li J, Liu F, Li Z (2018) Grid-side flexibility of power systems in integrating large-scale renewable generations: a critical review on concepts, formulations and solution approaches. Renew Sustain Energy Rev 93:272–284

    Google Scholar 

  129. [129]

    Henao A, Sauma E, Gonzalez A (2018) Impact of introducing flexibility in the Colombian transmission expansion planning. Energy 157:131–140

    Google Scholar 

  130. [130]

    Li J, Li Z, Liu F (2018) Robust coordinated transmission and generation expansion planning considering ramping requirements and construction periods. IEEE Trans Power Syst 33(1):268–280

    Google Scholar 

  131. [131]

    Khanabadi M, Ghasemi H, Doostizadeh M (2013) Optimal transmission switching considering voltage security and N-1 contingency analysis. IEEE Trans Power Syst 28(1):542–550

    Google Scholar 

  132. [132]

    Nikoobakht A, Aghaei J, Mardaneh M (2017) Securing highly penetrated wind energy systems using linearized transmission switching mechanism. Appl Energy 190:1207–1220

    Google Scholar 

  133. [133]

    Sang Y, Sahraei-Ardakani M, Parvania M (2018) Stochastic transmission impedance control for enhanced wind energy integration. IEEE IEEE Trans Sustain Energy 9(3):1108–1117

    Google Scholar 

  134. [134]

    Migliavacca G (2012) Advanced technologies for future transmission grids. Springer, Berlin

    Google Scholar 

  135. [135]

    Henriot A (2015) Economic curtailment of intermittent renewable energy sources. Energy Econ 49:370–379

    Google Scholar 

  136. [136]

    Brandstätt C, Brunekreeft G, Jahnke K (2011) How to deal with negative power price spikes? Flexible voluntary curtailment agreements for large-scale integration of wind. Energy Policy 39(6):3732–3740

    Google Scholar 

  137. [137]

    Brijs T, De Vos K, De Jonghe C et al (2015) Statistical analysis of negative prices in European balancing markets. Renew Energy 80:53–60

    Google Scholar 

  138. [138]

    Bird L, Lew D, Milligan M et al (2016) Wind and solar energy curtailment: a review of international experience. Renew Sustain Energy Rev 65:577–586

    Google Scholar 

  139. [139]

    Ahmadyar AS, Riaz S, Verbič G et al (2018) A framework for assessing renewable integration limits with respect to frequency performance. IEEE Trans Power Syst 33(4):4444–4453

    Google Scholar 

  140. [140]

    Muljadi E, Gevorgian V, Hoke A (2016) Short-term forecasting of inertial response from a wind power plant. In: Proceedings of IEEE energy conversion congress and exposition, Milwaukee, USA, 18–22 September 2016, p 5

  141. [141]

    Xin H, Liu Y, Wang Z et al (2013) A new frequency regulation strategy for photovoltaic systems without energy storage. IEEE Trans Sustain Energy 4(4):985–993

    Google Scholar 

  142. [142]

    Kawabe K, Ota Y, Yokoyama A et al (2017) Novel dynamic voltage support capability of photovoltaic systems for improvement of short-term voltage stability in power systems. IEEE Trans Power Syst 32(3):1796–1804

    Google Scholar 

  143. [143]

    Doostizadeh M, Khanabadi M, Ettehadi M (2018) Reactive power provision from distributed energy resources in market environment. In: Proceedings of Iranian conference on electrical engineering, Mashhad, Iran, 8–10 May 2018, p 6

  144. [144]

    De Vos K, Driesen J (2015) Active participation of wind power in operating reserves. IET Renew Power Gener 9:566–575

    Google Scholar 

  145. [145]

    Lorenz C, Gerbaulet C (2017) Wind providing balancing reserves: an application to the german electricity system of 2025. DIW Berlin, Berlin

    Google Scholar 

  146. [146]

    Xu B, Dvorkin Y, Kirschen DS et al (2016) A comparison of policies on the participation of storage in us frequency regulation markets. In: Proceedings of IEEE PES general meeting, Boston, USA, 17–21 July 2016, p 5

  147. [147]

    Li W, Du P, Lu N (2018) Design of a new primary frequency control market for hosting frequency response reserve offers from both generators and loads. IEEE Trans Smart Grid 9(5):4883–4892

    Google Scholar 

  148. [148]

    Rothleder M, Loutan C (2017) Case study–renewable integration: flexibility requirement, potential over generation, and frequency response challenges. Academic Press, London

    Google Scholar 

  149. [149]

    Miller N, Clark K, Shao M (2011) Frequency responsive wind plant controls: impacts on grid performance. In: Proceedings of IEEE PES general meeting, San Diego, USA, 24–29 July 2011, p 8

  150. [150]

    Kroposki B, Johnson B, Zhang Y (2017) Achieving a 100% renewable grid: Operating electric power systems with extremely high levels of variable renewable energy. IEEE Power Energy Mag 15(2):61–73

    Google Scholar 

  151. [151]

    Dreidy M, Mokhlis H, Mekhilef S (2017) Inertia response and frequency control techniques for renewable energy sources: a review. Renew Sustain Energy Rev 69:144–155

    Google Scholar 

  152. [152]

    Schmidt O, Hawkes A, Gambhir A et al (2017) The future cost of electrical energy storage based on experience rates. Nat Energy 2(8):17110

    Google Scholar 

  153. [153]

    Ipakchi A, Albuyeh F (2009) Grid of the future. IEEE Power Energy Mag 7(2):52–62

    Google Scholar 

  154. [154]

    Speer B, Miller M, Schaffer W (2015) Role of smart grids in integrating renewable energy. https://www.nrel.gov/docs/fy15osti/63919.pdf. Accessed 18 March 2019

  155. [155]

    D’hulst R, Labeeuw W, Beusen B (2015) Demand response flexibility and flexibility potential of residential smart appliances: experiences from large pilot test in Belgium. Appl Energy 155:79–90

    Google Scholar 

  156. [156]

    Schuller A, Flath CM, Gottwalt S (2015) Quantifying load flexibility of electric vehicles for renewable energy integration. Appl Energy 151:335–344

    Google Scholar 

  157. [157]

    Li R, Wei W, Mei S et al (2018) Participation of an energy hub in electricity and heat distribution markets: an mpec approach. IEEE Trans Smart Grid. https://doi.org/10.1109/TSG.2018.2833279

    Article  Google Scholar 

  158. [158]

    Alipour M, Zare K, Abapour M (2018) MINLP probabilistic scheduling model for demand response programs integrated energy hubs. IEEE Trans Ind Inform 14(1):79–88

    Google Scholar 

  159. [159]

    Zhang X, Che L, Shahidehpour M (2017) Reliability-based optimal planning of electricity and natural gas interconnections for multiple energy hubs. IEEE Trans Smart Grid 8(4):1658–1667

    Google Scholar 

  160. [160]

    Moeini-Aghtaie M, Farzin H, Fotuhi-Firuzabad M et al (2017) Generalized analytical approach to assess reliability of renewable-based energy hubs. IEEE Trans Power Syst 32(1):368–377

    Google Scholar 

  161. [161]

    Baldick R, Helman U, Hobbs BF et al (2005) Design of efficient generation markets. Proc IEEE 93(11):1998–2012

    Google Scholar 

  162. [162]

    Conejo AJ, Carrión M, Morales JM (2010) Decision making under uncertainty in electricity markets. Springer, New York

    Google Scholar 

  163. [163]

    Kirby B (2013) The value of flexible generation. http://www.consultkirby.com/files/PowerGen-2013_The_Value_of_Flexible_Generation_Nov_2013.pdf. Accessed 18 March 2019

  164. [164]

    Wang Q, Wu H, Tan J (2016) Analyzing the impacts of increased wind power on generation revenue sufficiency. In: Proceedings of IEEE PES general meeting, Boston, USA, 17–21 July 2016, p 5

  165. [165]

    MISO (2017) Day-ahead margin assurance payment tariff discrepancy. https://cdn.misoenergy.org/20170309%20MSC%20Item%2003%20DA%20Margin%20Assurance%20Payment%20Tariff74921.pdf. Accessed 18 March 2019

  166. [166]

    NYISO (2012) Determination of day-ahead margin assurance payments: NYISO. https://nyisoviewer.etariff.biz/ViewerDocLibrary/Filing/Filing237/237FilingSections/MST%2025%20FID237%20Fqncy%20Rgltn%20-%20redline_6180.pdf. Accessed 18 March 2019

  167. [167]

    TenneT (2017) Market review 2017; electricity market insights. https://energeia-binary-external-prod.imgix.net/jxfrSKuCu0u0P8iAnp20sr80M9o.pdf?dl=Tennet%20Market%20Review%202017.pdf. Accessed January 2018

  168. [168]

    Carroll R (2014) PJM/NYISO coordinated transaction scheduling. https://www.pjm.com/~/media/committees-groups/committees/mic/20140707-energy/20140707-item-09-cts-education.ashx. Accessed 18 March 2019

  169. [169]

    Carroll R (2015) MISO/PJM coordinated transaction scheduling. https://www.pjm.com/-/media/committees-groups/committees/mic/20150108/20150108-item-06a-pjm-miso-coordinated-transaction-scheduling.ashx. Accessed 18 March 2019

  170. [170]

    White M, Pike R (2011) ISO-NE and NYISO Inter-regional interchange scheduling. In: Joint stakeholder meeting. https://www.iso-ne.com/static-assets/documents/committees/comm_wkgrps/mrkts_comm/mrkts/mtrls/2011/mar72011/a1_a8_iso_ne_nyiso_presentation_03_07_11.pdf. Accessed 18 March 2019

  171. [171]

    CONSENTEC (2014) Description of load-frequency control concept and market for control reserves. Study commissioned by the German TSOs. https://www.consentec.de/wp-content/uploads/2014/08/Consentec_50Hertz_Regelleistungsmarkt_en__20140227.pdf. Accessed 18 March 2019

  172. [172]

    Hogan WW (2015) Electricity market design energy and capacity markets, and resource adequacy. In: Proceedings of EUCI conference on capacity markets: gauging their real impact on resource development and reliability, Cambridge, USA, 1 September 2015, p 30

  173. [173]

    Frew BA, Milligan M, Brinkman G et al (2016) Revenue sufficiency and reliability in a zero marginal cost future. https://www.nrel.gov/docs/fy17osti/66935.pdf. Accessed 18 March 2019

  174. [174]

    Jenkin T, Beiter P, Margolis R (2016) Capacity payments in restructured markets under low and high penetration levels of renewable energy. https://www.nrel.gov/docs/fy16osti/65491.pdf. Accessed 18 March 2019

  175. [175]

    ACER (2013) Capacity remuneration mechanisms and the internal market for electricity. http://www.acer.europa.eu/official_documents/acts_of_the_agency/publication/crms%20and%20the%20iem%20report%20130730.pdf. Accessed 18 March 2019

  176. [176]

    Surendran R, Hogan WW, Hui H et al (2016) Scarcity pricing in ERCOT. https://www.ferc.gov/CalendarFiles/20160629114652-3%20-%20FERC2016_Scarcity%20Pricing_ERCOT_Resmi%20Surendran.pdf. Accessed 18 March 2019

  177. [177]

    California ISO (2017) Flexible resource adequacy criteria and must-offer obligation, phase 2 draft flexible capacity framework proposal. https://www.caiso.com/Documents/DraftFlexibleCapacityFrameworkProposalFlexibleResourceAdequacyCriteria_MustOfferObligationPhase2.pdf. Accessed 18 March 2019

  178. [178]

    Hawker G, Bell K, Gill S (2017) Electricity security in the European Union—the conflict between national capacity mechanisms and the single market. Energy Res Soc Sci 24:51–58

    Google Scholar 

  179. [179]

    Ziegler H, Mennel T, Hülsen C (2017) Demand response activation by independent aggregators as proposed in the draft electricity directive. https://euagenda.eu/upload/publications/untitled-97330-ea.pdf. Accessed 20 January 2018

  180. [180]

    Stoll B, Jain R, Martinez-Anido CB et al (2017) Reserve estimation in renewable integration studies. In: Du P, Baldick R, Tuohy A (eds) Integration of large-scale renewable energy into bulk power systems, power electronics and power systems. Springer, Cham

    Google Scholar 

  181. [181]

    Chen Y, Leonard R, Keyser M et al (2015) Development of performance-based two-part regulating reserve compensation on miso energy and ancillary service market. IEEE Trans Power Syst 30(1):142–155

    Google Scholar 

  182. [182]

    Wang Z, Zhong J, Li J (2017) Design of performance-based frequency regulation market and its implementations in real-time operation. Int J Electr Power Energy Syst 87:187–197

    Google Scholar 

  183. [183]

    Zhou Z, Levin T, Conzelmann G (2016) Survey of US ancillary services markets. http://www.ipd.anl.gov/anlpubs/2016/01/124217.pdf. Accessed 18 March 2019

  184. [184]

    Ela E, Tuohy A, Milligan M et al (2012) Alternative approaches for incentivizing the frequency responsive reserve ancillary service. Electr J 25(4):88–102

    Google Scholar 

  185. [185]

    Ahlstrom M, Ela E, Riesz J et al (2015) The evolution of the market: Designing a market for high levels of variable generation. IEEE Power Energy Mag 13(6):60–66

    Google Scholar 

  186. [186]

    ERCOT(2013) Future ancillary services in ERCOT. https://www.ferc.gov/CalendarFiles/20140421084800-ERCOT-ConceptPaper.pdf. Accessed 18 March 2019

  187. [187]

    Ofgem (2014) Bidding zones literature review, FTA Team. https://www.ofgem.gov.uk/sites/default/files/docs/2014/10/fta_bidding_zone_configuration_literature_review_1.pdf. Accessed 18 March 2019

  188. [188]

    Stoft S (2002) Power system economics: designing markets for electricity. IEEE Press, New York

    Google Scholar 

  189. [189]

    MISO (2010) Convex hull pricing extended LMP (“ELMP”). https://cdn.misoenergy.org/20161004%20MSC%20Item%2006e%20Convex%20Hull%20Pricing%20Workshop%202010050374769.pdf. Accessed 18 March 2019

  190. [190]

    Gribik PR, Chatterjee D, Navid N et al (2011) Dealing with uncertainty in dispatching and pricing in power markets. In: Proceedings of IEEE PES general meeting, San Diego, USA, 24–29 July 2011, p 6

  191. [191]

    Abdul-Rahman KH, Alarian H, Rothleder M et al (2012) Enhanced system reliability using flexible ramp constraint in CAISO market. In: Proceedings of IEEE PES general meeting, San Diego, USA, 22–26 July 2012, p 6

  192. [192]

    CAISO (2012) Annual report on market issues and performance. http://www.caiso.com/Documents/2012AnnualReport-MarketIssue-Performance.pdf. Accessed 18 March 2019

  193. [193]

    CAISO (2013) Report on market issues and performance. http://www.caiso.com/Documents/2013AnnualReport-MarketIssue-Performance.pdf. Accessed 18 March 2019

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Farrokh AMINIFAR.

Additional information

CrossCheck date: 13 February 2019

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

AKRAMI, A., DOOSTIZADEH, M. & AMINIFAR, F. Power system flexibility: an overview of emergence to evolution. J. Mod. Power Syst. Clean Energy 7, 987–1007 (2019). https://doi.org/10.1007/s40565-019-0527-4

Download citation

Keywords

  • Power system flexibility
  • Renewable energy resources
  • Ancillary services
  • Electricity market design