1 Introduction

Blessed with diverse and plentiful renewable and non-renewable energy resources, Australia has the distinction of being world’s ninth-largest energy producer country. By country, Australia currently ranks 11th in the world for wind generation per capita ahead of countries like China and France. At jurisdictional level, Australia’s wind generation is heavily skewed towards states like South Australia (SA) and Tasmania, which have some of the highest per capita wind generation in the world alongside leading U.S. states like Iowa and Texas [1]. With 39 percent of its total generation supplied by wind and solar plants, SA of National Electricity Market (NEM) region has second per capita capacity of renewable wind and solar energy in world after Iowa. The SA renewables experiment is more significant given most other high renewables penetration regions and countries – Iowa, Denmark and Germany – are much more integrated into larger grids with complementary (dispatchable) generation technologies. The SA grid is partially constrained, connected to Victoria by two transmission lines which allows it to source a maximum of around 20 percent of peak load from Victoria. By contrast, Denmark has interconnections that allow it to source its entire peak load from other countries.

With a new energy policy, current Australian electricity network is at biggest transition stage with insufficient and lagging operational policy settings. NEM regions have historically attracted frequency regulation services from synchronous generation but their displacement in presence of high wind energy penetrated system tends to create system wide shortfall in frequency regulation services and security. Design and operation of region like islanding prone SA and Tasmania in presence of wind energy is one of the major grid integration issues in NEM. The challenge of maintaining security and cost in a wind penetrated system was more prominently highlighted during the recent longest SA blackout on September 28, 2016. After the initial blame on intermittent wind farms for this outage, it was later identified that weather and fault ride through settings aggravated the conditions to outage. This outage highlighted the failure of a NEM region to completely integrate renewable non-synchronous technology and complete utilization of frequency responsive ancillary services. Due to high wind energy target for future NEM and lesson learnt from recent outage, NEM operator now acknowledge the need for urgent reforms in its structure and operating policies and strives to take more cautious approach and working towards improved mitigation measures.

2 Australian energy market structure

Under the banner of NEM, Australia owns the largest geographical interconnected electricity network in the world. Transmission lines and associated infrastructure extends approximately 51000 km from Port Douglas in Queensland to Port Lincoln in SA and across the Bass Strait to Tasmania [2]. NEM jurisdiction comprises five participating states acting as price regions – Queensland, New South Wales (including the Australian Capital Territory), Victoria, SA and Tasmania. There are approximately 270 registered generators in NEM and 16 major distribution networks for mutually supplying electricity to consumers. Regional reference nodes in NEM are interconnected through transmission flow-paths called as interconnectors and consist of transmission infrastructure traced on each side of a regional boundary, connected by a set of high-voltage transmission lines or cables [3]. NEM commenced operation as a wholesale electricity spot market in December 1998 [2], after successful implementation of linear programming optimization solver based National Electricity Market Dispatch Engine (NEMDE) in 1995.

Australian Energy Market Operator (AEMO) as transmission system and retail electricity market operator is responsible for NEM’s reliable and secure operation. Operating on cost reclamation basis, AEMO exclusively recovers its operational expenses through market participants and network service providers compensated fees. As presented in Fig. 1, AEMO operates in conjunction with the Australian Energy Regulator (AER), which is responsible for economic regulation and national rules compliance in the NEM, the Australian Energy Market Commission (AEMC), which is rules maker for energy market regulation and the Council of Australian Governments’ (COAG) Standing Council on Energy and Resources (SCER) which is policies developer for electricity markets. The NEM operates within the framework of national electricity rules (NERs) under joint legislation enacted by participating states. These rules are maintained and developed by AEMC and enforced by the AER.

Fig. 1
figure 1

AEMO interactions with other regulators

AEMO regulates NEM through two control centers where all NEM connected generation performance is supervised. Co-optimization of energy market and ancillary services through NEMDE is performed to derive dispatch commands for all scheduled generators, semi-scheduled generators (less than 30 MW and intermittent generation like solar and wind), scheduled network services and scheduled loads. Derived dispatch targets are issued through the automatic generation control (AGC) system or the AEMO electricity market management system (EMMS) interfaces. In case of generation deficiency, AEMO can instruct for load cut off to some customers to maintain balance between generation and consumption.

All financial transactions related to electricity traded in the NEM is settled based on spot price. NEM spot market matches real time instantaneous demand with power supply through a centralized dispatch process. A specified amount of electricity at specified prices is offered by generators to be supplied to market for agreed time periods. Market operator scrutinizes all bids and decides the deployment of specific generators to produce electricity according to cost-efficient methodology with dispatching of the cheapest generator. Spare generating capacity is kept as reserve after matching electricity consumption with power generation. Every 5-minute target is applied to determine a dispatch price based on highest or the marginal bid for electricity delivery. Spot price is determined for each NEM region by averaging dispatch prices over every half hour period. Market price cap denoting maximum spot price and market floor price denoting minimum spot price are set according to the NERs. Market price cap was set at 13800 $/MWh and market floor price was set − 1000 $/MWh at January 2015 [4]. Market price cap highly rewards generation supply for meeting demand in need while negative market floor price strongly encourages power reduction by all generation, including wind in case supply exceeds demand. These two price settings are reviewed every four years by the reliability panel set up by AEMC to safeguard the NEM reliability standard.

AEMO prepares forecasts of the available capacity of all semi-scheduled generators, to schedule sufficient generation in the dispatch process and unconstrained intermittent generation forecasts (UIGF) for reserve assessment purposes. UIGF forecasts for individual unconstraint semi-scheduled wind generators dictates the available capacity which refers to the generation capability of a wind generator that is available for dispatch (without consideration of network limitations, price bids etc.). Australian wind energy forecasting system (AWEFS) which has less than 1.5% normalized mean absolute error in the 5-minute band, produces generation forecasts for all NEM connected wind farms including semi-scheduled and non-scheduled ones. Maximum wind generation is included in central dispatch process as forecasted intermittent generation for timeframes ranging from five minutes’ advance to two years’ advance for balance between load and generation. Wind farm integration in NEM market and its interactions between AWEFS and NEMDE in NEM dispatch market is shown in Fig. 2.

Fig. 2
figure 2

Wind farm integration in NEM market

Semi-scheduled wind generators participating in NEM dispatch process must mandatory follow the NEMDE generated dispatch levels only when the semi-dispatch cap (SDC) is active otherwise wind farms are free to generate to any level. For an unconstraint wind farm, UIGF is based upon actual megawatt output received through supervisory control and data acquisition (SCADA) system while in case of constraint wind farms, UIGF is based on meteorological forecast based upon number of available wind turbine, wind speed and megawatt set point. AWEFS performs three validation checks prior to every dispatch interval to determine if a wind farm’s output is being limited below its wind speed-based forecast (potential power): ① Is the wind farm control system set point less than the registered capacity of the wind farm? ② Is the wind farm control system set point less than active power plus 5% of registered capacity? ③ Is the wind farm control system set point less than potential power? If all three validation checks pass, AWEFS produces a wind speed-based UIGF. If any of these checks fail, AWEFS reverts to producing a UIGF based on the active power SCADA output.

3 Challenges to NEM operation due to increased wind penetration

Despite of having sufficient feasibilities for wind power integration in Australian NEM, it still lacks various technical and commercial aspects that require further investigations to test the system adequacy for reliable and secure operation. Intermittent wind power generation and asynchronous generators are two key characteristics of wind generation technology affecting its integration in NEM grid and spot market. Even though NEM-wide challenges are not identified by AEMO as each NEM region has a different generation mix, network configuration, and demand characteristics, which lead to different challenges or different timing; future increased wind power penetration will bring some adverse operational challenges as discussed below.

3.1 Merit order effect and reduced spot prices

Merit order effect in spot market infers that the marginal generator meeting market demand sets the market clearing price. In simple terms, merit order refers to the lowering of wholesale electricity price by subsidized generators by adding reduced short run cost generation to total supply. Merit order effect has an indirect impact on frequency regulation in NEM. Displacement of conventional generation in spot market during wind availability, may lead to low inertia situation and may aggravate any contingency event. Wind farms bid their output in NEM as price takers with low marginal cost. Subjected to available wind and any network limitation, present wind farms in Australia run at full capacity always. Wind farms get connection to the grid through user pays scheme while their access to the market is not always guaranteed [5]. Without storage options, wind farm operators typically dispatch electricity into the market regardless of price. They are incentivized by the RET for every unit of electricity they produce. Wind farms quite often oversupply the market and hence cause downward pressure on the wholesale electricity price. Under favorable wind conditions, the wind farms dislodge thermal or gas plants by bidding their low marginal cost to clear the market, lowering spot prices through the merit order effect. As per a Deloitte study [6] on merit order effect in SA, increased wind penetration in spot market resulted in backing off the dispatch of fossil fuel based marginal generator that would have set the marginal price leading to the reduction of traded electricity ultimately effecting the economics of generators in the form of reduced prices.

3.2 Reduced inertia and high rate of change of frequency issues

All synchronized rotating generators and motors constitute inertia of a power system. Higher the system inertia, lesser is the frequency volatility due to the disturbance. The amount of a conventional generator’s inertia is dependent upon its size and design, and is expressed in megawatt seconds. It is very difficult to maintain frequency within acceptable limits for low inertia based power system as it will slow down or speed up very quickly. Like demand level, power system inertia is also only an observed characteristic and currently AEMO has no control over it in any form. AEMO presently operates the power system around the requirements that arise from the present inertia levels. With synchronous generators (especially thermal power plant) providing majority of energy, each NEM region has sufficient inertia adequacy without any effect on system security needs. However, impending renewable energy targets has increased the probability of reduced power system inertia due to increasing renewable generation and displacement of conventional generation, particularly in SA and Tasmania. Current system inertia in SA is around 18725 MWs but a low inertia value of around 1000 MWs has also been observed [7]. Although some non-synchronous generation, like wind, also has rotating turbines, these technologies are increasingly connected to the power system via power electronic converters, so the mechanical movement is decoupled from the power system. According to [8], system inertia would be below acceptable levels for 30%-40% of the time in Tasmania, and 30% of the time in SA by year 2010. Victoria also experiences low inertia sometimes, but can rely on inertia from other NEM regions due to its strong interconnections.

Rate of change of frequency (ROCOF) df/dt management is critical to grid frequency regulation up to the frequency operating standard (FOS) [9]. The initial ROCOF measured soon after generating unit loss (∆P) is related to system inertia as given below:

$$\frac{{{\text{d}}f}}{{{\text{d}}t}} = - \frac{{f_{0} }}{{2H_{\text{conv}} }}\frac{{\Delta P}}{{S_{\text{conv}} }}$$
$$\frac{{{\text{d}}f}}{{{\text{d}}t}} = - \frac{1}{2}\frac{{\Delta P}}{{I_{\text{R}} }}f_{0}$$

where IR represents system inertia; Hconv represents conventional generators inertia; Sconv represents the mega volt-ampere rating of generators; f0 is the frequency set point. The amount of inertia required to maintain a ROCOF under different contingency is proportional to the contingency size. Lower inertia leads to a higher ROCOF than higher inertia system. That means the frequency changes faster following a disturbance in a power system with less synchronous generation, and this could result in the loss of additional generation or load to arrest the frequency deviation when it occurs. Reduced system inertia can challenge the effectiveness of existing frequency control mechanisms, which can reduce under high ROCOF.

The higher ROCOF will require stabilizing control systems to respond more rapidly to contain the change. For example, for a contingency event resulting in a ROCOF of 1 Hz/s, the frequency drops from 50 to 49 Hz in 1 s. A ROCOF of 2 Hz/s would reduce this time to 500 ms. Table 1 shows ROCOF variation of time it takes for under frequency load shedding (UFLS). High ROCOF will lead to additional tripping for the same size imbalance within a short duration meaning much faster action would be required to prevent the system frequency from breaching the FOS for a credible contingency event.

Table 1 ROCOF and time to UFLS comparison

Relays and protection schemes on generators and feeders have inherent delays and so may not respond quickly enough to high ROCOF. Critical schemes such as UFLS become compromised in maintaining the FOS operating successfully to prevent system collapse.

Increased wind penetration would make management of ROCOF after contingency events more challenging. The current standards are automatically met if a generating unit can withstand a ROCOF of ±4 Hz/s for quarter of a second. Generators can negotiate a lower standard, but the minimum standard is ±1 Hz/s for one second. There is no obligation on generators to remain connected to the system through an event where ROCOF exceeds those levels, even if the frequency remains within the bounds of the FOS. Present NEM faces an inconsistency in NERs with no specific power system operating standard for ROCOF maintenance at 1 Hz/s or better. Historically ROCOF following a separation between SA and Victoria has been below 3 Hz/s as shown in Table 2 but low level of inertia is likely to increase ROCOF and frequency deviations as observed during recent SA blackout in 2016 which happened due to loss of 966 MW brining ROCOF values up-to 6.2 Hz/s leading to UFLS failure in quick triggering.

Table 2 ROCOF variation during contingency event in SA

3.3 FCAS issues

Frequency control ancillary service (FCAS) is indispensable for the secure operation of large interconnected power system like NEM. Role of FCAS is to keep system within prescribed frequency bounds under various real-time conditions of demand forecasting errors, generators either non-scheduled or simply not following their schedules and short-term load variations. Fast ramping ancillary service proficiencies crucial to manage frequency in NEM are determined by the FOS as given in Table 3. According to NEM FOS [10] for normal system operation, the frequency must be maintained within the normal operating frequency band (49.85 to 50.15 Hz in both Tasmania and the NEM mainland) for no less than 99 percent of the time. NEM frequency is required to be within the normal operating frequency excursion band for more than five minutes on any occasion during credible contingency event. In case of region islanding or multiple contingency event, region frequency should not surpass the normal operating frequency excursion band for more than ten minutes.

Table 3 FOS for NEM

Unlike energy, FCAS is procured on a megawatt “enablement” basis. These services include 2 regulating FCAS services for normal operating conditions and 6 fast responding contingency FCAS services following any contingency events as represented in Table 4.

Table 4 NEM FCAS classification

Each FCAS is procured competitively each 5 min though a bidding process integrated with the energy dispatch process, managed and optimized witahin the NEMDE. 130 MW is procured for raise FCAS services while 120 MW is procured for FCAS lower services within a 5-minute dispatch interval. An accumulated time error of greater than ± 1.5 s may require additional regulation support of an extra 60 MW/s deviation for mainland. NEM mainland regulation requirement in the form of dispatch raise requirement is calculated as \(\hbox{min} \left( {250,130 + \left( { - 1 \times \hbox{min} \left( { - 1.5,T_{\text{error}} } \right) - 1.5 \, } \right) \times 60} \right)\)while dispatch lower requirement is calculated as \(\hbox{min} \left( {250,120 + \left( {\hbox{max} \left( {1.5,T_{\text{error}} } \right) - 1.5 \, } \right) \times 60} \right)\), where Terror denotes time error. Regulation for Tasmania is nominally set to 50 MW. The NEM mainland contingency FCAS requirement is determined within the dispatch algorithm considering together the largest contingency size and the network load forecast. All types of large FCAS requirement is calculated as: FCAS requirement is equal to contingency risk (megawatt change due to generator or load loss) minus load relief (demand change due to frequency deviation). Load relief factor is 1.5% for mainland in NEM while it is 1% for Tasmania [9]. As given in Table 5, total regulation FCAS capacity registered in the NEM for 2016 was 7245 MW (raise) and 7213 MW (lower) while SA has a minimum regulation FCAS enablement of 35 MW during island condition. An example of FCAS operation in NEM is represented in Fig. 3 during excess generation due to sudden loss of a large load [11].

Table 5 Low load NEM simulation scenario
Fig. 3
figure 3

NEM FCAS response during load contingency event

6-second contingency FCAS operational within 6 s is used to arrest the steep frequency excursion before it exceeds the operational frequency tolerance band of 51 Hz. The 60-second contingency FCAS operational within 60 s stabilizes the frequency, tracked by the 5-minute FCAS action to improve the frequency to within the normal frequency operating band. A power system with increased penetration of intermittent, non-scheduled wind generation would necessitate a larger dependence on regulation FCAS for operation over the 5-minute cycle. In a lower inertia based power system having high ROCOF, frequency deviations will take less time to reach the threshold contingency frequency range, thereby decreasing the stabilizing systems operational response time, and potentially growing the ancillary services requirements to return to normal operating conditions [11]. Effect of inertia and disturbance changes on FCAS requirement is presented in Fig. 4 [12].

Fig. 4
figure 4

Effect of inertia and disturbance changes on FCAS requirement

Historically synchronous generations have provided regulation and contingency FCAS in NEM but with exit of base load dispatch-able generators from islanding prone SA region, both FCAS availability at local level will be more challenging with increasing wind penetration level. According to a study, FCAS regulation service required capacity in NEM will increase by approximately 20% by 2020 due to increased renewable generation [13] which is likely to increase the magnitude of minute to minute generation deviations. In another study [14] based on 1% probability of exceedance (POE) metric for wind generation change in 5 min, beyond 6-10 GW of installed wind capacity, wind variability may cause enablement of more regulation FCAS in some periods. In case of insufficient FCAS availability, NEM frequency maintenance within the required standards will be difficult for AEMO, and NEM may collapse under big contingency events at worst scenario. AEMO is reviewing the procurement of regulation ancillary services, especially in smaller systems with high wind penetration to ensure their frequency control within required limits.

3.4 Interconnectors performance issues

Interconnectors serve to exploit the geographic diversity of intermittent generation sources, leading to more efficient generation siting decisions from a resource perspective, and smoothing the intermittency in aggregate across the NEM. Power import-export in NEM interconnected regions through interconnectors are limited by transient or voltage stability due to the contingent trip of the largest generating unit and potential thermal over-loadings. SA’s transmission network is connected to the rest NEM via the Murray link (DC link of 220 MW transfer capability) and Heywood interconnectors (AC link upgraded from 460 to 600 MW). The Heywood interconnector is the only link that provides synchronous connection between SA and the rest of the NEM. DC transmission line Bass link connects Tasmania to mainland.

According to NEM rules, connection costs are part of any new generator including wind farm construction cost while shared transmission network cost is funded by customers. Attributing additional shared transmission cost resulting from increased wind integration in NEM is challenging for AEMO in terms of market benefit economy. Most of the wind farms in Australia are in remote locations that will require significant transmission investment for improved transmission network infrastructure otherwise around 35% and 15% of the Victoria’s and SA’s wind energy respectively might be curtailed due to network limitations [15]. Therefore, balancing wind penetration around the NEM necessitates robust interconnections to yield geographical diversity benefits. Australia is unable to transfer FCAS transfer between the mainland and Tasmania. FCAS is reduced to minimum when interconnector reaches the maximum export limit or maximum import limit or when it is transferring power within a ‘dead-zone’ between − 50 to 50 MW.

In highly penetrated wind energy network, existing interconnector transfer limits is highly reduced under low demand and high wind speeds [1516]. Increased wind generation in states like SA and Tasmania has increased energy imports/export through interconnectors putting more thermal stress especially on Heywood interconnectors during the period of high demand and low wind generation and vice versa. Any contingency at interconnector level may have high contingency impact on NEM spot market and FCAS market operation due to high ROCOF. A combined effect of spot market price variation and interconnectors dependence was observed recently during a contingency event which occurred on 1 November 2015 when SA was islanded for 26 min due to a transmission line tripping and Heywood interconnector being unavailable due to upgrading works. Sudden fall of interconnector capacity resulted in FCAS procurement within SA region resulting in price hikes.

4 Simulation for frequency response indicators for a NEM region

To support above mentioned claims, author simulated an augmented form of 14-generator NEM model [17] to study wind integration effect on frequency response indicators in Area 5 which can be held representative of SA. Areas 1 to 4 represent Tasmania, New South Wales, Victoria, and Queensland, respectively. In the original model, there are 14 generators, 5 static var compensators (SVCs), 59 buses and 104 lines with voltage levels ranging from 15 to 500 kV. The automatic voltage regulator (AVR) excitation system and power system stabilizer (PSS) of generators are adopted from [16]. Also, it is assumed that all thermal and hydro power plants have a standard steam turbine governor (i.e. IEEEG1) and hydro turbine governor (i.e. HYGOV), respectively. Test network is studied when doubly fed induction generator (DFIG) based wind farms are integrated into Area 5. Area 5 has three lumped generators: G503 at bus 503, G502 at bus 502 and G501 at bus 501. G503 and G502 are connected at 15 kV while G501 is connected at 20 kV. Several generator and interconnector contingency case studies were performed under low load scenario as given in Table 5 to analyze the technical problems due to wind farm integration in low load scenario under low levels of conventional synchronous generation in operation. Wind penetration level (LWP) is defined as ratio of total wind generated power (Pw) by total generated power which includes synchronous generator power (PSG), inter-area power flow (P1A) and wind power.

$$L_{\text{WP}} = \frac{{P_{\text{W}} }}{{P_{\text{SG}} + P_{\text{W}} + P_{\text{IA}} }}$$

Simulation results for Area 5 are presented in Table 6. Linear polynomial surface view for wind penetration effect on ROCOF and number of active synchronous unit effect on ROCOF and frequency nadir FNadir is shown in Fig. 5. A directly proportional relationship is observed between ROCOF, FNadir with wind penetration and megawatt loss while an inverse proportional relationship is observed for number of active synchronous generator with ROCOF and FNadir point. It is also observed that ROCOF is maintainable within 1 Hz/s range up to 400 MW generator contingencies and increases sharply within 2 s range with higher megawatt loss. In the Australian NER, no standard is set for a maximum level of ROCOF on the power system. Generation, on the other hand, is required by their access standards to remain connected through an event where ROCOF reaches ± 1 Hz/s. NEM needs to have clear ROCOF standard for the correct operation of emergency protection frequency relay that manages multiple contingency events should be based on maximum ROCOF.

Table 6 Simulation results for low load scenario for Area 5
Fig. 5
figure 5

Linear polynomial surface view for wind penetration effect

Based on simulation results, a linear polynomial regression model for ROCOF, FNadir variation with wind penetration, amount of megawatt loss and number of active synchronous generators was formulated as:

$$F(x,y) = p_{00} + p_{10} F_{\text{Nadir}} + p_{01} \frac{{{\text{d}}f}}{{{\text{d}}t}}$$

Coefficients (with 95% confidence bounds) for linear polynomial regression model are given in Table 7.

Table 7 Coefficients for linear polynomial regression model

Following results are concluded for Area 5 operation from the simulation studies:

  1. 1)

    Up to 45% wind penetration in Area 5 has a negligible impact on strongly interconnected Area 3 regarding changes in ROCOF and frequency nadir. Thus, a strongly interconnected network with power sharing can be a key to sustainable wind energy integration.

  2. 2)

    NEM has a minimum standard of ±1 Hz/s for one second for generators to withstand ROCOF. With increasing wind penetration, there is a requirement for ROCOF dependent relay operation to consider the time duration also for which there is a change in ROCOF values.

  3. 3)

    Islanded network calls for more local regulation FCAS from local generating units’ including wind energy participation in centrally managed AGC.

  4. 4)

    Three tired classification level can be formulated from simulation results for ROCOF for wind integrated power system like Area 5: ① Green (df/dt ≤ 1 Hz/s), FOS of 47-52 Hz can be easily met under wind penetration as high as 40% for both low load and high load scenarios. ② Orange (1 Hz/s < df/dt ≤ 4 Hz/s), FOS of 47-52 Hz cannot be met under wind penetration more than 20% and high contingency event for both low load and high load scenario. With higher wind penetration and generation loss, ROCOF remains under 4 Hz/s, but frequency nadir drops beyond set level. ③ Red (df/dt > 4 Hz/s), FOS of 47-52 Hz will not be met. Even though with wind penetration as high as 45% and megawatt loss as high as 740 MW, ROCOF remains under 4 Hz/s. If ROCOF goes above 4 Hz/s, FOS standard will be violated for more than a second leading to generation disconnection.

5 Frequency control mitigation methods

Maintaining a minimum synchronous inertia is an approach for frequency regulation, but it may become expensive with depleting level of synchronous generation and increasing renewable generation. The scale and type of response required to make a useful contribution to the low inertia power system is still unknown. Understanding the dynamic performance of the future low inertia Australian NEM requires an extensive investigation and analysis. Currently, AEMO manages these power system impacts with short term mitigation measures like applying constraint equations in the central dispatch process to limit wind generation, UFLS or market intervention as last option where AEMO issues instructions to synchronous generators to guarantee adequate power system inertia level maintenance for satisfactory control of power system frequency.

5.1 Current short-term mitigation methods

5.1.1 Automatic UFLS

UFLS gets initiated in the absence of sufficient R6 FCAS. The basic design premise of the scheme is that any frequency drop in response to credible and non-credible contingency events should be limited to 47 Hz by the controlled disconnection of load through frequency sensing relays. UFLS operates only during rare events, usually following a non-credible contingency, where a drop-in frequency has not been arrested by FCAS. Market customers with expected peak demand at their connection point more than 10 MW are required to provide automatic interruptible load to a minimum of 60% of their expected demand [18].

5.1.2 Constraint equations

AEMO can intervene in frequency reserve trading process based on linear programming based medium term and short term projected assessment of system adequacy (PASA) which indicates low reserve condition or lack of reserve. In case of generation demand imbalance being outside reliability standard, low reserve condition is declared while lack of reserve level 1, 2 or 3 is declared if capacity reserves reduces below the level required to manage credible contingency events. AEMO administers PASA process under 10% POE and 50% POE demand conditions for medium- and short-term system security. AEMO maintains the minimum local generation required in each region targeting 0.002% unserved energy under NEM reliability standard which is transformed into operational commands in the form of minimum reserve level (MRL) equations which are given in Table 8, where RVIC and RSA denote reserves of Vitoria and SA, respectively. Static MRL equations are applied in medium term PASA for Queensland, New South Wales, and Tasmania regions while shared MRL equations are applied to Victoria and SA regions with net import limits (0 into Queensland and SA, 330 MW into New South Wales, 940 MW into Victoria). Any planned or unplanned outage is handled by AEMO through constraint equations. In case of any PASA related generating/load unit failing to provide required services, AEMO declares them as non-conforming, put penalty and apply constraint equations to generator dispatch or load shedding.

Table 8 MRL for NEM

Constraint equations are used to define the mathematical restrictions translated from a physical transmission network representation. These constraint equations may be grouped into constraint sets to simplify the constraint management process. Any system security issue arising during system normal conditions or network outage conditions is countered through predefined generic constraints and following the occurrence of a contingent event through network outage constraint set. Discretionary constraints may be used with routine planned network outages where a constant limit on power flow on a single network element is required. AEMO staff may generate thermal constraints using EMS based constraint automation application.

5.2 Prospective mitigation methods

Besides above mentioned short term action plans, AEMO is working on long term mitigation methods. Some of the potential technical solutions in terms of their capabilities and limitations are discussed below for future NEM reliability.

5.2.1 Synchronous condensers

The synchronous condenser has been used as a conventional solution for reactive power regulation but they have been losing value due to growth in power electronics based reactive power compensation scheme. However, during the global trend towards renewable energies, synchronous condenser has been experiencing a renaissance since last 5 years as frequency regulation solution. High inertia synchronous condensers are estimated to cost in the order of $50 million for the addition of 1000 MWs of inertia [19]. Non-profitable or disengaged power plant conversion to synchronous condensers is presently perceived as the utmost cost-effective route. Synchronous condensers are synchronous machines integrated to the electricity network. The condenser when synchronized with the electricity network, will act as a motor, turned by the energy taken from the grid. Because of the nature of the synchronous machine, reactive power can be consumed and generated by controlling the excitation of the rotor. Generator of synchronous condenser with its rotating mass also always acts against a variation of network frequency, thereby acting as support for stable system frequency. According to KEMA report, only 1%-4% of the nominal power rating is consumed during inertia provision of approximately 1 s. Synchronous condensers can respond immediately if on otherwise take less 15 min start up [20]. Francis turbines based hydro plant when operated in tail water depression mode can easily be operated as synchronous condensers. For Francis turbines, this is achieved by ‘dewatering’ using high pressure air to force the water level below the turbine so that it can spin freely and with minimal hydraulic resistance [12]. Operating Pelton turbines in synchronous condenser operation is generally easier, as the turbine is not submerged during normal operation. A synchronous condenser has the benefit of providing a synchronous inertia response to support ROCOF management capabilities and providing fault level and voltage support services beneficial to the power system.

The impact of synchronous condenser effect on the frequency response of 14-generator NEM model’s Area 5 under contingency events is investigated. Synchronous generator with the same number of units as that of isolated generating plants is connected to the 275 kV high-voltage transmission network via a step-up transformer. In the first simulation, Area 5 G503 is isolated at 10 s and G501 isolated at 20 s from Area 5. Total wind farm penetration in the area is taken as 44% with 585 MW DFIG based wind farm connected to bus 509 and 561 MW DFIG based wind farm connected to bus 507. In the second simulation, synchronous condensers of equivalent ratings of G503 and G501 are introduced in the test network. First, only one synchronous condenser connected at bus 508 is activated to analyze frequency response. Second simulation test has both synchronous condensers activated in the network. Under 44% wind penetration and 710 MW loss in Area 5 under low load scenario, a clear improvement in frequency response indicators; ROCOF and frequency nadir point are observed as shown in Fig. 6 when synchronous condenser 1 (150 MW) and synchronous condenser 2 (710 MW) are added to Area 5. ROCOF observed at G502 during generator trip contingency starting at 0 s and synchronous condenser added in network show a variation from − 2.68 Hz/s to 1.2 Hz/s in 0.6 s while it varies from − 2.35 Hz/s to 0.875 Hz/s in 0.6 s with synchronous condenser 1 added and vary from − 1.68 Hz/s to 0.5 Hz/s in 0.8 s with synchronous condenser 2 added. Similar improvement is observed for frequency nadir variation also. Additional frequency support through additional inertia support from synchronous condensers can enhance frequency response performances, which in turn reduces the amount of UFLS. ROCOF can be maintained within its acceptable limit by employing a certain number of synchronous condensers which depends on active and committed synchronous machines to the system. This frequency improvement suggests that if large generators of retiring plants are converted as synchronous condensers, the appropriate level of inertia is possible to achieve with desired ROCOF.

Fig. 6
figure 6

ROCOF comparison and FNadir comparison with/without synchronous condensers

Our simulation results support the current situation in Tasmania. The current position for Tasmania is that the minimum demand can be as low as 900 MW, Basslink may be importing up to 478 MW and wind can contribute up to 308 MW. Under these conditions, there is little room left for synchronous generation. If the minimum system technical requirements for ROCOF and FCAS availability cannot be met within the central dispatch process, AEMO’s constraints will limit Basslink flow and/or wind farm output so that more on-island synchronous generation is provided. Tasmania has currently around 1470 MWs of synchronous condenser capability comprising of 14 hydro units, 3 open cycle gas turbine (OCGT) units and 2 local synchronous condensers installed at Musselroe wind farm [12]. System constraints can be alleviated by dispatching selected hydro generators in synchronous condenser mode. Inertia addition and contingency reduction can reduce fast FCAS requirements. However, under the existing rules, AEMO does not have a mechanism to dispatch this service and the service is provided by Hydro Tasmania on a voluntary basis. The cost of energy used to operate in this mode, along with the associated operation and maintenance costs, is ignored by the market.

5.2.2 New interconnectors

To present the importance of interconnectors on frequency response, interconnection separation contingency was investigated by authors for Area 5, 14-generator NEM model under low load and high load scenario. Table 9 presents frequency response for wind penetrated Area 5 with interconnector contingency under high load scenario. G502 terminal ROCOF changes from − 1.17 to 0.57 Hz/s in 0.6 s soon after G503 isolation. At second contingency of interconnector loss, ROCOF changes as − 3.37 to 0.14 Hz/s in 0.9 s. G501 terminal ROCOF changes from − 1.09 to 0.57 Hz/s in 0.6 s soon after G503 isolation. At second contingency of interconnector loss, ROCOF changes as − 2.85 to 0.25 Hz/s in 0.9 s. G302 terminal ROCOF changes from 0.1 to − 0.16 Hz/s in 5 s soon after G503 isolation. At second contingency of interconnector loss, ROCOF changes as 0.4 to − 0.37 Hz/s after 10 s. During G503 only contingency, not much deterioration in ROCOF values is observed due to increased power export from Area 3 to Area 5 from 500 to 700 MW while frequency nadir is decreased. With interconnector contingency included, all frequency operation standards are violated. Similar results are obtained under low load scenario also where under low generator contingency along with interconnector contingency, NEM frequency standard is violated as frequency goes below 49.5 Hz. These results match the SA jurisdiction which has recommended AEMO for permitting larger frequency variations (47-52 Hz) as frequency can go beyond 49.5-50.5 Hz for the credible loss of the Heywood Interconnector. In all the contingencies above 500 MW, we observed ROCOF breaches the standard of 1 Hz/s for around 1-2 s. However, the maintenance of FOS is possible up to 40 % wind penetration with sufficient local FCAS and strong interconnectors in NEM region.

Table 9 Area 5 frequency response indicators when G503 (436 MW) is isolated at 5 s, and interconnector is isolated at 10 s, wind farm added (436 MW)

An interconnector provides greater access to lower-cost fuel supplies at times when intermittent generation within the region is low delivering potential generation dispatch efficiency benefits. An additional interconnector may alleviate high ROCOF concerns and reduce the likelihood of a widespread blackout in a region like Area 5 represented by SA or Area 3 represented by Tasmania by mitigating the possibility of electrical separation from the rest of the NEM.

There are new interconnector proposals like a new interconnector linking SA with either New South Wales or Victoria from 2021. Augmenting the existing interconnector linking New South Wales with both Queensland and Victoria in the mid to late 2020s, particularly as coal-fired generation retires. A second Bass Strait interconnector from 2025, when combined with augmented interconnector capacity linking New South Wales. Besides new interconnector, additional regional solutions would be required to address low system strength concerns and minimize the potential contingency size [12]. Operating an interconnector below its maximum transfer limit can enable the interconnector(s) to remain connected following a larger contingency size, which decreases the risk of separation, but this can reduce available market benefits. A new frequency control system protection scheme (FCSPS) is being applied to mitigate the effects of a credible contingency and optimizing the import and export capability of the Tasmania Basslink interconnector with very positive results [9]. The scheme continuously monitors the interconnector flow, and system load demand over 4 s cycle and calculates the required load or generation tripping that is necessary to mitigate the contingent loss of the interconnector. Loads or generators are tripped within hundreds of milliseconds of protection clearance time to allow system frequency to be maintained within the operational frequency tolerance band limits, even though Basslink could be operating at up to 630 MW export or 478 MW import.

5.2.3 Augmented wind turbine technology with frequency response capability

Grid code compatible frequency responsive wind turbines have the potential for fast frequency response (FFR). Higher permissible ROCOF during contingency events requires fast acting FCAS, also known FFR service in the range of 0.5-2 s [21]. FFR service from wind farm is mandated in UK (response time 1 s), ERCOT (response time 0.5 s), EirGrid (response time 2 s) [21] but frequency control in Australia has traditionally only been provided from synchronous thermal generation. There has been little incentive for wind farms to contribute to frequency regulation so wind farms prefer not to participate in FCAS.

Frequency-active power control model is an auxiliary control algorithm implemented in individual wind turbine generator control loop for providing controllable power reserve on demand in form of spinning reserve or power ramp rate limit in response to system frequency deviations. A frequency responsive WTG model is shown in Fig. 7a while Fig.7b shows simulated results for hydro-wind based control area frequency response with different integrated wind plant configurations. Highest deviation and longest settling time are observed when wind plant is just feeding power to control area and smallest deviation with lowest settling time is observed for control area without wind plant. Grid code responsive wind plant with AGC participation shows the best performance with frequency response comparable to control area without wind plant with identical settling time. Integration of simplified droop based variable speed wind power plant VSWPP [22] produces highest frequency deviations and longer settling time.

Fig. 7
figure 7

Frequency responsive wind turbine model and frequency observed at bus 506 under different scenarios

Authors investigated frequency grid code responsive wind turbine model integration effect on frequency of Area 5 of NEM model [17] under low load scenario. Figure 7c presents frequency observed at bus 506 of Area 5 when 436 MW of normal wind farm is integrated and Fig. 7d presents frequency observed at bus 506 of Area 5 when frequency responsive wind farm is integrated. A clear improvement in frequency deviation from 49.1 to 49.97 Hz can be observed.

Despite of good power frequency support technical capabilities [23] from new wind turbines technology, their provision in ancillary market is limited due to regulatory policies and some operational challenges. New frequency responsive wind plants face a challenge for adaptation from present grid structure and seek regulatory certainty and stability from government for economic gains. Currently, there are no operational frequency responsive wind turbine generator based farms in Australia, however, things are changing now with AEMO supporting first trial for FCAS from 100 MW Hornsdale 2 wind farm [24]. Figure 8 demonstrates the speed at which the Hornsdale power reserve responded to a contingency FCAS – to an incident when a Loy Yang unit tripped on December 14. Since it began offering FCAS in December 2017, the 100 MW/129 MWh lithium-ion Tesla battery based Hornsdale power reserve project has also been effectively responding to AEMO AGC signals and has been consistently enabled in all eight FCAS markets [25,26,27].

Fig. 8
figure 8

Frequency response from Hornsdale wind farm power reserve on December 14, 2017

5.2.4 Battery storage for FFR

Energy storage is viewed as the most substantial complimentary technology which can either smooth or shift intermittent renewable generation to match demand profiles and improve renewable energy ability in ancillary services participation for frequency regulation. Storage has technical advantage over any form of generation is that it is a two-way process; it can both export energy and import energy. An association of diverse energy storage technologies with their corresponding discharge rates, power ratings and efficiencies [31] is presented in Fig. 9. As can be noticed, modular and scalable batteries are proficient in short-to-medium term storage with a comprehensive output capacity range. A major benefit of batteries is the scalable nature of the technology. Duration can be extended to support renewable energy sources. Improved battery technologies like Li-ion have efficiency range from 85%-98% with lifetimes of 5-15 years and have capability for both fast and slow discharge rates.

Fig. 9
figure 9

Storage technologies

Mass production and continual innovation has brought the cost of lithium-ion consumer batteries down 90% over 16 years from 3185 $/kWh in 1995 to 320 $/kWh in 2011. Southern California Edison and San Diego Gas & Electric [28], U.S. Investment Bank Lazard [29] and the International Renewable Energy Agency (IRENA) [30] have confirmed the competitiveness of batteries with conventional peak plants for ancillary service provision. According to AECOM calculation report, energy storage paired with solar off-grid remote locations is 120 $/MWh which is half of the cost of diesel-only conventional generation at 346 $/MWh [31]. According to a 2017 report, energy storage requirement for system adequacy in Australia under high renewable energy penetration of 75% will be 105 GWh [32]. Under the assumption that the security requirement is met by batteries providing two hours of storage, the need for energy storage for adequacy is reduced by two thirds at 2030 under 52% renewable integration [32]. Operational thermal and hydro/pumped hydro system generators are currently responsible for FCAS in NEM, with each service providers required to provide more than 1 MW capacity. There are over 1.5 GW of pumped hydro storage operating in NEM and no examples of CAES, Sodium-Sulphur or liquid metal technology installations. Most of battery energy storage systems (BESSs) are in operation for off-grid customers in NEM. While projects like 10.4 MW of solar photovoltaic (PV) with 1.4 MW/5.3 MWh of lithium-ion battery storage are under construction for improved power quality and supply at a fringe-of-grid location in Queensland, still there is little deployment for battery storage specifically for FCAS. Bloomberg New Energy Finance (BNEF) projects over 3 GW of cumulative installed residential BESS asset by 2030 [33]. As with utility scale BESS assets, this represents an asset base of fully controllable generation and customer load that can provide critical reliability in a changing market. The existing market framework is not, however, well set up to extract energy, FCAS or demand response services from these assets, without the involvement of a retailer.

Though battery technologies are capable of faster response times for frequency regulation, like Tesla power pack which has a response time of less than 200 milliseconds, current frequency regulation NEM market mechanism has no provision of financial reward for response faster than required under the 6 s FCAS market. Currently, a BESS asset currently must register as both a generator and market load to provide both charging and discharging services. As a result, a more conservative approach to bidding is required as the BESS operator or market participant is required to estimate whether a charge or discharge service is likely to be more valuable to the market within a given dispatch period [26]. A distinct market participant classification should be defined for BESS assets that will allow for single dispatch bids for both generation and load services. A new small generator aggregator (SGA) market classification may provide alternative means to meet the measurement and monitoring requirements associated with energy and FCAS market participation. With increased intermittent generation, future NEM spot market may seek increased FCAS services pricing promoting ancillary services from battery storage. Lack of integration standard is a current barrier for deployment of this technology on grid but Australia is seeking grid connection standards development for energy storage systems in form of AS4777.

5.2.5 OFGS and ROCOF adjustment in emergency UFLS

In view of effect of ROCOF and low inertia on UFLS failure in arresting frequency drop, a new hybrid graded UFLS is now operational in Tasmania and is in implementation stage in SA in which 15% of the load available to the UFLS scheme is tripped based on ROCOF. These relays are tripped if the ROCOF is greater than or equal to 1.5 Hz/s and the absolute frequency is less than or equal to 49.4 Hz. This scheme presents a smaller contingency with lower residual ROCOF to the remaining blocks of UFLS. AEMO along with service provider ElectraNet, has designed an over frequency generator scheduling (OFGS) to limit the frequency rise in SA to 52 Hz in line with the FOS and will be implemented by July 2017. The objective of the scheme is to coordinate the tripping of generation in a pre-determined manner, tripping low inertia generators first, to maximize the inertia online. This seeks to minimize the impacts of exacerbated ROCOF that would result from disconnecting synchronous generators that provide system inertia during an extreme frequency event. The scheme would only operate for frequency excursions above the upper limit of the “operational frequency tolerance band” of 51 Hz. Generation to be tripped is split into eight blocks, each with around 150 MW of wind generation, set to trip between 51 and 52 Hz.

5.2.6 Demand response

Services reducing total electricity consumption or altering load curve by load reduction during peak times or shifting loads to off-peak time comes under demand-side management (DSM). Demand response refers to second condition of load reduction during high wholesale electricity market prices or in case of endanger to system reliability. Despite the development and implementation of many initiatives since 1992, demand-side contribution level in the NEM has been fairly low. NEM does not contain an explicit demand response mechanism for purchasing demand response as a substitute for generation. The load centered ancillary services in the NEM wholesale market is presently restricted to registered customers in the wholesale market with large loads that can respond quickly such pumped hydro. Wholesale demand response is achieved through direct exposure of loads to spot prices or via a retailer. According to AEMO estimate in 2016, there is 700 MW of price responsive load across NEM which corresponds to 2% of peak demand [34]. Another study estimates 3.8 GW of industrial demand response potential corresponding to 12% of peak demand [35]. The presence of unscheduled DR, forecasting error and absence of demand response mechanism is the current problem faced by NEM. NEM dispatch targets are based on load forecasting which does not directly account for price responsive demand due to practical difficulty of smart metering and energy consumers survey. DR remains invisible to the market operator in the absence of a proper market based DR mechanism.

However, under the new unbundling rule 2016 [36, 37] commencing from 1 July 2017, a new group of market participant called demand response aggregator (DRA) which would be registered with AEMO, would be able to provide ancillary services to the market in addition to demand response mode (DRM) participation. This will be accomplished without requiring the DRA to be a market customer in the spot market, thereby effectively unbundling the provision of these services from the purchase of energy in the spot market. The DRA would be able to register a load or aggregation of loads as ancillary services load and provide FCAS. Future demand response will be categorized by the nature of load (e.g. mining, manufacturing, transport and storage) and the DRM provision mechanism (e.g. electricity generation, plant shutdown, batteries etc.). Price settlement during demand response in NEM trading interval is represented in Fig. 10. AEMO will use utilized metered energy and baseline energy to separate energy use from demand. End users become eligible for financial compensation through demand-side response mechanisms in case they decide on switching off or rescheduling their energy consumption in reaction to market pointers.

Fig. 10
figure 10

Demand response participation mechanism in NEM

Figure 11 shows that the supply mix for the contingency raise service is changing as more non-synchronous generators are enabled in that market. Deeper and more diverse FCAS markets have the potential to provide improved system security services by increasing the competition among suppliers of ancillary service in FCAS markets and so leading to more efficient FCAS prices. More and greater diversity in providers of ancillary services would supplement the increased intermittent and non-synchronous generation penetration in the NEM. Management of flexible loads used by residential and industries will be a key element to improve network utilization. Controlling these loads enables the reshaping of the load profile on the network in a way that has minimal impact on the customer by matching load to generation. There are increasing number of examples of new technologies and approaches being integrated and trailed in the NEM for demand response. According to a study [38], interruptible load services as part of demand response can deliver 135 MW of frequency raise service within 1 s of the trigger event, or 70 MW within 0.2 s.

Fig. 11
figure 11

Changing supply mix for FCAS

PeakSmart air-conditioner as part of positive payback program is an example of successful demand response program being implemented in Queensland. Currently, there are 4 ranges of the demand meter in Queensland: ① low, up to 1999 MW; ② moderate, from 2000 to 2999 MW; ③ high, from 3000 to 3999 MW; ④ extreme, above 4000 MW [39]. PeakSmart air-conditioner units operate when peak demand on network rises above 3000 MW. There are 3 DRMs for air-conditioners, depending on how extreme the need for demand management becomes. Compressor gets off in DRM1. Air conditioning unit is capped to operate at 50% in DRM2 while in DRM3, it is capped to operate at 75%. PeakSmart demand management solutions rely on audio frequency load control (AFLC) system, which operates by the injection of a high frequency (1042 Hz) coded signal onto the high voltage network at substations, to send signals to participating households. All PeakSmart air-conditioners are equipped with a signal receiver. A signal is sent remotely from the operator via power supply that tells the air-conditioner to cap its energy consumption on occasions when the network reaches peak demand [39]. Another successful example is 770000 residential customers hot water systems connected to a controlled load tariff in Queensland. Load control of hot water reduces peak demand [40] and can be used to increase load during the day to absorb solar PV output.

NEM currently have 13 registered market ancillary service providers. EnerNOC as one of the demand response providers participates in 6-second, 60-second and 5-minute raise FCAS markets by offering a reduction in load. Their FCAS resource is comprised of distributed, aggregated switching controllers installed at commercial and industrial energy users’ facilities throughout the NEM. Participating customers come from the cold storage, industrial, and forest products manufacturing sectors, majority provide a FFR in less than 250 ms [41]. To date, EnerNOC has offered and cleared as much as 14/60/71 MW in the R6/R60/R5 FCAS markets [42]. Under a three-year trail program funded by Arena, electricity users would be paid up to $12.5 million a year to have 160 MW capacity on standby to take offline to help manage pek demand. United energy’s demand response program will remotely reduce the voltage at 47 zone substations by 3% on average to deliver at least 30 MW of demand response within 10 min when called upon.

AEMO is currently developing systems and procedures to implement a new rule where any load wanting to provide FCAS services will be classified as ‘ancillary service’ load, as long as AEMO’s technical requirements are met. Permission of separate ancillary services supply from the retail electricity supply will facilitate rise in demand response participation in FCAS markets. DRAs with aggregated ancillary services load will have similar FCAS payment and recovery mechanisms as that of generation resources. AEMO will pay the DRA for dispatched FCAS based on enabled megawatt power into corresponding FCAS clearing price. Intermittent wind energy shares an increasing proportion of NEM electricity, so it calls for improved automatic demand response with capabilities surpassing customary peak load-reducing demand response. Fast frequency responsive demand response decreasing or increasing load during over-generation and under-generation period will improve the increased renewable resources utilization and will thus, strongly improve system stability.

6 Conclusion

Current Australian NEM is undergoing continuous shift from centrally dispatched large scale synchronous generation towards scheduled and non-scheduled sustainable distributed generation. This change in generation mix due to high renewable energy penetration challenges the whole interconnected system designs built under different network configuration, operational strategies and the regulatory framework within which it operates.

The Australian NEM was originally established on the postulation of an incessant demand growth but current market with a projected flat demand is facing critical system security challenge. Australia national electricity model and its operational policies are studied in this paper for real time frequency regulation challenges associated with large scale integration of wind plant. Even though NEM-wide challenges are not currently identified as each NEM region has a different generation mix, network configuration, and demand characteristics, leading to different challenges or different timing; future increased wind power penetration will bring some adverse operational frequency regulation challenges in the whole NEM.

Key finding shows that currently applied mitigation policies for FCAS control along with automatic load frequency shedding and changing constraint equations for frequency control may not be sufficient and satisfactory under increased wind energy penetration. NEM FOS management is becoming difficult due to increasing ROCOF under increased wind penetration and reduced inertia due to synchronous generator losses and requires visionary reforms in its structure and operating policies. Fast adoption of proven technologies like synchronous condensers, frequency responsive wind plants, battery storage and demand response in NEM will provide greater support for future FCAS market. Consistent integration of renewable energy like wind calls requires improved policies development and encouragement for efficient diverse frequency regulation ancillary services market from governing authority to keep pace with current grid transition and security maintenance.