A new design for market power monitoring in the electricity market. A simulation for Italy

Abstract

The liberalization of electricity markets has been dominated by conditions of oligopoly and market power, as shown in numerous studies in empirical literature on the supply side. However, regulators have used statistical measurements to monitor the extent of market power, making little reference to founded theoretical approaches. This paper provides a new contribution to the literature on the electricity market by presenting a theoretical and empirical model to construct competitive equilibrium, and estimating market power on both the supply and demand side of the day-ahead electricity market. We implement an accurate measurement of the welfare loss associated with non-competitive market conditions, based on ex-ante demand and supply behavior.This model provides a useful analytical tool for regulators and policy-makers in order to implement pro-competitive regulation. We perform an empirical simulation to show the effects of non-competitive equilibria on the Italian hourly markets over the period 2013–2014. In an ideal competitive market, prices would be lower than historical prices by about 2–5% and quantities would be higher by about 0.5–1%.

1 Introduction

The extensive changes to the electricity sector introduced with the reforms of the 80 s and 90 s have gradually constructed an independent supply and demand market and the consequent formation of a decentralized price mechanism. Abandoning the previous structure of vertically integrated monopolists and government-owned companies, the new market design has been striving to create a competitive environment, promote new investments in efficient technologies for the generation sector, and familiarize the consumer with the free market. In Italy, following EU Directives, the reform was characterized by three markets: the Day Ahead Market (DAM) open to suppliers and purchasers based on an implicit auctions mechanism; the adjustment market and the dispatching market, reserved for suppliers and the dispatcher providing network security services. In the DAM, a Walrasian auctioneer separately ranks the individual bids of each supplier and purchaser for every hour, to construct a stepwise supply and demand function. The equilibrium price resulting from this intersection is the system marginal price (SMP) paid by purchasers to all dispatched suppliers.

The new reform was based on the premise that market competition would increase social welfare, thereby reducing the final price for consumers. Unfortunately, the initial deregulation revealed just how many organized electricity markets in Europe and the USA were affected by anti-competitive behavior, especially on the supply side (Bolle, 2001). The break-up of the formerly government-owned monopolist resulted in a new oligopolistic group, which exercises market power, as widely reported in the empirical literature (Amountzias et al., 2017; Anielski et al., 2002; Boffa et al., 2010; Green & Newbery, 1992; Kannan et al., 2011; Wolak, 2003). Suffice to recall, in Italy the monopolist was hence divided into four large generators: the former government-owned provider retained 50% of capacity, while three New Gencos made up the remaining 50% (roughly 20%, 20%, 10%). In other words, the market started with a Herfindhal of around 3400. Today it is around 1000 in the North and 3000 in the rest of Italy. Recently, empirical evidence of the existence of oligopsony market power was shown by Bigerna and Bollino (2016). A recent analysis of the critical aspects of the electricity market performance is given in Sapio and Spagnolo (2020) and Yang and Sharma (2020), with specific reference to market power in Bask et al. (2011), Nazemi et al. (2016) and Pham (2019).

This paper provides three new contributions to the literature on the electricity market, using the estimated market power on both the supply and demand side of the day-ahead electricity market to construct a competitive equilibrium.

First, we infer ex-ante demand and supply behavior from elementary bid data in the organized electricity market submitted by the individual market participants, unlike the aggregated market data traditionally used in the literature. Second, we relax the assumption of price-taking behavior on the demand side, thus assuming a Cournot oligopolistic and oligopsonistic behavior on the supply and demand side, respectively. Third, we estimate and compute the deviation of equilibrium prices and traded quantities from their competitive values, disentangling the two main components, i.e., the oligopolistic behavior effect on the supply side, and the oligopsonistic effect on the demand side.

This model shows an accurate empirical measurement of the dead-weight loss of welfare due to strategic behavior on both sides of the market. We have developed a comprehensive model to compare competitive and non-competitive equilibriums caused by agents exercising market power. In this way, our analysis can quantify how much the current welfare would increase if a competitive market structure were applied to both sides of the market. Deviations from the competitive equilibrium and the corresponding welfare loss are measured as follows: after constructing an empirical measure of market power, we build a counterfactual market auction from bids purged by market power, which allows us to derive the implied theoretical competitive equilibrium.¹ The total welfare loss is computed as the difference between simulated competitive welfare and welfare derived from the current market equilibrium.

The empirical results suggest that deviations from competitive welfare are significant, and that removing the oligopsonistic market power highlights a large increase in welfare. In addition, we break down the current welfare product into producer and consumer components, and simulate how these shares would change if a competitive structure were to be applied. We find evidence that both components affect welfare, confirming the need to analyze both supply and demand behavior in the market.

The paper is structured as follows. Section 2 briefly discusses some related literature, Sect. 3 presents the electricity market structure in Italy, Sect. 4 shows the theory and the methods applied, Sect. 5 presents the results and discussions and Sect. 6 presents the conclusions and the policy implications. Appendix A presents the analysis of robustness of the econometric estimation procedure.

2 Related literature

Electricity is a homogeneous and divisible good that fits well with being allocated according to a uniform price auction mechanism, where agents (both generators and purchasers) compete by simultaneously submitting bids to optimize their profits and the auctioneer clears the price. Despite the widespread use of these auctions, extensive literature has shown they are susceptible to strategic behavior that deviates from competitive equilibrium and shrinks total welfare, and might be exploited by large firms market power to raise prices.

Among the seminal papers that estimate unilateral market power in the electrcity market, we recall Klemperer and Meyer (1989) who provided the underlying conditions for the existence of a univocal Nash equilibrium in a symmetric oligopoly. Green and Newbery (1992) estimated unilateral market power in the British electricity spot market by assuming a non-cooperative game. Other theoretical contributions have implemented models that incorporate other assumptions. Newbery (1998) extended the model to the case of forward contracts; Athey and Haile (2006), Guerre et al. (2000), Wilson (1979) and Wolfram (1999) modelled the suppliers’ bidding behavior assuming a multi-unit auction. In this case, the main source of market power is the strategy of witholding capacity: bidders may hide/obscure their real valuations, inflating their bids after the first one, to increase their revenues for all infra-marginal units dispatched.²

A uniform price auction market is also analyzed by Cramton (2004) and Wolak (2003). Wolak (2003) estimated suppliers’ unilateral market power computing the Lerner index from the elasticity of residual demand. Cramton (2004) instead investigated the suppliers’ profit by maximizing bidding and the incentive to exercise unilateral market power under different conditions (the presence of forward contracts, the equilibrium price, the market structure).

The crucial role of firm behavior, and especially deviations from optimal competitive behavior have received increasing attention by scholars [see among others Borenstein et al. (2000, 2002), Ciarreta and Espinosa (2010), Hortaçsu and Puller (2008), Joskow and Kohn (2002), Mansur (2008); Senthilvadivu et al. (2019) and Wolfram (1999)].

These analyses refer to the so-called benchmark approach and were based on the comparison between the simulated outcomes of a competitive market and the actual outcomes. Borenstein et al. (2000, 2002) used market-level data (the market clearing prices and quantities traded) to estimate the suppliers’ market power in the restructured California electricity market. First, they simulated marginal industry costs and the competitive market, where no generator has the ability to exercise market power. Second, they compared these simulated prices with actual prices, using the Lerner index computed at industry level. Joskow and Kohn (2002) implemented this simulation model to estimate marginal costs, taking into better account emission allowance costs. Hortaçsu and Puller (2008) examined the bidding behavior of firms in the Texas electricity spot market. Assuming a non-cooperative game, the authors measured unilateral market power by comparing actual bidding behaviors to theoretical benchmarks. Ciarreta and Espinosa (2010) computed the lower bound measure of generators’ market power in the Spanish day-ahead electricity auction using hourly data. Computation was based on the behavioral differences between strategic generators with a high market share and small firms with competitive behavior. Senthilvadivu et al. (2019) simulated bidding strategy problems in the electricity market using a hybrid algorithm technique to address the problem of optimizing supplier and consumer accounts due to network constraints, the actual market-clearing price and the power generation limit.

The benchmark approach is effective in evaluating welfare losses as deviations from outcomes in a competitive market. However, it is less informative about the specific manifestations of market power. Other inefficiencies may impact market outcomes as highlighted, among others, by Harvey and Hogan (2002), such as start-up and minimum load costs, emission allowances, environmental constraints, outages, hydro-power availability, degree of vertical integration, and transmission congestion. For the Italian market, this issue (which issue?—these issues?) has been considered by Boffa et al. (2010), Bigerna et al. (2016a) and Sapio and Spagnolo (2016), who highlighted the relevance of transmission constraints on the rise in electricity prices. Lundin and Tangeras (2020) estimate the extent of market power in the Nord Pool. Lynch et al. (2021) and Teirila and Ritz (2019) analyze the effects of market power and strategic behavior in the Irish market. Newbery et al. (2018) propose a comprehensve analysis of the welfare and efficiency characteristics that are desirable for market design. Brehm and Zhang (2021) and Hortaçsu et al. (2019) discuss how market power reduces efficiency in the ERCOT Texas market. Marshall et al. (2021) discuss alternative measures for monitorng market power in Australia.

This study follows the benchmark approach and focuses on the bids of both suppliers and purchasers, avoiding restricted assumptions on the marginal cost function. Indeed, the market equilibrium computation requires tackling some issues. As shown in Athey and Haile (2006), in a sealed bid price auction, identifying the best equilibrium function (BEF) of each agent may be impractical if the distribution of all scheduled bids is not observed. We overcome this issue by using the real bid data of each unit-plant and comparing the actual bidding behavior to the theoretical competitive benchmark. In this context the best bidding strategy is unequivocally identified, since all the bids are independent (Borenstein et al., 2000). Operationally, we use unit-specific bid data for suppliers and purchasers, measuring both oligopolistic and oligopsonistic market powers. For each agent, the empirical residual demand (supply) curve can be derived to compute the Lerner index, i.e. the mark-up (mark-down) with respect to the competitive bid (Bigerna & Bollino, 2016; Wolak, 2003). Subsequently, we compare actual market equilibrium with theoretical, competitive and simulated outcomes to measure welfare loss in accordance with the competitive benchmark approach. The crucial assumption is that market power can be exercised in both demand and supply sides, which differentiates this study from the traditional literature.

Electricity markets are far from static and new challenges are emerging with the ongoing transformation of the electricity industry. The main forces driving change are the expansion of renewables, demand response, distributed generation, smart homes, and battery storage. Several authors have discussed these current challenges [see e.g. Cramton (2017), Joskow (2019) and Newbery et al. (2018)] and most of them have identified that there is considerable room for improvement in current policies to better address changes in the power Markets, and policy makers are invited to define new and extended regulatory interventions to ensure the security of supply and decarbonization. This study aims to provide policy makers with a ready-to-use analytical instrument tailored to these new challenges, that is, when new market designs change the market structure giving rise to anticompetitive behaviors. The proposed measurement of market power and the subsequent loss of welfare helps policy makers design and evaluate new policy interventions: until now regulators have not really tackled this issue. The present market monitoring tools³ are limited to computing the traditional herfindahl–hirschman (HH) index (Table 1) and its variants such as the HH pivotal or the new index NERSI proposed by Marshall et al. (2021) for the Australian electricity market. Other measures include the residual supplier index, (Hakam, 2019), the return on withholding capacity, (Bataille et al., 2019), imposition of price caps, (Moutinho et al., 2014), reccomendation to use demand side management, (Yoo et al., 2017) and real time market schemes, (Woo et al., 2019). Our proposal is to use a behavioral model with sound theoretical foundations to measure the exercise of market power.

Table 1 HH Index by zones and load periods

3 Market structure monitoring

The widespread wave of power market reforms has also involved the Italian electricity industry where generation has shifted from a state-owned monopoly to competitive companies, and most of the electricity has been allocated through the Day-ahead market. The day-ahead electricity market is a short term hedge market where hourly blocks of electricity are exchanged one day in advance of the actual physical delivery of power.

In this environment, the injection/withdraw schedules for the next day are the result of a double auction mechanism where suppliers and purchasers submit simple bids for each hour of the day ahead. The “simple” bid format consists of a pair of (hourly) values: quantity (MWh) and price (euro/MWh). In this way each selling/buying participant defines its offer/demand curve.⁴

Bids are managed ‘centrally’ by the trasmission system operator (TSO) that clears the market by finding the equilibrium price and the injection/withdraw programs (commitments to supply and to purchase electricity) according to the economic merit order.⁵ Suppliers get the uniform equilibrium price and dispatch the generators output appropriately, while purchasers pay the National Single Price (PUN).⁶ When programs exeeed capacity constraints, the Italian power system is segmented into zones that record different zonal SMP, the exporting zones, upstream of the constraints with lower prices, and the importing zones, downstream of the constraint with higher prices.⁷

Table 2 Relative frequency of zone segmentation (%): years 2004–2020

In 2013 and 2014 congestions split the electricity market into two-zones roughly 60% of times, and the division between Sicily and mainland Italy represents more than 50% of the total hours (Table 2). Since 2016, the Sorgente Rizziconi undersea cable has dramatically reduced the number of hours in which Sicily is separated from the rest of Italy, which in 2019 occurred only for 25% of total hours.

For this reason, our empirical data consists of all the hours in which a single market occurs, or there is a split of Sicily from the mainland. These represent 65.5% and 62.9% of the total hours in 2013 and 2014, respectively.

In 2005 DAM became fully operational with the participation of almost 1,000 main generation units on the supply side, and about 100 purchasers on the demand side. This included big industries, traders, retailers, and high energy-intensive companies, together with single buyers, and the state company which guarantees to cover the demand from non-eligible customers.

We assume that unilateral market power may be exercised by the main operators trading larger quantities of hourly electricity. Other participants constitute the competitive fringe.

4 Theory and methods

4.1 The theoretical model

We measure the unilateral market power assuming a standard Cournot model where operators are divided in two categories: agents with the highest market shares acting as strategic players, and agents belonging to the competitive fringe acting as price-takers. This procedure is applied to both market sides, supply and demand.

Starting with the supply side, for any hour of the day, each supplier submits a bid to maximize its profit function ${\pi }_{ih}$, given the price p_h, its marginal cost curve $M{C}_{ih}$, its expectations of market demand $D({p}_{h})$, its expectations of the competitors’ supply curves ${Q}_{-ih}^{\star }={\sum }_{j\ne i}{q}_{jh}^{\star }$ and, in turn, its expectations of its residual demand curve $R{D}_{ih}({p}_{h})$.

For each supplier, the optimal bidding strategy is represented by the pair of quantity and price according to which the marginal revenues associated with the period’s demand realization equals the short-run marginal cost.

Formally, the profit maximization problem faced by supplier $i$ becomes to choose:

$$q_{Cih} = argmax\left\{ {\pi_{ih} \left( {q_{Cih} } \right)} \right\} = argmax{ }\left[ {p_{h} \left( {Q_{ - ih}^{ * } ,q_{Cih} } \right) - MC_{ih} } \right]q_{Cih}$$

(1)

where ${q}_{Cih}$ is the residual demand faced by firm $i$ at time h, net of contract cover⁸; ${\pi }_{ih}$ is the portion of variable profits that are affected by the DAM bidding strategy.⁹

Applying the first order condition we derive the identity:

$$\frac{{p_{h} - MC_{ih} }}{{p_{h} }} = \frac{{\partial p_{h} }}{{\partial q_{Cih} }}\frac{{q_{Cih} }}{{p_{h} }} = \frac{1}{{\epsilon_{{RD_{Cih} }} }}$$

(2)

where $\frac{\partial {p}_{h}}{\partial {q}_{Cih}}$ is the inverse demand derivative and ${\epsilon }_{R{D}_{Cih}}=\frac{\partial {q}_{Cih}}{\partial {p}_{h}}\cdot \frac{{p}_{h}}{{q}_{Cih}}$ is the elasticity of the residual demand function faced by firm $i$.

Note that in the Italian electricity market, the forward contracts have not developed much. A reason can be found in the regulatory framework, which sets standard offer regime contracts. These are implicit reference benchmarks for the whole market throughout time, thus weakening the need for independent retailers to hedge long term positions. Forward contracts are sold in the Italian forward electricity market (MTE) but are unobservable. Following Reguant (2014), we assume that agents hedge a given percentage of their output and empirically we find that the overall quantity traded in the MTE was about 3.86% in 2013 and 9.90% in 2014 of the total electricity sold in the spot market.¹⁰ For this reason, we infer that the inverse elasticity of the residual demand curve $R{D}_{ih}$ slightly overestimates the incentive to raise market prices.

Rearranging the equation allows measurement of the market power exercised by each player using the Lerner index ($L{S}_{ih}$) derived as the inverse of the residual demand elasticity faced by firm $i$ in hour$h$ ¹¹;

$$LS_{ih} = \frac{{p_{h} - MC_{ih} }}{{p_{h} }} = \frac{1}{{\epsilon_{{RD_{Cih} }} }}$$

(3)

Symmetrically, in line with what we did on the supply side, we derive the Lerner index in the oligopsony market, starting from the optimizing strategy applied by purchasers:

$$LD_{ih} = \frac{{z_{h} \frac{{\partial f_{i} q_{ih}^{D} }}{{\partial q_{ih}^{D} }} - p_{h} }}{{p_{h} }} = - \frac{{\partial p_{h} }}{{\partial q_{Cih}^{D} }}\frac{{q_{Cih}^{D} }}{{p_{h} }} = \frac{1}{{\epsilon_{{RS_{ih} }} }}$$

(4)

where ${\epsilon }_{R{S}_{ih}}$ denotes the elasticity of the residual supply faced by buyer i in hour h, $\frac{\partial {f}_{i}{q}_{ih}^{D}}{\partial {q}_{ih}^{D}}$ is the electricity marginal product and z_h is the selling price for the purchaser. $L{D}_{ih}$ is, therefore, the inverse of this elasticity and represents a Lerner measure of the buyer’s mark-down over its willingness to pay, that is, a measure of the unilateral market power of buyer$j$.¹² The elasticity of the residual supply incorporates all relevant information on how a change in buyer $j$’s quantity would change the market price by affecting the behavior of other buyers. The residual supply curves faced by fringe buyers are assumed to be vertical, inhibiting the exercise of market power and involving a zero Lerner.

Since market demand and supply curves (and the corresponding residual demand and supply curves) are step functions, computing the elasticity at a given point requires the finite difference approach using the following formula (Wolak, 2003):

$$\epsilon_{Fih} = \frac{{RF_{ih} \left( {p_{h} \left( {high} \right)} \right) - RF_{ih} \left( {p_{h} \left( {low} \right)} \right)}}{{p_{h} \left( {high} \right) - p_{h} \left( {low} \right)}} \times \frac{{p_{h} \left( {high} \right) + p_{h} (\left( {low} \right)}}{{RF_{ih} \left( {p_{h} \left( {high} \right)} \right) + RF_{ih} \left( {p_{h} \left( {low} \right)} \right)}}$$

(5)

where $F=D, S$ denotes the demand and supply step functions respectively, $R{F}_{ih}({p}_{h}(high))$ and $R{F}_{ih}({p}_{h}(low))$ are respectively the lower and upper limits on the steps of the residual curve for quantities, as ${p}_{h}(low)$ and ${p}_{h}(high)$ are the lower and upper limits on prices.

Equations (4) and (5) highlight the trade-off faced by suppliers and buyers when they exercise market power. On the supply side, the incentive to deviate from the competitive equilibrium is inversely related to the elasticity in the competitors’ supply and that of market demand. Inflating bid prices involves increasing the risk that either other competitors will step in to serve the demand, or that buyers will curtail their demand; i.e., there is a trade-off between the marginal gains from a higher bid curve against marginal losses from foregone output. The same line of reasoning applies to the demand side.

4.2 Empirical methodology

We constructed the aggregate market supply and demand functions by designing a five-step procedure using the individual bids of market participants. First, for each hour we divided the sample distribution of prices into 30 quantiles (i.e., price observations are divided into 30 groups of the same size) in every year. The cutoff point of each quantile is used as the break point to aggregate both hourly supply and demand. This obviously results in a numerical approximation of the supply and demand into 30-step functions for every hour. Given that there are on average about 50 bids on the demand side and 200 bids on the supply side, there are no empty steps, so that the two step-functions are acceptably accurate approximations of the true market behavior.¹³

Second, we constructed the residual demand and supply curves for each agent, applying the Cournot model to derive its BEF. Unit level bids allow derivation of the empirical residual demand (supply) curve for each agent, to unequivocally identify its BEF, and to directly compute the residual demand (supply) elasticity.

Third, for each strategic firm i and buyer j we recovered from their BEF the corresponding $L{S}_{i}$ and $L{D}_{j}$ expressed by Eqs. (3) and (4), respectively. Using the formula of arc elasticity in Eq. (5), the two indexes are computed at the quantile of the price distribution where the clearing price lies. In this way, for each player we obtain an empirical distribution of the Lerner index as a function of the price level.¹⁴ In this framework, the inverse of this elasticity can be thought of as the Lerner index measuring the incentive to bid below or above the competitive benchmark.

Fourth, we used the derived mark-up (mark-down) to correct the price of the bid submitted by each strategic firm in the supply (demand) side (Cramton, 2004). These new bids are purged from oligopolistic/oligopsonistc market power, so they represent the BEF that would result in a competitive market. As would be done by a Walrasian auctioneer, we re-order the corrected bids to sell (purchase) so as to construct a new merit order ascending (descending), and we recover the supply (demand) curves to derive a new market equilibrium. This new equilibrium is the intersection of the simulated demand and supply derived from competitive behavior and can therefore be viewed as the competitive equilibrium that would prevail when agents cannot exercise unilateral market power.¹⁵

As an example, we show the procedure using two real observations randomly extracted from our sample in Figs. 1 and 2, referring to 13 January 2013, 7 a.m. (off-peak hour) and 1 p.m (peak hour). In both figures, the real supply and demand step functions are labeled “Empirical Supply” and “Empirical Demand” and are the blue and red lines, respectively. The “Simulated supply” and “Simulated demand” funtions are instead the green and yellow lines, respectively.

Fig. 1

Empirical and simulated market equilibriums. Off-peak hour. Observations related to the empirical curves refers to 7 a.m.—12 January 2013

Fig. 2

Empirical and simulated market equilibriums. Peak hour. Observations related to the empirical curves refers to 1 p.m. 12 January 2013

Looking at Fig. 1, we see the original historical equilibrium price at 50 euro/MWh (point A labelled “Old Equilibrium”). First, we corrected the suppliers' bids to remove only the oligopolistic strategic behavior. We then recomputed the merit order and the SMP. This entails a reduction (or non-increase) of the equilibrium SMP with respect to the historical SMP, at point B roughly equal to 45 euro/MWh, because the aggregate supply function can only shift downward (or remain unchanged). The interpolation of the adjusted supplier bids near the new equilibrium allows derivation of the slope and the intercept of the new supply curve. Second, we also removed strategic demand behavior, which entails the upward (non-downward) shift of the demand curve. The slope and intercept of the new demand function were estimated using the interpolation procedure. This new clearing price was then derived at point C (labelled “Simulated Equilibrium), at the intersection of the two simulated curves. Compared to point B, the new clearing price slightly increased, shifting from 45 to 47 euro/mWh. This is the estimated competitive equilibrium, which we deem the ideal competitive electricity market. The market configuration of the peak hour in Fig. 2 has similar movements. The clearing price shifts from its historical value of 60 euro/MWh to the simulated final value equal to 59 euro/MWh.

Comparing the two alternative equilibriums (points B and C in Fig. 1) to the historical outcomes (points A) assesses the loss in efficiency measured in terms of social welfare. We are interested in investigating how much strategic behaviors affect the total welfare, and what their re-distributive effects are. We computed the social welfare resulting from the different market outcomes as the sum of producer and consumer surplus. Moreover, we computed the weight of the producer and consumer surpluses on the total welfare, showing how these percentages change when we remove first the oligopolistic and then the oligopsonistic market power.

Consumer surplus is the difference between what the purchaser is willing to pay for electricity, and the current clearing prices. The consumer surplus is computed in two different ways. First, we assume that the maximum willingness to pay is the institutional price-cap imposed by the Energy Authority, equal to 3000 euro/MWh; the corresponding consumer surplus is measured as the area below the downward-sloping demand curve and above the equilibrium market price (depicted with a horizontal line drawn between the y-axis and the demand curve). Second, we consider the maximum willingness to pay, as given by the maximum accepted price bid in each hour. In this case, the resulting consumer surplus is smaller than in the previous case, because the vertex of the area below the demand curve (the maximum price accepted) is usually lower than the price-cap. In both cases, consumer surplus increases as the equilibrium price falls and vice-versa.

The producer surplus is the extra-profit gained when the market price is higher than the marginal cost of production. Graphically, the producer surplus for all suppliers in the market is the area below the equilibrium price line and above the aggregated supply curve. The size of the producer surplus increases as market price increases and vice-versa.

5 Results and discussions

We summarize the estimation results in Table 3. We show the average historical equilibrium prices and quantities (col. 1), their changes after correction to remove market power on the supply curves (col. 2), on demand curves (col. 3) and the total effect (col. 4).

Table 3 Hourly current clearing prices and quantities and their estimated variations: years 2013–2014

The results in Table 3 highlight that the market power exercised in both supply and demand sides mainly affects clearing prices rather than the quantities traded: for excample, when quantities increase by 0.77%, the clearing prices fall by 3.29%. These relative magnitudes reflect the empirically estimated low elasticity levels of both the supply and demand curves.

The deviation from the competitive equilibrium is broken down into two components, namely, the effects of the strategic behavior of suppliers and purchasers. Note that oligopolistic strategic behavior affects the market equilibrium more than the oligopsonistic one does. Removing the oligopolistic market power alone resulted in a decrease of 5.88% and 3.16% in 2013 and 2014 respectively, whilst the increase in the quantities traded was marginal. On the other hand, when we adjust the purchasers’ bids to remove their oligopsonistic market power, the changes in the clearing prices are meaningful, but lower—on average 1.33%.

These effects are more pronounced in 2013 than in 2014. In 2013, prices are on average lower by 4.58%, while the equilibrium quantities are on average higher by 0.96%, whereas in 2014 prices are lower by 2.01% and quantities higher by 0.59%.

There is no well-defined pattern if we distinguish between the results for peak hours (from 8:00 AM to 7:00 PM) and off-peak hours (from 8:00 PM to 7:00 AM). Looking at the figures for 2013 and 2014 (Tables 4 and 5), the supply side market power affects the equilibrium values for off-peak hours, where deviations from the competitive equilibrium are slightly larger than those referring to peak load periods. Even if the oligopsonistic effects seem greater during peak hours, they do not affect the overall pattern in either years. Note that in the central part of the year 2014, there is a peak price lower than the off-peak price: the progressive increase of renewables leads to this price reversal, Bigerna et al. (2016b).

Table 4 Hourly current clearing prices and quantities and their estimated variations by quarters and load periods: years 2013

Table 5 Hourly current clearing prices and quantities and their estimated variations by quarters and load periods: years 2014

Table 6 shows the total social welfare and the breakdown of the total welfare between consumer and producer surplus (col. 1), in order to analyze the re-distributive effects (between wholesalers and suppliers) and the changes caused by the shift in the new competitive market structure (changes due to: removal of only suppliers’ market power in col 2; removal of only purchasers market power in col. 3; total effect in col. 4). When we remove market power the welfare increases, and the total average welfare gain lies between 0.55% in 2013 and 0.39% in 2014 (Table 6, col. 4). Breaking down the overall market power effect in the oligopolistic and oligopsonistic markets shows that the increment in total welfare due to the suppliers’ market power is negligible, ranging between 0.19 and 0.07%, while the increase in social welfare due to the strategic behavior of wholesalers is larger, between 0.31 and 0.36%. This presents a new result for the Italian market, showing that it is important for the regulator to monitor also the demand side.

Table 6 Hourly current welfare, consumers’ and producers’ surplus shares and their estimated variations: years 2013–2014

The total surplus is mainly held by wholesalers, whose average share amounts to around 81%. Removing the suppliers’ strategic behavior slightly increases the wholesalers’ shares by a percentage ranging between 0.7 and 1.7%; changes in consumer surplus are even more marginal when we neutralize the oligopsonistic market power, where the deltas settle around 0.33%.

Compared to the average share held by consumers, the producers’ surplus undergoes major changes, decreasing by 5.5% in 2013, and by 2.56% in 2014; this is essentially caused by the greater elasticity of the supply curve. When we correct the bid prices on the supply side only, the suppliers’ shares decrease considerably, by 3.59% and 6.69%. On the other hand, when we also eliminate the oligopsonistic market power, the producers shares increase by 1.06% and 1.64%. Note that when we take into account the effects of purchasers’ strategic behavior, the reduction in the share of consumers is lower than the increase in the share of the suppliers, given that the supply curve is more elastic than the demand curve.

This analysis is broken down by quarters and peak-off-peak periods in Tables 7 and 8. Social welfare increases more during off-peak hours in both periods, confirming the previous findings. The overall effect on the surplus shares (for both purchasers and suppliers) is the same as it was in total welfare—the shares of purchasers increase more during off-peak hours (by about 1.45%) while shares of suppliers decrease on average by 5.61%.

Table 7 Hourly current welfare, consumers’ and producers’ surplus shares and their estimated variations by quarters and load periods: year 2013

Table 8 Hourly current welfare, consumers’ and producers’ surplus shares and their estimated variations by quarters and load periods: year 2014

These results support two main considerations. Firstly we should note that our analysis of demand and supply on the DAM only includes the wholesale sector of the market, accounting for about 2/3 of the final price paid by consumers, as the rest is taxation, compensation for retail services and so on. Therefore, our analysis relates to the total effect on the supply side and the largest portion of the effects on the final consumer. Secondly, our analysis confirms the need for regulators to also monitor competitive behavior during off-peak hours, partly changing the conventional view that non-competitive behavior is more likely when demand peaks and thus tight supply brings about a higher mark-up.

Our results show that operators are apt to exercise market power i.e., deviation from perfect competition, also in non-tight market conditions. For instance, Mansur (2008) reports that “estimates of deadweight loss are small” and values of welfare loss are up to 5% for the JPM in the pre-competitive market period. During the period in the Italian case considered here, the average hourly volume of generation was around 32 GWh and the average hourly price around 58 Euro/MWh. From this we can infer that on an annual basis, the estimated effect on welfare was about 80 million euros.

6 Conclusions

This paper provides an empirical measurement of the market power exercised both in the supply and demand side and of the dead-weight loss of welfare in the Italian electricity market due to the strategic behavior deviating from the competitive equilibrium. The paper offers a counterfactual simulation of the competitive market solution, correcting the historical bids with a measure of the market power of main suppliers and purchasers, the Lerner Index. The simulation model implemented recovers the competitive equilibrium, taking into account forward contracts and the presence of a competitive fringe, allowing us to measure the deviation of the actual market equilibrium from that of the counterfactual competitive one.

These Results highlight that the wholesale Italian electricity market recorded a welfare dead-weight loss in both 2013 and 2014. The deviation from competitive equilibrium appears to be more pronounced in clearing prices rather than exchange quantities: counterfactual competitive prices are on average 3.29% lower than the recorded clearing prices, while the divergence between historical and competitive quantities is about 0.77%.

The gap/distance from the competitive equilibrium is essentially due to the oligopolistic behavior of strategic suppliers, but our results have highlighted that even purchasers are able to hold some market power. The lower prices and the higher sales volumes derived in the simulated competitive framework would retrieve the social welfare dead-weight loss, which is on average 0.46%. This latter loss is greater during off-peak hours when market power is increased [because in the evening there is less supply of renewables, as shown by Bigerna et al. (2016b)] and strategic players have a greater ability to deviate from competitive bids.

Removing market power has essentially indicated that purchasers would increase their share of welfare, which over the two years increases on average by 0.92%. On the other hand, the supply side of market recorded larger losses, since its yearly average share of total welfare decreased by 4.03%.

We are aware that this study has some limitations. We assume a Cournot behavior, but other strategies could be adopted by market agents. In addition, we have not considered in our welfare measurement the existence of environmental externalities. We also believe that analyzing strategic behavior for forward contracts constitutes a promising line for future research.

In conclusion, despite these methodological limitations and recent policy achievements toward better competitive conditions, which have increased efficiency and reduced prices in the electricity market, these results highlight that there is still a deficit regarding the attainment of optimum competitive market conditions in order to maximize consumers’ welfare. Regulators can complement their market monitoring actions utilizing the tool developed in this paper. They need to measure potential distortions from the competitive equilibrium to be credible in their strive towards greater benefits from the liberalization of the electricity market, in order to effectively enact mechanisms that mitigate or avoid market power. The non-regulated component of tariffs, i.e. the electricity price, is still at risk of being manipulated by the strategic players in the wholesale electricity market. We advocate that liberalizing the electricity market will be completed when this likelihood of manipulation is minimized or totally removed, thus rendering an electricity price equal to its marginal cost to the consumer.

Appendix: Theoritcal model and Robustness analysis (to be shown ONLINE)

1.1 Theoretical model

The theoretical model represents then behavior of the market participants.On the supply side there are large generators equipped with fossil fuel plants and small genrators with renewable energy sources, such as photoviltaic, wind and bionass, Geothermal genration in Italy has been historically develpoed by ENEL.

On the demand side, there are some energy intensive firm buying directly in th ewholesale market and then several retailers, some large (eth subsidiaries of large intgrartd operotors) and many smll independent retailers, which blossomed after the libelarization of the beginnig of the 2000’s.

On the supply side, our theoeretical model allows, from Eq. (2) in the text, defining ${q}_{Cih}({p}_{h})={q}_{ih}({p}_{h})-{q}_{iC}$ as the net of contract cover residual demand faced by firm $i$,¹⁶ to write the portion of profits that are affected by the DAM bidding strategy as follows:

$$\pi_{ih}^{ * } \left( {q_{Cih} } \right) = \left[ {p_{h} \left( {Q_{ - ih} ;q_{Cih} } \right) - MC_{ih} } \right]q_{Cih}$$

(6)

Showing that only ${q}_{Cih}$ is the only relevant quantity determining the incentive of the suppliers to drive the market price.

Formally, the profit maximization problem faced by supplier $i$ becomes to choose:

$$q_{Cih} = argmax\left\{ {\pi_{ih} \left( {q_{Cih} } \right)} \right\} = \left[ {p_{h} \left( {Q_{ - ih}^{ * } ,q_{Cih} } \right) - MC_{ih} } \right]q_{Cih}$$

(7)

Applying the first order condition we derive the identity:

$$\frac{{\partial \pi_{ih} }}{{\partial q_{Cih} }} = \left[ {\frac{{\partial p_{h} }}{{\partial q_{Cih} }}} \right]q_{Cih} + p_{h} - MC_{ih} = 0$$

(8)

where $\frac{\partial {p}_{h}}{\partial {q}_{Cih}}$ is the inverse demand derivative. From the first order condition we derived the identity¹⁷:

$$\frac{{p_{h} - MC_{ih} }}{{p_{h} }} = \frac{{\partial p_{h} }}{{\partial q_{Cih} }}\frac{{q_{Cih} }}{{p_{h} }} = \frac{1}{{\epsilon_{{RD_{Cih} }} }}$$

(9)

where ${\epsilon }_{R{D}_{Cih}}=\frac{\partial {q}_{Cih}}{\partial {p}_{h}}\cdot \frac{{p}_{h}}{{q}_{Cih}}$ is the elasticity of the residual demand function faced by firm $i$ on the supply side.

On the demand side, recalling that the residual supply $R{S}_{Cih}={q}_{Cih}^{D}={q}_{ih}^{D}-{q}_{iC}^{D}$ is the best response quantity purchased with contract cover, the profit maximization problem buyer i has to solve is give by the following first order conditions:

$$\frac{{\partial \pi \left( {q_{Cih} } \right)}}{{\partial q_{Cih}^{D} }} = z_{h} \frac{{\partial f_{i} \left( {q_{ih}^{D} } \right)}}{{\partial q_{ih}^{D} }}\frac{{\partial q_{ih}^{D} }}{{\partial q_{Cih}^{D} }} - \left[ {\frac{{\partial p_{h} }}{{\partial q_{Cih}^{D} }}} \right]q_{Cih}^{D} - p_{h} = 0$$

(10)

where ${p}_{h}$ is the industry price for electricity, $\frac{\partial {p}_{h}}{\partial {q}_{ih}^{D}}$ is the industry factor demand derivative, z_h is the purhcaser selling price, $\frac{\partial {f}_{i}({q}_{ih}^{D})}{\partial {q}_{ih}^{D}}$ is the electricity marginal product.

The elasticity of residual supply incorporates all relevant information to characterize how a change in buyer $j$’s quantity would change the market price by affecting the behavior of other buyers. The residual supply curves faced by fringe buyers are supposed to be vertical, inhibiting the exercise of market power and involving a zero Lerner.

1.2 Robustness analysis

All the data are available upon request at a repository at the University of Perugia.

This section tests the robustness of our analysis, by comparing previous results to alternative estimation methods of the slope and intercept of the demand and supply curves, as described around Fig. 1 in the text.

The first estimation method we applied involves non-parametric local regression techniques where the supply and demand predictions are derived using kernel weights leading to a much smoother regression function. The derived coefficients are the LOWESS estimates that minimize the weighted average least square, where the weights are the tricubic kernel weights down-weighting large residuals. Compared to the traditional kernel regression estimator, the LOWESS estimator is more robust to outliers using a variable bandwidth. The graphs below (Fig. 3) depict the linear predictors and the LOWESS estimators for both, demand and supply curves. 

Fig. 3

Lowess Demand and Supply Linear Prediction. First Column: Lowess Demand Linear Prediction: a 2013—Off-Peak Hours; b 2013—Peak Hours; c 2014—Off-Peak Hours: d 2014—Off-Peak Hours. Second Column: Lowess Supply Linear Prediction: a 2013—Off-Peak Hours; b 2013—Peak Hours; c 2014—Off-Peak Hours: d 2014—Off-Peak Hours

The line plots refer to the OLS estimators, while the scatter plots refer to the LOWESS estimators. Both kinds of predictions were performed for the old curves, and for the new supply and demand functions, adjusted by the Lerner indexes.

Since the two samples are not independent the diagnostic procedure involves performing the Wilcoxon signed rank test, that is a paired difference test checking if the two population mean ranks of the repeated measurements on the single sample are the same.

The null hypothesis, and the linear prediction for the supply and demand curves adjusted and not for the Lerner index, do not show significant differences with the LOWESS predictions, and they come from the same distribution. The null hypothesis assumes that the two matched samples (the linear and the LOWESS predictions) arise from the same distribution, and the difference between the pairs follows a symmetric non-canonical distribution around zero. As the sample size increases, the non-conventional distribution of the test statistic converges to the normal distribution with mean zero. Therefore, the test statistics have to show absolute values that are roughly smaller than 1,96 in order not to fall in the rejection region with a significance level of 5%. In both the years, the test infers that in roughly 90% of the hours, the linear and the LOWESS predictions are derived from the same distributions. We reported the main descriptive values for the Wilcoxon test statistics that we computed for 2013 and 2014 in Tables 9 and10.

Table 9 Wilcoxon test statistic linear vs lowess predictions

Table 10 Wilcoxon test statistic linear vs lowess predictions

Second, we perform a robust regression using the iteratively re-weighted least square method, where the weight assigned to each observation depends on its residual.¹⁸ This alternative estimation method, instead, begins by fitting the regression, and then calculating Cook’s distance, and then excluding observations whose distances are larger than 1. Thereafter, the regression is performed iteratively. The iteration starts using the Huber weighting function until the convergence, then from that residual, the iteration computes the estimator using the bi-weight function. The program uses both, the Hubert and the bi-weighted functions since the first ones have a problem dealing with outliers data, while the second ones may have multiple solutions or come short of convergence.

After running the robust regression, we compared them with our previous results. The derived estimates of demand and supply curves are then compared to the linear predictions using the Wilcoxon signed rank test again. Even for these estimations, we depict the two different prediction curves. Again, the line plots refer to the OLS estimators, while the scatter plots refer to the re-weighted least square estimators. Both kinds of predictions were performed for the old curves and the new supply and demand functions adjusted by the Lerner indexes (Figs. 4, 5, 6 and 7).

Fig. 4

Kernel density estimates of the Lerner Indexes computed for the main strategic suppliers. Rows: a A2A; b Edison; c Enel; d ENI; e Sorgenia. First column: 2013; Second column: 2014. Kernel function—Epanechnikov

Fig. 5

Kernel density estimates of the Lerner Indexes computed for the main strategic purchaser. Rows: a Alpiq; b Danske; c EDF; d Electrade; e GDF. First column: 2013. Second column: 2014. Kernel function—Epanechnikov

Fig. 6

Average Lerner Indexes for the main strategic suppliers according to the price quantiles. Rows: a A2A; b Edison; c Enel; d ENI; e Sorgenia. First column: 2013. Second column: 2014

Fig. 7

Average Lerner Indexes for the main strategic purchasers according to the price quantiles. Rows: a Alpiq; b Danske; c EDF; d Electrade; e GDF. First column: 2013. Second column: 2014

As before, the re-weighted predictions are compared with the linear supply and demand curves (see Tables 11 and 12).

Table 11 Wilcoxon test statistic: linear vs robust predictions

Table 12 Wilcoxon test statistic: linear vs robust predictions

Conclusions do not change, and the Wilcoxon tests confirm that the linear and the re-weighted predictions are sourced from the same distribution.