1 Introduction

By 1999, the electricity and natural gas sector in Great Britain was open to retail competition, including to all residential customers. Over the next decade, the energy regulator Ofgem (Office of Gas and Electricity Markets) removed an initial transitional price cap, addressed a variety of concerns, and reported positively on the evolution of competition.

Then in 2008 Ofgem began a series of regulatory interventions—notably a non-discrimination condition and then imposing a restriction to four “simple tariffs”—that reversed its previous policy of minimal intervention in the market. In 2014 Ofgem referred the energy market for investigation by the Competition and Markets Authority (CMA). In its 2016 Final Report the CMA criticised Ofgem’s two regulatory interventions but made recommendations for different interventions. There was a demand for more substantial action. In 2018 an Act of Parliament required Ofgem to introduce a widespread residential energy tariff cap.

Retail energy policy has thus reversed. Britain is not alone here. Other jurisdictions including Australia, New Zealand and New York are also taking or contemplating potentially radical interventions.

The opening of regulated monopoly markets to competition spurred numerous theoretical analyses of price discrimination, including in markets with switching costs. For early contributions see (e.g.) Borenstein (1985), Klemperer (1987a, b, 1995), Brennan (1990, 2007), Armstrong and Vickers (1993), Borts (1998). There has been increasing study of competitive retail energy markets and their successes and limitations, both in GB e.g. Green and McDaniel (1998), Otero and Waddams Price (2001), Littlechild (2002), Giulietti et al. (2005), Hviid and Price (2012), Waddams Price and Zhu (2016), and elsewhere e.g. Joskow (2005), Littlechild (2006), Simshauser (2018), Simshauser and Whish-Wilson (2017). Some papers have focused on customers’ limitations e.g. Wilson and Waddams Price (2010), Kleit et al. (2012), Giulietti et al. (2014), Hortascu et al. (2017). Others have explored regulatory policy e.g. Joskow (2000), Littlechild (2003, 2014a, 2016, 2018b), Waddams Price (2018).

The present paper focuses on retail regulatory policy in Britain, and more precisely the changes and reasons for changes over the second decade compared to the first. The main body of the paper summarises Ofgem’s regulatory interventions after 2008 and the debates about them. Many regulatory economists including myself felt strongly about these policies. The paper draws upon their responses to Ofgem and CMA consultations. The paper conjectures possible factors that might have led Ofgem to change its previous policy. Finally, it examines what the CMA had to say about these interventions and about the regulatory framework that led to what the CMA considered were inappropriate interventions.

2 Ofgem’s regulatory policy until 2008

The prospective privatisation, restructuring and introduction of wholesale competition into the GB nationalised electricity sector raised the question of how the potential benefits of wholesale competition would be transferred to customers. As an alternative to regulating retail prices, the possibility of creating retail competition, via competing private retailers with access to local distribution networks, was mooted. (Beesley and Littlechild 1983) Privatisation arrangements provided for opening the electricity market to all retail customers on a phased basis over the eight years 1990–1998. By the end of that period, the privatised natural gas market, too, was wholly open to retail competition.

A substantial proportion of GB residential customers consume both fuels. Competition was soon based on the former regional electricity monopolists and the former gas monopolist trying to break into each others’ markets by offering “dual fuel” tariffs.Footnote 1

The structure of the energy market evolved over the years following privatisation, particularly around the time of full market opening. Between 1998 and 2002 the 14 incumbent electricity distribution companies, each of which originally had had a regional monopoly, consolidated with some of the generating companies into five vertically integrated retail supply businesses. (Their network businesses were operated and regulated separately.) By 2002 these five electricity suppliers had in aggregate taken nearly 40% of the residential gas market from the incumbent and former monopoly gas supplier British Gas. Meanwhile British Gas (also trading as Centrica) had acquired generation capacity and become the largest residential electricity supplier, with 22% of the residential electricity market. These six suppliers, often referred to as the Big 6, between them accounted for 99 per cent of residential customers. There were occasional new entrants into the residential retail market, many of them transient, and never supplying more than about 1 per cent of the market in aggregate before 2009.

Transitional price caps had been put in place when the residential markets first opened. Ofgem (2002) abolished them because competition was now effective “across all social groups and all methods of payment”. Competition could deliver more benefits for customers than regulation. Indeed, regulation could jeopardise competition, especially regulation of prices to particular customer groups. Ofgem proposed to focus instead on the use of competition and consumer law powers. It outlined a “substantial programme of work to remove continuing obstacles to competition (for example, reviewing the transfer process, tackling misselling, reforming rules for objecting to switches, improving competition for dynamically teleswitched customers, as well as enforcing competition law)”. (Ofgem 2003) It also sought to improve customer information and billing standards. But it imposed essentially no restrictions on what suppliers could offer or charge.

From 2002 to 2007, Ofgem issued further retail market reports fairly regularly, noting the continued development of competition. For example, Ofgem (2007) reported vigorous price competition for all customers; innovation by suppliers in terms of fixed and capped price deals, cheaper online deals and green tariffs; improving customer service; and customer switching rates at their highest in four years. There had been price increases, but on 16 January 2008, responding to Government and public concerns, Ofgem issued a press release headed “Market is sound—Ofgem assures Chancellor”.

3 Ofgem’s energy supply probe 2008

In contrast, on 21 February 2008 Ofgem issued another press release headed “Ofgem launches probe into energy supply markets”. This was said to be “in response to public concern about whether the market is working effectively”.

Ofgem’s Energy Supply Probe Initial Findings in October 2008 noted that household energy bills had more than doubled since 2004. (Ofgem 2008, para 1.1, p 5) But competition was developing well, suppliers competed actively, and there was no evidence of a cartel, although no “competitive fringe” offering a competitive threat to the Big 6 had yet emerged. The level of customer engagement was among the highest of any retail energy market in the world, and higher than in other retail services in the UK. But active consumers were in a minority, and switching might not yet be exerting sufficient pressure on suppliers’ prices.

The Probe did not find excess profits. “1.19 Any assessment of suppliers’ aggregate profitability is fraught with difficulties, given the complexities that result from vertical integration, hedging and the recent volatility in wholesale energy prices.”

However, instances of differential pricing were a concern. Specifically

  • the five former incumbent electricity suppliers charged electricity customers in their former monopoly areas higher prices than comparable “out-of-area” customers, without any cost basis for this premium;

  • suppliers charged higher prices to customers taking only electricity under a single fuel arrangement (around a third of their electricity customers), 4.3 m of whom were not connected to the gas main and so were unable to move to a more competitively priced dual fuel deal;

  • many price differentials between payment methods did not appear to have a cost justification, particularly for those customers who paid by Standard Credit (quarterly in arrears by cash or cheque on receipt of bill) and for some customers paying in advance by prepayment meter (PPM), relative to customers who paid by Direct Debit (to their bank accounts).

Ofgem commented that “1.18 … The ability to price differentially in this way means that pressure on prices in the most competitive segments of the market does not always constrain prices for all other consumers.”

What was the extent of these price differentials? Ofgem took as a benchmark of the competitive price the standard dual-fuel direct debit (DF-DD) product supplied to 10.4 m out-of-area customers, at a price of £1184 per annum for a customer with average usage. By comparison with this benchmark, Ofgem calculated that various groups of other customers paid premiums totalling around £1 bn per annum. Ofgem acknowledged that this annual premium exceeded the average annual margin earned by the Big 6 retail energy businesses between 2005 and 2007. Ofgem did not say what this average margin was. I note that it was transitioning from negative to positive but on average was about zero. (Littlechild 2014b, fn 25) Ofgem concluded that “if these differentials were eroded [as a result of regulatory action], it may be through a re-balancing between prices rather than by a straightforward decrease in price for the most impacted customers.” (para 8.77)

Ofgem then calculated that, to rebalance the differentials while holding total revenue constant, “the annual cost of energy to a DF-DD customer would have to rise by around £40 per customer, or 3.5 per cent on the average dual fuel bill”. Using this higher level of DF-DD price as the new benchmark for assessing differentials, “the impact would fall from £1 billion to around £550 million”. (para 8.78) It said that “Although a rebalancing of this type would not reduce average prices paid by consumers it would disproportionately benefit vulnerable groups. It would also improve the prospects for new entrants and small suppliers because margins for the most price sensitive and active segments would improve to levels that may make market entry profitable.” (para 8.79)

The report analysed and quantified the impact of competition on various groups of customers, especially prepayment meter customers and various vulnerable customers (older, low income and rural). It concluded that many customers were not effectively engaged in the market and that competition was not developing fast enough. It therefore proposed a package of measures to “accelerate the transition” to effective competitive markets. These measures included actions to promote more active customer engagement and to help consumers make well-informed choices (e.g. via more and better information), and to reduce barriers to entry and expansion (e.g. via suppliers publishing segmental accounts and by improving market liquidity).

To address its concern about what it now called “unfair price differentials”, Ofgem proposed a requirement that differences in charges for different payment methods should be cost-reflective. Ofgem was “also considering placing a further new condition in the licences of the Big 6 suppliers that would either impose a prohibition on undue price discrimination or introduce a form of relative price control”. This condition would “seek to ensure that price differentials are objectively justified by cost differences”.

4 Reactions to the probe initial findings

Ofgem’s new thinking was controversial, both in the energy community and within Ofgem. It was not that Ofgem was arguing that introducing competition could reduce the welfare of some customers (cf Brennan 1990, 2007): Ofgem’s declared aim was to accelerate the transition to competition. The finding of price differentials was not new: previous Ofgem reviews had reported this, but had explained how competition was nonetheless protecting all groups of customers.Footnote 2 The concern with vulnerable customers was not new: Ofgem had previously drawn up a Social Action Plan “to ensure efficiency, choice and fairness in the provision of gas and electricity to disadvantaged customers” (Offer/Ofgas 1999); it had explained various additional steps that it was takingFootnote 3; and it now described Ofgem’s “long history of involvement in these issues”, including a lengthy list of current initiatives to help vulnerable customers. (Ofgem 2008, paras 2.34, 9.18) Nor was the possibility of imposing a non-discrimination condition new—Ofgem had repeatedly considered this in assessing whether or how to continue the initial transitional price caps.

What was surprising was that Ofgem was focusing on price differentials when the overwhelming concern was with price increases, and what was different was that Ofgem had previously rejected a non-discrimination condition because it could reduce competition.Footnote 4 Now Ofgem was open to imposing one on grounds of fairness. Although a fair rate of return is embedded in US regulation, the concept of “fairness” itself was more elusive,Footnote 5 and had not previously been invoked by Ofgem.Footnote 6

Ofgem put the Probe Initial Findings out to consultation. Even before this was complete, suppliers began to adjust their prices to meet the prospective requirements. By December 2008 “Ofgem’s energy market probe is on track to remove more than £500 million in unfair premiums from suppliers’ tariffs, but Ofgem wants speedier action”.Footnote 7 In January 2009 Ofgem (2009a) set out a range of proposed formulations for the licence conditions. Appendix 2 (this point was not made in the main text) indicated that “1.7 … the erosion of price differentials may be through a rebalancing of prices rather than by a straightforward decrease in price for the most impacted customers”.

With some qualifications, consultation respondents broadly accepted—and many welcomed—Ofgem’s proposals for encouraging customer engagement, providing better information and reducing barriers to competition.

As to the non-discrimination conditions, large suppliers variously argued that their tariffs were broadly compliant with them and/or were being made so, that price discrimination was common in competitive markets, and that enforcing these conditions unduly strictly would reduce competition. Consumer and fuel poverty groups were most concerned about the price increases. Some groups (but not all) were concerned about some price differentials: Consumer Focus (2008, para 68) said “We recommend that Ofgem concentrate initial efforts on prepayment and standard credit cost differentials against direct debit and on the treatment of single fuel electricity consumers.” These groups made virtually no mention of in-area versus out-of-area differentials. Waddams (2008) at the University of East Anglia Centre for Competition Policy (CCP) warned that “The remedies [for unfair price differentials] run the risk of introducing considerable distortions into the market, and even of producing perverse effects of raising prepayment prices above the level which the competitive market might deliver.”

In response to a further consultation in early 2009, suppliers repeated their concerns. Perhaps cost-reflective payment methods could be made to work, though some suppliers suggested that it would increase rather than reduce their PPM charges, but reducing area differentials would reduce competition, and a strict enforcement of non-discrimination could limit innovation. They wanted Ofgem to give clearer guidance on interpretation. In contrast, consumer and fuel poverty groups were increasingly concerned that Ofgem might not do anything at all. Many were sympathetic to some kinds of non-discrimination conditions, but this was not enough: Consumer Focus wanted a “widespread rollout of mandatory social tariffs”. Many argued that suppliers should equalise their prepayment tariffs with other tariffs. Energy Action Scotland wanted a change in focus of policy, from competition to vulnerable customers.Footnote 8 This organisation did not support cost-reflective pricing and was concerned that the proposal could lead to increases in tariffs for those vulnerable customers paying by standard credit and direct debit. Waddams (2009a) argued that “There is a considerable body of economic analysis and literature which indicates that prevention of such price differences is likely to harm competition in the market; and there is little evidence that it will necessarily help vulnerable customers.”Footnote 9

In March, the Gas and Electricity Markets Authority GEMA (the governing body of Ofgem) held a special meeting to discuss and agree its policy on retail regulation. Professor Yarrow, the economist Non-Executive Director on GEMA, did not support the no undue discrimination condition, and resigned.

5 Final proposals

In April, Ofgem (2009b, c, d) issued for consultation its two final proposals. The requirement that differences in terms and conditions for different payment methods should be cost-reflective was intended to be a permanent condition, as a means of better enforcing a European Directive to that effect.

The more general prohibition on undue discrimination in any terms and conditions would have a sunset clause of three years. Ofgem explained the statutory basis for putting more weight on the interests of vulnerable customers.Footnote 10 It discussed some of the theoretical literature on price discrimination, taking a critical and arguably rather narrow and theoretical approach.Footnote 11 It noted submissions that the condition could reduce competitive pressure by reducing price differentials and thereby reducing customers’ incentive to switch supplier. However, it argued that other factors could also have an effect, and the condition “would not prevent suppliers from pricing differently based on cost or some strategic considerations”. (Ofgem 2009d, para 5.50) It acknowledged the potential downsides of the proposed policy but said there were many potential considerations, and claimed “the net effect on welfare is ambiguous”. (Ofgem 2009d, para 5.34) It concluded: “Given our statutory duty to protect the interests of customers, and to have regard to the interests of vulnerable customers in particular, we have attached more weight to the benefits to vulnerable customers.” (Ofgem 2009d, paras 11.2–11.5)

Ofgem had undertaken to provide Guidelines on its interpretation and enforcement of the conditions. So as not to limit suppliers’ ability to innovate in the retail market, the Guidelines made exceptions for time limited or “initial” offers, innovative offers or new-customer acquisition targeted offers. (Ofgem 2009c, para 3.18)

Suppliers, by now, were more concerned to clarify the Guidelines. Consumer groups were generally supportive of the proposals, though several basically wanted lower energy prices in general, and for prepayment and standard credit customers in particular. Energy Action Scotland (2009b) noted Ofgem’s repeated acknowledgement that some vulnerable customers would be negatively affected. It calculated that “more than 560,000 households already in fuel poverty [would] suffer electricity price increases …[and] almost 430,000 fuel poor households may see gas bills increase”. Furthermore, “up to 2.5 million households classed as ‘vulnerable’ may suffer increased electricity costs and a further 1.9 million may see gas bills increase”. In response to this last point, Ofgem (2009f, para 6.10) later noted that (in round terms) 40% of fuel poor households paid by direct debit but 60% paid by standard credit or prepayment, so more such households would benefit than would lose.

Regulatory economists did not accept that the impact of the non-discrimination clause on competition was ambiguous or that this policy would be in the interests of customers generally or of vulnerable customers. Professor Waddams, then a member of the Competition Commission, reinforced her earlier submission, saying that the proposed clause “is likely to stifle competition”.Footnote 12 Professor Sir John Vickers, recently Director General of Fair Trading (whom Ofgem had cited), was equally forthright about this “bad policy”.Footnote 13 Professor Yarrow, former Non-Executive Director of GEMA, said there was “almost a consensus among economists” that the proposals would have “harmful consequences for consumers and for competition”.Footnote 14 Summing up, Ofgem (2009e, para 1.8) noted that “three academics” were strongly opposed to the licence condition, but its Final Impact Assessment was essentially unchanged.

In September 2009 the two new obligations were formally implemented as Standard Licence Conditions SLC 27.2A and SLC 25A, the latter prohibiting undue discrimination (particularly between regions: in-area versus out-of-area prices). Ofgem had initially argued that the sunset clause was because it would take three years for the other remedial measures to take effect, but it now explained that this clause was “intended to limit the duration of any potential negative impact on competition”. (Ofgem 2009f, paras 5.71, 11.4)

Ofgem (2010) said that “We are also committed to a thorough review of the impact of the measures introduced as a result of the Probe, before SLC 25A terminates at the end of July 2012.” Such a thorough review would presumably evaluate the contrasting predictions of the effect of the non-discrimination conditions in light of the accumulating empirical evidence. In the event, this review did not happen.

6 Effects of the non-discrimination condition

Ofgem (2010, p 14) reported that “since the middle of 2008 the average differential between in-area and out-of-area electricity Standard Credit bills has nearly halved, falling from over £30 per customer per year to £18. This change has resulted from electricity suppliers reducing their in-area tariffs towards their out-of-area offerings and therefore represents a benefit to a large proportion of ‘sticky’ customers who remain with their ex-incumbent supplier.”

But Green (2012) noted “a large increase in the level of the average bill at almost exactly the same time that companies started to reduce their cross-region differences”. He suggested that, although further analysis was needed, the policy may have had some unintended consequences.

Hviid and Waddams Price (2012) commented that “because of the volatility of the wholesale electricity market, it is difficult to identify the counterfactual and so assess the direct effect of the non-discrimination clauses on price levels”. They cite Ofgem data to the effect that, from June 2008 to December 2010, dual fuel gross margins (charges net of wholesale costs and VAT) increased from £45 to £145 and net margins (net of retail operating costs) increased from (minus) £75 to (plus) £15: “although they are not conclusive, the increased margins suggest that they may reflect aggregate price rises (relative to costs) rather than the price falls which the regulator claimed”.

Taking other dates yields different numbers for gross and net margins, but the general picture is the same: a substantial increase in suppliers’ margins after 2008. There were corresponding increases in suppliers’ total profits. (Littlechild 2014b, s2.10)

A caveat is in order here (as others have noted). Household energy prices doubled over the period 2004 to 2008, largely reflecting increases in wholesale costs. There is some evidence that retail prices nonetheless lagged wholesale costs.Footnote 15 This could have yielded artificially low retail prices, margins and profits in 2008. So the higher retail prices, margins and profits since then may to some extent have been a return to competitive normality (and/or a consequence of subsequently falling wholesale prices) rather than an indication of how far the non-discrimination condition restricted competition and increased prices and profits (though the condition may have affected the speed or nature of adjustment).

Hviid and Waddams Price (2012) looked in more detail at the implications for vulnerable customers of the two different non-discrimination conditions.Footnote 16 They argued that low income customers paying by standard credit could benefit from non-discrimination by payment type, provided that rivalry was maintained among direct debit customers. However, this rivalry was likely to be undermined by non-discrimination with respect to area, which was likely to raise all prices, and which they noted seemed to have occurred. They commented further that the exemptions allowed under the non-discrimination condition had led to significant increases in discounts for online tariffs, but that vulnerable groups generally have less access to broadband, hence would have benefited less from this.

Waddams Price and Zhu (2016) found that “the nature of competition in the industry has changed, with less effective rivalry between the regional incumbents and large regional competitors following the intervention; companies seem to have ‘retreated’ to their home regions, leaving a market where pricing behaviour resembles more closely a duopoly between British Gas and the regional incumbent”. They found in particular that the average difference between the incumbent supplier’s price and the best-priced non-incumbent roughly halved after 2009.

They and others remarked on the significant reduction in customer switching after the non-discrimination conditions were introduced. “Switching rates have fallen, confirming widespread evidence that potential price gains are the main drivers of consumer activity in the residential energy market.” (Waddams Price and Zhu 2016) By 2013, switching rates had halved.Footnote 17

Here again, the non-discrimination condition was not the only factor: Ofgem also put pressure on the major suppliers to cease doorstep selling, which was (until then) the most cost-effective means of attracting customers, and particularly effective in reaching customers that might not otherwise engage in the market. A couple of the major suppliers began to reduce their door-to-door sales force around 2009. However, it was not until July 2011 that the first major supplier finally abandoned doorstep selling completely, by which time the decline in switching had been underway for over two years.

The non-discrimination condition impacted on competitive tactics. For example, SSE’s aggressive pricing policy from 2004 to 2008 (aided by a favourable generation contract) took that company from fourth to second largest supplier. But from then until 2012 relative positions hardly changed. SSE subsequently explained that it modified its pricing strategy in 2008 as a result of Ofgem’s change in policy.Footnote 18

In sum, several factors determined the nature of competition and the path of prices following the introduction of the non-discrimination condition. However, the contrast between Ofgem’s aspirations and the subsequent outcome is evident. Ofgem proclaimed that it was removing “unfair differentials” which would save about £500 m for some customers. (It also envisaged that suppliers’ revenue would remain constant, and that other customers would pay higher prices to the extent of about £500 m, but its press releases did not highlight this). In the event, differentials did indeed reduce but suppliers’ revenue seems to have increased by about £1 billion. Not all of this was due to the non-discrimination condition, but several other impairments of competition can be attributed to it.

It is not clear that vulnerable customers, for whose benefit the condition was allegedly introduced, were in general or on average better off. Some or many may have been worse off. Indeed, the very mixed nature of Ofgem’s own estimates of distributional impacts (in one case cited above, 60% of fuel poor households would be better off but 40% worse off) make this sound more like a defensive line to take than a reason for adopting the policy in the first place.

7 Ofgem’s retail market review (RMR)

In presenting its September 2009 reforms, Ofgem said that it would take at least three years for them to take effect. However, little over a year later, in November 2010, it noted a 38% increase in net retail profit margins and announced its intention to review again the effectiveness of the retail energy market.

In March 2011 Ofgem’s Retail Market Review (referred to as RMR) found a mixed picture. There was evidence of the removal of what Ofgem regarded as large unjustified price differences between some payment types, there was one new entrant in the market and a small rise in the market share of small suppliers. (Ofgem 2011a) But there was no change across much of the market. In several respects Ofgem perceived a deterioration, including:

  • “complex pricing structures are contributing to lower customer engagement”

  • “an increase in the number of tariffs available may also be contributing to lower customer engagement”

  • “an increase in the number of passive consumers and fall in the number of active consumers”

  • “evidence that energy prices have tended to rise in response to wholesale cost increases more quickly than they fall with decreases”. (p 4)

Ofgem’s analysis centred on the following argument about customer engagement.

3.4. We regard consumer activity as a key driver of competition among suppliers and to stimulate new entry. The greater the number of engaged consumers, the greater the competitive pressure on suppliers to make efforts to retain them. We have found a number of market features that lower consumer engagement, the most significant being complex tariff information and the prevalence of sticky customers.

Ofgem concluded that “further radical actions are required”. To facilitate new entry it proposed steps to increase liquidity in the wholesale market. It also proposed to strengthen the Probe remedies and to improve reporting transparency. But in its most significant innovation, Ofgem proposed measures “to make it far easier for domestic customers to compare prices and get a better deal”.

These measures focused particularly on what Ofgem called standard evergreen tariffs (now called standard variable tariffs or default tariffs), on which most residential customers were (and still are) supplied. Standard tariffs comprise a monthly or quarterly fixed charge (commonly called a standing charge, or in the US a customer charge) plus a unit or usage rate (in pence per kWh) applied to the amount of energy consumed.Footnote 19 This tariff is variable in that it can be modified at the supplier’s discretion, subject to a minimum period of notice (30 days). Evergreen means that the tariff is for a period of indefinite length: there is no specified termination date as there is for a fixed-price fixed-term contract.

Ofgem argued that the solution to the problem was to simplify tariffs.

3.15. We intend to address confusion in the domestic market by a proposal to restrict the number of tariffs for standard evergreen products from each supplier to only one per payment method. We also propose to standardise the format of these tariffs across suppliers, with suppliers allowed to compete on a single ‘per unit’ price. Consumers would then be able to tell at a glance whether they can save money either by switching supplier or by moving to a new deal. This would be a major reform impacting the 75% of customers currently on standard evergreen products.

Ofgem clarified the proposal as follows:

  • “All suppliers limited to one evergreen product per payment method.

  • Compulsory standardised element set annually by Ofgem and identified separately on consumers’ bills.

  • Standardised element designed to cover pass through costs, such as Transmission & Distribution charges and some environmental and social charges.

  • All consumers in each region to have the same standardised element.

  • All other revenue recovered through a single unit charge, set on a p/kWh format.”

Ofgem’s logic was that customers would have only one number to compare among suppliers, namely the unit rate. This would simplify the market and encourage customer engagement.Footnote 20

8 Discussion of Ofgem’s RMR proposals

Respondents to Ofgem’s consultation generally agreed that many customers had difficulty with complexity and that it was important to assist them to engage effectively in the market. There was some sympathy for simplifying tariffs and providing more effective information. But most suppliers considered that Ofgem’s proposed simplifications went too far, and would limit competition and innovation.Footnote 21 Consumer groups too were sceptical: they felt that more needed to be done for consumers generally and for fuel poor, low income and vulnerable customers, but they were apprehensive about these particular proposals. One found the proposals an indictment of Ofgem’s energy Probe proposals; another suggested a regulated tariff for specified categories of the most vulnerable customers.

CCP economists initially offered qualified support for Ofgem’s proposal. They encouraged the move towards tariff simplification while warning that, in providing more opportunity to segment the market, it could lead to more detrimental outcomes for disengaged customers even while it improved outcomes for those who engaged. (Ball et al. 2011) They wondered why the proposal did not go further, and prohibit fixed-term tariffs. They urged Ofgem to undertake field experiments with actual customers and different types of intervention before committing to universal changes.

In December 2011, Ofgem confirmed its RMR proposals, with two additional information measures, but warned that there could be more intervention.Footnote 22 Waterson (2012) welcomed the proposals because customer search costs appeared high. But he warned of unintended consequences (facilitating cartel-like behaviour), and made his welcome conditional on creating more competition in the wholesale supply market to facilitate entry of new retail suppliers.

CCP economists, after further consideration, had “become more pessimistic” about the net effect of the proposals” and now argued against the core proposal. (Hviid et al. 2012) Ofgem’s “heavy intervention” was likely to dampen competition.Footnote 23 They reported evidence that better information for customers was a necessary but not sufficient condition for more engagement. They reported “anecdotal evidence … that some consumers don’t change supplier because they feel it would commit them to an endless treadmill of checking the market and reswitching”. Research showed that a substantial minority of switchers ended up with a worse deal. (Wilson and Waddams Price 2010) A better understanding of psychological factors was a priority, and “the provision of simplified tariffs alone is very unlikely to have the desired effect”.

My response in January 2012 referred to this proposal as Ofgem’s Procrustean Bed.Footnote 24 (Littlechild 2012a) I challenged the need for regulatory action. Discounts for those who shopped around were of the order of 10–15%, as they always had been, and it was a remarkable achievement for retail competition to ensure that even customers who made no attempt to change supplier received a price that was within 10 to 15% of the very best price in the market. The reliability of Ofgem’s consultants’ customer research, and the claim that a common standing charge would lead to far more customer engagement, were both questionable. Uniform standing charges would be inconsistent with the flexibility and innovation associated with smart metering, would preclude much-valued tariffs with zero standing charge, would increase costs and risks to suppliers, would increase regulatory costs, and would make setting energy prices once again a regulatory and/or political decision.

In February (Ofgem 2012a, b) Ofgem proposed to prevent the sunset clause coming into effect in July 2012, and to renew the non-discrimination condition for two years, without any review. It confirmed this decision in April. I objected that the condition had had adverse effects which Ofgem had not reviewed, contrary to its previous commitment. (Littlechild 2012b, c)

There was a long pause while Ofgem considered submissions on its RMR proposals. There was much political debate. The Labour Party proposed that energy companies should be required to put all over-75 s on their cheapest tariff. The Prime Minister said that “we will be legislating so that energy companies have to give the lowest tariff to their customers”. He apparently meant that each supplier would have to give its customers the lowest price in the market.

9 Change of plan: simple tariffs

On 26 October 2012, Ofgem changed tack on both outstanding policies. It announced that it would not seek to renew the non-discrimination condition because “The aim of the RMR package of proposals is to empower consumers so the threat of switching places an effective constraint on supplier pricing and other behaviour.” (Ofgem 2012a) Nevertheless, it gave a strong warning to suppliers not to resume such “unjustified pricing practices”. There was still no regulatory review of the impact of the condition.

On the same day, Ofgem announced that it would maintain the customer information parts of its RMR proposals (including the requirement to provide a price comparison guide and tariff information label). But it abandoned its key proposal to set a uniform standing charge, having “listened to concerns” and recognised some “practical difficulties”, though it did not explain what these concerns and difficulties were.

Instead, Ofgem now proposed a Simple Tariffs policy. Each supplier would be limited to a maximum of 4 tariffs per fuel per payment method.Footnote 25 All tariffs had to have a simple two-part structure, i.e. one standing charge and one unit rate. The standing charge could be zero, but the unit rate could no longer vary with the level of consumption (i.e. declining block tariffs were prohibited). Discounts were only allowed if expressed in pounds, not as percentages. Suppliers could no longer keep customers on “dead” tariffs (tariffs no longer available to prospective new customers), and had to move such customers to that supplier’s cheapest “live” or open tariff.

CCP economists welcomed the end of the non-discrimination condition but not the Simple Tariffs policy. Their empirical research led them to doubt that making comparisons easier would have a substantial effect on customer switching or engagement rates. The forced simplification of tariffs would exacerbate concerns about coordinated effects, could lead to reduced rivalry, smaller gains from switching, hence less switching and less pressure to reduce prices. (Waddams and Loomes 2012)

I argued that Ofgem’s claim that tariff complexity hindered switching was inconsistent with its own customer research.Footnote 26 Ofgem’s proposed policy would have unintended consequences (via impacts on tariffs offered) and would make customers worse off rather than better off. Later, I argued that Ofgem’s policy aim of fairness logically implied restricting rather than promoting competition.Footnote 27

Debate on this policy and its implementation continued well into 2013 (despite the political impatience noted below). In August 2013, Ofgem announced that the simple tariff conditions would be in place by end 2013 and most other provisions by March 2014. With some modifications (e.g. to prohibit all but two kinds of discount) these tariff restrictions generally came into effect on 1 January 2014, although most suppliers had adjusted their range of tariffs earlier.

10 Consequences of the simple tariffs policy

Given the four tariff limit, suppliers chose to keep their most popular and profitable tariff types and phase out their minority preference tariffs. They substantially withdrew green tariffs. Suppliers were less innovative: they could only introduce a new product if they withdrew one of their four allowed tariffs, but existing tariffs were tried and tested, whereas new tariffs were necessarily more risky and liable to have a low take-up in early years. The Big 6 suppliers soon limited themselves to one Standard Variable Tariff and two or three fixed-price tariffs for different periods ahead, mostly between one and two years.

Supplier E.On previously had a StayWarm tariff that offered customers over 60 years of age a fixed monthly bill regardless of how much energy the customer used (though the bill could be adjusted on a forward-looking basis depending on actual usage). This tariff was popular with customers, and highlighted by Ofgem in 2001 as one of a small number of major initiatives by fuel companies to address the needs of the fuel poor. This tariff was withdrawn because it did not meet the prescribed rules. The also-popular tariffs with zero standing charge were withdrawn because they were not viable with a single unit rate. (They were uneconomic for suppliers at low usage and too expensive for customers at high usage.)

Supplier SSE had earlier introduced a new tariff that, for several months, price comparison websites deemed “the best offer in the market”. That tariff was now prohibited because the discount was greater in the first year than in the second. Introductory and “cash-back” discounts that some suppliers used were now banned. Prompt payment discounts, and all but two other types of discounts, were disallowed too.

Some supermarkets and other organisations had previously competed in the retail energy market by offering products on a “white label” basis, whereby an energy supplier provides the product and backup service under the supermarket’s name. Previously these white label products could be tailored to the supermarket’s specification. Now, Ofgem proposed that they should count against the supplier’s limit of four products, hence the supermarket could only offer the same product as the supplier itself offered. Some supermarkets considered entering the market on their own account, in competition with energy suppliers, instead of using a white label. Until then, they could have competed by offering discounts on grocery bills, but the new rules prohibited that.

Not surprisingly, given the extent, detail and severity of Ofgem’s restrictions, there were disputes about whether particular products were compliant with the rules. There were also applications for derogations in exceptional cases. Ofgem granted several of these.Footnote 28 This raised the question whether it was sensible to prohibit such variations and innovations in the first place.

11 Reappraisal of policy: the CMA investigation

Ofgem’s Simple Tariffs policy did not end debate about the retail energy market. There were still concerns about rising prices and perceived profitability of the suppliers, and also increasing complaints, mostly about billing. Customer switching in Q3 2013 was less than half the level it was in 2008 and apparently still declining. It was not obvious to customers that tariffs actually were simpler. And customer support for Ofgem’s policy may have depended on not appreciating its full implications.Footnote 29

A new chairman of Ofgem took office in October 2013, followed shortly by a new Chief Executive. A market investigation reference would enable the CMA to assess and address such concerns about the market. It would also judge Ofgem’s own performance as regulator.Footnote 30 The Government, Ofgem, OFT and CMA coordinated a framework for assessing competition (published in December 2013) and then the assessment itself in March 2014. (Littlechild 2016, pp 46–49) This assessment found various problematic features of the market.Footnote 31 In June 2014 Ofgem referred the energy market to the CMA. The CMA’s task was to consider “whether any features of the relevant markets prevent, restrict or distort competition”—so-called Adverse Effects on Competition (AECs)—and if so, to decide whether the CMA should take remedial action or recommend that others should do so.Footnote 32

In July 2014 the CMA put out an Issues Statement, which included “initial theories on what might be adversely affecting competition”. (CMA 2014) For the first time, “regulatory interventions”—specifically the non-discrimination licence condition and the simple tariffs conditions—were mentioned as a potential source of “weak incentives to compete”. The CMA invited comments.

12 Professor Yarrow’s submission

In response, Professor Yarrow and I separately suggested that the CMA first examine why Ofgem had changed course in 2008 and thereafter. Why had it implemented these controversial regulatory interventions that seemed to have had adverse effects?

Yarrow (2014a) suggested that the change in direction was associated with a change in Ofgem personnel.

  1. 12.

    … Suppose that, having looked at the matter, the CMA concludes that—as the Gas and Electricity Markets Authority (GEMA) was advised would likely be the case—the 2009 measure did indeed serve to restrict competition in retail markets and thereby harmed consumer interests. Suppose further that it is concluded that Ofgem’s more recent simplification reforms contain elements that can also be expected to restrict competition. …

  2. 13.

    … Specifically, it [the CMA] needs to assess why the mistakes occurred. …Why were these decisions taken? Why was the economic advice of the time ignored? Why were internal legal opinions ignored? Why was a regulatory approach that had been developed over a period of nearly twenty years up to 2008 abandoned more or less overnight? …

  3. 16.

    In specific relation to the 2009 undue discrimination decision, the CMA will l think find a sharp change of direction in Ofgem’s approach to retail markets between 2007 and 2008…

  4. 18.

    The regulatory shift correlates precisely with a board-level change in Ofgem personnel and responsibilities on 1 January 2008, but “because different people were in charge” is not in itself a satisfactory answer to the question. … looking across the various regulated sectors in the UK, changes in senior regulatory personnel occur on a relatively frequent basis and it is by no means common to find U-turns as clear cut as the Ofgem 2008 episode. This suggests that there were other factors at work.

Yarrow’s “prime candidate” for another factor at work was “rapidly increasing world energy prices”. This generated a demand to know why prices are rising, who is to blame and what is to be done about it. Explaining that no one may be to blame and it may not be sensible to do anything about it “invites the inference that … the regulator is to blame and something should be done about the regulator”. He continued,

  1. 21.

    The reality is that strong, upward cost pressures tend to place strains on the institutional structures of markets that are partly governed by a regulator who has powers to do things in those markets when stuff happens. In a system based on independent regulation, it is a large part of the job of regulators to protect and sustain these institutional structures in the face of the pressures and strains.

  2. 22.

    The evidence … indicates that strong upward cost pressures in GB energy markets started to occur significantly before 2008 and, in the face of these pressures, Ofgem/GEMA performed its delegated tasks well up until 31 December 2007. From then on it didn’t. The boundaries that independent regulators are supposed to defend were abandoned and Ofgem/GEMA started pointing the finger of blame for the higher prices that energy consumers were facing …. The narrative changed: the market was (somehow) failing, suppliers were (“possibly”, in vague and unspecified ways) tacitly coordinating prices, prices were ‘unfair’, hobnail boots were needed, consumers were depicted as victims.

  3. 23.

    There are other diagnostic questions that could be asked about the relevant episode: Why did GEMA ignore the option of making a reference to the Competition Commission? (The option was raised at a board meeting in early 2008.) Were the personnel changes on 1 January 2008 a cause or effect of the policy shift? (My view is that they were largely instrumental, a means of effecting a change in direction.)

My own explanation for the change of approach was along similar lines: the increase in energy prices, the reduced involvement of economists in senior roles at Ofgem, and systematic changes in Government policy and the statutory regulatory framework. (Littlechild 2014b) The following three sections briefly summarise and illustrate these three factors.

13 The path of energy prices

Figure 1 shows that, from about the mid-1980s to the early 2000s, the UK enjoyed steadily falling domestic energy prices (in real terms). But by 2008 energy prices nearly doubled in real terms. In the subsequent five years they fell but increased again.

Fig. 1
figure 1

Source: House of Commons Library, Energy Prices, January 2014

Index of domestic energy prices since 1970

The theoretical and empirical literature on public choice and economic regulation suggest that regulators and governments feel the need to Do Something when there is concern about prices. The point has been made in the present context.Footnote 33 There were certainly obvious political pressures on Ofgem—for example, from the Select Committee on Business and Enterprise, which (not satisfied with Ofgem’s 16 January 2008 reassurance) announced on 5 February its own investigation into Energy Prices, Fuel Poverty and Ofgem.

Until early 2008, Ofgem explained that cost and price increases were not the fault of the retail suppliers.Footnote 34 There is no reason to think that the causal factors suddenly changed. But thereafter Ofgem left this question more open, redirected attention to relative rather than absolute prices, and intervened to reduce unfair price differentials in general—even though, of the concerns then voiced about the energy market, price increases, fuel poverty and prepayment meter surcharges were frequently mentioned but the in-area versus out-of-area price differentials do not seem to feature at all (Littlechild 2016 pp 26–30).

14 Economists and non-executive directors at GEMA

Yarrow referred to board-level changes at Ofgem. To put this in broader context, in 2013 the retiring Chairman of GEMA/Ofgem looked back over the ten years that he and the Chief Executive (also then retiring) had served, and paid tribute to the latter’s “energetic, expert and transformational role in leading Ofgem from a body that was essentially an economic regulator ten years ago to its current and far more multi-dimensional purpose and activities.” (Ofgem 2013, Yarrow (2014), p 5) One reflection of this was a significant reduction in the number of economists on the GEMA Board.

From Ofgem’s inception in January 1999 until September 2003, four of the most senior posts on the GEMA Board were filled by economists (trained at graduate level), viz Chair and Chief Executive (at that time combined), Head of Markets Division (responsible for retail markets) and one Non-Executive Director. In 2003 the economist Chair/Chief Executive was replaced by two non-economists. In January 2008 the economist Head of Markets was replaced by a non-economist. For the next five years none of these three key Executive Director posts was held by an economist. When Professor Yarrow resigned in March 2009 he was not replaced by another economist Non-Executive Director until November 2012. From January 2009 until October 2010 another Executive Director was an economist, but for the two-year period from November 2010 to October 2012 there were no economists at all on the GEMA board.

More generally, from 1999 to 2008, all Non-Executive Directors of GEMA had been from the world of business and finance, plus one professor of economics. In January 2009 this changed when the Minister appointed four new Non-Executive Directors. Only one was from business and finance. Another was a scientist. The other two were from consumer bodies who had been critics of Ofgem’s pre-2008 policy and supporters of the Probe and its remedies.

In 2011 a Government review of Ofgem drew attention to imbalances in non-executive expertise—notably the absence of economic regulation expertise since 2008 but also the absence of social and consumer expertise before that.Footnote 35 In contrast, as of 1 October 2018, the Chair, Chief Executive, Head of Markets and one Non-Executive Director are once again all trained economists. In addition there is now a Chief Economist on the Board (see below).

This is not to argue that trained economists are a necessary or sufficient condition for making good regulatory decisions. But they should be better able and more interested to assess the economic issues and evidence, and are likely to give more weight to economic reasoning in coming to decisions on policy, than non-economists. Specifically, in 2008/9 economists would have been more familiar with varieties of price differentials and discrimination in competitive markets; less likely to introduce the term “unfair price differentials” and to see their replacement by “price differentials objectively justified by cost differences” as a means of “accelerating the transition” to more competitive markets; more alert to the risks and limitations of non-discrimination conditions; and arguably more reluctant to countenance measures leading to significant income redistribution whereby (e.g.) 60% of fuel poor customers would benefit but 40% would be worse off.

In parallel with its RMR Findings and Initial Proposals on 21 March 2011, Ofgem issued a paper on what behavioural economics could say about energy consumers. (Ofgem 2011b). It explained how this had informed its RMR approach, and how biases affected energy consumer decision-making. It seems questionable whether behavioural economics would have driven policy as dramatically had Ofgem still been headed by Directors trained in conventional economics. An alternative scenario is that in 2011–13, economist Directors would have been less likely to see an increased number and variety of tariffs as a deterioration in competition; more likely to have associated the decline in switching with the reduction in price differentials; and more sensitive to the reduction in customer choice, innovation and competition associated with the “Procrustean Bed” and Simple Tariff policies (and the regulatory vulnerability of the former).Footnote 36 Policy would have been different. Further evidence for this is that independent regulatory economists expressed their views strongly to Ofgem and the CMA.

15 Government policy and Ofgem’s duties

Board level appointments at GEMA are made by the Secretary of State, after consulting the Chair. Could changes in governments, ministers, energy policy and Ofgem’s statutory responsibilities have contributed to Ofgem’s change in policy? There were four distinct phases of Government over this period.

Conservative Governments (May 1979 to April 1997) privatised the gas and electricity industries, and created the gas and electricity regulators as independent bodies, responsible to Parliament not to ministers. They gave regulators relatively simple statutory duties, primarily to promote competition and, subject to this, to protect the interests of consumers.Footnote 37 These Conservative Governments largely rejected the concept of an energy policy.

Until September 2008, Labour Governments (May 1997–April 2010) were moderate and pro-market, but with a gradually more active approach to energy policy. They diluted the duty to promote competition, put more weight on vulnerable customers, added environmental duties, and took powers to guide the regulator in the discharge of its social and environmental duties.Footnote 38 In March 2008 the Government wanted energy company spending on social tariffs to increase from about £50 m a year to at least £150 m. Also, the Government “is looking to Ofgem and the suppliers to bring forward proposals for treating prepayment customers more fairly. However, if sufficient progress is not made by next winter, the Secretary of State … is prepared to use his statutory powers with a view to reducing the differential between prepayment and other forms of payment”. (H M Treasury 2008, p 66)

On 3 October 2008 the Labour Government created a new Department of Energy and Climate Change (DECC), headed by Ed Miliband, who had a more interventionist perspective. The minutes of GEMA’s meeting of 20 November 2008 (following Ofgem’s Probe Initial Findings on 6 October 2008) refer to “the parallel interest within Government in securing rapid and effective responses to the issues identified in the Ofgem study and the remedies proposed.” On 9 December 2008 (while Ofgem was consulting on the non-discrimination condition) Mr Miliband’s “first major speech on UK energy policy” emphasised his concern about vulnerable customers, said that the unfair price differentials could not continue, that the Government would work with Ofgem, and that if necessary the Government itself would act. (Miliband 2008) In October 2009 (just after Ofgem had implemented the non-discrimination condition) the Government welcomed Ofgem’s measures, stating that “the regulator should be active in protecting consumers and their interests, whether through the promotion of effective competition or through other, more direct, interventions”. (DECC 2009) In November 2009 the Bill that became the Energy Act 2010 redefined the interests of electricity consumers to include their interests in reduced emissions of greenhouse gases; provided that, before promoting competition, the Secretary of State and Ofgem should consider whether there were other approaches that would better protect the interests of consumers; enabled the Government to direct suppliers to implement schemes for reducing fuel poverty; and gave the Government power to adjust charges if it considered that some customers of a supplier were treated less favourably than others.

The Conservative-Liberal Democrat Coalition Government (May 2010–May 2015) had an Energy Market Reform policy to radically increase the use of renewable energy. It expected energy costs to rise, and saw strong regulation as part of the process for avoiding unnecessarily high prices. It committed to review Ofgem “to ensure that the regulatory framework … would continue to deliver against our strategic objectives” and to deliver “confidence that the regulator’s decisions would be aligned with the Government’s strategic policy framework”. Its Energy Act 2013 required Ofgem to further the delivery of the Government’s policy outcomes, to explain each year whether it had succeeded in this, and to say how it would do better next year.

Ofgem’s RMR Findings and initial proposals, announced on 21 March 2011, were discussed in advance with the Coalition Government. The Secretary of State issued a statement of support. In September 2011 the Secretary of State proposed half a dozen new measures to protect customers. The Government had agreed in principle with energy suppliers (apparently independently of Ofgem) to provide information about their best tariffs, an arrangement that was firmed up in April 2012. In October 2012 the Secretary of State strongly supported Ofgem’s revised RMR proposals (simple tariffs), describing them as “the sort of measures I have been urging for some time”. The Prime Minister’s office said that the Government “had been working with them [Ofgem] for some time”. The Government also announced that it would take powers to require suppliers to do precisely the things that Ofgem proposed to implement via licence amendment. Its policy objective was to encourage retail competition through greater engagement of customers, but its urgent aim was to keep energy prices down, time was of the essence and a new Act would speed up the process. It also referred to “uncertainty in the retail market due partly to the length of time Ofgem has been considering reform”, and “uncertainty … created by a long RMR process”. The Energy Act 2013 gave the Government the promised power to impose simple tariffs on suppliers. It also gave the Government power to designate a strategy and policy statement, with GEMA/Ofgem having a duty to best further delivery of the specified policy outcomes.

Thus, two quite distinct government approaches to GB retail energy policy and regulation have obtained during the last 35 years. From 1979 to 2008, successive governments created and promoted competition in energy markets and intervened relatively little in regulation. In contrast, from 2008 onwards, governments intervened substantially and increasingly, modified Ofgem’s statutory duties to subordinate the promotion of competition to more direct intervention, and increasingly required or pressed Ofgem to further the delivery of Government policy. (See also Deller and Waddams Price 2018, ch 2) Opposition parties did not demur.Footnote 39

16 Reappraisal of policy: the CMA investigation

The CMA published its Final Report in June 2016. (CMA 2016) The Assessment Framework, the Assessment itself and Ofgem’s reference had all drawn the CMA’s attention to concerns about weak customer response, vertical integration, barriers to entry and expansion, continuing incumbency advantage and market segmentation, and possible tacit collusion. To the surprise of many observers and critics, the CMA found that only the first of these had an Adverse Effect on Competition (AEC), the others did not. The CMA did, however, find that aspects of Ofgem’s regulatory policies since 2008 constituted an AEC, and furthermore that governance of the regulatory framework was itself an AEC.

On the non-discrimination licence condition SLC25A, the CMA found that “when Ofgem prohibited suppliers from offering out-of-area discounts for new customers, the effect was to increase prices for out-of-area customers and reduce the strength of competition”.Footnote 40 There was no need for a remedy because Ofgem had already removed that licence condition.

The CMA critically evaluated the ongoing RMR simple tariffs policy in some detail. It found no real evidence of improved customer engagement.Footnote 41 But it did find that “certain aspects of the ‘simpler choices’ component of the RMR rules (including the ban on complex tariffs, the maximum limit on the number of tariffs that suppliers are able to offer at any point in time, and the simplification of cash discounts)” had an Adverse Effect on Competition “through reducing retail suppliers’ ability to compete and innovate in designing tariff structures to meet customer demand, in particular, over the long term”. (CMA 2016, p 42) The CMA recommended that Ofgem remove these restrictions.Footnote 42

17 Governance of the regulatory framework

The CMA then sought to “identify aspects of the structure and governance of the regulatory framework—including the roles and responsibilities of institutions, the design of decision-making processes and the availability of appropriate information—which are likely to increase the risk of policies being developed in the future that are not in consumers’ interests or inhibit the development of policies that are in their interests.” (para 18.7) To that end it examined four main aspects of the regulatory framework:

  1. (a)

    Ofgem’s statutory objectives and duties

  2. (b)

    The mechanism for addressing disagreements between DECC and Ofgem

  3. (c)

    Communication on the impact of government and regulatory policies

  4. (d)

    The regulatory requirement for financial reporting.

    1. (a)

      Ofgem’s statutory objectives and duties

      The CMA observed that “Ofgem has expressed concerns with regard to its current objectives and duties, noting that its competition duty had been progressively downrated relative to other duties over the last ten years”. (para 18.11) It quoted different views on the impact of this.Footnote 43 It focused on the additional clause in the Energy Act 2010, to the effect that, before promoting competition, Ofgem should look first at other ways of protecting customers. It opined that this “is likely to constrain Ofgem’s ability to promote competition or lead to confusion as to its role in promoting competition”. (para 18.23) It recommended that this paragraph be deleted from Ofgem’s statutory duties.

    2. (b)

      The mechanism for addressing disagreements between DECC and Ofgem

      The CMA noted that policy reforms might be implemented by measures taken by DECC (mainly through legislation), Ofgem (mainly through licence conditions), and the industry (through code self-regulation). It was concerned that “this fragmentation of responsibility increases the risk of policy decisions being taken that are inconsistent, conflicting, or based on insufficient analysis. It also increases the difficulty both industry parties and other stakeholders have in navigating the regulatory framework”. (para 18.31)

      The CMA cited evidence that Government had put pressure on Ofgem by announcing its intention to act if Ofgem didn’t.Footnote 44 Unless there were clear formal processes for Ofgem and DECC to discuss transparently the merits of policy design, and strategies for implementation of DECC policies, DECC was more likely to pressure Ofgem. But “use of such an informal approach—if it encourages Ofgem to implement changes that it would not pursue in the absence of such pressure—is likely to harm transparency and the independence of regulation, undermine trust and transparency and ultimately lead to inefficient decision making.” (para 18.42) The CMA recommended a formal process for Ofgem to comment publicly on government proposals, and the issue of joint DECC-Ofgem statements on policy.Footnote 45

    3. (c)

      Communication on the impact of government and regulatory policies

      The CMA considered that there was insufficient analysis and communication of the impact of government and regulatory policies on energy prices and bills (including assessment of the trade-offs made within and between policies). The dramatic increase in prices between 2004 and 2014 was not understood and had contributed to the inappropriate regulatory decisions. Although there was no broad lack of information in the public domain on the price and bill impacts of policies, “some of the results are not communicated effectively and clearly to a broader audience”, and this could “contribute to an overall lack of transparency of, and trust in, such analysis, which in turn inhibits productive debate and effective policymaking.” (para 18.103)

      The CMA considered that “a more authoritative and independent assessment of the energy markets is necessary”. To provide this, the CMA recommended that Ofgem publish annually a State of the Market ReportFootnote 46 and create a new unit (e.g. an Office of the Chief Economist) within Ofgem to do this.

    4. (d)

      The regulatory requirement for financial reporting

      Following the Probe in 2008, Ofgem had required further information from the companies, but the CMA considered that this had limitations.Footnote 47 Since market opening there had been regular concerns about the competitiveness of retail prices, which had not been answered.Footnote 48 There was a “lack of a regulatory requirement for clear and relevant financial reporting concerning generation and retail profitability”. The remedy was for Ofgem to require the six Large Energy Firms to provide more information.Footnote 49 It should also ensure that its price monitoring regime “measures wholesale energy purchases on a relevant basis, such as the opportunity cost.” (para 20.30)

18 Responses to the CMA report

Ofgem accepted the CMA’s recommendations. It immediately removed the problematic Simple Tariffs restrictions. It set up an Office of the Chief Economist, which has already produced two informative State of the Energy Market reports. In November 2016 it issued a draft RFI [Request For Information] about more detailed financial reporting by large companies, with an intention to have licence amendments in place by November 2017. As of November 2018, nothing more seems to have become of this.

The Government was less convinced by the CMA’s recommendations for reform. It “agreed with the CMA, that clarity about the government’s work with Ofgem is desirable. … The right approach, however, may not always be a detailed joint statement, so we will agree this with Ofgem on a case-by-case basis.” (BEIS 2018) Nor was it convinced of the need to change Ofgem’s statutory duties: “the current objective already places sufficient emphasis on competition”. And there was no need for any new mechanism to address disagreements: “Ofgem is already able to comment on Government policy”.

19 Concluding remarks

Why did Ofgem change its regulatory stance from 2008 to 2016? During the mid- to late-2000s, there were significant increases in energy prices, a lack of public understanding of the reasons for this, public, political and government pressure on Ofgem to Do Something about this, an increasing wish of government to intervene in the market and to guide Ofgem, changes in Ofgem’s statutory duties to that end, corresponding changes in Ofgem board personnel, the removal of economists from senior positions—all these put increasing strain on Ofgem’s previous policy on retail regulation. In 2008 the dam burst. Ofgem began to intervene significantly.

The CMA concluded—consistent with the arguments of regulatory economists—that neither of Ofgem’s two subsequent interventions (the non-discrimination condition and simple tariffs) were in the interests of competition or customers. The CMA did not suggest that these policies were, nonetheless, in the interests of the subset of vulnerable customers to which Ofgem was increasingly required to pay greater heed. Nor was there ever agreement among customer and fuel poverty groups or other respondents that either of these policies was an effective way to advance the interests of vulnerable customers.

The CMA concluded that part of the problem was a lack of information about the market and the major companies, and a failure to explain the reasons for price increases and the impacts of policy. Major suppliers have not yet been asked to provide additional information about themselves, but Ofgem’s new Office of the Chief Economist is already providing valuable additional information about how the market is working. It might be noted, however, that when it chose to do so, Ofgem was quite capable in the past of explaining robustly that exogenous factors rather than retailers were the cause of energy price increases, even with the information then available to it.Footnote 50

The CMA considered that Ofgem was subject to pressure from Government, and that the solution was to formalise the relationship between Government and Ofgem. The Government disagreed. Whether implementing these CMA recommendations would have had any significant effect is questionable. (Deller and Waddams Price 2018, p 39). So the failure to implement them may not make much difference.Footnote 51

There is much evidence to indicate increasing Government involvement with Ofgem, as a matter of policy and practice. But the evidence does not suggest that either of these two particular Ofgem policies was designed and proposed by Government. Rather, Government wanted Ofgem do Do Something—and to be seen to be Doing Something—in the face of increasing retail energy prices.

The problem may be with public and political pressures more generally. In his evidence to the CMA cited above, Yarrow referred to “markets that are partly governed by a regulator who has powers to do things in those markets when stuff happens. In a system based on independent regulation, it is a large part of the job of regulators to protect and sustain these institutional structures in the face of the pressures and strains.” He suggested that “The boundaries that independent regulators are supposed to defend were abandoned.”

During the post-privatisation transitions to competition, in energy and in other utility markets, it was convenient to have sector regulators with “powers to do things”. But in most other competitive markets such powers are held by a general competition authority. Such an authority is not held responsible for the state of the market at any time, and hence is able to investigate on a more considered basis “when stuff happens” such as price increases. (Investigations of petrol/gasoline markets are often cited as a model.) Transferring powers from an energy sector regulator to the competition authority would better protect the competitive retail energy market from undue public and political pressures. There is of course a tension between such a policy and the view that sector regulators and competition authorities are an instrument and extension of government policy, and a means of dealing quickly with economic and political pressures.

Finally, transferring responsibility from a sector regulator to a competition authority would not in itself preclude controversial economic analysis by the competition authority or significant government intervention—witness the recent decisions to introduce energy tariff caps in GB and Australia. Further scrutiny of such economic analysis is therefore required. Were Ofgem’s “unfair price differentials” an indication that the market was or was not competitive, or that unequal prices may not be socially acceptable? The CMA basically agreed with Ofgem’s diagnosis that there was a competition problem as a result of “weak customer response”, but considered that Ofgem chose the wrong remedies in and after 2008, and proposed different remedies of its own. In turn, the Government, too, accepted the diagnosis of a competition problem but found the CMA majority’s remedies inadequate and imposed its own remedy in the form of the widespread tariff cap. A further paper drawing on previous analyses (Littlechild 2018a, b, c) will question the Ofgem/CMA analysis of this market and hence the need for competition remedies in the first place.