The European Natural Gas Market

Abstract

The European Union started the introduction of competition in the European market for natural gas. Today, mid-2016, the process of restructuring is still going on. In parallel, important changes in geopolitical, environmental and technological determinants can be observed in the European and global energy and gas markets. These changes have been highly influential in generating ‘reactive’ policies in the European Union, both in Brussels as well as in the Member States. The evolution of the European natural gas policy creating a strongly regulated version of a ‘well-functioning’ gas market remains a highly politicized and instable experiment. The values attached to natural gas are constantly shifting between the economics, to security of supply and sustainability. Moreover, the importance attached to these values and their operationalization are different in the various parts of Europe. Therewith, the creation of ‘well-functioning’ EU gas market will always remain a politicized and never ending story.

Introduction

Nearly 20 years ago, the European Union started the introduction of competition in the European market for natural gas. Thereupon, a process unfolded which has radically changed the gas market in terms of its organization and coordination and the role—or even the presence—of specific types of actors involved.

Today, mid 2016, the process of restructuring is still going on. The claim that ‘the market’ needs further developments is a recurrent message, and on a regular basis, new rules, regulations and interventions are announced. Initially, it was argued that these problems were a consequence of delays in the process of market restructuring. The continued exercise of market power by producers and wholesale and retail companies was keeping the EU gas market from fulfilling its function as an efficient system of coordination. Indeed, competition between a variety of suppliers has evolved mainly in north-west Europe, whereas other regions are still supplied by one single supplier, Russia, or at best by a few. The solution is seen in creating more competition by deeper restructuring, a more stringent overview and control on the behaviour of market actors and a more effective regulation of the transport and storage infrastructure [1].

In parallel, important changes can be observed in the European and global energy and gas markets and in the world in which these markets exist. First of all, in geopolitical terms, the EU has been expanded, incorporating new Member States in central Europe that formerly where members of the Warsaw Pact. Moreover, the formerly Soviet republics have gained independence and are playing out their own political and energy strategies; either or not influenced by internal struggles. Secondly, in terms of energy and environmental policy, the increasingly widespread acceptation of global warming as a consequence of the use of fossil energy resources has altered the preferences and priorities in respect of energy supply in the EU countries. Thirdly, in terms of technological development, the rapid maturing of both the LNG technology and unconventional gas (and oil) production has substantially altered the local availability of gas resources and their transportability. Particularly, the latter has had an impact in economically connecting the world’s three main continental gas markets: America, Eurasia and southwest Asia [2, 3].

As will be shown below, changes in these geopolitical, environmental and technological determinants have been highly influential in generating ‘reactive’ policies in the European Union, both in Brussels as well as in the Member States. In the meantime, the overall energy and gas market policy objectives remain framed into the particular EU conception of how a ‘well-functioning’ market can be created in an industry that at least partially exhibits the characteristics of a natural monopoly [4].

This raises the interesting question to what extent the current European gas policy is able to continue on the path of, primarily, creating a ‘well-functioning’ gas market by means of a more or less constant regulatory intervention. This, while in the second row, all kinds of changes are taking place that alter the evaluation of important values attached to natural gas as a main energy input in the European economy, security of supply, sustainability and the economics of gas supply. It is important to note that in the different parts of Europe these values are valued differently.

Below, we firstly will address a number of fundamental characteristics of the gas industry. Subsequently, we will outline briefly how the restructuring of the European gas market has taken shape, in the subsequent directives and interventions of the EU Commission. Thereupon, we will briefly discuss the structural changes in the European gas market and the global energy context. We conclude with a discussion on the interaction of these aspects and the consequence for gas in Europe.

The Gas Market

The natural gas industry consists of three main segments. Up-stream the exploration and the production of gas takes place. The mid-stream segment involves the transport of the gas to the local distribution grids and the large scale industrial users and power plants. On a continental scale, generally, the gas is transported via high pressure transmission pipelines. Overseas the gas is transported in tankers as LNG. Gas can be stored in salt caverns or in depleted gas fields. Down-stream the local distribution grids deliver the gas to small domestic and business consumers.

The development, the operation and the exploitation of these production, transport, distribution and storage systems are complex and risky activities. In the first place, they require large investments; capital expenditure involves the lion’s share of the overall cost. Secondly, the assets involved are highly specific; once constructed at some location, they cannot be removed or used for alternative purposes when either the supply or the demand of gas would come to a halt. These costs are literarily ‘sunk’. Thirdly, all parties are locked into some degree of inter-dependency vis-à-vis each other. Pipelines, LNG facilities and storages form indispensable essential facilities to producers, traders and end-users. So, access conditions to these facilities are a key determinant in the functioning of the supply systems and the economic well-being of the parties involved.

In this respect, volume and price risk are crucial. Gas producers and infrastructure operators will only generate a profitable return when their assets are used at a reasonable rate of throughput and at revenues that cover their costs over the longer term. They need security of demand. The consumers, by investing in specific gas-fired appliances and installations, are committed to using gas. They need security of supply, obviously at a price which is acceptable in the light of the cost of alternative sources of energy.

Generally, because of these circumstances and because of the economy scale associated with their technical and spatial characteristics, gas systems have been considered as natural monopolies, in which competition was not feasible [5]. In the past, the economic concepts of market imperfections, market failure and public goods have led governments to intervene and to regulate the industry, to ensure the public interest and the economic stability of the industry. In those countries where gas was produced, the public interest associated with resource management was an additional argument for state intervention [6].

In the USA, private firms in the production, the transport and the retail segment were regulated by the federal authorities and by the states [7, 8]. In Europe, public coordination generally took the shape of direct state intervention. International gas transmission and wholesale trade were undertaken by joint-ventures between gas producers and national and local authorities, while local distribution networks and retail trade were managed by municipal gas companies. Commercial transactions and market coordination were organized by means of long-term contracts, including take-or-pay and destination clauses, under which the gas price was linked to oil products as the main substitutes for gas, while the gas producers received their revenues according to the net-back principle. The net-back principle implies that producers (and the governments) receive a residual amount, after all costs have been paid for. In this context, public finance and economic coordination were important facets. Gas production and sales were subject to intricate regimes for exploration and production and tax collection. Often gas pricing was used as an instrument for regional and sectoral economic stimulation [9–12].

Restructuring the EU Gas Market

As from the end of the 1970s onwards, such forms of public intervention and market coordination were increasingly criticized, initially mainly in the Anglo-Saxon world. ‘Rolling back the state’ á la Margaret Thatcher and Ronald Reagan and the introduction of competition would allow for a more efficient provision of energy, water, public transport and other public services. The European Community, in 1985, adopted the Single European Market a general strategy for liberalization of the EU economy, followed in 1988 by ‘The Internal Energy Market’ [13], which advocated a similar restructuring in the energy markets. The primary argument was to dismantle the impediments to intra-communitarian trade in goods and services, but also the neo-liberal outlook on competition and efficiency and business interests played an important role [14].

On this basis, gradually, a perspective unfolded for a European gas market in which competition was to be created between gas producers and suppliers in the up-stream segment and between traders in the wholesale and retail segment. To this end, it was envisaged that the traditional long-term contracts would give way to short-term transactions. Scarcity conditions would determine the market price, thus balancing the supply and demand for gas. It was expected that liquid spot markets would arise in those places where different sources of supply would connect with demand. A precondition for this was that the competing traders would be given access to the essential facilities in transport, distribution and storage, to reach their customers. To this end, three basic regulatory principles were embraced in Europe.

Firstly, the essential facilities would have to be ‘unbundled’ from the production and trading activities, in the sense that the operators of such facilities would not have any commercial interest in manipulating the gas flows or in using the market insight that the operation of their systems would yield them in any trading activities. In respect of the transmission and distribution pipelines, over time, an increasingly stringent degree of legal and managerial unbundling was required. Most of the networks were separated from the former wholesale companies and local gas utilities, to be managed as either a transmission, or distribution, system operator (TSO or DSO). In respect of other facilities, for LNG, storage and conversion, depending on their position as a (local) monopoly in a ‘relevant’ market, different regimes were allowed with possible exemptions from third party access requirements, to be awarded on a case-by-case basis.

The second principle involved the provision of ‘un-discriminatory access’ for trading parties to these essential facilities. This, obviously, not only involved access to the transport systems but also to storage, LNG and quality conversion facilities when needed. Initially, this was arranged by means of relatively simple ‘first come–first served’ contracts, under which traders could book a specific amount of capacity for a specific time slot, at a specific tariff to be paid to the operator. Yet, over time, in several steps, an increasingly complex approach developed. Access to the transmission pipelines, generally, came to be organized as a so-called entry-exit model in which ‘shippers’ of gas book their entry and exit rights, for specific volumes of gas to be fed in or taken out of the transport system at specific locations [15].

The essence of this model is that it abstracts from the actual routes that the gas molecules take to their destination. Therewith, it provides maximal flexibility to shippers in buying and selling to market parties at no matter what location. Entry and exit tariffs at specific locations are equal for all shippers. In respect of the distribution networks, generally the so-called postage stamp tariffs are applied, in which a seller books his entry in a given zone, where his client is located, at a pre-specified tariff. Access conditions to other facilities depend on their status as being exempted or not; either their owners or operators have to provide access to any interested users at pre-specified conditions, or they are allowed to determine how they use their capacity themselves.

An important element in providing access, in combination with the tariff structure, is the allocation of the capacity available to the interested shippers. Starting out with simple first come–first serve rules, a complex set of mechanisms was developed in order, on the one hand, to maximally expand the use of the physically limited capacity available by as much shippers as possible, thus reducing so-called contractual congestion. On the other hand, the objective became to efficiently award scarce capacity to those shippers who put the highest value on that capacity. Therefore, a variety of procedures for the tendering and secondary trading capacity and the re-allocation of un-used capacities was developed. The underlying objective was to enable the shippers to match the commodity transactions they arranged in the gas market, with the appropriate handling arrangements to move it to their clients or storage facilities. This would enhance the functioning of the gas market and facilitate a maximally efficient allocating of volumes of gas at market prices, reflecting its value to producers and consumers.

A third principle, involved the notion that the gas infrastructure inherited from the traditional gas industry still reflected the consequences of its natural monopoly status. Main elements in this respect were ‘gold plated’ over-investment in assets, high operational costs, high tariffs that did not reflect the actual, economically efficient, costs of transport, and awkward practices in discriminating the different types of consumers and regions. To lower the cost of the infrastructure, tariff and/or revenue regulation would have to force the TSOs and DSOs to become more efficient in their operations and the use of their system. Eventually, this evolved into a wide variety of price cap, yardstick, or RPI-X regulation. To this end, each Member State had to establish a National Regulatory Authority (NRA) for the energy sector, with the responsibility to approve and control the tariffs (or methodologies) to ensure non-discriminatory access to the unbundled networks.

Implementation

After some preliminary attempts in the early 1990s, this liberal concept for a gas market was implemented in several steps. With three consecutive Gas Directives ([16--18]) the European Commission provided increasingly explicit and stringent rules, to be implemented by the Member States. Particularly, the 2009 Directive, also called the Third Package, was important in the sense that it unified the different national approaches. This was a reaction to the ‘Sector Enquiry’ carried by the Commission’s Directorate-General Competition [19], in which it concluded that lack of access to the infrastructure and dominant positions of only a few companies were blocking the development of a well-functioning gas market.

Another important element emanating from of the Third Package was the enhanced EU-wide cooperation. As regards the NRAs, in 2009, the Commission established the Agency for the Cooperation of National Energy Regulators (ACER). Since March 2000, the regulators had been cooperating voluntarily in the Council of European Energy Regulators (CEER), working together with the European Regulators Group for Electricity and Gas (ERGEG) established by the Commission. To fill the regulatory vacuum in cross-border situations and to facilitate EU-wide regulatory coordination, ACER was set up to align national market and network operation rules and facilitate investment in trans-European infrastructure. Also the unbundled national TSOs, in 2009, engaged in EU-wide cooperation in the European Network of Transmission System Operators for Gas (ENTSOG) in the promotion of cross-border gas trade and development of the European transmission network, for example, by providing a 10-year gas network development plan.

The results of the enhanced EU-wide cooperation were that further steps were taken towards achieving a ‘frictionless’ cross-border trade of gas in the EU by the gas target model (GTM) initiative (see [20–22]). The GTM is meant to provide cross-border linkages between the national or regional entry/exit areas, implying that instantaneous supply and demand will determine the price of gas in the area, as in a virtual spot market. The delivery of long-term contracted volumes of gas will then take place at the borders of the entry/exit area, forming the market price, also being influenced by occasional inter-area volumes purchased on a short-term basis.

Security of Supply

In the first decade of the twenty-first century, concerns about the security of gas supply were put on the agenda. In 2004, a Directive (2004/67/EC) was adopted, aiming to establish a common framework for the Member States security-of-supply policies, consistent with the requirements of a single gas market. The disputes between Russia and Ukraine in 2006 and 2009 caused supply problems in central south-eastern Europe and led to the adoption of a new regulation (994/2010). This regulation sought to harmonize national measures and introduced common minimum standards for preparedness, to enhance the solidarity between states in case of a crisis.

As the tensions between Russia, the Ukraine and the EU continued to increase, in 2014, a so-called stress test was undertaken; security of gas supply and recently (in 2016) a draft proposal for a more extensive security of supply regulation was published. This draft addresses the industry, the Member States grouped in specific regions, as being responsible for safeguarding the supply to protected customers, and the European Commission in charge of general coordination and consistency. The draft highlights a stronger cooperation in regional preventive action plans and emergency plans. Moreover, it proposes an infrastructure standard, which guarantees the possibility of supplying gas even if the largest infrastructure is not available, while enabling permanent bi-directional transmission capacity [23].

Gas in the Energy Union Policy

In February 2015, the Commission announced its Energy Union policy in which it announced a fundamental transformation of Europe’s energy system. In the light of an ambitious EU climate policy, it aims to give EU consumers secure, sustainable, competitive and affordable energy [24]. Main components of this policy are as follows: (1) energy security, solidarity and trust; (2) a fully integrated European energy market; (3) energy efficiency contributing to moderation of demand; (4) decarbonising the economy and (5) research, innovation and competitiveness. It is stated that the EU has ‘to move away from an economy driven by fossil fuels, an economy where energy is based on a centralized, supply-side approach and which relies on old technologies and outdated business models.’ ([1]: 2).

Observations and objectives as regards the role of gas are mainly formulated under the heading of Energy security, solidarity and trust, with a strong focus on the need to diversify energy sources, suppliers and transport routes. Key elements are as follows: firstly, the development of a Southern Gas Corridor to enable central Asian countries to deliver gas to Europe; secondly, the establishment of liquid gas hubs with multiple suppliers in northern Europe, as well as in central and eastern Europe and the Mediterranean area; thirdly, the construction of additional transport infrastructure, supported by community funding instruments European financial institutions; fourthly, the preparation of an LNG strategy to provide for back-up in crisis situations and including the transport infrastructure connecting LNG access points and the creation of gas storages in Europe. Finally, as regards the domestic energy production to reduce import dependence, the plan refers ‘notably’ to renewables, needed for decarbonisation, as well as conventional and—for those Member States that choose it—unconventional oil and gas ‘provided that issues of public acceptance and environmental impact are adequately addressed’ ([1]: 5).

Developments in the Gas Market

Parallel to the implementation of the consecutive gas directives, and particularly since the turn of the century, important economic and political changes have taken place on the supply as well as on the demand side of the European gas market and in the international gas market. These developments have considerably altered the context in which the European gas market functions.

On the supply side, the European indigenous gas reserves, in the Netherlands, the UK and Germany, and France and Italy are slowly but surely in decline. In the Netherlands, moreover, gas production-induced earthquakes in the huge Groningen field have forced the government to limit the annual production to 24 Bcm; about half the amount it produced in 2013. Apart from Norway, the potential for supply expansion is minimal within the EU area. The role of unconventional gas seems limited in Europe, as a consequence of popular resistance and, possibly, the regulatory difficulties in creating adequate infrastructures.

Current European gas imports are mainly transported via pipelines from Russia, Norway, Algeria and Libya, being produced at fields that were taken in production in the 1980s. These imports are governed by the traditional long-term contracts, although both Norway and Russia do accept renegotiations of the main parameters, in order to keep their supply competitive versus potential alternative suppliers. Although there is a definite potential to expand the supply capacity of these countries and to bring new reserves on-stream, this will require huge investments in production and transport mission facilities. These fields are more remote and expensive to develop than the traditional field, as they are located in ‘difficult’ areas in Russia and off-shore Norway in the North Atlantic.

New potential pipeline gas suppliers are located to the south-east of Europe, in countries like Turkmenistan, Kurdistan, Iran, Azerbaijan and the East Mediterranean area, involving Cyprus, Israel and Egypt. Yet, most of these supply options face complex political circumstances, either because of internal instabilities or conflictive international relations. Also the transit issue is at stake here, particularly in respect of the position of Turkey, but also regarding the many other states involved. These complexities, including the relations with Russia, have hampered the construction of any pipelines in the Southern corridor, so far.

An option that is often proclaimed as the ‘ideal’ solution to Europe’s energy and gas dependency is the evolving global LNG market. A huge growth has taken place the LNG supply potential from traditional and new reserves, particularly in the wake of the US shale gas revolution. This gas could be imported into the EU market via LNG re-gasification capacity along the European shores.

Also on the demand-side, the situation has changed significantly. Initially, the use of gas in Europe was growing slowly in those countries in Europe where gas is a traditionally important source of energy. In other countries, like Spain, gas was a relatively new fuel; rapidly gaining market share and raising great expectations, because of the advantageous environmental aspects, the comfort it provides and economic advantages. So in the early twenty-first century, the perspective for natural gas in Europe was bright. Yet, over time, the future perspective for natural gas has considerably weakened.

A main reason has been the post-2008 economic downturn that has caused an overall decline in the use of gas. Although the European economy is not growing fast anyway, it remains to be seen how much of this demand may return in the future. Particularly over the recent past, and definitely after the Paris agreements in December 2015, the public image of natural gas has weakened. Undeniably, the policies of some European countries, like Germany [4] and Spain, have stimulated the emergence of a huge potential of wind and solar power.

To an increasing extent, gas is considered a ‘dirty’ fossil fuel. In terms of its position in future energy scenarios, the role of gas as the ‘bridging fuel towards sustainability’ has lost credibility. This may have been a consequence of the widespread aversion of local communities against shale gas and ‘fracking’. In the Netherlands, the induced earthquakes in the huge Groningen field are generating a growing resistance against gas in general, including the exploration of other much smaller occurrences.

Shale gas extraction in the USA is also indirectly influential in the EU gas market. As a consequence of the substitution of low-cost unconventional gas for coal in the USA, low cost coal has become available to the EU power sector. Hence, even in the traditional gas consuming power sectors in the EU, coal has become the fuel of preference, driving gas out of the merit order.

Conclusions

The changes above have important consequences and affect the prevailing perspective of what constitutes a ‘well-functioning’ EU gas market. First of all, we observe a continuous drive towards the creation of a competitive gas market by means of sector regulation, as regards the increasingly unbundled pipeline, storage and LNG infrastructure, and by means of competition policy, as regards the dominant positions of large suppliers and traders in the industry.

The implication is that investments in new supply infrastructures are strongly dependent on, on the one hand, investor’s and TSO’s expectations of their usability in the light of future gas flows, and on the other, on regulatory decision-making. In respect of the huge requirements in transport, storage and LNG infrastructure emanating from the objectives to internally secure the gas supply to regions and to connect liquid market areas, as outlined in the Energy Union plan, it looks unlikely that private parties are willing to provide the required investments.

As regards the interest of producers and traders in supplying gas to the EU market, both demand and price expectations play a crucial role. In respect of the demand side, we have explained that the expectations regarding the future use of gas are highly uncertain because of the ambitious objective to move away from fossil fuels, the more or less steady state of the European economy and, at least currently, the disadvantage of gas in power generation, in competition with zero-marginal cost renewable and low cost coal.

As regards the supply-side, we have suggested that apart from the traditional suppliers, including Russia and Norway, there are only meagre alternatives present for pipeline supplies, as a consequence of the political instability in many potential producing countries and transit troubles. Indigenous supply is being depleted. This leaves LNG as the main option for future supply, either via reliable long-term contracts when that may prove feasible or via spot cargos at the world market price. The former arrangement, however, requires certainty of demand to justify the required investments and the willingness to pay the price. Both requirements are not easily met in the current context. The latter arrangement carries a risk. On the current over supplied gas market, the price is low. Yet, that may change.

Probably the main insight of this longer term perspective over the evolution of the European natural gas policy is that a strongly regulated version of a ‘well-functioning’ gas market remains a highly politicized and instable experiment. As stated in the introduction, and illustrated in the sections above, the values attached to natural gas as a source of energy in the European economy are constantly shifting between the economics, to security of supply and sustainability. Moreover, the importance attached to these values and their operationalization will be different in the various parts of Europe. Therewith, the creation of ‘well-functioning’ gas market will always remain a highly politicized and never ending story.