Skip to main content

Coordination of power and natural gas markets via financial instruments

Computational Management Science volume 18, pages 505–538 (2021)Cite this article

Abstract

Current electricity and natural gas markets operate with deterministic description of uncertain supply, and in a temporally and sectorally decoupled way. This practice in energy systems is being challenged by the increasing integration of stochastic renewable energy sources. There is a growing need for exchanging operational flexibility among energy sectors, which requires to improve the sectoral coordination between electricity and natural gas markets. In addition, the dispatch of flexible units in both sectors needs to be made in a more uncertainty-aware manner, requiring to strengthen the temporal coordination between day-ahead and real-time energy markets. We explore the use of existing financial instruments in the form of virtual bidding (VB) as a market-based solution to enhance both sectoral and temporal coordination in energy markets. It is established in the literature that VB by purely financial players is able to enhance the temporal coordination between deterministic day-ahead and real-time markets. By developing various stochastic equilibrium and optimization models, we show that VB by physical players, i.e., gas-fired power plants, at the interface of power and natural gas systems is of great potential to improve not only the temporal coordination between deterministic day-ahead and real-time markets, but also the sectoral coordination between deterministic electricity and natural gas markets. We exploit a fully stochastic co-optimization model as an ideal benchmark, and numerically illustrate the benefits of VB for increasing the overall market efficiency in terms of reduced expected operational cost of the entire energy system. We eventually show that flexible resources in both electricity and natural gas markets are dispatched more efficiently in the day-ahead stage when VB exists.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Notes

  1. By operational flexibility, we refer to the capability of a power system to modify its output or state in response to a change in renewable power production (Zhao et al. 2016).

  2. Note that this definition of loose temporal coordination should not be confused with the issue that the timing of gas nomination cycles is not necessarily harmonized with the needs of the power industry. We do not address such a harmonization issue in this paper.

  3. To relax this assumption, we will use later an out-of-sample analysis in our numerical study, where the realized RT prices are different than those within the stochastic program of the virtual bidders. This analysis shows that imperfect virtual bidding can still improve the coordination but to a limited extent.

  4. The merit order refers to placing the power plants with an ascending order of marginal production costs.

  5. Although \(\varvec{\Delta }{} \mathbf{v} ^{\mathrm{E}}_{r}\) is a recourse variable, it is not indexed by \(\omega \). The reason for this is that throughout all scenarios, the RT position of the explicit virtual bidder should be identical. Mathematically speaking, this variable can take a scenario index to become \(\varvec{\Delta }{} \mathbf{v} ^{\mathrm{E}}_{r,\omega }\). However, constraint (1b) would enforce again all those recourse variables over scenarios to take an identical value.

  6. In the rest of the manuscript we use the terms implicit virtual bidder and self-scheduler interchangeably.

References

  • Birge J, Hortaçsu A, Mercadal I, Pavlin M (2018) Limits to arbitrage in electricity markets: a case study of MISO. Energy Econ 75:518–533

    Article  Google Scholar 

  • Bobo L, Mitridati L, Taylor JA, Pinson P, Kazempour J (2021) Price-region bids in electricity markets. Eur J Oper Res. https://doi.org/10.1016/j.ejor.2021.03.024 (to be published)

    Article  Google Scholar 

  • Borraz-Sánchez C, Bent R, Backhaus S, Hijazi H, Van Hentenryck P (2016) Convex relaxations for gas expansion planning. INFORMS J Comput 28(4):645–656

    Article  Google Scholar 

  • Byeon G, Van Hentenryck P (2020) Unit commitment with gas network awareness. IEEE Trans Power Syst 35(2):1327–1339

    Article  Google Scholar 

  • Carter R, Backhaus S, Hollis A, Zlotnik A, Chertkov M, Giacomoni A, Daniels A (2016) Impact of regulatory change to coordinate gas pipelines and power systems. In: Pipeline Simulation Interest Group (PSIG 2016) conference. Vancouver, British Columbia, pp 1–15

  • Chen R, Wang J, Sun H (2017) Clearing and pricing for coordinated gas and electricity day-ahead markets considering wind power uncertainty. IEEE Trans Power Syst 33:2496–2508

    Article  Google Scholar 

  • Chen S, Conejo AJ, Sioshansi R, Wei Z (2019) Unit commitment with an enhanced natural gas-flow model. IEEE Trans Power Syst 34(5):3729–3738

    Article  Google Scholar 

  • Chen S, Conejo AJ, Sioshansi R, Wei Z (2020) Operational equilibria of electric and natural gas systems with limited information interchange. IEEE Trans Power Syst 35(1):662–671

    Article  Google Scholar 

  • Cleland N, Zakeri G, Pritchard G, Young B (2015) Boomer-consumer: a model for load consumption and reserve offers in reserve constrained electricity markets. CMS 12:519–537

    Article  Google Scholar 

  • Conejo AJ, Nogales FJ, Arroyo JM, García-Bertrand R (2004) Risk-constrained self-scheduling of a thermal power producer. IEEE Trans Power Syst 19(3):1569–1574

    Article  Google Scholar 

  • Correa-Posada CM, Sánchez-Martín P (2015) Integrated power and natural gas model for energy adequacy in short-term operation. IEEE Trans Power Syst 30(6):3347–3355

    Article  Google Scholar 

  • Craig M, Guerra OJ, Brancucci C, Pambour KA, Hodge BM (2020) Valuing intra-day coordination of electric power and natural gas system operations. Energy Policy 141:111470

    Article  Google Scholar 

  • Dall’Anese E, Mancarella P, Monti A (2017) Unlocking flexibility: Integrated optimization and control of multienergy systems. IEEE Power Energy Mag 15(1):43–52

    Article  Google Scholar 

  • Daraeepour A, Patino-Echeverri D, Conejo AJ (2019) Economic and environmental implications of different approaches to hedge against wind production uncertainty in two-settlement electricity markets: A PJM case study. Energy Econ 80:336–354

    Article  Google Scholar 

  • Doherty R, O’Malley M (2005) A new approach to quantify reserve demand in systems with significant installed wind capacity. IEEE Trans Power Syst 20:587–595

    Article  Google Scholar 

  • Dominguez R, Oggioni G, Smeers Y (2019) Reserve procurement and flexibility services in power systems with high renewable capacity: effects of integration on different market designs. Int J Electr Power Energy Syst 113:1014–1034

    Article  Google Scholar 

  • Dvorkin V, Delikaraoglou S, Morales JM (2019) Setting reserve requirements to approximate the efficiency of the stochastic dispatch. IEEE Trans Power Syst 34(2):1524–1536

    Article  Google Scholar 

  • Dvorkin V Jr, Kazempour J, Pinson P (2019) Electricity market equilibrium under information asymmetry. Oper Res Lett 47(6):521–526

    Article  Google Scholar 

  • Energinet (2021) Danish transmission system operator. https://en.energinet.dk/Gas

  • Facchinei F, Kanzow C (2007) Generalized Nash equilibrium problems. 4OR 5(3):173–210

    Article  Google Scholar 

  • Facchinei F, Pang JS (2007) Finite-dimensional variational inequalities and complementarity problems. Springer, Berlin

    Google Scholar 

  • Fleten SF, Nasakkala E (2010) Gas-fired power plants: investment timing, operating flexibility and CO2 capture. Energy Econ 32(4):805–816

    Article  Google Scholar 

  • Fukushima M (2011) Restricted generalized Nash equilibria and controlled penalty algorithm. CMS 8:201–218

    Article  Google Scholar 

  • Gil J, Caballero A, Conejo AJ (2014) Power cycling: CCGTs: the critical link between the electricity and natural gas markets. IEEE Power Energy Mag 12(6):40–48

    Article  Google Scholar 

  • Guo X, Beskos A, Siddiqui A (2016) The natural hedge of a gas-fired power plant. CMS 13:63–86

    Article  Google Scholar 

  • Hallack M, Vazquez M (2013) European union regulation of gas transmission services: challenges in the allocation of network resources through entry/exit schemes. Util Policy 25:23–32

    Article  Google Scholar 

  • Harker PT (1991) Generalized Nash games and quasi-variational inequalities. Eur J Oper Res 54(1):81–94

    Article  Google Scholar 

  • Harker PT, Pang JS (1990) Finite-dimensional variational inequality and nonlinear complementarity problems: a survey of theory, algorithms and applications. Math Program 48(1–3):161–220

    Article  Google Scholar 

  • Heinen S, Hewicker C, Jenkins N, McCalley J, O’Malley M, Pasini S, Simoncini S (2017) Unleashing the flexibility of gas: innovating gas systems to meet the electricity system’s flexibility requirements. IEEE Power Energ Mag 15(1):16–24

    Article  Google Scholar 

  • Hogan WW (2016) Virtual bidding and electricity market design. Electr J 29(5):33–47

    Article  Google Scholar 

  • Höschle H, Le Cadre H, Smeers Y, Papavasiliou A, Belmans R (2018) An ADMM-based method for computing risk-averse equilibrium in capacity markets. IEEE Trans Power Syst 33:4819–4830

    Article  Google Scholar 

  • Hua B, Baldick R (2017) A convex primal formulation for convex hull pricing. IEEE Trans Power Syst 32:3814–3823

    Article  Google Scholar 

  • Isemonger AG (2006) The benefits and risks of virtual bidding in multi-settlement markets. Electr J 19(9):26–36

    Article  Google Scholar 

  • Ito K, Reguant M (2016) Sequential markets, market power, and arbitrage. Am Econ Rev 106:1921–1957

    Article  Google Scholar 

  • Jha A, Wolak FA (2015) Testing for market efficiency with transaction costs: an application to financial trading in wholesale electricity markets. Stanford Working Paper. https://web.stanford.edu/group/fwolak/cgi-bin/sites/default/files/CAISO_VB_draft_VNBER_final_V2.pdf

  • Jonsson T, Pinson P, Madsen H (2010) On the market impact of wind energy forecasts. Energy Econ 32(2):313–320

    Article  Google Scholar 

  • Kazempour J, Hobbs BF (2018) Value of flexible resources, virtual bidding, and self-scheduling in two-settlement electricity markets with wind generation- Part I: principles and competitive model. IEEE Trans Power Syst 33(1):749–759

    Article  Google Scholar 

  • Kohansal M, Sadeghi-Mobarakeh A, Manshadi SD, Mohsenian-Rad H (2020) Strategic convergence bidding in nodal electricity markets: optimal bid selection and market implications. IEEE Trans Power Syst Publ. https://doi.org/10.1109/TPWRS.2020.3025098

    Article  Google Scholar 

  • Krawczyk J (2007) Numerical solutions to coupled-constraint (or generalised Nash) equilibrium problems. CMS 4:183–204

    Article  Google Scholar 

  • Kulkarni AA, Shanbhag UV (2012) On the variational equilibrium as a refinement of the generalized Nash equilibrium. Automatica 48(1):45–55

    Article  Google Scholar 

  • Li R, Svoboda AJ, Oren SS (2015) Efficiency impact of convergence bidding in the California electricity market. J Regul Econ 48(3):245–284

    Article  Google Scholar 

  • Liu Y, Holzer JT, Ferris MC (2015) Extending the bidding format to promote demand response. Energy Policy 86:82–92

    Article  Google Scholar 

  • Lo Prete C, Guo N, Shanbhag UV (2019a) Virtual bidding and financial transmission rights: an equilibrium model for cross-product manipulation in electricity markets. IEEE Trans Power Syst 34:953–967

    Article  Google Scholar 

  • Lo Prete C, Hogan WW, Liu B, Wang J (2019b) Cross-product manipulation in electricity markets, microstructure models and asymmetric information. Energy J 40(5):221–246

    Google Scholar 

  • Manshadi SD, Khodayar ME (2019) Coordinated operation of electricity and natural gas systems: a convex relaxation approach. IEEE Trans Smart Grid 10(3):3342–3354

    Article  Google Scholar 

  • Mather J, Bitar E, Poolla K (2017) Virtual bidding: equilibrium, learning, and the wisdom of crowds. IFAC-PapersOnLine 50(1):225–232

    Article  Google Scholar 

  • Mays J, Morton D, O’Neill R (2019) Asymmetric risk and fuel neutrality in capacity markets. Nat Energy 4:948–956

    Article  Google Scholar 

  • Meibom P, Hilger KB, Madsen H, Vinther D (2013) Energy comes together in Denmark: the key to a future fossil-free Danish power system. IEEE Power Energ Mag 11(5):46–55

    Article  Google Scholar 

  • Morales JM, Pineda S (2017) On the inefficiency of the merit order in forward electricity markets with uncertain supply. Eur J Oper Res 261:789–799

    Article  Google Scholar 

  • Morales JM, Conejo A, Liu K, Zhong J (2012) Pricing electricity in pools with wind producers. IEEE Trans Power Syst 27(3):1366–1376

    Article  Google Scholar 

  • Morales JM, Zugno M, Pineda S, Pinson P (2014) Electricity market clearing with improved scheduling of stochastic production. Eur J Oper Res 235(3):765–774

    Article  Google Scholar 

  • NERC (2010) Flexibility requirements and metrics for variable generation: implications for system planning studies. In: Technical report of North American Electric Reliability Corporation. https://www.nerc.com/files/IVGTF_Task_1_4_Final.pdf

  • Nicholson E, Quinn A (2019) Wholesale electricity markets in the united states: identifying future challenges facing commercial energy. IEEE Power Energ Mag 17(1):67–72

    Article  Google Scholar 

  • O’Connell N, Pinson P, Madsen H, O’Malley M (2016) Economic dispatch of demand response balancing through asymmetric block offers. IEEE Trans Power Syst 31:2999–3007

    Article  Google Scholar 

  • O’Malley C, Delikaraoglou S, Hug G (2019) Improving electricity and natural gas systems coordination using swing option contracts. In: IEEE PES PowerTech conference 2019. Italy, Milan, pp 1–6

  • Ordoudis C, Pinson P, Morales JM (2019) An integrated market for electricity and natural gas systems with stochastic power producers. Eur J Oper Res 272(2):642–654

    Article  Google Scholar 

  • Ordoudis C, Delikaraoglou S, Kazempour J, Pinson P (2020) Market-based coordination for integrated electricity and natural gas systems under uncertain supply. Eur J Oper Res 287(3):1105–1119

    Article  Google Scholar 

  • Orvis R, Aggarwal S (2018) Refining competitive electricity market rules to unlock flexibility. Electr J 31(5):31–37

    Article  Google Scholar 

  • Pang JS, Fukushima M (2005) Quasi-variational inequalities, generalized Nash equilibria, and multi-leader-follower games. CMS 2:21–56

    Article  Google Scholar 

  • Papavasiliou A, He Y, Svoboda A (2015) Self-commitment of combined cycle units under electricity price uncertainty. IEEE Trans Power Syst 30(4):1690–1701

    Article  Google Scholar 

  • Papavasiliou A, Smeers Y, de Maere d’Aertrycke G (2021) Market design considerations for scarcity pricing: a stochastic equilibrium framework. Energy J 42:195–220

    Article  Google Scholar 

  • Parsons JE, Colbert C, Larrieu J, Martin T, Mastrangelo E (2015) Financial arbitrage and efficient dispatch in wholesale electricity markets. MIT Center for Energy and Environmental Policy Research. http://www.mit.edu/~jparsons/publications/20150300_Financial_Arbitrage_and_Efficient_Dispatch.pdf

  • Pritchard G, Zakeri G, Philpott A (2010) A single-settlement, energy-only electric power market for unpredictable and intermittent participants. Oper Res 58:1210–1219

    Article  Google Scholar 

  • Ratha A, Schwele A, Kazempour J, Pinson P, Torbaghan SS, Virag A (2020) Affine policies for flexibility provision by natural gas networks to power systems. Electr Power Syst Res 189:106565

    Article  Google Scholar 

  • Roald LA, Sundar K, Zlotnik A, Misra S, Andersson G (2020) An uncertainty management framework for integrated gas-electric energy systems. Proc IEEE 108(9):1518–1540

    Article  Google Scholar 

  • Savelli I, Cornelusse B, Giannitrapani A, Paoletti S, Vicino A (2018) A new approach to electricity market clearing with uniform purchase price and curtailable block orders. Appl Energy 226:618–630

    Article  Google Scholar 

  • Schewe L, Schmidt M, Thürauf J (2020) Computing technical capacities in the European entry-exit gas market is NP-hard. Ann Oper Res 295:337–362

    Article  Google Scholar 

  • Schiro DA, Pang JS, Shanbhag UV (2013) On the solution of affine generalized Nash equilibrium problems with shared constraints by Lemke’s method. Math Progr 142(1–2):1–46

    Article  Google Scholar 

  • Schwele A, Ordoudis C, Kazempour J, Pinson P (2019) Coordination of power and natural gas systems: convexification approaches for linepack modeling. In: IEEE PES PowerTech conference 2019. Italy, Milan, pp 1–6

  • Schwele A, Ordoudis C, Kazempour J, Pinson P (2021) Electronic companion: coordination of power and natural gas markets via financial instruments. https://zenodo.org/badge/latestdoi/293420700

  • Sioshansi R, Oren S, O’Neill R (2010) Three-part auctions versus self-commitment in day-ahead electricity markets. Util Policy 18:165–173

    Article  Google Scholar 

  • Tabors RD, Englander S, Stoddard R (2012) Who’s on first? The coordination of gas and power scheduling. Electr J 25(5):8–15

    Article  Google Scholar 

  • Wang B, Hobbs BF (2016) Real-time markets for flexiramp: a stochastic unit commitment-based analysis. IEEE Trans Power Syst 31:846–860

    Article  Google Scholar 

  • Warrington J, Goulart P, Marithoz S, Morari M (2013) Policy-based reserves for power systems. IEEE Trans Power Syst 28:4427–4437

    Article  Google Scholar 

  • Yang J, Zhang N, Kang C, Xia Q (2018) Effect of natural gas flow dynamics in robust generation scheduling under wind uncertainty. IEEE Trans Power Syst 33:2087–2097

    Article  Google Scholar 

  • Zakeri G, Pritchard G, Bjorndal M, Bjorndal E (2019) Pricing wind: a revenue adequate, cost recovering uniform price auction for electricity markets with intermittent generation. INFORMS J Optim 1(1):35–48

    Article  Google Scholar 

  • Zavala VM, Kim K, Anitescu M, Birge J (2017) A stochastic electricity market clearing formulation with consistent pricing properties. Oper Res 65(3):557–576

    Article  Google Scholar 

  • Zhao B, Zlotnik A, Conejo AJ, Sioshansi R, Rudkevich AM (2019) Shadow price-based co-ordination of natural gas and electric power systems. IEEE Trans Power Syst 34(3):1942–1954

    Article  Google Scholar 

  • Zhao J, Zheng T, Litvinov E (2016) A unified framework for defining and measuring flexibility in power system. IEEE Trans Power Syst 31:339–347

    Article  Google Scholar 

  • Zlotnik A, Roald L, Backhaus S, Chertkov M, Andersson G (2016) Control policies for operational coordination of electric power and natural gas transmission systems. In: American control conference (ACC) 2016, pp 7478–7483

  • Zugno M, Conejo AJ (2015) A robust optimization approach to energy and reserve dispatch in electricity markets. Eur J Oper Res 247(2):659–671

    Article  Google Scholar 

Download references

Acknowledgements

We thank Benjamin F. Hobbs and Stefanos Delikaraoglou for thoughtful discussions about the initial model. We also thank Kenneth Bruninx and Anubhav Ratha for providing constructive feedback on previous drafts. We are very grateful to Uday V. Shanbhag, Maryam Kamgarpour and Jong Shi Pang for their fruitful inputs on generalized Nash equilibrium model. In addition, we thank Yves Smeers and Ibrahim Abada for their helpful comments at the 23rd International Symposium on Mathematical Programming (ISMP 2018). Finally, we thank the three anonymous reviewers for their constructive feedback. This research was supported by the Danish Energy Development Programme (EUDP) through the Coordinated Operation of Integrated Energy Systems (CORE) project (64017-0005).

Author information

Authors and Affiliations

  1. Center for Electric Power and Energy, Technical University of Denmark, Kongens Lyngby, Denmark

    Anna Schwele, Christos Ordoudis & Jalal Kazempour

  2. Department of Technology, Management and Economics, Technical University of Denmark, Kongens Lyngby, Denmark

    Pierre Pinson

Authors
  1. Anna Schwele
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christos Ordoudis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pierre Pinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jalal Kazempour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Schwele.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 322 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Schwele, A., Ordoudis, C., Pinson, P. et al. Coordination of power and natural gas markets via financial instruments. Comput Manag Sci 18, 505–538 (2021). https://doi.org/10.1007/s10287-021-00403-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10287-021-00403-x

Keywords

  • Electricity market
  • Generalized Nash equilibrium
  • Natural gas market
  • Sectoral and temporal coordination
  • Stochastic programming
  • Virtual bidding