Abstract
Current electricity and natural gas markets operate with deterministic description of uncertain supply, and in a temporally and sectorally decoupled way. This practice in energy systems is being challenged by the increasing integration of stochastic renewable energy sources. There is a growing need for exchanging operational flexibility among energy sectors, which requires to improve the sectoral coordination between electricity and natural gas markets. In addition, the dispatch of flexible units in both sectors needs to be made in a more uncertainty-aware manner, requiring to strengthen the temporal coordination between day-ahead and real-time energy markets. We explore the use of existing financial instruments in the form of virtual bidding (VB) as a market-based solution to enhance both sectoral and temporal coordination in energy markets. It is established in the literature that VB by purely financial players is able to enhance the temporal coordination between deterministic day-ahead and real-time markets. By developing various stochastic equilibrium and optimization models, we show that VB by physical players, i.e., gas-fired power plants, at the interface of power and natural gas systems is of great potential to improve not only the temporal coordination between deterministic day-ahead and real-time markets, but also the sectoral coordination between deterministic electricity and natural gas markets. We exploit a fully stochastic co-optimization model as an ideal benchmark, and numerically illustrate the benefits of VB for increasing the overall market efficiency in terms of reduced expected operational cost of the entire energy system. We eventually show that flexible resources in both electricity and natural gas markets are dispatched more efficiently in the day-ahead stage when VB exists.
Similar content being viewed by others
Notes
By operational flexibility, we refer to the capability of a power system to modify its output or state in response to a change in renewable power production (Zhao et al. 2016).
Note that this definition of loose temporal coordination should not be confused with the issue that the timing of gas nomination cycles is not necessarily harmonized with the needs of the power industry. We do not address such a harmonization issue in this paper.
To relax this assumption, we will use later an out-of-sample analysis in our numerical study, where the realized RT prices are different than those within the stochastic program of the virtual bidders. This analysis shows that imperfect virtual bidding can still improve the coordination but to a limited extent.
The merit order refers to placing the power plants with an ascending order of marginal production costs.
Although \(\varvec{\Delta }{} \mathbf{v} ^{\mathrm{E}}_{r}\) is a recourse variable, it is not indexed by \(\omega \). The reason for this is that throughout all scenarios, the RT position of the explicit virtual bidder should be identical. Mathematically speaking, this variable can take a scenario index to become \(\varvec{\Delta }{} \mathbf{v} ^{\mathrm{E}}_{r,\omega }\). However, constraint (1b) would enforce again all those recourse variables over scenarios to take an identical value.
In the rest of the manuscript we use the terms implicit virtual bidder and self-scheduler interchangeably.
References
Birge J, Hortaçsu A, Mercadal I, Pavlin M (2018) Limits to arbitrage in electricity markets: a case study of MISO. Energy Econ 75:518–533
Bobo L, Mitridati L, Taylor JA, Pinson P, Kazempour J (2021) Price-region bids in electricity markets. Eur J Oper Res. https://doi.org/10.1016/j.ejor.2021.03.024 (to be published)
Borraz-Sánchez C, Bent R, Backhaus S, Hijazi H, Van Hentenryck P (2016) Convex relaxations for gas expansion planning. INFORMS J Comput 28(4):645–656
Byeon G, Van Hentenryck P (2020) Unit commitment with gas network awareness. IEEE Trans Power Syst 35(2):1327–1339
Carter R, Backhaus S, Hollis A, Zlotnik A, Chertkov M, Giacomoni A, Daniels A (2016) Impact of regulatory change to coordinate gas pipelines and power systems. In: Pipeline Simulation Interest Group (PSIG 2016) conference. Vancouver, British Columbia, pp 1–15
Chen R, Wang J, Sun H (2017) Clearing and pricing for coordinated gas and electricity day-ahead markets considering wind power uncertainty. IEEE Trans Power Syst 33:2496–2508
Chen S, Conejo AJ, Sioshansi R, Wei Z (2019) Unit commitment with an enhanced natural gas-flow model. IEEE Trans Power Syst 34(5):3729–3738
Chen S, Conejo AJ, Sioshansi R, Wei Z (2020) Operational equilibria of electric and natural gas systems with limited information interchange. IEEE Trans Power Syst 35(1):662–671
Cleland N, Zakeri G, Pritchard G, Young B (2015) Boomer-consumer: a model for load consumption and reserve offers in reserve constrained electricity markets. CMS 12:519–537
Conejo AJ, Nogales FJ, Arroyo JM, García-Bertrand R (2004) Risk-constrained self-scheduling of a thermal power producer. IEEE Trans Power Syst 19(3):1569–1574
Correa-Posada CM, Sánchez-Martín P (2015) Integrated power and natural gas model for energy adequacy in short-term operation. IEEE Trans Power Syst 30(6):3347–3355
Craig M, Guerra OJ, Brancucci C, Pambour KA, Hodge BM (2020) Valuing intra-day coordination of electric power and natural gas system operations. Energy Policy 141:111470
Dall’Anese E, Mancarella P, Monti A (2017) Unlocking flexibility: Integrated optimization and control of multienergy systems. IEEE Power Energy Mag 15(1):43–52
Daraeepour A, Patino-Echeverri D, Conejo AJ (2019) Economic and environmental implications of different approaches to hedge against wind production uncertainty in two-settlement electricity markets: A PJM case study. Energy Econ 80:336–354
Doherty R, O’Malley M (2005) A new approach to quantify reserve demand in systems with significant installed wind capacity. IEEE Trans Power Syst 20:587–595
Dominguez R, Oggioni G, Smeers Y (2019) Reserve procurement and flexibility services in power systems with high renewable capacity: effects of integration on different market designs. Int J Electr Power Energy Syst 113:1014–1034
Dvorkin V, Delikaraoglou S, Morales JM (2019) Setting reserve requirements to approximate the efficiency of the stochastic dispatch. IEEE Trans Power Syst 34(2):1524–1536
Dvorkin V Jr, Kazempour J, Pinson P (2019) Electricity market equilibrium under information asymmetry. Oper Res Lett 47(6):521–526
Energinet (2021) Danish transmission system operator. https://en.energinet.dk/Gas
Facchinei F, Kanzow C (2007) Generalized Nash equilibrium problems. 4OR 5(3):173–210
Facchinei F, Pang JS (2007) Finite-dimensional variational inequalities and complementarity problems. Springer, Berlin
Fleten SF, Nasakkala E (2010) Gas-fired power plants: investment timing, operating flexibility and CO2 capture. Energy Econ 32(4):805–816
Fukushima M (2011) Restricted generalized Nash equilibria and controlled penalty algorithm. CMS 8:201–218
Gil J, Caballero A, Conejo AJ (2014) Power cycling: CCGTs: the critical link between the electricity and natural gas markets. IEEE Power Energy Mag 12(6):40–48
Guo X, Beskos A, Siddiqui A (2016) The natural hedge of a gas-fired power plant. CMS 13:63–86
Hallack M, Vazquez M (2013) European union regulation of gas transmission services: challenges in the allocation of network resources through entry/exit schemes. Util Policy 25:23–32
Harker PT (1991) Generalized Nash games and quasi-variational inequalities. Eur J Oper Res 54(1):81–94
Harker PT, Pang JS (1990) Finite-dimensional variational inequality and nonlinear complementarity problems: a survey of theory, algorithms and applications. Math Program 48(1–3):161–220
Heinen S, Hewicker C, Jenkins N, McCalley J, O’Malley M, Pasini S, Simoncini S (2017) Unleashing the flexibility of gas: innovating gas systems to meet the electricity system’s flexibility requirements. IEEE Power Energ Mag 15(1):16–24
Hogan WW (2016) Virtual bidding and electricity market design. Electr J 29(5):33–47
Höschle H, Le Cadre H, Smeers Y, Papavasiliou A, Belmans R (2018) An ADMM-based method for computing risk-averse equilibrium in capacity markets. IEEE Trans Power Syst 33:4819–4830
Hua B, Baldick R (2017) A convex primal formulation for convex hull pricing. IEEE Trans Power Syst 32:3814–3823
Isemonger AG (2006) The benefits and risks of virtual bidding in multi-settlement markets. Electr J 19(9):26–36
Ito K, Reguant M (2016) Sequential markets, market power, and arbitrage. Am Econ Rev 106:1921–1957
Jha A, Wolak FA (2015) Testing for market efficiency with transaction costs: an application to financial trading in wholesale electricity markets. Stanford Working Paper. https://web.stanford.edu/group/fwolak/cgi-bin/sites/default/files/CAISO_VB_draft_VNBER_final_V2.pdf
Jonsson T, Pinson P, Madsen H (2010) On the market impact of wind energy forecasts. Energy Econ 32(2):313–320
Kazempour J, Hobbs BF (2018) Value of flexible resources, virtual bidding, and self-scheduling in two-settlement electricity markets with wind generation- Part I: principles and competitive model. IEEE Trans Power Syst 33(1):749–759
Kohansal M, Sadeghi-Mobarakeh A, Manshadi SD, Mohsenian-Rad H (2020) Strategic convergence bidding in nodal electricity markets: optimal bid selection and market implications. IEEE Trans Power Syst Publ. https://doi.org/10.1109/TPWRS.2020.3025098
Krawczyk J (2007) Numerical solutions to coupled-constraint (or generalised Nash) equilibrium problems. CMS 4:183–204
Kulkarni AA, Shanbhag UV (2012) On the variational equilibrium as a refinement of the generalized Nash equilibrium. Automatica 48(1):45–55
Li R, Svoboda AJ, Oren SS (2015) Efficiency impact of convergence bidding in the California electricity market. J Regul Econ 48(3):245–284
Liu Y, Holzer JT, Ferris MC (2015) Extending the bidding format to promote demand response. Energy Policy 86:82–92
Lo Prete C, Guo N, Shanbhag UV (2019a) Virtual bidding and financial transmission rights: an equilibrium model for cross-product manipulation in electricity markets. IEEE Trans Power Syst 34:953–967
Lo Prete C, Hogan WW, Liu B, Wang J (2019b) Cross-product manipulation in electricity markets, microstructure models and asymmetric information. Energy J 40(5):221–246
Manshadi SD, Khodayar ME (2019) Coordinated operation of electricity and natural gas systems: a convex relaxation approach. IEEE Trans Smart Grid 10(3):3342–3354
Mather J, Bitar E, Poolla K (2017) Virtual bidding: equilibrium, learning, and the wisdom of crowds. IFAC-PapersOnLine 50(1):225–232
Mays J, Morton D, O’Neill R (2019) Asymmetric risk and fuel neutrality in capacity markets. Nat Energy 4:948–956
Meibom P, Hilger KB, Madsen H, Vinther D (2013) Energy comes together in Denmark: the key to a future fossil-free Danish power system. IEEE Power Energ Mag 11(5):46–55
Morales JM, Pineda S (2017) On the inefficiency of the merit order in forward electricity markets with uncertain supply. Eur J Oper Res 261:789–799
Morales JM, Conejo A, Liu K, Zhong J (2012) Pricing electricity in pools with wind producers. IEEE Trans Power Syst 27(3):1366–1376
Morales JM, Zugno M, Pineda S, Pinson P (2014) Electricity market clearing with improved scheduling of stochastic production. Eur J Oper Res 235(3):765–774
NERC (2010) Flexibility requirements and metrics for variable generation: implications for system planning studies. In: Technical report of North American Electric Reliability Corporation. https://www.nerc.com/files/IVGTF_Task_1_4_Final.pdf
Nicholson E, Quinn A (2019) Wholesale electricity markets in the united states: identifying future challenges facing commercial energy. IEEE Power Energ Mag 17(1):67–72
O’Connell N, Pinson P, Madsen H, O’Malley M (2016) Economic dispatch of demand response balancing through asymmetric block offers. IEEE Trans Power Syst 31:2999–3007
O’Malley C, Delikaraoglou S, Hug G (2019) Improving electricity and natural gas systems coordination using swing option contracts. In: IEEE PES PowerTech conference 2019. Italy, Milan, pp 1–6
Ordoudis C, Pinson P, Morales JM (2019) An integrated market for electricity and natural gas systems with stochastic power producers. Eur J Oper Res 272(2):642–654
Ordoudis C, Delikaraoglou S, Kazempour J, Pinson P (2020) Market-based coordination for integrated electricity and natural gas systems under uncertain supply. Eur J Oper Res 287(3):1105–1119
Orvis R, Aggarwal S (2018) Refining competitive electricity market rules to unlock flexibility. Electr J 31(5):31–37
Pang JS, Fukushima M (2005) Quasi-variational inequalities, generalized Nash equilibria, and multi-leader-follower games. CMS 2:21–56
Papavasiliou A, He Y, Svoboda A (2015) Self-commitment of combined cycle units under electricity price uncertainty. IEEE Trans Power Syst 30(4):1690–1701
Papavasiliou A, Smeers Y, de Maere d’Aertrycke G (2021) Market design considerations for scarcity pricing: a stochastic equilibrium framework. Energy J 42:195–220
Parsons JE, Colbert C, Larrieu J, Martin T, Mastrangelo E (2015) Financial arbitrage and efficient dispatch in wholesale electricity markets. MIT Center for Energy and Environmental Policy Research. http://www.mit.edu/~jparsons/publications/20150300_Financial_Arbitrage_and_Efficient_Dispatch.pdf
Pritchard G, Zakeri G, Philpott A (2010) A single-settlement, energy-only electric power market for unpredictable and intermittent participants. Oper Res 58:1210–1219
Ratha A, Schwele A, Kazempour J, Pinson P, Torbaghan SS, Virag A (2020) Affine policies for flexibility provision by natural gas networks to power systems. Electr Power Syst Res 189:106565
Roald LA, Sundar K, Zlotnik A, Misra S, Andersson G (2020) An uncertainty management framework for integrated gas-electric energy systems. Proc IEEE 108(9):1518–1540
Savelli I, Cornelusse B, Giannitrapani A, Paoletti S, Vicino A (2018) A new approach to electricity market clearing with uniform purchase price and curtailable block orders. Appl Energy 226:618–630
Schewe L, Schmidt M, Thürauf J (2020) Computing technical capacities in the European entry-exit gas market is NP-hard. Ann Oper Res 295:337–362
Schiro DA, Pang JS, Shanbhag UV (2013) On the solution of affine generalized Nash equilibrium problems with shared constraints by Lemke’s method. Math Progr 142(1–2):1–46
Schwele A, Ordoudis C, Kazempour J, Pinson P (2019) Coordination of power and natural gas systems: convexification approaches for linepack modeling. In: IEEE PES PowerTech conference 2019. Italy, Milan, pp 1–6
Schwele A, Ordoudis C, Kazempour J, Pinson P (2021) Electronic companion: coordination of power and natural gas markets via financial instruments. https://zenodo.org/badge/latestdoi/293420700
Sioshansi R, Oren S, O’Neill R (2010) Three-part auctions versus self-commitment in day-ahead electricity markets. Util Policy 18:165–173
Tabors RD, Englander S, Stoddard R (2012) Who’s on first? The coordination of gas and power scheduling. Electr J 25(5):8–15
Wang B, Hobbs BF (2016) Real-time markets for flexiramp: a stochastic unit commitment-based analysis. IEEE Trans Power Syst 31:846–860
Warrington J, Goulart P, Marithoz S, Morari M (2013) Policy-based reserves for power systems. IEEE Trans Power Syst 28:4427–4437
Yang J, Zhang N, Kang C, Xia Q (2018) Effect of natural gas flow dynamics in robust generation scheduling under wind uncertainty. IEEE Trans Power Syst 33:2087–2097
Zakeri G, Pritchard G, Bjorndal M, Bjorndal E (2019) Pricing wind: a revenue adequate, cost recovering uniform price auction for electricity markets with intermittent generation. INFORMS J Optim 1(1):35–48
Zavala VM, Kim K, Anitescu M, Birge J (2017) A stochastic electricity market clearing formulation with consistent pricing properties. Oper Res 65(3):557–576
Zhao B, Zlotnik A, Conejo AJ, Sioshansi R, Rudkevich AM (2019) Shadow price-based co-ordination of natural gas and electric power systems. IEEE Trans Power Syst 34(3):1942–1954
Zhao J, Zheng T, Litvinov E (2016) A unified framework for defining and measuring flexibility in power system. IEEE Trans Power Syst 31:339–347
Zlotnik A, Roald L, Backhaus S, Chertkov M, Andersson G (2016) Control policies for operational coordination of electric power and natural gas transmission systems. In: American control conference (ACC) 2016, pp 7478–7483
Zugno M, Conejo AJ (2015) A robust optimization approach to energy and reserve dispatch in electricity markets. Eur J Oper Res 247(2):659–671
Acknowledgements
We thank Benjamin F. Hobbs and Stefanos Delikaraoglou for thoughtful discussions about the initial model. We also thank Kenneth Bruninx and Anubhav Ratha for providing constructive feedback on previous drafts. We are very grateful to Uday V. Shanbhag, Maryam Kamgarpour and Jong Shi Pang for their fruitful inputs on generalized Nash equilibrium model. In addition, we thank Yves Smeers and Ibrahim Abada for their helpful comments at the 23rd International Symposium on Mathematical Programming (ISMP 2018). Finally, we thank the three anonymous reviewers for their constructive feedback. This research was supported by the Danish Energy Development Programme (EUDP) through the Coordinated Operation of Integrated Energy Systems (CORE) project (64017-0005).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Schwele, A., Ordoudis, C., Pinson, P. et al. Coordination of power and natural gas markets via financial instruments. Comput Manag Sci 18, 505–538 (2021). https://doi.org/10.1007/s10287-021-00403-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10287-021-00403-x