Engineering Energy Markets: The Past, the Present, and the Future

Since the beginning of the energy sector liberalization, the design of energy markets has become a prominent field of research. Markets nowadays facilitate efficient resource allocation in many fields of energy system operation, such as plant dispatch, control reserve provisioning, delimitation of related carbon emissions, grid congestion management, and, more recently, smart grid concepts and local energy trading. Therefore, good market designs play an important role in enabling the energy transition toward a more sustainable energy supply for all. In this chapter, we retrace how market engineering shaped the development of energy markets and how the research focus shifted from national wholesale markets to more decentralized and location-sensitive concepts.


Introduction
Power markets rank among the most complex marketplaces operated at present. Several interrelated markets have evolved in many countries, which together facilitate efficient and reliable operation of the electricity system. Starting from wholesale power exchanges and control reserve procurement platforms, new products and trading platforms came into play, such as emission allowance exchanges, marketplaces for redispatch, and other flexibility products or local energy markets.
Following the postulation formulated by Roth (2002) or Weinhardt et al. (2003), among others, markets should be designed using engineering methods. The complexity of the electricity sector and its high importance for a competitive economy calls for modeling methods that help gain insights into the dynamics of power markets and that are capable of properly representing the relevant complex aspects. Computational methods and experiments are useful engineering tools that support analyses in the process of designing complex markets. Figure 1 summarizes important periods and events of the energy market development. These can be seen as milestones on the way toward more sustainable energy systems. In the European Union, the vision of Clean energy for all Europeans nicely phrases the direction into which the energy system is planned to evolve, and it relies to a large extent on markets to ensure efficiency, facilitate emissions management, and allow consumers to actively participate in energy system operation.
This contribution recaps the history of the most prominent research topics in energy market engineering. The methods applied are reflected in the context of these research questions. It is shown how the market engineering framework  contributes valuable elements to power market design. Finally, an outlook on the most urgent research challenges in the next years is given.

Energy Sector Developments
In the following, some prominent market design questions related to energy sector developments are recapped, and the methods applied in the respective context are reviewed. As sketched in Fig. 1, the development starts with market liberalization (Sect. 2.1) and the later introduction of carbon trading (Sect. 2.2). It continues with integrating the demand side and flexibility (Sect. 2.3) and with managing congestion in highly renewable energy systems (Sect. 2.4). A very recent focus, finally, is peerto-peer energy trading (Sect. 2.5), which fosters consumer/prosumer participation.

Emergence of Power Markets
For the power sector, the 1990s were marked by deregulation policies. In many regions, formerly state-owned energy suppliers were transformed into private companies, network and generation were separated, and the networks were placed under state supervision by regulatory authorities. In this way, a market economy framework was created, which also characterizes the current structure of the energy sector. All this was done with the aim of increasing cost efficiency.
One important development after electricity sector liberalization was the establishment of open electronic power exchanges, which broadened and gradually replaced the formerly prevalent mutual exchange of electric energy among vertically integrated utilities (Strecker and Weinhardt 2001). The double-auction format became the dominant market institution for electricity trading. In a sealed-bid double auction, both buyers and sellers submit bids specifying the prices at which they are willing to buy or sell a certain good. Buying bids are then ranked from the highest to the lowest, selling bids from the lowest to the highest bid price. The intersection of the so-formed supply and demand functions determines the market clearing quantity and gives a range of possible prices from which the market clearing price is chosen according to some arbitrary rule (McAfee and McMillan 1987). One prominent research stream in energy market design was which settlement rulemostly pay-as-bid vs. uniform price-performs best in terms of efficiency, consumer surplus, profits, welfare, and average prices (Federico and Rahman 2003;Kahn et al. 2001;Son et al. 2004) and came to different results.
In many countries, incumbent market players continued to have high market shares, making them the dominant players in power markets. This left large companies with some potential to act strategically and gave rise to research streams focusing on (the potential for) market power exertion, mainly applying game theoretic approaches (Stoft 1999) or agent-based simulation (Borenstein et al. 1999;Nicolaisen et al. 2001;Weidlich and Veit 2008). In comparison, game theoretic approaches to study power markets started to be conducted earlier and remained a bit more popular than agent-based approaches, if the number of published papers in either field is taken as an indicator (as illustrated in Fig. 2). However, Fig. 2 Relative number of publications in power market research; papers with the given keywords in either title or abstract were set in relation to all publications of the same year; plots show development relative to the year with the highest publication count for the respective keyword; the number in brackets shows the total number of papers; data taken from https://app.dimensions. ai/ while game theoretic models allow for analyzing strategic bidding in very specific market settings, agent-based computational models provide a lot of flexibility in representing the system under study, allowing to tackle both long-term decisions (e.g., Czernohous et al. 2003) and short-term reoccurring trading situations (e.g., Weidlich 2008).
In the early years of the current millennium, many governments started to support power generation from renewable energy sources and, in parts, other forms of decentralized energy resources, such as small-scale combined heat and power generation. Since then, the number of generators has been on the rise globally, and system operations became more challenging. Therefore, decentralized markets as coordination mechanisms moved into the focus of research, addressing a number of both technical and economic questions, e.g., Can decentralized markets solve the coordination problem of increasingly dispersed electricity generation? (How) can information and communication technology help organize decentralized markets? To solve these questions, researchers started to use the analogy of decentralized energy systems to the inherently decentralized organization principles of the Internet, providing interesting insights. These studies can be seen as pioneering work in the newly emerging research field of energy informatics. The SESAM ("Self-organization and spontaneity in liberalized and harmonized markets," BMBF, 2003-2007) project was one prominent example for these kinds of research endeavors in the German energy market (Kamper et al. 2005).

Introduction of Emissions Trading
In Europe, the establishment of the first and largest cross-country greenhouse gas emissions trading scheme (EU-ETS, Directive 2003/87/EC of the European Union (EU)) had a considerable impact on electricity market prices. It introduced a limit on the total amount of greenhouse gases that the industries covered by the EU-ETS were allowed to emit. Other countries also introduced similar mechanisms. Trading emission certificates is supposed to efficiently reduce CO 2 emissions at the lowest overall cost (Dales 1968).
One key market design challenge related to emissions trading was the optimal allowance allocation mechanism for existing power plants. Allowances might either be auctioned among the emitters or they can be allocated free of charge, according to a fixed allocation method. Allocation rules can be based on past emissions in a selected base year or on the output that an emitter would have achieved using the best available technology in a reference period. The method of allowance allocation to existing installations does not basically affect the static efficiency of emissions trading. Investment patterns under an emissions trading system depend mainly on the stringency of the overall emission cap and, thus, on the resulting price of allowances. Practice has shown that the emissions cap in the EU-ETS was set rather laxly, which led to low allowance prices and reduced incentives to reduce emissions.
While the allowance method does not affect the mechanism's static efficiency, it influences the competitive situation of companies. Some companies that would be net sellers under grandfathering could become net buyers under benchmarking. Besides, free allocation leads to windfall profits for power generators (Betz et al. 2010), as they would still reflect the allowance's opportunity cost in their bid prices. Moreover, many researchers (e.g., Jung et al. 1996) argued that auctioned allowances would create greater incentives for technology diffusion and adoption than allowances allocated free of charge, since that reduces allowance prices. The innovator can benefit from this price decrease, since he will not have to pay as much for his remaining emissions. In the case of free allocation, however, the price decrease due to innovation would lower the value of the innovator's allowances, which makes innovation less attractive. The EU-ETS opted for free allocation in the pilot phase and introduced auctioning for the power sector in the second trading period.

Empowering the Demand Side
Various European policy initiatives are related to demand-side participation, in particular the Electricity Directive 2009/72/EC, the Energy Efficiency Directive 2012/27/EU, and the e-Directive 2019/944/EU. They stipulate that the national energy regulatory authorities must encourage concepts such as demand response and demand-side market participation. Beginning in 2021, electric mobility solutions must also be encouraged by the EU Member States. Similarly, they must ensure that aggregators can offer aggregation contracts to customers and that citizen energy communities to offer energy services to its members or shareholders can be formed. These regulatory underpinnings have seen a prelude of research activities aiming to model and assess the mechanics of demand response in smart distribution networks.
While key findings date back to the 1980s, the bulk of publications emerged over in the 2010s. Here, we cluster these contributions along the dimensions modeling, marketing, and aggregation.
In the research community, topics addressing load flexibility in the context of energy markets and smart charging of electric vehicles are notably reflected in a similar manner in the number of publications. Figure 3 shows that these topics started to be covered shortly before 2010, had a first peak in 2013, and are now gaining momentum again. Research related to congestion management has been on the agenda for a longer time. Consequently, publications on this topic increased substantially in the last years and are covered further in Sect. 2.4.

Modeling Flexibility
Keshav and Rosenberg (2011) and Ramchurn et al. (2012) illustrate how smart grid design can leverage on concepts from the domains of Internet communication and Relative number of publications on load flexibility and smart charging, following the same approach and data source as in Fig. 2; total number of papers in brackets artificial intelligence. They stress that smart grids are constituted by a multitude of individual actors which are individually small relative to the aggregate system. Therefore, decentralization is a key feature of smart grid systems. It follows that successful coordination of dispersed entities is crucial for enabling effective and efficient systems. Adopting the ideas from experimental and computational economics, these individuals should be interpreted as economic agents (Holland and Miller 1991). By combining appropriate agent models, the emergent aggregate behavior of an agent population can be used to characterize the likely system behavior (Smith 1982). This is of particular interest when new market platforms need to be evaluated before their deployment (Bichler et al. 2010;Weinhardt et al. 2003).
The fundamental building blocks of modeling and managing decentral energy market actors were already identified in the 1980s: In particular, Constantopoulos et al. (1983) and Schweppe et al. (1988) laid out the general idea of modeling customers facing varying electricity prices. Models of particular load types included space heating (O'Rourke and Schweppe 1983), electric vehicle charging (Heydt 1983), and decentral storage (Daryanian et al. 1989).
In the absence of the Internet and ubiquitous computing, it was not until the late 2000s that many of these ideas were re-discovered and put into action in the context of smart grid research. Taking an engineering perspective, Stamminger et al. (2008) rigorously quantified the potentials of smart devices in smart households. In particular, they established demand response potentials of appliances such as refrigerators or washing machines. Building upon these insights, Gottwalt et al. (2011) characterized the system-wide impact of rolling out smart devices at scale. With significant efforts of decarbonizing the transportation sector, the emerging potentials of electric vehicles were put forward by Kempton and Letendre (1997). As these loads are both large and shiftable, they can also make major contributions to the power system. The established approach to model these loads is to combine a storage optimization logic for the vehicle battery with driving profiles, as well as characterizing the available charging locations (see, e.g., Flath et al. 2012;Schuller et al. 2014Schuller et al. , 2015. Combining household, storage, and electric vehicle flexibility models to assess their load shifting potentials, Gottwalt et al. (2016) provide a holistic treatise of flexibility modeling in smart grids.

Marketing Flexibility
Flexibility is the ability to deviate from a plan. In the case of the power system, the plan is the load schedule that needs to be served. Traditionally, this load was not flexible or had no incentive to be so. Nowadays, the power system has become increasingly inverted in this respect, as there is a considerable share of volatile generation from renewable energy. So the plan changes-generation becomes less flexible, and demand must better fit the supply.
Demand flexibility shares a main characteristic with most volatile energy sources: it is distributed across the whole system. Therefore, it must be aggregated in order to be marketable in the different power markets, since one source of demand flexibility will usually not meet the capacity limits that still apply for most power markets (Quinn et al. 2010). Flexibility can be offered on different markets, e.g., in ancillary services markets and intraday (regular) markets or in a rather bilateral manner, for instance, for the relief of distribution grids in a regional context. The value of flexibility can thus vary considerably, given the local grid context and the ability of the flexibility resource to be combined with other complementary sources. Studies thus aimed at assessing the relative value of demand flexibility in a given context and at identifying the incentives and market structures needed to harvest the flexibility. Dauer et al. (2015), for instance, propose an auction-based market for the regional allocation of demand-side flexibility that can help the distribution system operator to locally balance the power grid. This work incorporates the specific requirements for flexibility procurement, while it remains incentive compatible for the bidders. Salah and Flath (2016) and  provide insights into how to create price-based incentives for the provision of demand-side flexibility while mitigating renewable generation uncertainty in a local context. Finally, in Salah et al. (2017), the idea of quality of service differentiation is applied to the energy sector. The general idea is to differentiate demand requirements by the service quality, i.e., the supply security and the demand pattern that need to be fulfilled.

Group Formation for Flexibility Aggregation
In order to gain impact, micro-flexibility must be aggregated for both trading and putting to use. In this, the success and the possibilities of scheduling loads are strongly affected by the composition of a demand flexibility aggregator's customer portfolio and by the corresponding customers' flexibility provision (Gärttner 2016). However, flexibility does not come for free. Domestic customers that offer flexibility must be compensated for their discomfort from demand-side management (Haring and Andersson 2014). Discomfort results from changes in daily environmental and behavioral preconditions, e.g., room temperature, starting times of semiautomatically controlled appliances, or available electric vehicle driving ranges. Therefore, the portfolios must be designed carefully. This is realized via the design of tariffs that incentivize customers to offer flexibility . To enable the aggregator to properly design incentives, knowledge about both customer flexibility and the optimal composition of a customer portfolio is needed. Considering the aggregator's decision dilemma, cost for scheduling flexible supply from conventional power plants or the spot market must be limited. Alike, demand flexibility contracting and dispatching costs must be restrained.
The process of designing flexibility portfolios spans a wide range of time. Hence, energy retailers face a complex multistage decision problem. They can procure demand response capacities from heterogeneous retail customers and also need long-term supply contracts. Procured capacities need to be dispatched in response to fluctuating renewable power generation and stochastic demand (Zugno and Conejo 2015). The attainable scheduling quality (with respect to a given objective) critically hinges on the structural composition and capacities of the portfolio. Electricity retailers also need to manage the composition of the customer portfolio. Customers whose flexibility is contracted for demand-side management may receive more favorable electricity rates. Similarly, the energy retailer has to engage in forward transactions for conventional power supply. These long-term actions need to cope with the uncertainty of possible future realization of scenarios with respect to renewable generation, as the portfolio design decisions have to be taken in advance, i.e., without recourse. With a shorter lead time, supply and flexible demand must be scheduled. The recourse scheduling decisions are obviously taken in the absence of uncertainty, as these are typically taken a day ahead or potentially even in real time. Customer flexibility endowments in the load scheduling problem hence depend on the decisions in the first-stage portfolio composition problem.
Expanding the question of how to optimally design demand response portfolios, flexibility aggregators must focus on the framing of mechanisms to incentivize the flexibility provision by household customers. This may be accomplished through offering tariffs that delineate the contractual conditions for scheduling flexibility as well as its remuneration. The development of demand tariffs builds upon knowledge about household characteristics, i.e., the availability of flexibility and the consumers' willingness to provide this flexibility (Gottwalt et al. 2016). Switching the perspective, consumers face a trade-off between remuneration payments and their perceived discomfort. The latter includes possible load adaptations which induce environmental and behavioral changes as well as risk aversion. In conclusion, the interplay between flexibility aggregators and electricity consumers resembles a never-ending game of acting and reaction. No doubt, however, the provision and utilization of demand flexibility come with a wide and strong set of advantages for all stakeholders. Not least, it allows for efficiently integrating renewable energy sources without taking the reliability of power supply at risk.

Renewable Energy Integration and Congestion Management
Recently, generation from new renewable sources (i.e., wind and solar) surpassed generation from conventional fossil or nuclear generation in several countries, including Germany. While being an important step in the agenda of the EU to bring clean energy to all citizens, this success didn't go without impact on the grid operation in Europe. The cost for redispatch measures to avoid surpassing grid capacity constraints rose to one billion euro in Germany, and neighboring countries were intervening at the European Union due to cross-order energy flows from northeastern Germany (Staudt et al. 2018c). At the same time, opposition was forming in Germany against transmission grid expansion, and skeptics of the energy transition pointed to curtailments of renewable generation and high redispatch costs (Galvin 2018), which had been discussed as the not-in-my-backyard phenomenon in other countries (Cotton and Devine-Wright 2010). The European Commission threatened to divide Germany into two price zones, as they had done in Sweden to advance the integration of the European electricity market (Bemš et al. 2016).
In parallel, governments began the mobility transition toward more electric mobility. Furthermore, the cost of installing photovoltaic modules was dropping, and self-consumption became a more attractive option than feeding the generated electricity into the grid (Bertsch et al. 2017). With the first installations to drop out of the feed-in tariff support scheme by 2020, households are looking for alternatives to use their renewable generation. In this time, blockchain emerged as a new concept that promised to facilitate (decentralized) peer-to-peer trading between households (Mengelkamp et al. 2018a). These developments fueled the further decentralization of the power sector that was promoted by the European Union. Through citizen energy communities, the EU intended to emphasize the role of citizens in energy systems. Neighborhoods and industry campuses started to use regulation that allowed them to share electricity at reduced rates . Researchers began to be concerned about capacity constraints in the distribution grid, and grid operators newly had to curtail renewable generation (Schermeyer et al. 2018).

Congestion Management in the Transmission Grid
Developments around renewable energy generation and grid congestion had a large impact on the research community. European scholars began to re-address research that had been performed years before with regard to nodal pricing or redispatch markets (Trepper et al. 2015). Congestion management, which had its peak in research output mainly from a US perspective in the early 2000s-according to Fig. 3-had a small revival caused by the large costs of redispatch in Germany in 2015 and by the European directive demanding market-based approaches for congestion management. This initiative included a change in the transaction object by adding a spatial component, opening new markets for redispatch as an ancillary service (Hirth et al. 2018). The European cost-based redispatch had first been described academically by Nüßler (2012) before it became a large concern in Europe.
As congestion raised more awareness in the European Union, previous research from the USA was re-discovered concerning nodal pricing (Hogan 1999). The graph in Fig. 3 shows that grid congestion as a research topic had its advent in the late 1990s and early 2000s and is now gaining momentum again. When the problem became apparent, researchers quickly published studies on the effects of the division of Germany into two price zones (Egerer et al. 2016;Trepper et al. 2015). Others proposed to use nodal pricing or any other market design with spatial components in the transaction object Richstein et al. 2018). In general, this period can be characterized by the discussion on different ways to address the reduced flexibility in the electricity grid. While some favored market mechanisms to cope with the problem (e.g., Staudt 2019), others made a case for more demand-side integration and flexibility (Huber et al. 2018). The approaches differed in the perspective on the market. While the first group intended to amend a spatial component to the transaction object on the energy-only market, others were in favor of a completely new market design. While market power had long been a concern, it re-surfaced with the discussion on regional redispatch markets because opponents of such markets feared that a regionalization of power markets would further encourage market power exertion (Staudt et al. 2018a). The desired market outcome was clear (a cost-optimal redispatch with regional investment incentives), but the main question was whether the agent behavior would be as desired, given the proposed market design. Finally, the discussion on redispatch markets went hand in hand with the question of transmission grid expansion. While the government emphasized the need for these expansions, some scholars argued that it might be inflated (Kemfert et al. 2016). However, besides one paper by Staudt and Oren (2020), no discussion on market-based solutions emerged, even though alternatives to grid expansion leveraging advances in machine learning exist (Aznarte and Siebert 2016).

Market-Based Congestion Management at the Distribution Level
Decentralized renewable power generation does not only affect the transmission grid but also challenges the distribution system. In Germany, grid operators currently manage congestion at the distribution level by curtailment and demandside management (EnWG §13.2 and § 14a). As these measures are cost-based and costs are not transparent, the discussion on market-based congestion management in the distribution grid emerged (Ecofys und Fraunhofer IWES 2017). Market-based solutions aim at coordinating flexibility deployment more efficiently and create a market outcome that prevents local congestion. Several examples of this were implemented in the German SINTEG demonstration projects (Huber et al. 2018). As with the market-based redispatch, problems with gaming (inc/dec) and local market power remain an unsolved challenge in these markets.
Another field of discussions are the transaction objects to be traded on these markets. Different mechanisms allow for coordinating the flexibility of decentralized renewable energy sources (Lehmann et al. 2019). While time-varying prices allow for a voluntary reaction, other mechanisms require the generators or consumers to describe and guarantee flexibility before delivery. In this description, more advanced models allow describing all possible future actions of the flexibility units, while simple models are the foundation for comparable products (Villar et al. 2018).
Depending on the desired market outcome, the transaction objects and the market micro-structure have to be adapted. In case the only purpose of the market is congestion management, with the system operator acting as a single buyer, there is no need to describe the flexibility in more complex models, as congestion is likely to occur only occasionally, and load adaption can solve the congestion. Simple products, however, cannot map temporal dependencies (Dauer et al. 2015) and other restrictions (e.g., ramping, must-run) and are not suited to coordinate the full flexibility potential of participants. This reduction in potential would be wasted in case the flexibility is to be used by other participants for other uses, e.g., portfolio management  or system services (Staudt et al. 2018b).
While market-based solutions might suffer from strategic behavior of agents (see redispatch inc/dec; Hirth et al. 2019), it seems likely that market participants at lower grid levels, who are often private households, will not behave in the same way as (more) rational agents on wholesale markets. For instance, households' energy consumption and electric vehicle users' flexibility provision are driven not only by monetary benefits but also by idealism and convenience ). While former research shows that different consumer types have different flexibility (Schuller et al. 2015) and motivation to use smart charging , recent research uses these insights to make individual users more flexible (Salah and Flath 2016) and sustainable in their decisions ).

Peer-to-Peer Energy Markets
In line with the decentralization of the energy system, local energy markets gained increasing attention in the last two decades as one of the opportunities to harness flexibility on an end customer (consumer, prosumer, and producer) level. Figure 4 shows how the rapidly growing amount of publications in the field of local energy markets represents the increasing significance of the topic in academia.
Right along the increasing share of renewable (mostly) volatile distributed generation, small-scale actors become increasingly involved in the overall energy system (Koirala et al. 2016). Local energy markets (LEMs) often represent a peerto-peer approach of distributing limited energy or flexibility among the market participants ). In the case of geographically limited LEM, a physical microgrid is likely to be the grid equivalent of the virtual market mechanism (Mengelkamp et al. 2018a). Figure 4 shows that the slope of academic publications starts to exponentially grow around 2010, which is about one decade later than the topic of LEMs. Nevertheless, the total amount of publications is larger than for LEMs. Related to this is the topic of energy communities (mostly between prosumers). While energy communities are nowadays often seen as the first step toward a microgrid of an LEM, their rise of academic attention is delayed by about 5 years compared to the microgrids, as Fig. 4 shows.
Within the last 5 years, the idea of bringing the decentralization of energy communities, LEMs, and microgrids toward the level of information systems has Fig. 4 Relative number of publications about blockchain, microgrids, and prosumers, following the same approach and data source as in Figs. 2 and 3, but limited to the research fields of information systems and electrical and electronic engineering; total number of papers in brackets reached an enormous scale. Distributed ledger technologies (often simply referred to as blockchain technology) represent the prevalent distributed and decentralized information technology in use. Figure 4 depicts the effect over time. Blockchain technology has since made its way in almost all distributed energy realms, e.g., vehicle-to-grid, smart charging, capacity markets, energy storage.
The first famous blockchain-based LEM project is the Brooklyn Microgrid (reviewed in Mengelkamp et al. 2018a). It began to gain attention when executing the (allegedly) first blockchain-based electricity trade in 2016 and has since expanded from a virtual to a (partly) physical microgrid and a large energy community . A large number of additional projects in the overlapping domains of microgrids (e.g., Block et al. 2008), LEMs (e.g., the LAMP project; Mengelkamp et al. 2018b), and energy communities (e.g., the Quartierstrom project; Brenzikofer et al. 2019) have since developed. In 2018, more than 120 organizations had started projects regarding LEMs (Metelitsa 2018). So far, however, most projects remain in the pilot or proof-of-concept phase. Their realworld applicability remains to be proven in the long term .
Increasing consumer interest in renewable and local energy generation (Mengelkamp et al. 2019) gave rise to so-called regional energy markets or energy communities that allow consumers to track energy balances throughout the communities and can ensure on-balance sheet evening up of generation and consumption of the participants. However, these products usually offer a fixed tariff and do not implement an actual market mechanism that incentivizes a change of generation and consumption on a decentralized level.

The Next Steps for Decentralized Energy Markets
In 2019, energy transition has picked up pace, partly spurred by the Fridays for Future demonstrations and their demand for decisive and fast action against climate change, which is strongly supported by different scientists (Hagedorn et al. 2019). Afterward, the COVID-19 pandemic caused national and international stimulus packages that further promote green initiatives, at least in Europe (Forster et al. 2020). In this process, hydrogen is proposed as one important pillar of a carbon emission-free economy (Dincer 2020). From a market engineering perspective, this development leads to new and interesting questions. Hydrogen is both a storage medium for excess green electricity and can provide value beyond the power system. By fueling air traffic or the energy-intensive industry, a new hydrogen economy could be created (Abe et al. 2019). This hydrogen economy interplays with the electricity market in a still unforeseen manner. Beyond the national markets, it is expected that international hydrogen trade will develop from countries with cheaply available renewable energy resources to countries with higher population densities and less favorable meteorological conditions (Boretti 2020). In regard to electricity market engineering, it will be important to consider system topologies when siting electrolyzers for hydrogen production in markets with single market clearing prices, as in the European Union. Electrolyzers are large consumers that can exacerbate existing mismatch between generation and consumption, if they are not carefully placed (vom Scheidt et al. 2021), which relates well to the already existing research on congestion management. Furthermore, regulatory decisions will be important. Relevant regulations from a market engineering perspective are rules on carbon leakage, which are supposed to protect industries from competitors with less severe regulations (Naegele and Zaklan 2019).
Furthermore, the public importance of the energy transition is recognized by large corporations. Google pursues a strategy of "24/7 carbon-free energy" which means that the company intends to use only green energy in real time (Miller 2020), which sets a new goal beyond the former 100% renewable pledge by many countries and promotes demand-side flexibility options. Also, demand response is increasingly implemented, as seen from the research presented in this chapter. Germany and California, for example, have integrated battery storage capacity aggregators into their electricity markets (Angenendt et al. 2020). In September 2020, FERC passed Order No. 2222, which allows distributed and aggregated resources to participate in the electricity market starting at a capacity of 1 kW. The Texan market operator ERCOT already allows time-of-use tariffs, critical peak pricing, or peak time rebates (Du et al. 2019); the first real-time pricing tariff has recently been launched in Germany by the company aWATTar (Basmadjian 2020).
The trend to deploy flexibility through appropriate regulation and incentives is supported by the ever-decreasing costs of battery storage. Cole and Frazier (2019) project a battery cost decrease of 32-80% from the 380 dollar per kWh in 2018, until 2050, which is mostly driven by the increase in electric vehicle deployment. In September 2020, the Tesla Model 3 was the most registered car in Switzerland, and Tesla's market share was the same as that of Audi. While car sales generally dropped in July and August 2020 in Germany, electric vehicle registrations increased. Additionally, Tesla announced in September 2020 to equip its cars with vehicle-to-grid technology. This comes at a time when researchers start to believe that vehicle-to-grid may be profitable, even when considering battery degradation (Ginigeme and Wang 2020). These developments show that the deployment of demand-side flexibility and local optimization will play an important role in research in the near future. This includes addressing public acceptance and trust and designing attractive schemes for aggregators and consumers.
Aggregators and consumers will continue to move into the spotlight of energy market engineering, with regard to the local coordination of supply and demand. The EU has recently proposed citizen energy communities as a regulatory concept that should allow neighborhoods to locally share and trade generated power (Lowitzsch et al. 2020). This moves peer-to-peer trading from a conceptual idea into deployment. Furthermore, many large utilities are now involved in research projects on citizen energy communities, such as E.ON (in the project IELECTRIX), and the real-world laboratory SmartQuart, funded by the German federal government. In the USA, similar concepts are now tested at large scale. For instance, Portland General Electric since 2018 experiments in the Portland Microgrid Testbed that covers an area of more than 20,000 customers. These concepts are pursued at a time when photovoltaic power generation is increasingly curtailed worldwide, which causes the need for local solutions (O'Shaughnessy et al. 2020). New regulation on mandatory installations of solar panels on newly developed buildings, as discussed in Germany and implemented in California (Senet 2019), might further increase the need for smart energy solutions in densely populated areas. The newly developing local markets are not only intended to supply neighbors with cheap green electricity but will also play an important role in the implementation of local sector coupling, that is, the joint supply of heat, electric, and transportation energy demand (Arabzadeh et al. 2020). The research community has to support energy communities, aggregators, and infrastructure developers to develop tailored solutions for different neighborhoods (Golla et al. 2021), but also to design trusted interfaces that can be used to operate such communities (Golla et al. 2020).

Conclusion and Outlook
Since the beginning of energy sector liberalization, markets have become an important and widely applied concept to support a variety of processes in the operation of today's energy systems. The first research questions to which market engineering delivered valuable insights were related to auction designs for wholesale power trading. Very quickly, a plethora of further market-based instruments was established, facilitating control reserve procurement, carbon emissions restrictions, congestion management, and renewable energy support or involving prosumers in local energy markets, to mention a few. In view of the current goal of the European Union to provide "Clean energy for all Europeans," it can be said that many of the necessary developments to promote this goal can be supported by energy market engineering. The potential contributions of market engineering to a clean and participatory energy system can be concluded on the basis of some perceived ongoing trends, which we summarize in the following: • Energy markets evolve slowly toward decentralization. On the one hand, millions of small dispersed (renewable) generation units have to be integrated into power systems; on the other hand, smaller stakeholders increasingly participate in markets, and their spatial distribution becomes more relevant. As a consequence, regional markets are emerging, and national markets must better accommodate for the spatial characteristics of generation and demand in the future. • Energy markets are becoming more dynamic, as trading periods approach the real time. Therefore, markets must enable a high temporal resolution, and dynamic forecasting methods have to be available to facilitate shorter-term trading. • Diversity of stakeholders is increasing. Aggregators will play an important role in pooling diverse small devices such as batteries and thermal storage units. This has an impact on the modeling requirements of these agents and on their representation in the current market frameworks.
• The urgency of climate protection has catapulted energy and emissions markets to the top of global decision-makers' agenda. Proper integration of greenhouse gas emission reduction schemes into energy markets is crucial. Here, the regulatory framework must be properly designed too, so that the market mechanisms can unfold their potential. A well-designed carbon trading scheme is useless if the emissions cap is set too laxly. Therefore, market mechanisms must always be regarded in the broader context of energy system regulation. • New technological advancements, for example, in storage technologies and also in smart meter deployment, make flexibility potentials available. Markets can facilitate its efficient use for the benefit of the whole system. Research questions on where flexibility deployment can provide the largest benefit remain open, so more dynamic and highly temporally and spatially resolved models are needed to provide answers to questions about the coordination challenge of flexibility.
The majority of energy research from the last decade is now on the brink of being implemented, as it has been illustrated in this chapter with numerous examples. Increasing focus is currently put on the distribution grid and on congestion management. This underscores the importance of modeling individual actors instead of simply considering the system as a whole, for which this review gives multiple examples. At the same time, it summons up many interesting challenges for researchers in energy system modeling and energy market engineering in the coming years and opens the floor for propositions on how to cope with the most important problems of today's energy systems.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.