Computational Economics

, Volume 32, Issue 1–2, pp 73–98 | Cite as

Learning Agents in an Artificial Power Exchange: Tacit Collusion, Market Power and Efficiency of Two Double-auction Mechanisms

  • Eric Guerci
  • Stefano Ivaldi
  • Silvano Cincotti


This paper investigates the relative efficiency of two double-auction mechanisms for power exchanges, using agent-based modeling. Two standard pricing rules are considered and compared (i.e., “discriminatory” and “uniform”) and computational experiments, characterized by different inelastic demand level, explore oligopolistic competitions on both quantity and price between learning sellers/producers. Two reinforcement learning algorithms are considered as well—“Marimon and McGrattan” and “Q-learning”—in an attempt to simulate different behavioral types. In particular, greedy sellers (optimizing their instantaneous rewards on a tick-by-tick basis) and inter-temporal optimizing sellers are simulated. Results are interpreted relative to game-theoretical solutions and performance metrics. Nash equilibria in pure strategies and sellers’ joint profit maximization are employed to analyze the convergence behavior of the learning algorithms. Furthermore, the difference between payments to suppliers and total generation costs are estimated so as to measure the degree of market inefficiency. Results point out that collusive behaviors are penalized by the discriminatory auction mechanism in low demand scenarios, whereas in a high demand scenario the difference appears to be negligible.


Agent-based simulation Power exchange Market power Reinforcement learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Baldick R., Grant R., Kahn E. (2004) Theory and application of linear supply function equilibrium in electricity markets. Journal of Regulatory Economics 25(2): 143–167CrossRefGoogle Scholar
  2. Borenstein S. (2002) The trouble with electricity markets: Understanding California’s restructuring disaster. The Journal of Economic Perspectives 16(1): 191–211CrossRefGoogle Scholar
  3. Bower J., Bunn D. (2001) Experimental analysis of the efficiency of uniform-price versus discrimatory auctions in the england and wales electricity market. Journal of Economic Dynamics & Control 25(3–4): 561–592CrossRefGoogle Scholar
  4. Bunn D.W., Oliveira F.S. (2001) Agent-based simulation – an application to the new electricity trading arrangements of England and Wales. IEEE Transactions on Evolutionary Computation 5(5): 493–503CrossRefGoogle Scholar
  5. Bunn D.W., Oliveira F.S. (2003) Evaluating individual market power in electricity markets via agent-based simulation. Annals of Operations Research 121(1–4): 57–77CrossRefGoogle Scholar
  6. Commission, U. F. E. R. (2003a). Notice of white paper. Technical report, US Federal Energy Regulatory Commission.Google Scholar
  7. Commission, U. F. E. R. (2003b). Report to congress on competiton in the wholesale and retail markets for electric energy. Technical report, US Federal Energy Regulatory Commission.Google Scholar
  8. Fabra N., von der Fehr N.-H., Harbord D. (2006) Designing electricity auctions. The Rand Journal of Economics 37(1): 23CrossRefGoogle Scholar
  9. Green R., Newbery D. (1992) Competition in the british electricity spot market. The Journal of Political Economy 100(5): 929–953CrossRefGoogle Scholar
  10. Guerci E., Ivaldi S., Raberto M., Cincotti S. (2007) Learning oligopolistic competition in electricity auctions. Computational Intelligence 23(2): 197–220CrossRefGoogle Scholar
  11. Holmberg, P. (2005). Modelling bidding behaviour in electricity auctions: Supply function equilibria with uncertainty demand and capacity constraints. PhD thesis, UPPSALA University.Google Scholar
  12. Hu, J., & Wellman, M. P. (1998). Multiagent reinforcement learning: Theoretical framework and an algorithm. In Proceedings of 15th International Conference on Machine Learning, pp. 242–250. Morgan Kaufmann, San Francisco, CA.Google Scholar
  13. Joskow P. (2006) Markets for power in the united states: An interim assessment. Energy Journal 27(1): 1–36Google Scholar
  14. Kaelbling L., Littman M., Moore A. (1996) Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4: 237–285Google Scholar
  15. Kahn, A., Cramton, P., Porter, R., & Tabors, R. (2001). Uniform pricing or pay-as-bid pricing: A dilemma for california and beyond. The Electricity Journal, 70–79.Google Scholar
  16. Klemperer P.D., Meyer M.A. (1989) Supply function equilibria in oligopoly under uncertainty. Econometrica 57(6): 1243–1277CrossRefGoogle Scholar
  17. Marimon, R., & McGrattan, E. (1995). On adaptive learning in strategic games. In A. Kirman & M. Salmon (Eds.), Learning and rationality in economics, (pp. 63–101). Blackwell.Google Scholar
  18. Nicolaisen J., Petrov V., Tesfatsion L. (2001) Market power and efficiency in a computational electricity market with discriminatory double-auction pricing. IEEE Transactions on Evolutionary Computation 5(5): 504–523CrossRefGoogle Scholar
  19. Puterman, M. (1994). Markov decision processes: Discrete stochastic dynamic programming. Wiley.Google Scholar
  20. Shoham Y., Powers R., Grenager T. (2007) If multi-agent learning is the answer, what is the question?. Artificial Intelligence 171(7): 365–377CrossRefGoogle Scholar
  21. Sun J., Tesfatsion L. (2007) Dynamic testing of wholesale power market designs: An open-source agent-based framework. Computational Economics 30(3): 291–327CrossRefGoogle Scholar
  22. Tesfatsion, L. (2006). Ace research area: Restructured electricity markets. Website available at, hosted by the Economics Department, Iowa State University.
  23. Tesfatsion, L., & Judd, K. (2006). Handbook of computational economics: Agent-based computational economics, Vol. 2 of Handbook in economics series. North Holland.Google Scholar
  24. von der Fehr N., Harbord D. (1993) Spot market competition in the UK electricity industry. Economic Journal 103: 531–546CrossRefGoogle Scholar
  25. Watkins C., Dayan P. (1992) Q-learning. Machine Learning 8(3–4): 279–292Google Scholar

Copyright information

© Springer Science+Business Media, LLC. 2008

Authors and Affiliations

  1. 1.Department of Biophysical and Electronic EngineeringUniversity of GenoaGenoaItaly

Personalised recommendations