Autonomous Agents and Multi-Agent Systems

, Volume 21, Issue 2, pp 115–142 | Cite as

Automated bidding in computational markets: an application in market-based allocation of computing services

  • Nikolay Borissov
  • Dirk Neumann
  • Christof Weinhardt


Autonomous agents are widely applied to automate interactions in robotics, e.g. for selling and purchasing goods on eBay, and in financial markets, e.g. in the form of quote machines and algorithmic traders. Current research investigates efficient economic mechanisms that fully automate the provisioning and usage processes of Grid-based services. On the one hand, consumers want to allocate resources on demand for their various applications, e.g. data sharing, stream processing, email, business applications and simulations. On the other hand, providers of Grid services want to govern business policies to meet their utilization and profit goals. The above-mentioned processes are not manually manageable, however, because decisions need to be taken within milliseconds. Therefore, such processes have to be automated to minimize human interactions. Hence, market mechanisms and strategic behavior play important roles when it comes to achieving automated and efficient allocation of Grid services. The paper begins by presenting a framework for automated bidding, providing a methodology for the design and implementation of configurable bidding strategies. Second, it presents a novel bidding strategy based on a reinforcement learning technique. This strategy is designed to automate the bid generation processes of consumers and providers in various market mechanisms. Third, the behavior and convergence of the strategy is evaluated in a centralized Continuous Double Auction and a decentralized on-line machine scheduling mechanism against selected benchmark bidding strategies. Fourth, we define a bidding language for communicating consumer and provider preferences to the market as well as report back the match of the market-based allocation process.


Bidding agent framework Bidding strategy Bidding language 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ågotnes T., van der Hoek W., & Wooldridge M. (2009) Reasoning about coalitional games. Artificial Intelligence 173(1): 45–79zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Andreozzi, S., Burke, S., Ehm, F., Field, L., Galang, G., Konya, B., et al. (2008). GLUE specification v. 2.0. GLUE WG, Open Grid Forum.Google Scholar
  3. 3.
    Andrieux, A., Czajkowski, K., Dan, A., Keahey, K., Ludwig, H., Nakata, T., et al. (2007). Web services agreement specification (WS-agreement), Open Grid Forum.Google Scholar
  4. 4.
    Anjomshoaa, A., Brisard, F., Drescher, M., Fellows, D., Ly, A., McGough, S., et al. (2005). Job submission description language (JSDL) Specification, Version 1.0, Open Grid Forum.Google Scholar
  5. 5.
    Anthony P., Jennings N. (2003) Developing a bidding agent for multiple heterogeneous auctions. ACM Transactions on Internet Technology (TOIT) 3(3): 185–217CrossRefGoogle Scholar
  6. 6.
    Bapna, R., Das, S., Garfinkel, R., & Stallaert, J. (2005). A market design for grid computing. Technical report. USA: Department of Operations and Information Management, University of Connecticut.Google Scholar
  7. 7.
    Bergenti, F., & Poggi, A. (2002). LEAP: A FIPA platform for handheld and mobile devices. LNCS, 2333, 436–446.Google Scholar
  8. 8.
    Borissov, N. (2009). Q-strategy: Automated bidding and convergence in computational markets. In Twenty-first Innovative Applications of Artificial Intelligence (IAAI) Conference.Google Scholar
  9. 9.
    Borissov, N., & Wirström, N. (2008). Q-Strategy: A bidding strategy for market-based allocation of grid services. In OTM Conferences (1) (pp. 744–761).Google Scholar
  10. 10.
    Cliff, D. (1997). Minimal-intelligence agents for bargaining behaviors in market-based environments. TechnicalReport, New york: Hewlett Packard Labs.Google Scholar
  11. 11.
    Das R., Hanson J., Kephart J., Tesauro G. (2001) Agent-human interactions in the continuous double auction. Artificial Intelligence 17: 1169–1178Google Scholar
  12. 12.
    DMTF. (2008). Common information model (CIM) v2.19.1. Distributed Management Task Force (DMTF).
  13. 13.
    Endriss U., Maudet N. (2004) Welfare engineering in multiagent systems. Lecture Notes in Computer Science 3071: 93–106Google Scholar
  14. 14.
    Even-Dar E., Mansour Y. (2004) Learning rates for Q-learning. The Journal of Machine Learning Research 5: 1–25MathSciNetGoogle Scholar
  15. 15.
    Even-Dar E., Mannor S., Mansour Y. (2006) Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. The Journal of Machine Learning Research 7: 1079–1105MathSciNetGoogle Scholar
  16. 16.
    Fasli M., Michalakopoulos M. (2008) e-Game: A platform for developing auction-based market simulations. Decision Support Systems 44(2): 469–481CrossRefGoogle Scholar
  17. 17.
    Fatima S., Wooldridge M., Jennings N. (2006) Multi-issue negotiation with deadlines. Journal of Artificial Intelligence Research 27: 381–417zbMATHMathSciNetGoogle Scholar
  18. 18.
    Feitelson, D. (2009). Parallel workloads archive.
  19. 19.
    FIPA, T. (2002). Fipa abstract architecture specification. Tech. rep., Foundation for Intelligent Physical Agents.Google Scholar
  20. 20.
    Foster, I., Zhao, Y., Raicu, I., & Lu, S. (2008). Cloud Computing and Grid Computing 360-Degree Compared. Grid Computing Environments Workshop (pp. 1–10).Google Scholar
  21. 21.
    Friedman-Hill, E.-J. (2003). Jess, the rule engine for the java platform. Livermore, CA: Distributed Computing Systems, Sandia National Laboratories.Google Scholar
  22. 22.
    Gjerstad S., Dickhaut J. (1998) Price formation in double auctions. Games and Economic Behavior 22(1): 1–29zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Gode D., Sunder S. (1993) Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality. The Journal of Political Economy 101(1): 119–137CrossRefGoogle Scholar
  24. 24.
    Gomes, E., & Kowalczyk, R. (2007). Learning in market-based resource allocation. 6th IEEE/ACIS Int Conference on Computer and Information Science (pp. 475–482).Google Scholar
  25. 25.
    Green P., Rao V. (1971) Conjoint measurement for quantifying judgmental data. Journal of Marketing Research 8(3): 355–363CrossRefGoogle Scholar
  26. 26.
    Grosu, D., & Das, A. (2006). Auctioning resources in grids:model and protocols. Concurrency and Computation, 18(15), 1927.Google Scholar
  27. 27.
    Helsinger, A., Thome, M., Wright, T., Technol, B., & Cambridge, M. (2004). Cougaar: A scalable, distributed multi-agent architecture. IEEE International Conference on Systems, Man and Cybernetics, 2.Google Scholar
  28. 28.
    Heydenreich B., Müller R., Uetz M. (2006) Decentralization and mechanism design for online machine scheduling. Lecture Notes in Computer Science 4059: 136–147CrossRefGoogle Scholar
  29. 29.
    Howden, N., Ronnquist, R., Hodgson, A., & Lucas, A. (2001). JACK intelligent agents-summary of an agent infrastructure. 5th Int Conference on Autonomous Agents.Google Scholar
  30. 30.
    Iyer K., Huhns M.N. (2009) Negotiation criteria for multiagent resource allocation. Knowledge Engineering Review 24(2): 111–135CrossRefGoogle Scholar
  31. 31.
    Kaelbling, L., Littman, M., & Moore, A. (1996). Reinforcement learning: A survey. Arxiv preprint csAI/9605103.Google Scholar
  32. 32.
    Lai K., Rasmusson L., Adar E., Zhang L., Huberman B. (2005) Tycoon: An implementation of a distributed, market-based resource allocation system. Multiagent and Grid Systems 1(3): 169–182zbMATHGoogle Scholar
  33. 33.
    Li, J., & Yahyapour, R. (2006). Learning-based negotiation strategies for grid scheduling. Int Symposium on Cluster Computing and the Grid (CCGRID2006) (pp. 576–583).Google Scholar
  34. 34.
    Luce R., Tukey J. (1964) Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of Mathematical Psychology 1(1): 1–27zbMATHCrossRefGoogle Scholar
  35. 35.
    Macías, M., Rana, O., Smith, G., Guitart, J., & Torres, J. (2008). Maximizing revenue in grid markets using an economically enhanced resource manager. Concurrency and computation: Practice and experience. doi: 10.1002/cpe.1370.
  36. 36.
    Medernach, E. (2005). Workload analysis of a cluster in a grid environment. LNCS, 3834, 36–61.Google Scholar
  37. 37.
    Milano M., Roli A. (2004) MAGMA: A multiagent architecture for metaheuristics. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 34(2): 925–941CrossRefGoogle Scholar
  38. 38.
    Milojicic D. et al (1998) MASIF: The OMG mobile agent system interoperability facility. Personal and Ubiquitous Computing 2: 117–129Google Scholar
  39. 39.
    Myerson R., Satterthwaite M. (1983) Efficient mechanisms for bilateral trading. Journal of Economic Theory 29(2): 265–281zbMATHCrossRefMathSciNetGoogle Scholar
  40. 40.
    Nassif, L., Nogueira, J., & de Andrade, F. (2007). Distributed resource selection in grid using decision theory. 7th IEEE International Symposium on Cluster Computing and the Grid (pp. 327–334).Google Scholar
  41. 41.
    Nimis, J., et al. (2009). D2.2a: Final specification and design documentation of the sorma components f́b revised version. Tech. rep.
  42. 42.
    Nisan N. (2004). Bidding languages. USA: Combinatorial Auctions MIT Press.Google Scholar
  43. 43.
    Pardoe, D., & Stone, P. (2009). An autonomous agent for supply chain management. In G. Adomavicius & A. Gupta (Eds.), Handbooks in information systems series: Business computing. Elsevier, Emerald Group.Google Scholar
  44. 44.
    Parkes, D., Singh, S., & Yanovsky, D. (2004). Approximately efficient online mechanism design. Proc 18th Annual Conf on Neural Information Processing Systems.Google Scholar
  45. 45.
    Paurobally, S., Tamma, V., & Wooldridge, M. (2007). A framework for web service negotiation. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 2(4), 14.Google Scholar
  46. 46.
    Poslad, S., Laamanen, H., Malaka, R., Nick, A., Buckle, P., & Zipf, A. (2001). CRUMPET: Creation of user-friendly mobile services personalised for tourism. IEE Conference Publication (pp. 28–32).Google Scholar
  47. 47.
    Reeves D., Wellman M., MacKie-Mason J., Osepayshvili A. (2005) Exploring bidding strategies for market-based scheduling. Decision Support Systems 39(1): 67–85CrossRefGoogle Scholar
  48. 48.
    Robles, S., Borrell, J., Bigham, J., Tokarchuk, L., & Cuthbert, L. (2001). Design of a trust model for a secure multi-agent marketplace. Proceedings of the 5th international conference on autonomous agents (pp. 77–78).Google Scholar
  49. 49.
    Rosenschein J., Zlotkin G. (1994) Rules of encounter: Designing conventions for automated negotiation among computers. MIT Press, USAGoogle Scholar
  50. 50.
    Rothkopf M. (2007) Thirteen reasons why the Vickrey-Clarke-Groves process is not practical. Operations Research 55(2): 191–197zbMATHCrossRefGoogle Scholar
  51. 51.
    Saaty T. (1986) Axiomatic foundation of the analytic hierarchy process. Management Science 32(7): 841–855zbMATHCrossRefMathSciNetGoogle Scholar
  52. 52.
    Sandholm, T., Lai, K., & Clearwater, S. (2008). Admission control in a computational market. Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID) (pp. 277–286).Google Scholar
  53. 53.
    Satterthwaite M., Williams S. (1989) Bilateral trade with the sealed bid k-double auction: Existence and efficiency. Journal of Economic Theory 48: 107–133zbMATHCrossRefMathSciNetGoogle Scholar
  54. 54.
    Schnizler B., Neumann D., Veit D., Weinhardt C. (2006) Trading grid services-a multi-attribute combinatorial approach. European Journal of Operational Research 187: 943–961CrossRefGoogle Scholar
  55. 55.
    Shehory, O., & Sturm, A. (2001). Evaluation of modeling techniques for agent-based systems. Proceedings of the 5th international conference on autonomous agents (pp. 624–631).Google Scholar
  56. 56.
    Sherstov A., Stone P. (2005) Three automated stock-trading agents: A comparative study. LNCS 3435: 173Google Scholar
  57. 57.
    Sun R., Peterson T. (1999) Multi-agent reinforcement learning: weighting and partitioning. Neural Networks 12(4–5): 727–753CrossRefGoogle Scholar
  58. 58.
    Vytelingum P., Cliff D., Jennings N. (2008) Strategic bidding in continuous double auctions. Artificial Intelligence 172: 1700–1729zbMATHCrossRefGoogle Scholar
  59. 59.
    Watkins C., Dayan P. (1992) Q-learning. Machine Learning 8(3): 279–292zbMATHGoogle Scholar
  60. 60.
    Wellman M., Walsh W., Wurman P., MacKie-Mason J. (2001) Auction protocols for decentralized scheduling. Games and Economic Behavior 35(1–2): 271–303zbMATHCrossRefMathSciNetGoogle Scholar
  61. 61.
    Wellman M., Greenwald A., Stone P. (2007) Autonomous bidding agents: Strategies and lessons from the trading agent competition. MIT Press, USAGoogle Scholar
  62. 62.
    Whiteson S., Stone P. (2006) Evolutionary function approximation for reinforcement learning. The Journal of Machine Learning Research 7: 877–917MathSciNetGoogle Scholar
  63. 63.
    Wolski, R., Plank, J., Brevik, J., & Bryan, T. (2001). Analyzing market-based resource allocation strategies for the computational grid. International Journal of High Performance Computing Applications, 15, 258.Google Scholar
  64. 64.
    Wurman, P., Wellman, M., & Walsh, W. (1998). The michigan internet auctionbot: A configurable auction server for human and software agents. Proceedings of the 2nd international conference on Autonomous agents (pp. 301–308).Google Scholar
  65. 65.
    Wurman P., Wellman M., Walsh W. (2001) A parameterization of the auction design space. Games and Economic Behavior 35(1–2): 304–338zbMATHCrossRefMathSciNetGoogle Scholar
  66. 66.
    Yeo C., Buyya R. (2006) A taxonomy of market-based resource management systems for utility-driven cluster computing. Software:Practice and Experience 36(13): 1381–1419CrossRefGoogle Scholar

Copyright information

© The Author(s) 2009

Authors and Affiliations

  • Nikolay Borissov
    • 1
  • Dirk Neumann
    • 2
  • Christof Weinhardt
    • 1
  1. 1.Institute of Information Systems and ManagementKarlsruhe Institute of Technology (KIT)KarlsruheGermany
  2. 2.Chair of Information Systems ResearchUniversity of FreiburgFreiburgGermany

Personalised recommendations