Skip to main content
SpringerLink
Log in

Combining Futures and Spot Markets: A Hybrid Market Approach to Economic Grid Resource Management

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Economic forms of resource management in which users can express their valuations for service, offer new possibilities for optimizing resource allocations in Grids. If users are to correctly express these valuations, quality of service guarantees need to be given with respect to the turnaround time of their workloads. Market mechanisms that support bidding and allocations in future time are crucial for delivering such guarantees. To deal with the significant delays that these mechanisms introduce in the allocation process, we present a hybrid market approach in which a low-latency spot market coexists with a higher latency futures market. Based on simulated market scenarios, we show how this combination can significantly increase the total value realized by the Grid infrastructure. We also demonstrate how providers can react to price dynamics in such a hybrid market setting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abdelkader, K., Broeckhove, J.: Pricing computational resources in a dynamic Grid. International Journal of Grid and Utility Computing (IJGUC) 1, 205–215 (2009). doi:10.1504/IJGUC.2009.027648

    Article  Google Scholar 

  2. Abramson, D., Buyya, R., Giddy, J.: A computational economy for Grid computing and its implementation in the Nimrod-G resource broker. Future Gener. Comput. Syst. 18(8), 1061–1074 (2002). doi:10.1016/S0167-739X(02)00085-7

    Article  MATH  Google Scholar 

  3. Amazon: Elastic compute cloud. http://aws.amazon.com/ec2(2010). Accessed 22 September 2010

  4. Auyoung, A., Chun, B.N., Snoeren, A.C., Vahdat, A.: Resource allocation in federated distributed computing infrastructures. In: Proceedings of the 1st Workshop on Operating System and Architectural Support for the On-demand IT InfraStructure (2004). http://www.theether.org/papers/oasis04.pdf

  5. Bapna, R., Das, S., Garfinkel, R., Stallaert, J.: A market design for Grid computing. INFORMS J. Comput. 20(1), 100–111 (2007). doi:10.1287/ijoc.1070.0221

    Article  Google Scholar 

  6. Broberg, J., Venugopal, S., Buyya, R.: Market-oriented Grids and utility computing: the state-of-the-art and future directions. Journal of Grid Computing 6(2), 255–276 (2008). doi:10.1007/s10723-007-9095-3

    Article  Google Scholar 

  7. Brucker, P.: Scheduling Algortihms. Springer, Berlin (2004)

    Google Scholar 

  8. Buyya, R.: Economic-based Distributed Resource Management and Scheduling for Grid Computing. Ph.D. thesis, Monash University, Australia (2002)

    Google Scholar 

  9. Buyya, R., Abramson, D., Giddy, J.: Economic models for resource management and scheduling in Grid computing. Concurr. Comput. Pract. Exp. 14, 1507–1542 (2002). doi:10.1002/cpe.690

    Article  MATH  Google Scholar 

  10. Buyya, R., Abramson, D., Venugopal, S.: The Grid economy. Proc. IEEE 93(3), 698–714 (2005). doi:10.1109/JPROC.2004.842784

    Article  Google Scholar 

  11. Bykowsky, M.M., Cull, R.J., Ledyard, J.O.: Mutually destructive bidding: the FCC auction design problem. J. Regul. Econ. 17(3), 205–228 (2000). doi:10.1023/A:1008122015102

    Article  Google Scholar 

  12. Cameron, D., Millar, A., Nicholson, C., Carvajal-Schiaffino, R., Stockinger, K., Zini, F.: Analysis of scheduling and replica optimisation strategies for data Grids using Optorsim. Journal of Grid Computing 2(1), 57–69 (2004). doi:10.1007/s10723-004-6040-6

    Article  Google Scholar 

  13. Cheliotis, G., Kenyon, C., Buyya, R.: 10 Lessons from finance for commercial sharing of IT resources. In: Sumabramian, R., Goodman, B. (eds.) Peer-to-Peer Computing: Evolution of a Disruptive Technology, chap. 11, pp. 244–264. Idea Group Publishing, Hershey (2004)

    Google Scholar 

  14. Chun, B.N., Buonadonna, P., AuYoung, A., Chaki, N., Parkes, D., Shneidman, J., Snoeren, A., Vahdat, A.: Mirage: A microeconomic resource allocation system for sensornet testbeds. In: Proceedings of the Second IEEE Workshop on Embedded Networked Sensors, pp. 19–28. IEEE Computer Society (2005)

  15. Clearwater, S. (ed.): Market-Based Control: A Paradigm for Distributed Resource Allocation. World Scientific, Singapore (1996)

    Google Scholar 

  16. Cramton, P., Shoham, Y., Steinberg, R. (eds.): Combinatorial Auctions. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  17. EGEE project site: http://www.eu-egee.org/ (2008)

  18. Engelen, R.A.V., Gallivan, K.A., Walsh, B.: Parametric timing estimation with Newton–Gregory formulae. Concurr. Comput. Pract. Exp. 18(11), 1435–1463 (2006). doi:10.1002/cpe.v18:11

    Article  Google Scholar 

  19. Feldman, M., Lai, K., Zhang, L.: A price-anticipating resource allocation mechanism for distributed shared clusters. In: Proceedings of the 6th ACM Conference on Electronic Commerce (EC05), pp. 127–136. ACM, New York (2005). doi:10.1145/1064009.1064023

    Chapter  Google Scholar 

  20. Fu, Y., Chase, J., Chun, B., Schwab, S., Vahdat, A.: Sharp: An architecture for secure resource peering. In: SOSP ’03: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 133–148. ACM, New York (2003). doi:10.1145/945445.945459

    Chapter  Google Scholar 

  21. Gomoluch, J., Schroeder, M.: Market-based resource allocation for Grid computing: A model and simulation. In: Endler, M., Schmidt, D. (eds.) Int. Middleware Conference, Workshop Proceedings, pp. 211–218. PUC-Rio, Rio De Janeiro (2003)

    Google Scholar 

  22. Grosu, D., Das, A.: Auctioning resources in Grids: model and protocols. Concurr. Comput. Pract. Exp. 18(15), 1909–1927 (2006). doi:10.1002/cpe

    Article  Google Scholar 

  23. Huang, Z., Qiu, Y.: Resource trading using cognitive agents: A hybrid perspective and its simulation. Future Gener. Comput. Syst. 23(7), 837–845 (2007). doi:10.1016/j.future.2007.02.006

    Article  Google Scholar 

  24. Iverson, M.A., Özgüner, F., Follen, G.J.: Run-time statistical estimation of task execution times for heterogeneous distributed computing. In: Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing, pp. 263–270. IEEE Computer Society (1996). doi:10.1109/HPDC.1996.546196

  25. Iverson, M.A., Özgüner, F., Potter, L.C.: Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment. IEEE Trans. Comput. 48(12), 1374–1379 (1999). doi:10.1109/12.817403

    Article  Google Scholar 

  26. Lai, K.: Markets are dead, long live markets. SIGecom Exch. 5(4), 1–10 (2005). doi:10.1145/1120717.1120719

    Article  Google Scholar 

  27. Lehmann, D., Maller, R., Sandholm., T.: The winner determination problem. In: Cramton, P., Shoham, Y., Steinberg, R. (eds.) Combinatorial Auctions, chap. 12, pp. 297–317. MIT Press, Cambridge (2006)

    Google Scholar 

  28. Lenstra, J.K., Kan, A.H.G.R., Brucker, P.: Complexity of machine scheduling problems. Ann. Discrete Math. 1, 343–362 (1977)

    Article  Google Scholar 

  29. Lewis, R.M., Torczon, V.: A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12, 1075–1089 (2002). doi:10.1137/S1052623498339727

    Article  MathSciNet  MATH  Google Scholar 

  30. LLC, A.W.S.: Amazon ec2 spot instances (2009). http://aws.amazon.com/ec2/spot-instances/. Accessed 23 December 2009

  31. Mills, K.L., Dabrowski, C.: Can economics-based resource allocation prove effective in a computation marketplace? Journal of Grid Computing 6(3), 291–311 (2008). doi:10.1007/s10723-007-9094-4

    Article  Google Scholar 

  32. Mohammed, A.B., Altmann, J.: A funding and governing model for achieving sustainable growth of computing e-infrastructures. Annals of Telecommunications (2010). doi:10.1007/s12243-010-0188-9 (online)

  33. Myerson, R.B., Satterthwaite, M.A.: Efficient mechanisms for bilateral trading. J. Econ. Theory 29(2), 265–281 (1983). doi:10.1016/0022-0531(83)90048-0

    Article  MathSciNet  MATH  Google Scholar 

  34. Neumann, D., Stößer, J., Weinhardt, C.: Bridging the adoption gap—developing a roadmap for trading Grids. Int. J. Electron. Mark. 18(1), 65–74 (2008). doi:10.1080/10196780701797664

    Article  Google Scholar 

  35. Neumann, D., Stößer, J., Weinhardt, C., Nimis, J.: A framework for commercial Grids—economic and technical challenges. Journal of Grid Computing 6(3), 325–347 (2008). doi:10.1007/s10723-008-9105-0

    Article  Google Scholar 

  36. Opitz, A., König, H., Szamlewskax, S.: What does Grid computing cost? Journal of Grid Computing 6(4), 385–397 (2008). doi:10.1007/s10723-008-9098-8

    Article  Google Scholar 

  37. Regev, O., Nisan, N.: The POPCORN market—an online market for computational resources. In: Proceedings of the 1st International Conference on Information and Computation Economies, pp. 148–157. ACM, New York (1998). doi:10.1145/288994.289027

    Chapter  Google Scholar 

  38. Sandholm, T., Lai, K., Clearwater, S.H.: Admission control in a computational market. In: Proceedings of CCGrid 2008, pp. 277–286. IEEE Computer Society (2008). doi:10.1109/CCGRID.2008.82

  39. Schnizler, B.: Resource Allocation in the Grid—A Market Engineering Approach. Ph.D. thesis, University of Karlsruhe (2007)

  40. Schnizler, B., Neumann, D., Veit, D., Weinhardt, C.: Trading Grid services—a multi-attribute combinatorial approach. Eur. J. Oper. Res. 187(3), 943–961 (2008). doi:10.1016/j.ejor.2006.05.049

    Article  MATH  Google Scholar 

  41. Smale, S.: A convergent process of price adjustment and global newton methods. J. Math. Econ. 3(2), 107–120 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  42. StöSSer, J., Neumann, D.: Greedex-a scalable clearing mechanism for utility computing. Electronic Commerce Research 8(4), 235–253 (2008). doi:10.1007/s10660-008-9023-z

    Article  MATH  Google Scholar 

  43. Stößer, J., Neumann, D.: GreedEx—a scalable clearing mechanism for utility computing. In: Proceedings of the Networking and Electronic Commerce Research Conference (NAEC) 2007 (2007)

  44. Streitberger, W., Hudert, S., Eymann, T., Schnizler, B., Zini, F., Catalano, M.: On the simulation of Grid market coordination approaches. Journal of Grid Computing 6(3), 349–366 (2008). doi:10.1007/s10723-007-9092-6

    Article  Google Scholar 

  45. Stuer, G., Vanmechelen, K., Broeckhove, J.: A commodity market algorithm for pricing substitutable Grid resources. Future Gener. Comput. Syst. 23(5), 688–701 (2007). doi:10.1016/j.future.2006.11.004

    Article  Google Scholar 

  46. Vanmechelen, K., Broeckhove, J.: A comparative analysis of single-unit Vickrey auctions and commodity markets for realizing Grid economies with dynamic pricing. In: Altmann, J., Veit, D. (eds.) Proceedings of the 4th International Workshop on Grid Economics and Business Models (GECON 2007). Lecture Notes in Computer Science, vol. 4685, pp. 98–111. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74430-6_8

    Google Scholar 

  47. Vanmechelen, K., Depoorter, W., Broeckhove, J.: Economic Grid resource management for CPU bound applications with hard deadlines. In: Proceedings of CCGrid 2008, pp. 258–266. IEEE Computer Society (2008). doi:10.1109/CCGRID.2008.11

  48. Vanmechelen, K., Depoorter, W., Broeckhove, J.: A simulation framework for studying economic resource management in Grids. In: Proceedings of the International Conference on Computational Science (ICCS 2008), vol. 5101, pp. 226–235. Springer, Berlin (2008). doi:10.1007/978-3-540-69384-0_28

    Google Scholar 

  49. Vanmechelen, K., Depoorter, W., Broeckhove, J.: Market-based Grid resource co-allocation and reservation for applications with hard deadlines. Concurr. Comput. Pract. Exp. 21, 2270–2297 (2009)

    Article  Google Scholar 

  50. Vanmechelen, K., Stuer, G., Broeckhove, J.: Pricing substitutable Grid resources using commodity market models. In: Lee, H., Miller, S. (eds.) Proceedings of the 3rd Int. Workshop on Grid Economics and Business Models (GECON 2006), pp. 103–112. World Scientific, Singapore (2006)

    Chapter  Google Scholar 

  51. Waldspurger, C.A., Hogg, T., Huberman, B.A., Kephart, J.O., Stornetta, W.S.: Spawn: a distributed computational economy. IEEE Trans. Softw. Eng. 18(2), 103–117 (1992). doi:10.1109/32.121753

    Article  Google Scholar 

  52. Wolski, R., Brevik, J., Plank, J., Bryan, T.: Grid resource allocation and control using computational economies. In: Berman, F., Fox, G.C., Hey, A.J. (eds.) Grid Computing: Making the Global Infrastructure a Reality, chap. 32, pp. 747–772. Wiley, New York (2003)

    Google Scholar 

  53. Xiao, L., Zhu, Y., Ni, L.M., Xu., Z.: GridIS: An incentive-based Grid scheduling. In: Proceedings of IPDPS’2005, pp. 65–74. IEEE Computer Society, Los Alamitos (2005). doi:10.1109/IPDPS.2005.237

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Sciences, University of Antwerp, Middelheimlaan 1, 2020, Antwerp, Belgium

    Kurt Vanmechelen, Wim Depoorter & Jan Broeckhove

Authors
  1. Kurt Vanmechelen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wim Depoorter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Broeckhove
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kurt Vanmechelen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vanmechelen, K., Depoorter, W. & Broeckhove, J. Combining Futures and Spot Markets: A Hybrid Market Approach to Economic Grid Resource Management. J Grid Computing 9, 81–94 (2011). https://doi.org/10.1007/s10723-010-9174-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-010-9174-8

Keywords

Search

Navigation