Journal of Grid Computing

, Volume 9, Issue 1, pp 81–94 | Cite as

Combining Futures and Spot Markets: A Hybrid Market Approach to Economic Grid Resource Management

Article

Abstract

Economic forms of resource management in which users can express their valuations for service, offer new possibilities for optimizing resource allocations in Grids. If users are to correctly express these valuations, quality of service guarantees need to be given with respect to the turnaround time of their workloads. Market mechanisms that support bidding and allocations in future time are crucial for delivering such guarantees. To deal with the significant delays that these mechanisms introduce in the allocation process, we present a hybrid market approach in which a low-latency spot market coexists with a higher latency futures market. Based on simulated market scenarios, we show how this combination can significantly increase the total value realized by the Grid infrastructure. We also demonstrate how providers can react to price dynamics in such a hybrid market setting.

Keywords

Resource management Grid economics Hybrid markets Futures markets Spot markets 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abdelkader, K., Broeckhove, J.: Pricing computational resources in a dynamic Grid. International Journal of Grid and Utility Computing (IJGUC) 1, 205–215 (2009). doi:10.1504/IJGUC.2009.027648 CrossRefGoogle Scholar
  2. 2.
    Abramson, D., Buyya, R., Giddy, J.: A computational economy for Grid computing and its implementation in the Nimrod-G resource broker. Future Gener. Comput. Syst. 18(8), 1061–1074 (2002). doi:10.1016/S0167-739X(02)00085-7 MATHCrossRefGoogle Scholar
  3. 3.
    Amazon: Elastic compute cloud. http://aws.amazon.com/ec2(2010). Accessed 22 September 2010
  4. 4.
    Auyoung, A., Chun, B.N., Snoeren, A.C., Vahdat, A.: Resource allocation in federated distributed computing infrastructures. In: Proceedings of the 1st Workshop on Operating System and Architectural Support for the On-demand IT InfraStructure (2004). http://www.theether.org/papers/oasis04.pdf
  5. 5.
    Bapna, R., Das, S., Garfinkel, R., Stallaert, J.: A market design for Grid computing. INFORMS J. Comput. 20(1), 100–111 (2007). doi:10.1287/ijoc.1070.0221 CrossRefGoogle Scholar
  6. 6.
    Broberg, J., Venugopal, S., Buyya, R.: Market-oriented Grids and utility computing: the state-of-the-art and future directions. Journal of Grid Computing 6(2), 255–276 (2008). doi:10.1007/s10723-007-9095-3 CrossRefGoogle Scholar
  7. 7.
    Brucker, P.: Scheduling Algortihms. Springer, Berlin (2004)Google Scholar
  8. 8.
    Buyya, R.: Economic-based Distributed Resource Management and Scheduling for Grid Computing. Ph.D. thesis, Monash University, Australia (2002)Google Scholar
  9. 9.
    Buyya, R., Abramson, D., Giddy, J.: Economic models for resource management and scheduling in Grid computing. Concurr. Comput. Pract. Exp. 14, 1507–1542 (2002). doi:10.1002/cpe.690 MATHCrossRefGoogle Scholar
  10. 10.
    Buyya, R., Abramson, D., Venugopal, S.: The Grid economy. Proc. IEEE 93(3), 698–714 (2005). doi:10.1109/JPROC.2004.842784 CrossRefGoogle Scholar
  11. 11.
    Bykowsky, M.M., Cull, R.J., Ledyard, J.O.: Mutually destructive bidding: the FCC auction design problem. J. Regul. Econ. 17(3), 205–228 (2000). doi:10.1023/A:1008122015102 CrossRefGoogle Scholar
  12. 12.
    Cameron, D., Millar, A., Nicholson, C., Carvajal-Schiaffino, R., Stockinger, K., Zini, F.: Analysis of scheduling and replica optimisation strategies for data Grids using Optorsim. Journal of Grid Computing 2(1), 57–69 (2004). doi:10.1007/s10723-004-6040-6 CrossRefGoogle Scholar
  13. 13.
    Cheliotis, G., Kenyon, C., Buyya, R.: 10 Lessons from finance for commercial sharing of IT resources. In: Sumabramian, R., Goodman, B. (eds.) Peer-to-Peer Computing: Evolution of a Disruptive Technology, chap. 11, pp. 244–264. Idea Group Publishing, Hershey (2004)Google Scholar
  14. 14.
    Chun, B.N., Buonadonna, P., AuYoung, A., Chaki, N., Parkes, D., Shneidman, J., Snoeren, A., Vahdat, A.: Mirage: A microeconomic resource allocation system for sensornet testbeds. In: Proceedings of the Second IEEE Workshop on Embedded Networked Sensors, pp. 19–28. IEEE Computer Society (2005)Google Scholar
  15. 15.
    Clearwater, S. (ed.): Market-Based Control: A Paradigm for Distributed Resource Allocation. World Scientific, Singapore (1996)Google Scholar
  16. 16.
    Cramton, P., Shoham, Y., Steinberg, R. (eds.): Combinatorial Auctions. MIT Press, Cambridge (2006)MATHGoogle Scholar
  17. 17.
    EGEE project site: http://www.eu-egee.org/ (2008)
  18. 18.
    Engelen, R.A.V., Gallivan, K.A., Walsh, B.: Parametric timing estimation with Newton–Gregory formulae. Concurr. Comput. Pract. Exp. 18(11), 1435–1463 (2006). doi:10.1002/cpe.v18:11 CrossRefGoogle Scholar
  19. 19.
    Feldman, M., Lai, K., Zhang, L.: A price-anticipating resource allocation mechanism for distributed shared clusters. In: Proceedings of the 6th ACM Conference on Electronic Commerce (EC05), pp. 127–136. ACM, New York (2005). doi:10.1145/1064009.1064023 CrossRefGoogle Scholar
  20. 20.
    Fu, Y., Chase, J., Chun, B., Schwab, S., Vahdat, A.: Sharp: An architecture for secure resource peering. In: SOSP ’03: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 133–148. ACM, New York (2003). doi:10.1145/945445.945459 CrossRefGoogle Scholar
  21. 21.
    Gomoluch, J., Schroeder, M.: Market-based resource allocation for Grid computing: A model and simulation. In: Endler, M., Schmidt, D. (eds.) Int. Middleware Conference, Workshop Proceedings, pp. 211–218. PUC-Rio, Rio De Janeiro (2003)Google Scholar
  22. 22.
    Grosu, D., Das, A.: Auctioning resources in Grids: model and protocols. Concurr. Comput. Pract. Exp. 18(15), 1909–1927 (2006). doi:10.1002/cpe CrossRefGoogle Scholar
  23. 23.
    Huang, Z., Qiu, Y.: Resource trading using cognitive agents: A hybrid perspective and its simulation. Future Gener. Comput. Syst. 23(7), 837–845 (2007). doi:10.1016/j.future.2007.02.006 CrossRefGoogle Scholar
  24. 24.
    Iverson, M.A., Özgüner, F., Follen, G.J.: Run-time statistical estimation of task execution times for heterogeneous distributed computing. In: Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing, pp. 263–270. IEEE Computer Society (1996). doi:10.1109/HPDC.1996.546196
  25. 25.
    Iverson, M.A., Özgüner, F., Potter, L.C.: Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment. IEEE Trans. Comput. 48(12), 1374–1379 (1999). doi:10.1109/12.817403 CrossRefGoogle Scholar
  26. 26.
    Lai, K.: Markets are dead, long live markets. SIGecom Exch. 5(4), 1–10 (2005). doi:10.1145/1120717.1120719 CrossRefGoogle Scholar
  27. 27.
    Lehmann, D., Maller, R., Sandholm., T.: The winner determination problem. In: Cramton, P., Shoham, Y., Steinberg, R. (eds.) Combinatorial Auctions, chap. 12, pp. 297–317. MIT Press, Cambridge (2006)Google Scholar
  28. 28.
    Lenstra, J.K., Kan, A.H.G.R., Brucker, P.: Complexity of machine scheduling problems. Ann. Discrete Math. 1, 343–362 (1977)CrossRefGoogle Scholar
  29. 29.
    Lewis, R.M., Torczon, V.: A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12, 1075–1089 (2002). doi:10.1137/S1052623498339727 MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    LLC, A.W.S.: Amazon ec2 spot instances (2009). http://aws.amazon.com/ec2/spot-instances/. Accessed 23 December 2009
  31. 31.
    Mills, K.L., Dabrowski, C.: Can economics-based resource allocation prove effective in a computation marketplace? Journal of Grid Computing 6(3), 291–311 (2008). doi:10.1007/s10723-007-9094-4 CrossRefGoogle Scholar
  32. 32.
    Mohammed, A.B., Altmann, J.: A funding and governing model for achieving sustainable growth of computing e-infrastructures. Annals of Telecommunications (2010). doi:10.1007/s12243-010-0188-9 (online)
  33. 33.
    Myerson, R.B., Satterthwaite, M.A.: Efficient mechanisms for bilateral trading. J. Econ. Theory 29(2), 265–281 (1983). doi:10.1016/0022-0531(83)90048-0 MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Neumann, D., Stößer, J., Weinhardt, C.: Bridging the adoption gap—developing a roadmap for trading Grids. Int. J. Electron. Mark. 18(1), 65–74 (2008). doi:10.1080/10196780701797664 CrossRefGoogle Scholar
  35. 35.
    Neumann, D., Stößer, J., Weinhardt, C., Nimis, J.: A framework for commercial Grids—economic and technical challenges. Journal of Grid Computing 6(3), 325–347 (2008). doi:10.1007/s10723-008-9105-0 CrossRefGoogle Scholar
  36. 36.
    Opitz, A., König, H., Szamlewskax, S.: What does Grid computing cost? Journal of Grid Computing 6(4), 385–397 (2008). doi:10.1007/s10723-008-9098-8 CrossRefGoogle Scholar
  37. 37.
    Regev, O., Nisan, N.: The POPCORN market—an online market for computational resources. In: Proceedings of the 1st International Conference on Information and Computation Economies, pp. 148–157. ACM, New York (1998). doi:10.1145/288994.289027 CrossRefGoogle Scholar
  38. 38.
    Sandholm, T., Lai, K., Clearwater, S.H.: Admission control in a computational market. In: Proceedings of CCGrid 2008, pp. 277–286. IEEE Computer Society (2008). doi:10.1109/CCGRID.2008.82
  39. 39.
    Schnizler, B.: Resource Allocation in the Grid—A Market Engineering Approach. Ph.D. thesis, University of Karlsruhe (2007)Google Scholar
  40. 40.
    Schnizler, B., Neumann, D., Veit, D., Weinhardt, C.: Trading Grid services—a multi-attribute combinatorial approach. Eur. J. Oper. Res. 187(3), 943–961 (2008). doi:10.1016/j.ejor.2006.05.049 MATHCrossRefGoogle Scholar
  41. 41.
    Smale, S.: A convergent process of price adjustment and global newton methods. J. Math. Econ. 3(2), 107–120 (1976)MathSciNetMATHCrossRefGoogle Scholar
  42. 42.
    StöSSer, J., Neumann, D.: Greedex-a scalable clearing mechanism for utility computing. Electronic Commerce Research 8(4), 235–253 (2008). doi:10.1007/s10660-008-9023-z MATHCrossRefGoogle Scholar
  43. 43.
    Stößer, J., Neumann, D.: GreedEx—a scalable clearing mechanism for utility computing. In: Proceedings of the Networking and Electronic Commerce Research Conference (NAEC) 2007 (2007)Google Scholar
  44. 44.
    Streitberger, W., Hudert, S., Eymann, T., Schnizler, B., Zini, F., Catalano, M.: On the simulation of Grid market coordination approaches. Journal of Grid Computing 6(3), 349–366 (2008). doi:10.1007/s10723-007-9092-6 CrossRefGoogle Scholar
  45. 45.
    Stuer, G., Vanmechelen, K., Broeckhove, J.: A commodity market algorithm for pricing substitutable Grid resources. Future Gener. Comput. Syst. 23(5), 688–701 (2007). doi:10.1016/j.future.2006.11.004 CrossRefGoogle Scholar
  46. 46.
    Vanmechelen, K., Broeckhove, J.: A comparative analysis of single-unit Vickrey auctions and commodity markets for realizing Grid economies with dynamic pricing. In: Altmann, J., Veit, D. (eds.) Proceedings of the 4th International Workshop on Grid Economics and Business Models (GECON 2007). Lecture Notes in Computer Science, vol. 4685, pp. 98–111. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74430-6_8 Google Scholar
  47. 47.
    Vanmechelen, K., Depoorter, W., Broeckhove, J.: Economic Grid resource management for CPU bound applications with hard deadlines. In: Proceedings of CCGrid 2008, pp. 258–266. IEEE Computer Society (2008). doi:10.1109/CCGRID.2008.11
  48. 48.
    Vanmechelen, K., Depoorter, W., Broeckhove, J.: A simulation framework for studying economic resource management in Grids. In: Proceedings of the International Conference on Computational Science (ICCS 2008), vol. 5101, pp. 226–235. Springer, Berlin (2008). doi:10.1007/978-3-540-69384-0_28 Google Scholar
  49. 49.
    Vanmechelen, K., Depoorter, W., Broeckhove, J.: Market-based Grid resource co-allocation and reservation for applications with hard deadlines. Concurr. Comput. Pract. Exp. 21, 2270–2297 (2009)CrossRefGoogle Scholar
  50. 50.
    Vanmechelen, K., Stuer, G., Broeckhove, J.: Pricing substitutable Grid resources using commodity market models. In: Lee, H., Miller, S. (eds.) Proceedings of the 3rd Int. Workshop on Grid Economics and Business Models (GECON 2006), pp. 103–112. World Scientific, Singapore (2006)CrossRefGoogle Scholar
  51. 51.
    Waldspurger, C.A., Hogg, T., Huberman, B.A., Kephart, J.O., Stornetta, W.S.: Spawn: a distributed computational economy. IEEE Trans. Softw. Eng. 18(2), 103–117 (1992). doi:10.1109/32.121753 CrossRefGoogle Scholar
  52. 52.
    Wolski, R., Brevik, J., Plank, J., Bryan, T.: Grid resource allocation and control using computational economies. In: Berman, F., Fox, G.C., Hey, A.J. (eds.) Grid Computing: Making the Global Infrastructure a Reality, chap. 32, pp. 747–772. Wiley, New York (2003)Google Scholar
  53. 53.
    Xiao, L., Zhu, Y., Ni, L.M., Xu., Z.: GridIS: An incentive-based Grid scheduling. In: Proceedings of IPDPS’2005, pp. 65–74. IEEE Computer Society, Los Alamitos (2005). doi:10.1109/IPDPS.2005.237 Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Kurt Vanmechelen
    • 1
  • Wim Depoorter
    • 1
  • Jan Broeckhove
    • 1
  1. 1.Department of Mathematics and Computer SciencesUniversity of AntwerpAntwerpBelgium

Personalised recommendations