Skip to main content
Book cover

International Workshop on Adaptive Resource Management and Scheduling for Cloud Computing

ARMS-CC 2015: Adaptive Resource Management and Scheduling for Cloud Computing pp 60–77Cite as

User-Guided Provisioning in Federated Clouds for Distributed Calculations

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 9438)

Abstract

Cloud Computing promises to be more flexible, usable, available and simple than Grid, covering also much more computational needs than the ones required to carry out distributed calculations. However, the diversity of cloud providers, the lack of standardised APIs and brokering tools prevent the massive portability of legacy applications to cloud environments. In this work a new framework to effectively schedule distributed calculations in cloud federations is presented. The system takes account of the experience in large and collaborative grid federations to provide several basic features that differentiates it from other approaches, such as the decentralisation, middleware independence, dynamic brokering, on-demand provisioning of specific virtual images, compatibility with legacy applications, efficient accomplishment of short tasks, etc. In this sense, the mechanisms that allow users to consolidate their own resource provisioning in cloud federations are the focus of this work. To demonstrate the suitability of the new approach, a common application to model radiation beams has been scheduled into the EGI FedCloud.

Keywords

  • Virtual Machine
  • Cloud Provider
  • Cloud Resource
  • Virtual Cluster
  • Legacy Application

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-28448-4_5
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   39.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-28448-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   49.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.

Notes

  1. 1.

    http://aws.amazon.com.

  2. 2.

    http://cloudinit.readthedocs.org.

  3. 3.

    https://www.egi.eu/infrastructure/cloud/.

  4. 4.

    http://sixsq.com/products/slipstream.html.

  5. 5.

    http://www.compatibleone.org.

  6. 6.

    https://appdb.egi.eu/store/vo/image/de355bfb-5781-5b0c-9ccd-9bd3d0d2be06.

References

  1. Anastasi, G.F., Carlini, E., Coppola, M., Dazzi, P.: BROKAGE: a genetic approach for QoS cloud brokering. In: 7th IEEE International Conference on Cloud Computing (IEEE CLOUD 2014), 27 June–2 July, Alaska, USA, pp. 304–311 (2014). doi:10.1109/CLOUD.2014.49

  2. Edmonds, A., Metsch, T., Papaspyrou, A., Richardson, A.: Toward an open cloud standard. IEEE Internet Comput. 16(4), 15–25 (2012)

    CrossRef  Google Scholar 

  3. Foster, I., Kesselman, C., Nick, J., Tuecke, S.: Grid services for distributed system integration. Computer 35(6), 37–46 (2002)

    CrossRef  Google Scholar 

  4. Garey, M., Johnson, D.: Computers and Intractibility: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1979)

    MATH  Google Scholar 

  5. Graciani, R., Casajús, A., Carmona, A., Fifield, T., Sevior, M.: Belle-DIRAC setup for using amazon elastic compute cloud. J. Grid Comput. 9(1), 65–79 (2011)

    CrossRef  Google Scholar 

  6. Grozev, N., Buyya, R.: Inter-cloud architectures and application brokering: taxonomy and survey. Softw. Pract. Experience 44, 369–390 (2014)

    CrossRef  Google Scholar 

  7. Huedo, E., Montero, R.S., Llorente, I.M.: A modular meta-scheduling architecture for interfacing with pre-WS and WS grid resource management services. Future Gener. Comput. Syst. 23(2), 252–261 (2007)

    CrossRef  Google Scholar 

  8. Jain, R.: The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley-Interscience, New York (1991)

    MATH  Google Scholar 

  9. Juve, G., Deelman, E.: Automating application deployment in infrastructure clouds. In: Third International Conference on Cloud Computing Technology and Science (CloudCom), 9 November–1 December, pp. 658–665 (2011). doi:10.1109/CloudCom.2011.102

  10. Kertesz, A.: Characterizing Cloud Federation Approaches, chap. 12. Computer Communications and Networks, pp. 277–296. Springer (2014). doi:10.1007/978-3-319-10530-7_12

    Google Scholar 

  11. Kovács, J., Marosi, A.C., Visegrádi, A., Farkas, Z., Kacsuk, P., Lovas, R.: Boosting gLite with cloud augmented volunteer computing. Future Gener. Comput. Sys. 43–44, 12–23 (2015)

    CrossRef  Google Scholar 

  12. Lorca, A., Martín-Caro, J., Núez-Ramirez, R., Martínez-Salazar, J.: Merging on-demand HPC resources from amazon EC2 with the grid: a case study of a Xmipp application. Comput. Inf. 31(1), 17–30 (2012)

    Google Scholar 

  13. Lordan, F., Tejedor, E., Ejarque, J., Rafanell, R., Álvarez, J., Marozzo, F., Lezzi, D., Sirvent, R., Talia, D., Badia, R.M.: ServiceSs: an interoperable programming framework for the cloud. J. Grid Comput. 12(1), 67–91 (2014)

    CrossRef  Google Scholar 

  14. Luckow, A., Santcroos, M., Zebrowski, A., Jha, S.: Pilot-data: an abstraction for distributed data. J. Parallel Distrib. Comput. (2014). doi:10.1016/j.jpdc.2014.09.009

    Google Scholar 

  15. Méndez, V., Casajús, A., Fernández, V., Graciani, R., Merino, G.: Rafhyc: an architecture for constructing resilient services on federated hybrid clouds. J. Grid Comput. 11, 753–770 (2013)

    CrossRef  Google Scholar 

  16. Mhashilkar, P., Tiradani, A., Holzman, B., Larson, K., Sfiligoi, I., Rynge, M.: Cloud bursting with glideinwms: means to satisfy ever increasing computing needs for scientific workflows. In: 20th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2013), Journal of Physics: Conference Series, vol. 513, p. 032069. IOP Publishing (2014). doi:10.1088/1742-6596/513/3/032069

    Google Scholar 

  17. Michon, E., Gossa, J., Genaud, S., Frincu, M., Burel, A.: Porting grid applications to the cloud with schlouder. In: IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), 2–5 December, pp. 505–512, Bristol, UK (2013). doi:10.1109/CloudCom..73

  18. Montero, R.S., Moreno-Vozmediano, R.: I.M. Llorente: an elasticity model for high throughput computing clusters. J. Parallel Distrib. Comput. 71(6), 750–757 (2011)

    CrossRef  Google Scholar 

  19. Moreno-Vozmediano, R., Montero, R.S., Llorente, I.M.: Multi-cloud deployment of computing clusters for loosely-coupled mtc applications. IEEE Trans. Parallel Distrib. Syst. 22(6), 924–930 (2011)

    CrossRef  Google Scholar 

  20. Parák, B., Šustr, Z., Feldhaus, F., Kasprzakc, P., Srbac, M.: The rOCCI project: providing cloud interoperability with OCCI 1.1. In: International Symposium on Grids and Clouds (ISGC), 23–28 March, Taipei, Taiwan, pp. 1–15. SISA PoS (2014)

    Google Scholar 

  21. Riedel, M., Laure, E., Soddemann, T., Field, L., et al.: Interoperation of world-wide production e-Science infrastructures. Concurrency Comput. Pract. Experience 21(8), 961–990 (2009)

    CrossRef  Google Scholar 

  22. Rodríguez, M., Tapiador, D., Fontan, J., Huedo, E., Montero, R., Llorente, I.: Dynamic provisioning of virtual clusters for grid computing. In: César, E., Alexander, M., Streit, A., Larsson, J., Cérin, C., Knüpfer, A., Kranzlmüller, D., Jha, S. (eds.) Euro-Par 2008 Workshops. LNCS, vol. 5415, pp. 23–32. Springer, Heidelberg (2009)

    CrossRef  Google Scholar 

  23. Rogers, D.W.O., Faddegon, B.A., Ding, G.X., Ma, C.M., Wei, J., Mackie, T.R.: BEAM: a monte carlo code to simulate radiotherapy treatment units. Med. Phys. 22, 503–524 (1995)

    CrossRef  Google Scholar 

  24. Rubio-Montero, A.J., Castejón, F., Huedo, E., Mayo-García, R.: A novel pilot job approach for improving the execution of distributed codes: application to the study of ordering in collisional transport in fusion plasmas. Concurrency Comput. Pract. Experience 27(13), 3220–3244 (2015)

    CrossRef  Google Scholar 

  25. Rubio-Montero, A.J., Huedo, E., Castejón, F., Mayo-García, R.: GWpilot: enabling multi-level scheduling in distributed infrastructures with GridWay and pilot jobs. Future Gener. Comput. Syst. 45, 25–52 (2015)

    CrossRef  Google Scholar 

  26. Rubio-Montero, A.J., Montero, R.S., Huedo, E., Llorente, I.M.: Management of virtual machines on globus grids using gridway. In: 21st IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 1–7 (2007). doi:10.1109/IPDPS.2007.370548

  27. Rubio-Montero, A.J., Rodríguez-Pascual, M.A., Mayo-García, R.: Evaluation of an adaptive framework for resilient monte carlo executions. In: 30th ACM/SIGAPP Symposium On Applied Computing (SAC 2015), 13–17 April, Salamanca, Spain, pp. 448–455 (2015). doi:10.1145/2695664.2695890

  28. Sehgal, S., Erdelyi, M., Merzky, A., Jha, S.: Understanding application-level interoperability: scaling-out MapReduce over high-performance grids and clouds. Future Gener. Comput. Syst. 27(5), 590–599 (2011)

    CrossRef  Google Scholar 

  29. Sheikhalishahi, M., Wallace, R., Grandinetti, L., Vázquez-Poletti, J.L., Guerriero, F.: A multi-dimensional job scheduling. Future Generation Computer Systems (2015). doi:10.1016/j.future.2015.03.014

    Google Scholar 

  30. Simón, A., Freire, E., Rosende, R., Díaz, I., Feijóo, A., Rey, P., López-Cacheiro, J., Fernández, C.: EGI FedCloud task force. In: 6th Grid Iberian Infrastructure Conference (IBERGRID 2012), 7–9 November, Lisbon, Portugal, pp. 183–194 (2012)

    Google Scholar 

  31. Torberntsson, K., Rydin, Y.: A Study of Configuration Management Systems. Solutions for Deployment and Configuration of Software in a Cloud Environment (June 2014), B.S. Thesis. Uppsala University, Sweden

    Google Scholar 

  32. Tordsson, J., Montero, R.S., Moreno-Vozmediano, R., Llorente, I.M.: Cloud brokering mechanisms for optimized placement of virtual machines across multiple providers. Future Gener. Comput. Syst. 28(2), 358–367 (2012)

    CrossRef  Google Scholar 

  33. Tröger, P., Merzky, A.: Towards standardized job submission and control in infrastructure clouds. J. Grid Comput. 12, 111–125 (2014)

    CrossRef  Google Scholar 

  34. Vázquez, C., Huedo, E., Montero, R.S., Llorente, I.M.: On the use of clouds for grid resource provisioning. Future Gener. Comput. Syst. 27(5), 600–605 (2011)

    CrossRef  Google Scholar 

  35. Walker, E., Gardner, J., Litvin, V., Turner, E.: Personal adaptive clusters as containers for scientific jobs. Cluster Comput. 10(3), 339–350 (2007)

    CrossRef  Google Scholar 

  36. Wang, L., Tao, J., Kunze, M., Castellanos, A.C., Kramer, D., Karl, W.: Scientific cloud computing: early definition and experience. In: 10th IEEE International Conference on High Performance Computing and Communications (HPCC 2008), 25–27 September, Dalian, China, pp. 825–830 (2008). doi:10.1109/HPCC.2008.38

  37. Yangui, S., Marshall, I.J., Laisne, J.P., Tata, S.: CompatibleOne: the open source cloud broker. J. Grid Comput. 12(1), 93–109 (2014)

    CrossRef  Google Scholar 

Download references

Acknowledgements

This work was supported by the COST Action BETTY (IC 1201) and partially funded by the Spanish Ministry of Economy and Competitiveness project CODEC (TIN2013-46009-P).

Author information

Authors and Affiliations

  1. Centro de Investigaciones Energéticas Medioambientales y Tecnológicas (CIEMAT), Madrid, Spain

    A. J. Rubio-Montero & R. Mayo-García

  2. Dep. de Arquitectura de Computadores y Automática, Facultad de Informática, Universidad Complutense de Madrid, Madrid, Spain

    E. Huedo

Authors
  1. A. J. Rubio-Montero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. Huedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Mayo-García
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. J. Rubio-Montero .

Editor information

Editors and Affiliations

  1. University Politehnica of Bucharest, Bucharest, Romania

    Florin Pop

  2. Université Pierre et Marie CURIE, Paris, France

    Maria Potop-Butucaru

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Rubio-Montero, A.J., Huedo, E., Mayo-García, R. (2015). User-Guided Provisioning in Federated Clouds for Distributed Calculations. In: Pop, F., Potop-Butucaru, M. (eds) Adaptive Resource Management and Scheduling for Cloud Computing. ARMS-CC 2015. Lecture Notes in Computer Science(), vol 9438. Springer, Cham. https://doi.org/10.1007/978-3-319-28448-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28448-4_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28447-7

  • Online ISBN: 978-3-319-28448-4

  • eBook Packages: Computer ScienceComputer Science (R0)