Constructing Virtual Private Supercomputer Using Virtualization and Cloud Technologies

  • Ivan Gankevich
  • Vladimir Korkhov
  • Serob Balyan
  • Vladimir Gaiduchok
  • Dmitry Gushchanskiy
  • Yuri Tipikin
  • Alexander Degtyarev
  • Alexander Bogdanov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8584)

Abstract

One of efficient ways to conduct experiments on HPC platforms is to create custom virtual computing environments tailored to the requirements of users and their applications. In this paper we investigate virtual private supercomputer, an approach based on virtualization, data consolidation, and cloud technologies. Virtualization is used to abstract applications from underlying hardware and operating system while data consolidation is applied to store data in a distributed storage system. Both virtualization and data consolidation layers offer APIs for distributed computations and data processing. Combined, these APIs shift the focus from supercomputing technologies to problems being solved. Based on these concepts, we propose an approach to construct virtual clusters with help of cloud computing technologies to be used as on-demand private supercomputers and evaluate performance of this solution.

Keywords

virtualization supercomputer virtual cluster cloud computing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Smarr, L., Catlett, C.E.: Metacomputing. Communications of the ACM 35(6), 44–52 (1992)CrossRefGoogle Scholar
  2. 2.
    Korkhov, V.V., Moscicki, J.T., Krzhizhanovskaya, V.V.: The user-level scheduling of divisible load parallel applications with resource selection and adaptive workload balancing on the grid. IEEE Systems Journal 3(1), 121–130 (2009)CrossRefGoogle Scholar
  3. 3.
    Figueiredo, R.J., Dinda, P.A., Fortes, J.A.B.: A case for grid computing on virtual machines. In: Proceedings of the 23rd International Conference on Distributed Computing Systems (2003)Google Scholar
  4. 4.
    Matsunaga, A.M., Tsugawa, M.O., Adabala, S., Figueiredo, R.J., Lam, H., Fortes, J.A.B.: Science gate- ways made easy: the In-VIGO approach. Concurrency and Computation: Practice and Experience 19(6), 905–919 (2007)CrossRefGoogle Scholar
  5. 5.
    Krsul, I., Ganguly, A., Zhang, J., Fortes, J.A.B., Figueiredo, R.J.: VMPlants: Providing and manag- ing virtual machine execution environments for grid computing. In: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing (2004)Google Scholar
  6. 6.
    Nishimura, H., Maruyama, N., Matsuoka, S.: Virtual clusters on the fly - fast, scalable, and flexible installation. In: CCGRID 2007: Seventh IEEE International Symposium on Cluster Computing and the Grid (May 2007)Google Scholar
  7. 7.
    Emeneker, W., Stanzione, D.: Dynamic virtual clustering. In: IEEE Cluster 2007, Austin, TX (September 2007)Google Scholar
  8. 8.
    Chase, J.S., Irwin, D.E., Grit, L.E., Moore, J.D., Sprenkle, S.E.: Dynamic virtual clusters in a grid site manager. In: HPDC 2003: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing, p. 90. IEEE Computer Society, Washington, DC (2003)Google Scholar
  9. 9.
    Shuming, S., Zhang, H., Yuan, X., Wen, J.-R.: Corpus-based semantic class mining: distributional vs. pattern-based approaches. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 993–1001 (2010)Google Scholar
  10. 10.
    Andrew, L., Gregor, D., Hendrickson, B., Berry, J.: Challenges in parallel graph processing, Parallel Processing Letters, vol. Parallel Processing Letters 17(01), 5–20 (2007)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Grzegorz, M., Austern, M.H., Bik, A.J., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 135–146 (2010)Google Scholar
  12. 12.
    Iordanov, B.: HyperGraphDB: A generalized graph database. In: Shen, H.T., Pei, J., Özsu, M.T., Zou, L., Lu, J., Ling, T.-W., Yu, G., Zhuang, Y., Shao, J. (eds.) WAIM 2010. LNCS, vol. 6185, pp. 25–36. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Kevin, K., Luk, S.K.: Building a large-scale knowledge base for machine translation. In: Proceedings of the National Conference on Artificial Intelligence, p. 773 (1994)Google Scholar
  14. 14.
    Ravi, K., Raghavan, P., Rajagopalan, S., Tomkins, A.: Extracting large-scale knowledge bases from the web. In: Proceeding of the International Conference on Very Large Data Bases, pp. 639–650 (1990)Google Scholar
  15. 15.
    Degtyarev, A., Gankevich, I.: Efficiency comparison of wave surface generation using OpenCL, OpenMP and MPI. In: Proceedings of 8th International Conference Computer Science & Information Technologies, Yerevan, Armenia, pp. 248–251 (2011)Google Scholar
  16. 16.
    Wibisono, A., Vasyunin, D., Korkhov, V.V., Zhao, Z., Belloum, A., de Laat, C., Adriaans, P.W., Hertzberger, B.: WS-VLAM: A GT4 based workflow management system. In: Shi, Y., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2007, Part III. LNCS, vol. 4489, pp. 191–198. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Peter, T., et al.: Standardization of an API for distributed resource management systems. In: Seventh IEEE International Symposium on Cluster Computing and the Grid, CCGRID 2007. IEEE (2007)Google Scholar
  18. 18.
    Douglas, T., Tannenbaum, T., Livny, M.: Distributed computing in practice: The Condor experience. Concurrency and Computation: Practice and Experience 17(2-4), 323–356 (2005)Google Scholar
  19. 19.
    Jeffrey, D., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Communications of the ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  20. 20.
    Ping, A., et al.: STAPL: An adaptive, generic parallel C++ library. In: Dietz, H.G. (ed.) LCPC 2001. LNCS, vol. 2624, pp. 193–208. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  21. 21.
    Bogdanov, A., Dmitriev, M.: Creation of hybrid clouds. In: Proceedings of 8th International Conference Computer Science & Information Technologies, Yerevan, Armenia, pp. 235–237 (2011)Google Scholar
  22. 22.
    Paul, B., et al.: Xen and the art of virtualization. ACM SIGOPS Operating Systems Review 37(5), 164–177 (2003)CrossRefGoogle Scholar
  23. 23.
    Hamlen, K., Kantarcioglu, M., Khan, L., Thuraisingham, B.: Security Issues for Cloud ComputingGoogle Scholar
  24. 24.
  25. 25.
    Resource Center Computer Center of St.Petersburg State University, http://cc.spbu.ru
  26. 26.
    Berendsen, H.J.C., van der Spoel, D., van Drunen, R.: GROMACS: A message-passing parallel molecular dynamics implementation. Computer Physics Communications 91(1-3), 43–56 (1995) ISSN 0010-4655Google Scholar
  27. 27.
    Rodríguez, M., Tapiador, D., Fontán, J., Huedo, E., Montero, R.S., Llorente, I.M.: Dynamic Provisioning of Virtual Clusters for Grid Computing. In: César, E., Alexander, M., Streit, A., Träff, J.L., Cérin, C., Knüpfer, A., Kranzlmüller, D., Jha, S. (eds.) Euro-Par 2008. LNCS, vol. 5415, pp. 23–32. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  28. 28.
    Bogdanov, A.V., Degtyarev, A.B., Gankevich, I.G., Gayduchok, V.Y., Zolotarev, V.I.: Virtual workspace as a basis of supercomputer center. In: Proceedings of the 5th International Conference on Distributed Computing and Grid-Technologies in Science and Education, Dubna, Russia, pp. 60–66 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Ivan Gankevich
    • 1
  • Vladimir Korkhov
    • 1
  • Serob Balyan
    • 1
  • Vladimir Gaiduchok
    • 1
  • Dmitry Gushchanskiy
    • 1
  • Yuri Tipikin
    • 1
  • Alexander Degtyarev
    • 1
  • Alexander Bogdanov
    • 1
  1. 1.St. Petersburg State UniversitySt. PetersburgRussia

Personalised recommendations