Abstract
Cloud infrastructures offer a wide variety of resources to choose from. However, most cloud users ignore the potential benefits of dynamically choosing cloud resources among a wide variety of VM instance types with different configuration/cost tradeoffs. We propose to automate the choice of resources that should be assigned to arbitrary non-interactive applications. During the first executions of the application, the system tries various resource configurations and builds a custom performance model for this application. Thereafter, cloud users can specify their execution time or financial cost constraints, and let the system automatically select the resources which best satisfy this constraint.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Tang, W., Desai, N., Buettner, D., Lan, Z.: Job scheduling with adjusted runtime estimates on production supercomputers. Elsevier Journal of Parallel and Distributed Computing (2013)
Amazon Elastic MapReduce, http://aws.amazon.com/elasticmapreduce/
Azure Batch, http://azure.microsoft.com/en-us/services/batch/
Allan, R.: Survey of HPC performance modelling and prediction tools. Technical Report DL-TR-2010-006, Science and Technology Facilities Council (July 2009)
Pllana, S., Brandic, I., Benkner, S.: A survey of the state of the art in performance modeling and prediction of parallel and distributed computing systems. IJCIR 4(1) (2008)
Beach, T.H., Rana, O.F., Avis, N.J.: Integrating acceleration devices using CometCloud. In: Proc. ORMaCloud Workshop (June 2013)
Vasic, N., Novakovic, D., Miucin, S., Kostic, D., Bianchini, R.: DejaVu: Accelerating resource allocation in virtualized environments. In: Proc. ACM ASPLOS (March 2012)
Fernandez, H., Pierre, G., Kielmann, T.: Autoscaling Web applications in heterogeneous cloud infrastructures. In: Proc. IEEE IC2E (March 2014)
Dejun, J., Pierre, G., Chi, C.H.: EC2 performance analysis for resource provisioning of service-oriented applications. In: NFPSLAM-SOC (November 2009)
Dejun, J., Pierre, G., Chi, C.H.: Resource provisioning of Web applications in heterogeneous clouds. In: Proc. USENIX WebApps (June 2011)
Farley, B., Juels, A., Varadarajan, V., Ristenpart, T., Bowers, K.D., Swift, M.M.: More for your money: exploiting performance heterogeneity in public clouds. In: SOCC (2012)
Oprescu, A.M., Kielmann, T., Leahu, H.: Budget estimation and control for bag-of-tasks scheduling in clouds. Parallel Processing Letters 21(2) (June 2011)
Verma, A., Cherkasova, L., Campbell, R.H.: ARIA: automatic resource inference and allocation for mapred uce environments. In: Proc. ICAC (2011)
Tian, F., Chen, K.: Towards optimal resource provisioning for running MapReduce programs in public clouds. In: Proc. IEEE CLOUD (2011)
Amazon Web Services, http://aws.amazon.com/blogs/aws/choosing-the-right-ec2-instance-type-for-your-application/
CopperEgg: AWS sizing tool, http://copperegg.com/aws-sizing-tool/
Wikipedia.org: Simulated annealing
CGC: Reverse time migration, http://www.cgg.com/default.aspx?cid=2358
Sikka, V., Färber, F., Lehner, W., Cha, S.K., Peh, T., Bornhövd, C.: Efficient transaction processing in SAP HANA database – the end of a column store myth. In: SIGMOD (2012)
Grid’5000, http://www.grid5000.fr/
SciPy, http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.anneal.html#scipy.optimize.anneal
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 IFIP International Federation for Information Processing
About this paper
Cite this paper
Iordache, A., Buyukkaya, E., Pierre, G. (2015). Heterogeneous Resource Selection for Arbitrary HPC Applications in the Cloud. In: Bessani, A., Bouchenak, S. (eds) Distributed Applications and Interoperable Systems. DAIS 2015. Lecture Notes in Computer Science(), vol 9038. Springer, Cham. https://doi.org/10.1007/978-3-319-19129-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-19129-4_9
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19128-7
Online ISBN: 978-3-319-19129-4
eBook Packages: Computer ScienceComputer Science (R0)