Abstract
Many scientific applications are described through workflow structures. Due to the increasing level of parallelism offered by modern computing infrastructures, workflow applications now have to be composed not only of sequential programs, but also of parallel ones. Cloud platforms bring on-demand resource provisioning and pay-as-you-go billing model. Then the execution of a workflow corresponds to a certain budget. The current work addresses the problem of resource allocation for non-deterministic workflows under budget constraints. We present a way of transforming the initial problem into sub-problems that have been studied before. We propose two new allocation algorithms that are capable of determining resource allocations under budget constraints and we present ways of using them to address the problem at hand.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
van der Aalst, W., Barros, A., ter Hofstede, A., Kiepuszewski, B.: Advanced Workflow Patterns. In: Proc. of the 7th Intl. Conference on Cooperative Information Systems, pp. 18–29 (2000)
Altintas, I., Bhagwanani, S., Buttler, D., Chandra, S., Cheng, Z., Coleman, M., Critchlow, T., Gupta, A., Han, W., Liu, L., Ludäscher, B., Pu, C., Moore, R., Shoshani, A., Vouk, M.A.: A modeling and execution environment for distributed scientific workflows. In: Proc. of the 15th Intl. Conference on Scientific and Statistical Database Management, pp. 247–250 (2003)
Amazon Elastic Compute Cloud, Amazon EC2 (2012), http://aws.amazon.com/ec2/instance-types/
Bahsi, E.M., Ceyhan, E., Kosar, T.: Conditional Workflow Management: A Survey and Analysis. Scientific Programming 15(4), 283–297 (2007)
Caron, E., Desprez, F., Muresan, A., Suter, F.: Budget Constrained Resource Allocation for Non-Deterministic Workflows on a IaaS Cloud. Research report (May 2012), http://hal.inria.fr/hal-00697032
Couvares, P., Kosar, T., Roy, A., Weber, J., Wenger, K.: Workflow Management in Condor. In: Taylor, I., Deelman, E., Gannon, D., Shields, M. (eds.) Workflows for e-Science, pp. 357–375. Springer (2007)
Deelman, E., Singh, G., Su, M.H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J., Katz, D.: Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Scientific Programming Journal 13(3), 219–237 (2005)
Desprez, F., Suter, F.: A Bi-Criteria Algorithm for Scheduling Parallel Task Graphs on Clusters. In: Proc. of the 10th IEEE/ACM Intl. Symposium on Cluster, Cloud and Grid Computing, pp. 243–252 (2010)
Fahringer, T., Jugravu, A., Pllana, S., Prodan, R., Seragiotto Jr., C., Truong, H.L.: ASKALON: a Tool Set for Cluster and Grid Computing. Concurrency - Practice and Experience 17(2-4), 143–169 (2005)
Fernandez, H., Tedeschi, C., Priol, T.: A Chemistry Inspired Workflow Management System for Scientific Applications in Clouds. In: Proc. of the 7th Intl. Conference on E-Science, pp. 39–46 (2011)
Glatard, T., Montagnat, J., Lingrand, D., Pennec, X.: Flexible and Efficient Workflow Deployment of Data-Intensive Applications on GRIDS with MOTEUR. Intl. Journal of High Performance Computing Applications 3(22), 347–360 (2008); Special Issue on Workflow Systems in Grid Environments
Ludäscher, B., Altintas, I., Berkley, C., Higgins, D., Jaeger, E., Jones, M., Lee, E.A., Tao, J., Zhao, Y.: Scientific Workflow Management and the Kepler System. Concurrency and Computation: Practice and Experience 18(10), 1039–1065 (2006)
Mao, M., Humphrey, M.: Auto-Scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows. In: Proc. of the Conference on High Performance Computing Networking, Storage and Analysis (SC 2011) (2011)
Mayer, A., McGough, S., Furmento, N., Lee, W., Newhouse, S., Darlington, J.: ICENI Dataflow and Workflow: Composition and Scheduling in Space and Time. In: UK e-Science All Hands Meeting, pp. 627–634. IOP Publishing Ltd. (2003)
Oinn, T., Greenwood, M., Addis, M., Alpdemir, N., Ferris, J., Glover, K., Goble, C., Goderis, A., Hull, D., Marvin, D., Li, P., Lord, P., Pocock, M., Senger, M., Stevens, R., Wipat, A., Wroe, C.: Taverna: Lessons in Creating a Workflow Environment for the Life Sciences. Concurrency and Computation: Practice and Experience 18(10), 1067–1100 (2006)
Quinson, M.: Simgrid: a generic framework for large-scale distributed experiments. In: Peer-to-Peer Computing, pp. 95–96 (2009)
Radulescu, A., van Gemund, A.: A Low-Cost Approach towards Mixed Task and Data Parallel Scheduling. In: Proc. of the 15th Intl. Conference on Parallel Processing (ICPP), pp. 69–76 (2001)
Taylor, I., Shields, M., Wang, I., Harrison, A.: The Triana Workflow Environment: Architecture and Applications. In: Workflows for e-Science, pp. 320–339 (2007)
Viroli, M., Zambonelli, F.: A Biochemical Approach to Adaptive Service Ecosystems. Information Sciences 180(10), 1876–1892 (2010)
Werner, T.: Target Gene Identification from Expression Array Data by Promoter Analysis. Biomolecular Engineering 17(3), 87–94 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Caron, E., Desprez, F., Muresan, A., Suter, F. (2012). Budget Constrained Resource Allocation for Non-deterministic Workflows on an IaaS Cloud. In: Xiang, Y., Stojmenovic, I., Apduhan, B.O., Wang, G., Nakano, K., Zomaya, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2012. Lecture Notes in Computer Science, vol 7439. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33078-0_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-33078-0_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33077-3
Online ISBN: 978-3-642-33078-0
eBook Packages: Computer ScienceComputer Science (R0)