Abstract
Despite the popularity and widespread usage of cloud computing, the cost of resources is one of the most important issues in Infrastructure as a Service (IaaS) clouds. Therefore, dynamic pricing models presented by some IaaS service providers, offers considerable price savings on spot instances or low-priority virtual machines. This significant discount has increased the popularity of using such resources among users. However, some of the most important Quality of Service (QoS) metrics such as reliability and availability are eliminated by obtaining this discount. For instance, the availability criterion is influenced by issues such as the user’s bid, supply, and demand rate of that specific instance, etc. In this paper, an extensive survey has been conducted on the issue of cloud preemptible instances, and the challenges in this context are studied. Furthermore, we point out the challenges that have not yet been investigated and define future directions in this research area.
Similar content being viewed by others
References
Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: A view of cloud computing. Commun.ACM 53(4), 50–58 (2010)
Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Futur. Gener. Comput. Syst. 25(6), 599–616 (2009). https://doi.org/10.1016/J.FUTURE.2008.12.001
“EC2 Spot Testimonials and Case Studies—Amazon Web Services.” https://aws.amazon.com/ec2/spot/testimonials/ (Accessed Apr. 16, 2019).
“National Instruments Case Study—Amazon Web Services (AWS).” https://aws.amazon.com/solutions/case-studies/national-instruments/ (Accessed Apr. 16, 2019).
Mishra, A.K., Umrao, B.K., Yadav, D.K.: A survey on optimal utilization of preemptible VM instances in cloud computing. J. Supercomput. 74(11), 5980–6032 (2018). https://doi.org/10.1007/s11227-018-2509-0
Kumar, D., Baranwal, G., Raza, Z., Vidyarthi, D.P.: A Survey on spot pricing in cloud computing. J. Netw. Syst. Manag. 26(4), 809–856 (2018). https://doi.org/10.1007/s10922-017-9444-x
Grance, P.M.T.: The NIST definition of cloud computing recommendations of the national institute of standards and technology. Recomm. Natl. Inst. Stand. Techol. 145, 7 (2011). https://doi.org/10.1136/emj.2010.096966
Chang, V : An introductory approach to risk visualization as a service. Open J. Cloud Comput. 1:1–9 (2014) [Online]. Available: www.ronpub.com/journals/ojcc%5Cnhttp://www.ronpub.com/publications/OJCC-v1i1n01_Chang.pdf.
Agarwal, S., Mishra, A.K., Yadav, D.K.: Forecasting price of amazon spot instances using neural networks. Int. J. Appl. Eng. Res. 12(20), 10276–10283 (2017)
“Amazon EC2 Instance Types—Amazon Web Services.” https://aws.amazon.com/ec2/instance-types/ (Accessed Mar. 13, 2019).
“Pricing - Windows Virtual Machines|Microsoft Azure.” https://azure.microsoft.com/en-us/pricing/details/virtual-machines/windows/ (Accessed Mar. 13, 2019).
Rodero-Merino, L., et al.: From infrastructure delivery to service management in clouds. Fut. Gen. Comput. Syst. 26(8), 1226–1240 (2010). https://doi.org/10.1016/j.future.2010.02.013
Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1(1), 7–18 (May 2010). https://doi.org/10.1007/s13174-010-0007-6
Deldari, A., Naghibzadeh, M., Abrishami, S.: CCA: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud. J. Supercomput. 73(2), 756–781 (2017)
Li, X., Salehi, M.A., Bayoumi, M., Tzeng, N.-F., Buyya, R.: Cost-efficient and robust on-demand video transcoding using heterogeneous cloud services. IEEE Trans. Parallel Distrib. Syst. 29(3), 556–571 (2018). https://doi.org/10.1109/TPDS.2017.2766069
Zhu, Z., Zhang, G., Li, M., Liu, X.: Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans. Parallel Distrib. Syst. 27(5), 1344–1357 (May 2016). https://doi.org/10.1109/TPDS.2015.2446459
Netto, M.A.S., Calheiros, R.N., Rodrigues, E.R., Cunha, R.L.F., Buyya, R.: HPC cloud for scientific and business applications: taxonomy, vision, and research challenges. ACM Comput. Surv. 51(1), 8 (2018). https://doi.org/10.1145/3150224
“Amazon EC2 Reserved Instances.” https://aws.amazon.com/ec2/pricing/reserved-instances/ (Accessed Apr. 15, 2019).
“Dedicated Hosts.” https://aws.amazon.com/ec2/dedicated-hosts/ (Accessed Apr. 15, 2019).
Zhang, Q., Zhu, Q., Boutaba, R.: Dynamic Resource Allocation for Spot Markets in Cloud Computing Environments. In 2011 Fourth IEEE International Conference on Utility and Cloud Computing, pp. 178–185 (2011). https://doi.org/10.1109/UCC.2011.33.
“Preemptible VM Instances|Compute Engine Documentation | Google Cloud.” https://cloud.google.com/compute/docs/instances/preemptible (Accessed Mar. 13, 2019).
“Preemptible VMs—Compute Instances|Google Cloud.” https://cloud.google.com/preemptible-vms/ (Accessed Apr. 15, 2019).
“Amazon EC2 Spot Two-Minute Warning is Now Available via Amazon CloudWatch Events.” https://aws.amazon.com/about-aws/whats-new/2018/01/amazon-ec2-spot-two-minute-warning-is-now-available-via-amazon-cloudwatch-events/ (Accessed Apr. 14, 2019).
Harlap, A., Tumanov, A., Chung, A., Ganger, G.R., Gibbons, P.B.: “Proteus: agile ML elasticity through tiered reliability in dynamic resource markets. ‘EuroSys’ 17 (2017). https://doi.org/10.1145/3064176.3064182
Huang, B., Jarrett, N.W.D., Babu, S., Mukherjee, S., Yang, J.: Cumulon: matrix-based data analytics in the cloud with spot instances. Proc. VLDB Endow. 9(3), 156–167 (2015). https://doi.org/10.14778/2850583.2850590
Sharma, P., Guo, T., He, X., Irwin, D., Shenoy, P.: Flint: batch-interactive data-intensive processing on transient servers”. EuroSys’ 16 (2016). https://doi.org/10.1145/29013182901319
Sharma, P., Lee, S., Guo, T., Irwin, D., Shenoy, P.: SpotCheck: designing a derivative IaaS cloud on the spot market. Proc Tenth Eur Conf Comput Syst EuroSys ‘15 (2015). https://doi.org/10.1145/2741948.2741953
Subramanya, S., Guo, T., Sharma, P., Irwin, D., Shenoy, P.: SpotOn: a batch computing service for the spot market. Proc sixth ACM Symp. Cloud Comput. SoCC’15 (2015). https://doi.org/10.1145/2806777.2806851
Yan, Y., Gao, Y., Chen, Y., Guo, Z., Chen, B., Moscibroda, T.: TR-Spark: Transient computing for big data analytics. Proc 7th ACM Symp Cloud Comput SoCC (2016). https://doi.org/10.1145/2987550.2987576
“New—EC2 Spot Blocks for Defined-Duration Workloads|AWS News Blog.” https://aws.amazon.com/blogs/aws/new-ec2-spot-blocks-for-defined-duration-workloads/ (Accessed May 12, 2019).
Fahringer, T., et al.: ASKALON: A grid application development and computing environment. Proc. IEEE/ACM Int. Work. Grid Comput 2005, 122–131 (2005). https://doi.org/10.1109/GRID.2005.1542733
Koo, R., Toueg, S.: Checkpointing and rollback-recovery for distributed systems. IEEE Trans. Softw. Eng SE-13(1), 23–31 (1987). https://doi.org/10.1109/TSE.1987.232562
Cao, G., Singhal, M.: On coordinated checkpointing in distributed systems. IEEE Trans. Parallel Distrib. Syst. 9(12), 1213–1225 (1998). https://doi.org/10.1109/71.737697
Bhargava, B., Lian, S.R.: Independent checkpointing and concurrent rollback for recovery in distributed system—an optimistic approach, pp. 3–12 (2003) https://doi.org/10.1109/reldis.1988.25775.
Jangjaimon, I., Tzeng, N.-F.: Adaptive incremental checkpointing via delta compression for networked multicore systems. 2013 IEEE 27th Int Symp Parall Distrib Proc (2013). https://doi.org/10.1109/IPDPS.2013.33
Di, S., Robert, Y., Vivien, F., Kondo, D., Wang, C.-L., Cappello, F.: Optimization of cloud task processing with checkpoint-restart mechanism. Proc Int Conf High Perform Comput Netw Storage Ana SC 13 (2013). https://doi.org/10.1145/2503210.2503217
Jung, D., Chin, S., Chung, K., Yu, H., Gil, J.: An efficient checkpointing scheme using price history of spot instances in cloud computing environment. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 6985 LNCS, pp. 185–200 (2011) https://doi.org/10.1007/978-3-642-24403-2_16.
Jangjaimon, I., Tzeng, N.F.: Effective cost reduction for elastic clouds under spot instance pricing through adaptive checkpointing. IEEE Trans. Comput. 64(2), 396–409 (2015). https://doi.org/10.1109/TC.2013.225
Zhou, J., Zhang, Y., Wong, W.-F.: Fault Tolerant Stencil Computation on Cloud-based GPU Spot Instances. IEEE Trans Cloud Comput. 7161(c), 1–1 (2017). https://doi.org/10.1109/tcc.2017.2710311
Poola, D., Ramamohanarao, K., Buyya, R.: Fault-tolerant workflow scheduling using spot instances on clouds. Procedia Comput. Sci. 29, 523–533 (2014). https://doi.org/10.1016/j.procs.2014.05.047
Song, K., Yao, Y., Golubchik, L.: Exploring the profit-reliability trade-off in Amazon’s spot instance market: A better pricing mechanism. IEEE Int Work Qual Serv IWQoS (2013). https://doi.org/10.1109/IWQoS.2013.6550272
Khatua, S., Mukherjee, N.: Application-centric resource provisioning for amazon EC2 spot instances, pp. 267–278. Springer, Berlin, Heidelberg (2013)
Khatua, S., Mukherjee, N.: “A novel checkpointing scheme for amazon EC2 spot instances. 2013 13th IEEE/ACM Int Symp Cluster Cloud Grid Comput. (2013). https://doi.org/10.1109/CCGrid.2013.71
Yi, S., Kondo, D., Andrzejak, A.: Reducing costs of spot instances via checkpointing in the amazon elastic compute cloud. 2010 IEEE 3rdInt Conf Cloud Comput (2010). https://doi.org/10.1109/CLOUD.2010.35
Gärtner, F.C.: Fundamentals of fault-tolerant distributed computing in asynchronous environments. ACM Comput. Surv. 31(1), 1–26 (2002). https://doi.org/10.1145/311531.311532
Poola, D., Ramamohanarao, K., Buyya, R.: Enhancing reliability of workflow execution using task replication and spot instances. ACM Trans. Auton. Adapt. Syst. 10(4), 1–21 (2016). https://doi.org/10.1145/2815624
Mosse, D., Melhem, R. and Ghosh, S.: Analysis of a fault-tolerant multiprocessor scheduling algorithm, pp. 16–25 (2002) https://doi.org/10.1109/ftcs.1994.315661.
Chervenak, A., et al.: Data placement for scientific applications in distributed environments. Proc IEEE/ACM Int Work Grid Comput (2007). https://doi.org/10.1109/GRID.2007.4354142
Qu, C., Calheiros, R.N., Buyya, R.: A reliable and cost-efficient auto-scaling system for web applications using heterogeneous spot instances. J. Netw. Comput. Appl. 65, 167–180 (2016). https://doi.org/10.1016/j.jnca.2016.03.001
Mazzucco, M, Dumas, M.: Achieving performance and availability guarantees with spot instances. In Proc- 2011 IEEE Int Conf HPCC 2011–2011 IEEE Int. Work. FTDCS 2011—Workshops 2011 Int. Conf. UIC 2011—Work. 2011 Int. Conf. ATC 2011, pp. 296–303 (2011) https://doi.org/10.1109/HPCC.2011.46.
Shastri, S., Irwin, D.: HotSpot: automated server hopping in cloud spot markets supreeth. Proc. 2017Symp Cloud Comput SoCC’ 17 (2017Symp). https://doi.org/10.1145/3127479.3132017
Dawoud, W., Takouna, I., Meinel, C.: Increasing spot instances reliability using dynamic scalability. Proc. - 2012 IEEE 5th Int. Conf. Cloud Comput. CLOUD 2012 (2012). https://doi.org/10.1109/CLOUD.2012.58
Yi, S., Andrzejak, A., Kondo, D.: Monetary cost-aware checkpointing and migration on amazon cloud spot instances. IEEE Trans. Serv. Comput. 5(4), 512–524 (2012). https://doi.org/10.1109/TSC.2011.44
Gong, Y., He, B., Zhou, A.C.: Monetary cost optimizations for MPI-based HPC applications on Amazon clouds. Proc Int Conf High Perform Comput Netw Storage Anal SC 17’. (2015). https://doi.org/10.1145/2807591.2807612
Voorsluys, W., Buyya, R.: Reliable provisioning of spot instances for compute-intensive applications. Proc Int. Conf. Adv. Inf. Netw. Appl. AINA (2012). https://doi.org/10.1109/AINA.2012.106
Jia, Q., Shen, Z., Song, W., van Renesse, R., Weatherspoon, H.: Smart spot instances for the supercloud. Proc Works Cross Cloud Infrastruct Platforms 1(212), 1–6 (2016). https://doi.org/10.1145/2904111.2904114
Jung, D., Chin, S.H., Chung, K.S., Yu, H.C.: VM migration for fault tolerance in spot instance based cloud computing. Notes Comput. Sci Lect (2013). https://doi.org/10.1007/978-3-642-38027-3_15. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics)
Weng, B., Lu, L., Wang, X., Megahed, F.M., Martinez, W.: Predicting short-term stock prices using ensemble methods and online data sources. Expert Syst. Appl. 112, 258–273 (2018). https://doi.org/10.1016/J.ESWA.2018.06.016
Patel, J., Shah, S., Thakkar, P., Kotecha, K.: Predicting stock market index using fusion of machine learning techniques. Expert Syst. Appl. 42(4), 2162–2172 (2015). https://doi.org/10.1016/J.ESWA.2014.10.031
Wang, F., et al.: Daily pattern prediction based classification modeling approach for day-ahead electricity price forecasting. Int. J. Electr. Power Energy Syst. 105, 529–540 (2019). https://doi.org/10.1016/J.IJEPES.2018.08.039
Contreras, J., Espinola, R., Nogales, F.J., Conejo, A.J.: ARIMA models to predict next-day electricity prices. IEEE Power Eng. Rev. 22(9), 57–57 (2002). https://doi.org/10.1109/MPER.2002.4312577
Tan, C.W., Chiang, M., Wang, X., Zheng, L., Joe-Wong, C.: How to Bid the Cloud. ACM SIGCOMM Comput. Commun. Rev. 45(5), 71–84 (2015). https://doi.org/10.1145/2829988.2787473
Salehan, A., Deldari, H., Abrishami, S.: An online valuation-based sealed winner-bid auction game for resource allocation and pricing in clouds. J. Supercomput. 73(11), 4868–4905 (2017). https://doi.org/10.1007/s11227-017-2059-x
Tang, S., Yuan, J., Wang, C., Li, X.-Y.: A Framework for Amazon EC2 Bidding Strategy under SLA Constraints. IEEE Trans. Parallel Distrib. Syst. 25(1), 2–11 (2013). https://doi.org/10.1109/tpds.2013.15
“AWS Spot Pricing Market|Kaggle.” https://www.kaggle.com/noqcks/aws-spot-pricing-market (Accessed Apr. 07, 2019).
“Spot Instance Pricing History—Amazon Elastic Compute Cloud.” https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances-history.html (Accessed Apr. 07, 2019).
Ben-Yehuda, O.A., Ben-Yehuda, M., Schuster, A., Tsafrir, D.: Deconstructing Amazon EC2 spot instance pricing. Proc.2011 3rd IEEE Int. Conf. Cloud Comput. Technol. Sci. CloudCom 2011 1(3), 304–311 (2011). https://doi.org/10.1109/CloudCom.2011.48
Xu, H., Li, B.: Dynamic cloud pricing for revenue maximization. IEEE Trans. Cloud Comput. 1(2), 158–171 (2013). https://doi.org/10.1109/TCC.2013.15
Chen, J., Wang, C., Zhou, B.B., Sun, L., Lee, Y.C., Zomaya, A.Y.: Tradeoffs between profit and customer satisfaction for service provisioning in the cloud. Proc 20th Int Symp High Perf Distrib Comput HPDC ’11 (2011). https://doi.org/10.1145/1996130.1996161
Kushwaha, V., Simmhan, Y.: Cloudy with a spot of opportunity: analysis of spot-priced vms for practical job scheduling. IEEE Int Conf Cloud Comput Emerg Mark (CCEM) 2014, 1–8 (2014). https://doi.org/10.1109/CCEM.2014.7015488
Javadi, B., Thulasiram, R.K., Buyya, R.: Characterizing spot price dynamics in public cloud environments. Futur. Gener. Comput. Syst. 29(4), 988–999 (2013). https://doi.org/10.1016/j.future.2012.06.012
Wee, S.: Debunking real-time pricing in cloud computing. Proc. 11th IEEE/ACM Int. Symp. Clust. Cloud Grid Comput. CCGrid 2011 (2011). https://doi.org/10.1109/CCGrid.2011.38
Liu, D., Cai, Z., Li, X.: Hidden markov model based spot price prediction for cloud computing. Proc. 15th IEEE Int. Symp. Parallel Distrib. Process. with Appl. 16th IEEE Int. Conf. Ubiquitous Comput. Commun. ISPA/IUCC 2017 (2018). https://doi.org/10.1109/ISPA/IUCC.2017.00152
Cai, Z., Li, X., Ruiz, R., Li, Q.: Price forecasting for spot instances in Cloud computing. Futur. Gener. Comput. Syst. 79, 38–53 (2018). https://doi.org/10.1016/j.future.2017.09.038
Song, Y., Zafer, M., Lee, K.W.: Optimal bidding in spot instance market. Proc IEEE INFOCOM (2012). https://doi.org/10.1109/INFCOM.2012.6195567
Wang, P., Qi, Y., Hui, D., Rao, L., Liu, X.: Present or future: optimal pricing for spot instances. 2013 IEEE 33rd Int Conf Distrib Comput Syst (2013). https://doi.org/10.1109/ICDCS.2013.68
Zhao, H., Pan, M., Liu, X., Li, X., Fang, Y.: Optimal resource rental planning for elastic applications in cloud market. 2012 IEEE 26th Int Parall Distrib Process Symp (2012). https://doi.org/10.1109/IPDPS.2012.77
Baughman, M., Chard, K., Foster, I., Wolski, R., Haas, C.: Predicting amazon spot prices with LSTM networks. Prov 9th Workshop Sci Cloud Comput (2018). https://doi.org/10.1145/3217880.3217881
Sarah, A., Lee, K., Kim H.: LSTM model to forecast time series for EC2 cloud price, In 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), 2018, pp. 1085–1088, 10.1109/DASC/PiCom/DataCom/CyberSciTec. 2018.00067
Al-Theiabat, H., Al-Ayyoub, M., Alsmirat, M., Aldwair, M.: A deep learning approach for amazon EC2 spot price prediction. 2018 IEEE/ACS 15th Int Conf Comput Syst Appl (AICCSA) (2018). https://doi.org/10.1109/AICCSA.2018.8612783
Domanal, S.G., Reddy, G.R.M.: An efficient cost optimized scheduling for spot instances in heterogeneous cloud environment. Futur. Gener. Comput. Syst. 84, 11–21 (2018). https://doi.org/10.1016/j.future.2018.02.003
Turchenko, V., et al.: Spot price prediction for cloud computing using neural networks. Int. J. Comput. 12(4), 348–359 (2014)
Khandelwal, V., Chaturvedi, A., Gupta, C.P.: Amazon EC2 Spot price prediction using regression random forests. IEEE Trans Cloud Comput (2017). https://doi.org/10.1109/TCC.2017.2780159
Arevalos, S., Lopez-Pires, F., Baran, B.: A comparative evaluation of algorithms for auction-based cloud pricing. Predict IEEE Int Conf Cloud Eng (IC2E) (2016). https://doi.org/10.1109/IC2E.2016.45
Wallace, R.M., et al.: Applications of neural-based spot market prediction for cloud computing. 2013 IEEE 7th Int Conf Intell Data Acquisit Adv Comput Syst (IDAACS) (2013). https://doi.org/10.1109/IDAACS.2013.6663017
Singh, V.K., Dutta, K.: Dynamic price prediction for amazon spot instances. 2015 48th Hawaii Int Conf Syst Sci (2015). https://doi.org/10.1109/HICSS.2015.184
Fabra, J., Ezpeleta, J., Álvarez, P.: Reducing the price of resource provisioning using EC2 spot instances with prediction models. Futur. Gener. Comput. Syst. 96, 348–367 (2019). https://doi.org/10.1016/j.future.2019.01.025
Amekraz, Z., Youssef, M.: Prediction of amazon spot price based on chaos theory using ANFIS model. 2016 IEEE/ACS 13th Int Conf Comput Syst Appl (AICCSA) (2016). https://doi.org/10.1109/AICCSA.2016.7945632
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Deldari, A., Salehan, A. A survey on preemptible IaaS cloud instances: challenges, issues, opportunities, and advantages. Iran J Comput Sci 4, 1–24 (2021). https://doi.org/10.1007/s42044-020-00071-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42044-020-00071-1