A time-sensitive learning-to-rank approach for cloud simulation resource prediction

Predicting the computing resources required by simulation applications can provide a more reasonable resource-allocation scheme for efficient execution. Existing prediction methods based on machine learning, such as classification/regression, typically must accurately predict the runtime of simulation applications and select the optimal computing resource allocation scheme by sorting the length of the simulation runtime. However, the ranking results are easily affected by the simulation runtime prediction accuracy. This study proposes a time-sensitive learning-to-rank (LTR) approach for cloud simulations resource prediction. First, we use the Shapley additive explanation (SHAP) value from the field of explainable artificial intelligence (XAI) to analyze the impact of relevant factors on the simulation runtime and to extract the feature dimensions that significantly affect the simulation runtime. Second, by modifying the target loss function of the rankboost algorithm and training a time-sensitive LTR model based on simulation features, we can accurately predict the computing resource allocation scheme that maximizes the execution efficiency of simulation applications. Compared with the traditional machine learning prediction algorithm, the proposed method can improve the average sorting performance by 3%–48% and can accurately predict the computing resources required for the simulation applications to execute in the shortest amount of time.

system simulation applications [1]. Parallel discrete event simulations (PDES) are often used to improve the efficiency of complex system simulation applications. It is typically necessary to specify the amount of computing resources (referred to in this study as the number of CPU cores) required for the complex system simulation applications based on PDES before they are executed and divide the simulation task into different CPU cores for parallel computation [2]. Cloud computing technology can realize collaborative management and on-demand distribution of computing/storage and other resources to provide efficient resource allocation tools for complex system simulation applications based on parallel and discrete simulation (PADS) [3]. However, a high communication delay occurs in the cloud environment because computing and storage resources are often distributed among different computing nodes [4]. During the execution of the simulation task, communication is frequently synchronized, and numerous messages are sent and received. The execution efficiency of the simulation application in the cloud environment will be affected if there is an excessive allocation of computing resources, which will increase communication overhead between the simulation entities [5]. Therefore, a method is required to predict the priority order of the simulation runtime corresponding to the number of different simulation computing resources before the simulation application is executed and to reasonably allocate computing resources for the simulation application according to the priority order.
Several researchers have recently investigated classification/regression machine learning-based simulation resource prediction methods in cloud environments [6,7]. These methods collect the characteristic data of simulation applications and use classification/regression algorithms to predict the runtime of simulation applications using various computing resources. Finally, these methods rank simulation computing resources based on the prediction results (simulation runtime) and expect to obtain those that will enable the simulation applications to execute in the shortest time. However, these methods typically base their prediction models on highprecision prediction indicators, such as root mean square error (RMSE) [8] or mean absolute percentage error (MAPE) [9]. The optimization objective is to minimize the difference between the predicted and actual runtimes, which makes the simulation's ordination susceptible to prediction accuracy. Compared with the prediction method that directly optimizes the overall ordination accuracy of computing resources, a method based on the prediction before ordination requires a higher time and space computational complexity and makes it difficult to achieve higher sorting accuracy. The learningto-rank (LTR) method uses the relative ranking between pairwise samples in the dataset as feedback information and reduces the error ranking probability between sample pairs through iterative learning to produce the best ranking possible [10]. Therefore, during the task of simulation computing resource prediction, the method that directly learns the ranking of resource usage can explain the relationship between different resource usage and simulation application runtime at a deeper level than the method that predicts the simulation runtime under different computing resources and re-ranking.
To address the above problems, this study proposes a time-sensitive LTR approach for resource prediction in cloud simulation. The method collects static feature and dynamic monitoring data from simulation applications, and analyzes the impact of various factors on the execution efficiency of the simulation based on the feature extraction method of SHapley additive exPlanation (SHAP) interpretation framework. Second, based on the analysis of the factors that significantly affect the runtime of simulation applications, the objective loss function of the proposed method was designed, and the RankBoost model was used to construct the simulation runtime-sensitive predictor. The method uses the shortest simulation runtime as the training goal and adds a time penalty factor to the loss function of the LTR model to ensure that more penalty is added when the best resources are not correctly ranked to accurately predict the computing resources with the shortest simulation runtime.
The main contributions of this study are as follows: 1. This study used the feature extraction method of the SHAP interpretability framework to quantitatively evaluate the importance of factors that influence the runtime of a complex system simulation. First, the set of factors that affect the efficiency of the simulation was analyzed, and the static characteristics before the execution of the simulation application and the dynamic characteristics during the dynamic execution of the simulation application were extracted as the feature data of the training samples. Second, the significant contribution of each feature was calculated based on the SHAP value, and the features that significantly affect the simulation application's runtime are extracted as input to improve the performance of the predictor. 2. This study proposes a time-sensitive learning-to-rank approach for resource prediction in cloud simulation. The method designs the target loss function of the RankBoost algorithm and trains a time-sensitive predictor based on the simulation features to accurately predict the computing resource allocation scheme that maximizes the execution efficiency of simulation applications. Compared with other eight machine learning methods, the experimental results show that our method can achieve the shortest simulation runtime and improve the accuracy of the optimal computing resource ranking.

Related work
Several studies have been conducted on the acceleration technology of complex system simulation applications combined with the advantages of on-demand sharing and elastic scaling in cloud computing environments. However, although cloud computing-based simulation technology has several advantages, the random resource allocation process in the cloud environment may also reduce simulation performance [11]. To address this problem, regression/classification prediction models based on machine learning, such as linear regression [12,13], nearest neighbor [14], regression tree [15], support vector machine [16], Bayesian [17], neural networks [18,19], and ensemble learning models [20,21], are the widely accepted solutions to this problem. These methods attempt to accurately predict the dynamic properties (such as computing resource requirements, runtime, or load fluctuations) of simulation applications using deep learning in many historical feature datasets. Bin [22] modeled computational resource planning in cloud environments as a classification problem using a weighted support vector machine (SVM) model to predict future workload changes, thus minimizing the cost of cloud resource allocation and virtual machine provisioning. To overcome the limitations of reactive cloud autoscaling, Kee proposed an online framework for cloud computing resource prediction using multiple predictors to create an ensemble model for accurate prediction of actual workloads [6]. To improve the responsiveness of cloud management systems and improve the quality of cloud services, Nawrocki [23] combined anomaly detection and machine learning methods to allocate cloud resources to reduce cost and improve the quality of service (QoS). To cope with unexpected events in the cloud environment, Shen [24] proposed an efficient method for predictive resource allocation in the cloud. The approach uses machine learning models to predict contingencies and normal events and execute different resource allocation schemes. To improve the allocation efficiency of virtual machines in heterogeneous cloud environments, Mahfoudh [25] used a diffusion convolutional recurrent neural network model to predict the fluctuation of computational load in the next time period in real time and validated the model on a real-world dataset of CPU usage traces from Plan-etLab. For time-series data in cloud computing environments, Jing [26] integrated bidirectional and lattice-based long-and short-term memory network models to achieve high-quality prediction of workload and resource usage in cloud environments.
The above machine learning methods typically aim to build a prediction model while improving the absolute accuracy (that is, reducing the RMSE, MAE, and other indicators). To obtain an effective resource ranking, these methods must predict the simulation runtime under various resource conditions and then rank the various resources according to the prediction results. However, compared with learning-to-rank method that directly optimize the overall ranking, classification/regression models that indirectly rank based on absolute accuracy tend to obtain poor ranking results [27].
Compared with classification/regression-based prediction methods, research on ranking computing resources for simulation applications using the LTR method is still very limited. Currently, most mainstream LTR algorithms adopt the pairwise approach, which considers a sample pair as a single training sample and focuses on the relative order of two samples rather than the specific value of a single sample. Some common methods include RankSVM [28], RankBoost [29], RankNet [?], and others [30,31]. Nguyen [32] empirically demonstrated using five datasets that the ranking accuracies of two LTR methods (RankSVM and RankBoost) were 4%-21% higher than that of the liner regression model (LR). Yang [33] proposed an LTR method to develop a software defect prediction model by directly optimizing the ranking performance. The method's accuracy in ranking the metrics was demonstrated. Jacky [27] analyzed the cost and data imbalance problems, and proposed a defect prediction method based ranking SVM model to improve the accuracy of software defect prediction. Burges et al. [34] proposed an ensemble prediction algorithm that combines the Lamb-daMART and LambdaRank algorithms and demonstrated through experimental datasets that the algorithm produced better prediction results.
The LTR algorithm of AdaBoost selects a feature variable each time to learn the weak sorters, which are then combined to form a strong sorter. The RankBoost algorithm inherits the advantages of the AdaBoost algorithm and has a fast convergence speed and strong generalization ability [35]. Therefore, this study combines the advantages of the Rank-Boost algorithm and proposes a time-sensitive LTR approach for resource prediction in cloud simulation. The approach selects the number of features that significantly affect the simulation efficiency based on the SHAP interpretability framework and then designs the target loss function of the RankBoost algorithm to train a time-sensitive LTR model. Compared with other LTR models, the proposed model adds more penalties when the ranking of the best resources is incorrect.

The time-sensitive learning-to-rank approach for resource prediction
Cloud computing technology could greatly improve the efficiency of resource allocation for complex system simulation applications [36]. Compared with the traditional method of using classification/regression technology to indirectly predict the use of computing resources for simulation applications, the LTR method can better explain the relationship between different resource usages and the execution efficiency. Therefore, this study proposes a timesensitive learning-to-rank approach (TSLTR) to support optimal resource allocation for simulation applications. The fundamental concept is to use an LTR algorithm to predict the priority of the computing resources used before a simulation application is executed.
This method aims to reduce the number of resource pairs with incorrect rankings, obtain the highest ranking of resource usage, and improve the execution efficiency of simulation applications. In practice, the PADS application focuses more on resource usage, which speeds up the simulation application's execution. Therefore, this study adds a penalty factor to the loss function of the model to improve the ranking accuracy of top-ranked resources. The architecture of the proposed approach is shown in Fig. 1 During the initial phase of the approach, we deployed resource monitors on cloud computing nodes to collect realtime operational information of the emulated applications. In this study, we considered two sets of simulation parameters: pre-execution and runtime parameters. The pre-execution parameters, which reflect the static characteristics of the simulation application, are set before the application is executed. The runtime parameters are collected while the simulation application is being executed and reflect the performance differences under different cloud computing resources. The resource monitor collects information every 3 s and deposits it into the cloud application database with the specific parameters shown in Table 1. During the second stage, the collected dataset was processed using feature extraction methods to obtain relevant features. First, data preprocessing and data cleaning methods were performed to remove the null values and negative values. Then, standardization was used to normalize all data to a distribution with a mean of 0 and a variance of 1 so that eliminating the negative effects of abnormal samples. Second, based on the SHAP explainable framework, we analyzed the impact of each feature dimension on the operational efficiency of the simulation application and selected strongly relevant features to reduce the model error and achieve a more accurate prediction result. Finally, the extracted relevant features were used as inputs to train the LTR model to predict the priority order of various cloud resource usage. The highest ranked resource was used as the final prediction result.

The LTR model
The LTR algorithm aims to use the data from the previous execution to predict the priority of the computing resource usage for the next execution. This method uses the relative ranking between sample pairs in the training sample set as feedback information and reduces the probability of incorrect ranking between sample pairs by iteratively learning to produce the best ranking for all samples. Therefore, this study uses the LTR algorithm to predict the overall ranking of computing resources as well as to provide an optimal resource allocation scheme for achieving the shortest simulation runtime.
(1) Problem description Consider the ranking objective named Set X, which ranks a set of applications S i with the same simulation static parameters (except for the CPU cores), where X represents the CPU cores assigned to the simulation execution. The optimal resource requirements for cloud-based simulation applications can be expressed as represents the d-dimensional feature vector of the simulation application, y i represents the simulation runtime, and f denotes the ranking function. The LTR model aims to learn from the dataset to obtain a sorting function that correctly sorts the number of resources to be used. Therefore, the feedback function is defined as follows: For any pair of samples S j and S k in any set of applications, S j x j , y j > S k (x k , y k ) indicates that the simulation runtime when using P j resources is higher than the simulation runtime when using P k resources. S j x j , y j < S k (x k , y k ) implies that the simulation runtime when using P j resources is lower than that when using P k resources. S j x j , y j = S k (x k , y k ) indicates that the simulation runtime when using P j resources is equal to that when using P k resources. The above LTR task was converted to a binary classification task that involves learning the relative order between sample pairs. Based on this, a weight D is assigned to each sample pair, indicating the importance of accurately judging the sample pairs. The distribution of D is defined as follows: where c represents a positive constant and satisfies the following: Therefore, the LTR algorithm aims to use positive feedback sample pairs with weights D to learn a final ranking function H . The probability of a ranking error between the best ranking and the predicted ranking in the learning process is called the ranking loss (Rloss), and its utility function Func and loss function Rloss are defined as follows: where r denotes the number of iterations, α r denotes the weight of the weak classifier obtained in the current iteration, and h r denotes a function with 0-1 values. α r > 0 indicates that the ranking effect of h r is positive, that is, the correct rate is more than half. Therefore, by selecting the appropriate α r and h r in each weak learning process, we can reduce the Rloss of the final ranking result. In this study, we set the α r weight value by referring to [24,30], defined as follows: (2) Algorithm description The specific execution steps are shown in Algorithm 1. Find weak learner h r that maximizes r (h) in Eq.(4) 5:

Algorithm 1 A resource prediction algorithm based on LTR
Choose Choose Rloss to make S j , S k D r S j x j , y j , S k (x k , y k ) = 1 7: Update D r +1 = D r S j x j , y j , S k (x k , y k ) × ex p(α r (h r (S j x j , y j − h r (S k (x k , y k ))/Rloss 8: Calculate H (x) = R r =1 α r hr 9: return H (x) 10: end for In this algorithm, S(X , Y ) denotes the input simulation application features and label data and D denotes the initialized sample pair weight distribution. The Final Sort Values H (x) are calculated by four steps.
(1) Initialize D 1 = D and ensure that the initial D value is the same for each sample pair by setting the value of c in Eq. (2). (2) During the iterative process, the maximized utility function r (h) is used as the learning objective, weak learning iterative training with D r is performed to generate multiple weak learners h r , and the α r value is obtained by calculation. Finally, the ranking result is calculated as where R denotes the number of iterations and H(x) denotes the ranking score, the higher score indicates the better ranking.
(3) The D distribution value is updated according to the line 7, where Rloss denotes the minimum objective function value in the current iteration calculated using Eq. (5). (4) Steps 2 and Steps 3 are repeated until the combined ranking order obtained in two to three consecutive iterations does not change, which is then considered the final predicted ranking, and the optimal order of resource usage is the output.

The time-sensitive LTR method for resource prediction
The simulation application resource prediction problem requires a problem to be considered to modify the LTR algorithm. Assume that the optimal resource ranking required for one simulation run is the correct ranking = {4, 2, 3, 1} cores. There are two predicted ranking results for Ranking a = {2, 4, 3, 1} cores and Ranking b = {4, 2, 1, 3} cores. Rankings a and b are incorrectly ranked. However, in simulation practice, one is more concerned with the number of resources that are used to complete the simulation in the shortest time, that is, the number of resources ranked first. Therefore, the time cost of Ranking a is significantly higher than that of Ranking b, where the shortest simulation run time is guaranteed in accordance with the ranking outcome. In other words, ranking a needs to be allotted a larger penalty. However, the RankBoost LTR method does not consider this and assumes that the time cost of Rankings a and b is equal. Therefore, a penalty factor ε(x j,k ) is added to ensure that more penalties are added when the best resource is ranked incorrectly. Therefore, the loss function in Eq. (5) can be converted into the following equation: The definition of the penalty factor is the main issue with time-sensitive LTR. In this study, we used heuristicstocalculate the value of ε(x j,k ), which is executed in Algorithm 2.
First, the algorithm establishes a correct ranking based on all computational resource requirements in the group of simulation applications and uses mean reciprocal rank (Eq. 8) as a criterion for ranking performance and for calculating the optimal score M R R best .
Second, for each sample pair Q = S j (x j , y j ), S k (x k , y k ) in the simulation application group, the positions of their module pairs are exchanged and the new ranking scores M R R j,k are calculated.

Algorithm 2 The calculation of penalty factor
Require: Dataset S(X , Y ) Ensure: penalty factor ε(x j,k ) 1: Obtain a correct ranking for all sample pairs in S(X , Y ) 2: Calculate the M R R best value of the correct ranking 3: Iteration = 0 4: Dec = 0 5: for each pair Q = S j (x j , y j ), S k (x k , y k ) do 6: Exchange S j x j , y j and S k (x k , y k ) 7: Calculate the MRR value of the new ranking M R R j,k 8: Dec = Dec + M R R best /M R R j,k 9: Iteration++ 10: end for 11: ε(x j,k ) = Dec/I teration 12: return ε(x j,k ) Finally, the standard decline ratio of the MRR values of all sample pairs after the swap relative to the correctly ranked MRR values was calculated and used as the value of the penalty factor ε(x j,k ).

Evaluation metrics
To test the accuracy of the ranked learning prediction method, two metrics were considered: mean reciprocal rank (MRR) and normalized discounted cumulative gain (NDCG). The indicator metrics are formally defined as follows: (1) M R R is concerned with the position of the first relevant element in the sorting result. If the first relevant element appears in the first position, the MRR will be larger. The MRR is calculated as follows: where K denotes the number of samples and rank i denotes the position where the best resource appears in the predicted computed resource ranking list for the ith sample. (2) By introducing a location influence factor, N DCG considers that more advanced samples in the ranking are more valuable and have a significant impact on the assessment results. The N DCG@n is calculated as follows: where Z n denotes the generalization factor such that the range of NDCG values is limited to

Application and experimental settings
Phold is a benchmark program used to evaluate the performance of simulation applications [37]. The Phold includes many simulation entities, and each entity will initialize multiple simulation events. When running the Phold, the simulation entities randomly executes three types of events and sends messages to another objects. Public Opinion Dissemination Model (PODM) is a typical complex system simulation model, that studies the impact on public opinion dissemination when hot events occur. The PODM consist of four types simulation entities: individuals, hot events, cities, and medias.
In this study, we repeat running simulation application with different parameters and collect feature data every 3 s to train multiple predictors. As shown in Table 2, the preexecution parameters of the simulation application were specified to run in a real cloud environment. To ensure the effectiveness of the algorithm, we repeated each experiments 10 times, and calculated the average value as the experimental result. Then, we built a cloud environment to execute complex simulation application and collect the feature data. The proposed approach was tested on Intel Xeon Gold 6338 CPU with 32 cores and 64 GB of RAM. In this study, the simulation application was built by Docker, an open container engine, and the simulation tasks were offloaded to no more than 32 containers (each container was assigned 1 core and 2 GB RAM) for parallel execution.

Feature extraction method based on the SHAP interpretable framework
The static features of the simulation application and the dynamic features at runtime are among the factors that affect the operational efficiency of complex system simulation applications in the cloud computing environment. Most current feature importance calculation methods can only indicate which features are important without indicating how these features affect the prediction results. Inspired by cooperative game theory, SHAP develops a method to explain individual predictions based on the game theory [38]. Therefore, to extract the significant factors affecting the operational efficiency of simulation applications, this study analyzed the degree of influence of each factor on the opera- tional efficiency of simulation applications based on SHAP's interpretable feature extraction method. Figure 2 shows an overall feature summary analysis based on SHAP interpretability model. Each point shown in the figure represents the SHAP values of the sample points corresponding to all features. The colors of the points, which range from blue to red, represent the low to high values of each feature. The x-axis in the figure represents the SHAP values; when the SHAP values of the features are greater than 0, it indicates a positive impact on the output of the model; when they are less than 0, it has a negative impact on the output of the model. According to Fig. 2, the feature "Entity" has the largest value, and the feature value of samples is greater than 0. This indicates that the feature has the largest positive impact on the model's output, that is, the simulation runtime increases as the number of simulation entities increases. The feature "File" has the smallest value, which indicates it is the least important to the model and has the smallest impact on its output.
To further investigate the influence pattern of each factor on the simulation runtime, SHAP dependency diagrams of the top three characteristic variables that significantly affect the simulation runtime are drawn, as shown in Fig. 3. The contribution of the number of simulation entities "Entity" has an overall positive correlation, that is, the simulation computational loads and communication loads increases as the increase of the number of simulation entities and cause a positive impact on the simulation runtime. The contributions of the number of CPU cores and CPU usage were negatively correlated over a range, which means that predicting and allocating optimal computing resources to simulation applications can reduce simulation runtime.
To prove the significance of SHAP method, mutual information methods are used to compare the performance of    Fig. 4. In addition, the features that SHAP values less than 1 and the F score less than 0.1 are removed. Second, this study uses the top k feature dimensions in the importance ranking to train the LTR model and evaluates its performance based on the NDCG model performance evaluation metrics. As shown in Fig. 5, The model performance obtained by SHAP method is better than the mutual information method in almost all dimensions, and the highest performance was obtained using the SHAP method with top 16 feature dimensions for training the ranking learning model. Therefore, these features were selected to train the ranking learning model in the subsequent experiments in this study.

Model parameter setting and sensitivity analysis
In this study, eight prediction models such as linear regression (LR), extremely randomized trees (ETR), support vector regression (SVR), extreme gradient boosting (XGBoost), multilayer perceptron (MLP), restricted Boltzmann machine (RBM), ranking boost(Rankboost) and random forest (RF) are built to compare the ranking performance, and the parameters of each models were tuned by the grid search toolkit. The specific parameters are shown in Table 3.  In addition, we analyze the parameters sensitivity of TSLTR on Phold application in Fig. 6. The model performance can improve as learning rate increases and get the best result when learning rate = 0.08, as shown in Fig. 6(a). When setting different max depth and n_estimators, NDCG values with max depth = 6 and n_estimators = 600 are the best, as shown in Fig. 6(b). In subsequent experiments, the hyper-parameter learning rate is set to 0.08, and max depth and n_estimators are set to 6 and 600, respectively.

Model performance evaluation
This section evaluates the performance of the time-sensitive LTR approach for resource prediction of cloud simulation (TSLRT). Figures 7 and 8  17%, and of Rankboost by 7%, respectively. This is because the TSLRT model modifies the loss function by considering the simulation features, and uses the relative ranking between computing resources as feedback information to train a predictor for minimizing the error ranking probability. Compared to RankBoost and TSLRT, other methods tend to get worse performance in NDCG and MRR values, because their aim to minimize the error between the predicted value and the true value and does not account for the impact of computing resource ranking. After revising the loss function of RankBoost, the TSLRT gains 3%-8% higher average NDCG and MRR values, because the TSLRT takes into account the ranking cost issue and modifying the loss function to produce the best ranking possible for the optimal resource. In addition, different models have their own advantages for different indicators. For example, The NDCG value of MLP model is lower than that of LR, ETR models in Phold application; However, the MRR value of MLP model is higher than that of LR, ETR models. The RBM model gains the lowest NDCG values in Phold but the SVR model has the worst performance in PODM application. According to the experimental results in Figs. 7 and 8, We conclude that the TSLRT achieved a higher-ranking performance in term of NDCG and MRR. Table 4 shows the average training and testing times for all the sorting. Compared with the other methods, TSLRT requires the longest training time of 116.04 s. Additionally, the training times for Rankboost, XBG, MLP, and RBM were 86.683, 30.267, 55.101, and 60.325 s, respectively. This is because these methods also require several iterations to obtain the optimal training model. However, the TSLRT and Rankboost models, with prediction times of 0.55 s and 0.52 s, respectively, produced the quickest results. This is because the other models require multiple prediction runtimes and generate the results by sorting, thus incurring more overhead. In this study, we focused on the prediction of simulation runtime resources under static conditions and trained all prediction The Phold application was run in a cloud environment using two simulation applications with different parameters (the number of simulation entities was 3000 and 100) and executed in parallel using various CPU core counts. In the experiments, the proposed TSLRT method was used to predict the priority of the computational resources used dur-ing this run by collecting the feature data of the simulation application after running for 60 s as input and ranking them from the highest to lowest priority. As shown in Table 5, for the simulation application Phold-1, the TSLRT model can accurately predict the priority order of this application using different CPU core counts. For the simulation application Phold-2, the TSLRT model incorrectly predicts only the priority order with 24 and 12 CPU cores. Additionally,  the shortest simulation runtime was obtained when the highest priority number of CPU cores was used in both cases. Therefore, the experimental results show that the ranking learning method proposed in this study can effectively predict the computational resource priority of the simulation application, and by selecting the highest priority computational resource, the simulation application runtime can be significantly reduced and its efficiency can be improved.

Conclusion
This study proposes a method for predicting cloud simulation computing resource, which is characterized by predicting the overall ranking of computing resource usage during a simulation application operation and obtaining the shortest runtime when the simulation application is run with the computing resource with the highest predicted ranking. First, the simulation application is deployed in the cloud environment to generate the dataset. Second, the SHAP interpretable feature extraction method is used to quantitatively determine the importance of each feature dimension. Third, the fea- ture set that significantly applies the simulation application runtime is evaluated. Finally, this study proposes a cloud simulation computing resource prediction algorithm based on time-sensitive ranking learning, which features the factors that significantly affect the simulation application runtime obtained from the analysis, and constructs a simulation runtime sensitive ranking learning model based on RankBoost by designing the objective loss function in the ranking learning algorithm. Different machine-learning methods were evaluated in this study to demonstrate the advantages of the proposed method. The results demonstrate that the proposed method can effectively predict the overall ranking of simulation computational resources and accurately evaluate the computational resources required for the shortest runtime of a simulation application. In future research, we plan to consider scalability factors to predict the overall ranking of computational resources in different types of simulation applications and to evaluate the model performance on larger scale cloud platforms.

Author Contributions
We declare that this manuscript entitled "A time-sensitive learning-to-rank approach for cloud simulation resource prediction" is original, has not been published before and is not currently being considered for publication elsewhere we confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declarations
Conflict of interest we declare that we have no known competing financial interests or personal relationships or organizations that could have appeared to influence the work reported in this paper.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/.