Hybrid ant colony optimization in solving multi-skill resource-constrained project scheduling problem

In this paper, hybrid ant colony optimization (HAntCO) approach in solving multi-skill resource-constrained project scheduling problem (MS-RCPSP) has been presented. We have proposed hybrid approach that links classical heuristic priority rules for project scheduling with ant colony optimization (ACO). Furthermore, a novel approach for updating pheromone value has been proposed based on both the best and worst solutions stored by ants. The objective of this paper is to research the usability and robustness of ACO and its hybrids with priority rules in solving MS-RCPSP. Experiments have been performed using artificially created dataset instances based on real-world ones. We published those instances that can be used as a benchmark. Presented results show that ACO-based hybrid method is an efficient approach. More directed search process by hybrids makes this approach more stable and provides mostly better results than classical ACO.


Introduction
Resource-Constrained Project Scheduling Problem (RCPSP) is one of the most investigated types of scheduling problems.Its goal is to find the resource-to-task assignments to make the finite project plan the cheapest or shortest.Description of RCPSP in (Blazewicz et al., 1983) as combinatorial, NP-hard problem encouraged scientists to find good enough methods that would be able to produce approximate, (sub)optimal solutions in finite, polynomial computing time.Those methods are called (meta)heuristics and are used to solve problems for which finding optimal solution in an acceptable time is impossible.
Beside Evolutionary Algorithms (EA), Taboo Search (TS), Simulated Annealing (SA) and some other techniques, metaheuristics contain also a group of methods called swarm-intelligence methods, as Particle Swarm Optimization (PSO) or Ant Colony Optimization (ACO).Those methods assume that separate individuals, representing given problem solutions, can interact with each other and cooperate to achieve their common goals.In this point of view, swarm intelligence techniques are similar to EA.However, they assume that there is one, constant population of individuals that can evolve in time but cannot be replaced by new individuals.ACO, as the name stands, simulates the behaviour of ants, travelling between the ant's nest and the source of food.The optimization goal is to find the optimal path between food and nest, while definition of path's quality is varied and dependent on the considered problem.
The real-life nature of RCPSP comes from business.Project managers in companies struggle to build effective project schedule, meeting duration, cost and other constraints.What is more, many constraints have to be satisfied, while manual scheduling often leads to violating of those constraints.It is a common problem for project managers.Hence computer-aided, (semi-)automatic tools are desired by the industry.Furthermore, obtaining the project plan by computer-driven methods is less time-consuming than obtained manually.
Developing RCPSP to a more practical problem, we have introduced the skills domain, transforming it to the Multi-Skill RCPSP (MS-RCPSP) extension.In MS-RCPSP resources dispose of some given pool of skills, while every task requires some skill in a given level to be performed.It means not every resource is capable of performing every task.As solution space in MS-RCPSP is more constrained, it is more difficult to build good enough solution -project schedule.Furthermore, we have added another criterion -project schedule performance cost, transforming the classical singleobjective (duration) RCPSP into multi-objective (duration vs. cost) MS-RCPSP.
We have decided to create hybrid methods by combining ACO-based approach with some heuristics described in (Skowroński et al., 2013b).Therefore, classical heuristics have been also investigated.Based on results obtained in that paper, we have chosen given heuristics that could be used to obtain the initial solution for ACO mechanism and stand as a Hybrid Ant Colony Optimization (HAntCO).A very significant fact is that depending on optimized criterion (duration or cost) various priority rule could be used.Therefore, we are able to decide whether using HAntCO allows to get better solutions than using ACO mechanism not supported by any priority rule.
Investigating ACO-based approach was motivated by the willingness to compare results obtained using several collective intelligence methods and other metaheuristics, such as TS or SA (Myszkowski et al., 2013) to solve this problem.As we had researched EA-based approach before (Skowroński et al., 2013a), we made a comparison of different approaches in case of their robustness, effectiveness and stability, while those terms would be explained further.
The dataset for experiments has been created artificially, but instances are based on the real-world ones obtained from an international enterprise.What is more, presented MS-RCPSP could be generalized to the PSP-LIB (Kolisch et al., 1996) dataset model that is regarded as a benchmark for methods solving project scheduling problems.
The rest of the paper is organized as follows.Section 2 describes selected ways of solving the (MS-) RCPSP using metaheuristics, especially ACO.Section 3 presents the MS-RCPSP problem statement, while Section 4 describes the approaches proposed in this paper.Section 5 provides conducted experiments of proposed methods in a given dataset.Finally, section 6 presents the conclusions of obtained results and suggests some directions of future work.
However, there is still lack of papers regarding multiobjective Multi-Skill extension of RCPSP.Some approaches solving MS-RCPSP in project duration domain (Al-Anzi et al., 2010), (Santos et al., 2011) or project cost domain (Li et al., 2009) could be found.On the other hand, there are methods solving classical RCPSP extended by cost domain but without skills considerations.Such research has been presented in (Phruksaphanrat, 2014), (Jaberi et al., 2014), (Gonzalez et al., 2013), (Luna et al., 2013) and (Yannibelli et al., 2013).Hence we have decided to combine those two elements: multi-objective optimization and multi-skill domain for project scheduling problem.
Although classical RCPSP is deeply investigated and numerous approaches could be easily compared using PSPLIB instances, it is very hard to find multi-objective MS-RCPSP methods working on datasets that could be regarded as a benchmark.Some papers describe instances artificially generated ( (Hegazy et al., 2000), (Santos et al., 2011)), while some others propose methods of PSPLIB dataset adaptation ((Al-Anzi et al., 2010), (Drezet et al., 2008), (Kadrou et al., 2006), (Li et al., 2009)).However, both of those approaches for handling MS-RCPSP benchmark data are not supplied by any published dataset instances.Hence the need of proposing our own dataset has arisen.
ACO is inspired by the rules in the real environment of ants.Real ants are capable of finding the shortest path from the source of food to the ant's nest.Every ant from a population leaves a substance called pheromone while getting to the source of food.This substance attracts other ants to come into that direction.However, the pheromone evaporates gradually in every period.It means the shorter path is, the less pheromone would be evaporated and that path would be more attractive to other ants.In that way, more and more ants start to exploit the region of a surface where there was more pheromone -the path to the source of food was shorter.Finally, all ants move along the same path, what is regarded as the found solution of the problem.
A classical ACO approach with some modifications that made it more robust has been presented in (Merkle et al., 2002).Particularly, the following features have been proposed: combination of two pheromone updating methods, dynamic influence of those methods during ACO runtime and possibility of leaving the best obtained solution by an elitist ant to preserve sticking in local optima.The presented methods have been tested on PSPLIB instances.In many cases, the obtained results were better than the best found so far, what confirms the robustness of that approach.
Various improvements of ACO have been proposed in (Luo et al., 2003).A single solution, represented by a single ant, is obtained using serial generation scheme.If generated schedule turns out to be infeasible after adding a given task, the ant can reschedule some beginning fragment of a current schedule in order to make it feasible.The feasibility is lost when precedence constraints are violated.The following activities that should be added to a current schedule are chosen by combination of classical heuristics: most total successors, latest finish time (LFT) and resource scheduling method.The authors used UBO dataset from ProGen (Kolisch et al., 1996) to verify their approach.
A different ACO approach has been presented in (Zhou et al., 2009) as well.The combination of Ant Colony System (Dorigo, 1997) and Max-Min Ant System (Stutzle et al., 2000) called MMACS has been proposed.The following improvements have been proposed in this approach: pseudorandom proportional rule for choosing a next activity, updating pheromone only in the base of the best ant from given iteration and serial schedule generation scheme.Furthermore, an extended and RCPSP-adjusted 2opt local search method (Watson et al., 1998) called PS-2opt has been proposed.Results of experiments conducted on PSPLIB stated that PS-2opt and MMACS methods are robust in solving RCPSP.
Another ACO-based approach has been presented in (Liang et al., 2004) where activity-on-node task precedence relations representation is considered.Activity selection is performed by forward-parallel method, while the search space exploration and exploitation is performed by tuned online and offline pheromone updating procedure.Conclusions supported by performed experiments on PSPLIB datasets stand that the approach proposed in (Liang et al., 2004) gives competitive results in comparison to other (not-only) ACO-based approaches.

Problem statement
Before the description of the multi-skill extension for RCPSP, the fundamentals of classical RCPSP would be presented.The motivation to investigate RCPSP and its extensions came from industry and would be explained in detail in Subsection 3.3.

Classical RCPSP description
In RCPSP a set of tasks is given, while every task is described by its duration, start and finish dates.Tasks are non-preemptive.It means any task cannot be withdrawn if it has been started.Tasks are related to each other by precedence relations, describing which tasks are needed to be completed before some other could be started.Tasks that have to be finished before the start time of another task are called predecessors.In classical RCPSP resource units are provided.Every resource owns a finite number of units (represented as integer numbers) that could be assigned to various tasks while tasks require some number of units to be performed.Cumulative number of units of tasks assigned to specified resource in a given period cannot exceed a number of units owned by resource.Only one resource can be assigned to a given task but not only one task can be assigned to given resource in given timestamp.In classical RCPSP, two dummy activities are added -start and finish tasks.It is because, in RCPSP, every task besides the start one has predecessors.Hence finish time of the last, dummy finish task is the finish time of schedule and the duration of a project could be computed as duration between start time of dummy start task and finish time of finish dummy task.The goal of RCPSP is to find such task-to-resource assignments to make the final schedule feasible and as shortest as possible.Combinatorial nature of the RCPSP makes it NP-hard.
A solution of RCPSP is a feasible schedule -the one in which resource units and precedence constraints are preserved.

Multi-skill extension of RCPSP
MS-RCPSP extension adds the skills domain to classical RCPSP.Every task requires some skill at given familiarity level to be performed, while every resource disposes some skills pool -subset of skill types (e.g.developer, analyst, tester, architect, etc.) defined in a project with given familiarity level.Therefore, the resource R is capable of performing the task T only if R disposes skill required by T at the same or higher level.The capabilities of performing tasks by resources could be presented as skill matrix.Sample skill matrix has been shown in the Fig. 1.

Fig. 1 Example of skill matrix
In the skill matrix presented in the Fig. 1 skills required by task to be performed have been written over task definition, while skills owned by resources have been written next to resource definition.This figure presents sample resource capabilities: resource R1 disposes skills Q1 and Q2 with familiarity level 3 and 2 respectively.It is capable of performing task T 1, T 3 and T 4 because all of those mentioned tasks require skill owned by R1 at no higher level than it has.R1 cannot be assigned to T 2, because this task requires totally different skill that R1 does not dispose of, even at the lowest familiarity level.Analogously, resource R2 can be assigned to task T 2, resource R3 is a proper one for task T 3 and, finally, resource R4 can perform tasks T 1, T 2 and T 3.Even though R3 disposes of skill Q2, it cannot be assigned to T 1 and T 3 because those tasks require Q2 at higher familiarity level that this resource disposes.

Model adjustment
As a result of consultations with representatives of various enterprises, we decided to introduce some practical changes in classical RCPSP extended to MS-RCPSP model.Firstly, we introduced resource salary (as an hourly wage) paid for performed work.In that case, resources are regarded only as human ones varied by their salary.We also resigned from introducing start and finish dummy activities as our approach assumes that there could be some tasks that are not connected by precedence relations with any other.Hence we cannot define the project duration, start time and finish time based on dummy activities.
What is more, resources are not described by unitsany resource cannot be assigned to more than one task in an overlapping period -dedicated resources (Bianco et al., 1998).If such a situation occurs, the conflict is detected and should be resolved.The conflict fixing procedure is presented in Subsec.4.4.
Schedule feasibility for such modified problem is extended from classical RCPSP schedule feasibility definition by skills domain -only resources capable of performing given tasks can be assigned to them.

Problem formulation
Feasible Project Schedule (P S) consists of J = 1, ..., n tasks and K = 1, ..., m resources.A non pre-emptive duration d j , start time S j and finish time F j is defined for each task.Predecessors of given task j are defined as P j .Each resource is defined by its hourly rate salary s k and owned skills Q k = 1, ..., r, while pool of owned skills is a subset of all skills defined in project Q k ∈ Q. Value l q denotes the level of given skill, while h q describes its type and q j is a skill required by j to be performed.Therefore, by J k subset of tasks that can be performed by k resource is defined.Duration of a project schedule is denoted as τ .Cost of performing j task by k resource is denoted as c k j = d j * s k , where s k describes the salary of resource k assigned to j.For simplicity, we have modified the task's performance cost from c k j to c j , because only one resource can be assigned to given task.Hence there is no need to distinguish various costs for the same task.Moreover, we have introduced variable defining whether k is assigned to j in given time t: U t j,k ∈ {0; 1}.If U t j,k = 1, k is assigned to j in t.Analogously, k is not assigned to j in t if U t j,k = 0. Feasible project schedule (P S) belongs to the set of all feasible and non-feasible solutions (violating precedence-, resource-and skills-constraints) : P S ∈ P S all .
Formally, the problem could be regarded as optimization (minimization) problem and stated as follows: Subject to: Eq. 1 denotes the duration and cost optimization respectively.Depending on the evaluation function configuration (described below), various optimization modes could be used in an optimization process.f τ (P S) is an evaluation function of project schedule's duration, while f C (P S) is an evaluation function of project schedule's performance cost.
The first constraint (Eq.2) preserves the positive values of resource salaries and ability to perform at least one task by every resource.Eq. 3 states that every task has positive finish date and duration, while Eq. 4 shows the precedence constrains rule.Next two equations: Eq. 5 introduces skill constraints and transforms RCPSP into MS-RCPSP.Constraint (Eq. 6) describes that any resource can be assigned to no more than one task in given time during the project.The last constraint (Eq.7) says that each task must be performed in schedule P S by one resource assignment.

Evaluation function
As it was mentioned, the proposed approach allows to set various objectives of optimization: duration-or cost-oriented one.Those two aspects are normalized, weighted and summarized.Normalization is necessary because of different domains of both aspects that are in opposition to each other.Setting optimization more cost-oriented causes enlarging the project duration, while setting as more important the duration aspect of optimization could increase the cost of the project.
The detailed formulation of the evaluation function has been presented in Sec.4.2.

Solution space size
Because of NP-hard (combinatorial) nature of investigated problem, we have decided to present an estimation of solution space size (SS).It has been computed as follows: The above estimation is valid for all solutions, including non-feasible ones.Computing factorial of tasks number provides the number of combinations of ordering tasks within the timeline.It is easy to notice that such estimation allows to set any order, skipping precedence constraints.The second element of Eq. 8 provides the number of resource-to-task assignments, including a situation that the same resource is assigned to all tasks and no skill constraints are preserved.
To imagine how big the solution space could be, let's take into account a sample project schedule with 100 tasks and 20 resources.Using Eq. 8, the solution space size is equal to SS(100, 20) = 1.19 * 10 288 solutions, including both feasible and infeasible ones.

Proposed approach
Before we describe the details of the proposed approach, some basic ACO definitions in terms of MS-RCPSP should be introduced.Colony is represented as a set of ants: A = 1, ..., p, where p is a number of ants in population.Edge represents a given task and resources that are capable of performing it.Furthermore, edge stores information about the pheromone (P h j = 1, ..., p k j ) values for each resource capable of performing a given task.Surface is represented as a set of edges: E = 1, ..., jall possible task-to-resource assignments, while path represents the set of specified task-to-resource assignments.Path is assigned to a given ant that represents a single solution.Surface represents the solution space of skill-feasible solutions.
The pheromone value determines the probability of assigning given resource to given task.In the first step of classical ACO, the initial value of pheromone is given for each resource in every edge while for a heuristic initialization, pheromone value is the biggest for path reflecting solution found by heuristic.It means that, at the beginning of our approach run, the probability of choosing resource to be assigned to a task is equal in classical ACO or is close to 1 for path representing heuristic-found solution and close to 0 for remaining edges in the surface.
Firstly we have used heuristics from (Skowroński et al., 2013b) to find the best approach for duration optimization (DO) and cost optimization (CO) modes.Based on the obtained results, successors list size-based heuristic (SLS) (Skowroński et al., 2013b) with descending order has been used for DO and resource salarybased (RS) (Skowroński et al., 2013b) with ascending order has been used for CO.Output of scheduling project instances by those heuristics has been used as input for ACO method that has been run with the same parameters' configuration as ACO not boosted by heuristic.
The proposed hybrid ACO-based approach could be briefly described in the following steps: 1. Set initial ant population using heuristics to find good initial solution 2. Check the stopping condition.3. Select edge for each ant. 4. Evaluate solutions.5. Evaporate given amount of pheromone from each edge.6. Update solutions.
7. Update pheromone value in edges by selected ants.8.Return to 2.
The pseudocode of investigated HAntCO approach is presented in Alg. 1.
Algorithm 1 HAntCO pseudocode 1: A ← set initial solution 2: while stopping criterion not satisf ied do 3: for a ∈ A do 4: for e ∈ P ath(a) do 5: e ← selectEdge(J k ) 6: f (a) ← evaluate(a) 7: for e ∈ E do 8: p e ← decayP heromone() 9: A ← selectAnts(A) 10: for a ∈ A do 11: for e ∈ P ath(a ) do 12: p e ← updateP heromone(e) 13: In every iteration, some ants have to be selected (line 9 in Alg. 1) to update a pheromone on their edges.The decision which ant should be chosen depends on selected pheromone update methods.There could be all ants chosen, only the local and global best or the local best and worst.Choosing ants to update a pheromone has been described in detail in Subsection 4.3.After each iteration, pheromone values are updated.Then local (A b ) and global (A g ) best solutions are updated.After each iteration, solutions in ants are ordered ascending by their evaluation function value (line 13).The first ant from the list is set as the best one (A b ) while the last one -as the worst local one.If the evaluation function value of the best local solution (A b ) is smaller (minimization problem) than evaluation function value for the best global solution (A g ), the best global solution is updated (line 15).Analogously the global worst solution (A v ) is updated.The local worst solution (A w ) is used in DIFF pheromone update method.

HAntCO Colony initialization
In the first step of classical ACO, the surface of n edges is obtained.For each resource in each edge, the initial pheromone value is set.Then p ants are defined by choosing random capable resource to j task.To reduce the influence of non-determinism and make search more directed, we have decided to introduce a heuristic initialization in hybrid called HAntCO.In HAntCO, one ant has assigned schedule obtained by heuristic described in (Skowroński et al., 2013b).This ant is set as favourable -it can leave much more pheromone than any other ant in a colony.Other ants in the colony are defined in the same way as in classical ACO initial colony definition.
Heuristic used to obtain an initial solution is varied depending on the optimization mode.For the Duration Optimization mode (DO) the Successors' List Size (SLS) heuristic has been used, as it provided the best results for DO mode.In this method, tasks are sorted by a number of successors they have in ascending order.Then for every task from ordered list a resource is assigned.The decision which resource should be assigned is determined by the earliest time when given resource would finish its previous tasks it has been assigned to.
For Cost Optimization (CO) mode, resources are sorted ascending by their standard salary rate and then are assigned to tasks from the list given in project definition, preserving skill constraints and avoiding conflicts, by assigning a given task to resource no earlier than all previously assigned tasks to resource would be finished.
In the next step each solution is evaluated, to set the pheromone value for each ant in the next iteration.The amount of pheromone left in every iteration is set according to the ant chosen as the best.
As the stopping criterion, the number of iterations with no change of global best solution has been proposed in this approach.It is notated as γ.
The probability of selecting resource k to task j in edge selection bases on the roulette method and is computed as follows: Where α is a weight for pheromone values influence.This value is the parameter of ACO approach and should be provided by the user.p k j is a pheromone value stored in the edge containing information about k resource performing j task.

Evaluation solution method
Evaluation function is formulated as follows: where: w τ -weight of duration component, f τ (P S)duration evaluation component, f c (P S) -cost evaluation component.Both components are non-negative values, while w τ ∈ [0; 1].
Summing both components' weight to 1 ensures that changing the importance of one aspect would cause also some change of second aspect's importance.
The time component f τ (P S) is calculated as follows: where: τ max -maximal (pessimistic) possible duration of the schedule P S, computed as the sum of all tasks' duration.It occurs when all tasks are performed serially in project: one-by-one.No matter how many and how flexible resources are.
The cost component f c (P S) is defined as follows: where: c min -minimal schedule cost -the total cost of all tasks assigned to the cheapest resource, c maxmaximal schedule cost -a total cost of all tasks assigned to the most expensive resource.Note: c max and c min do not involve skill constraints.It means that c min value could be reached only for non-feasible solution.
Analogously to c max .

Update pheromone
Pheromone evaporates iterative.It means the pheromone value is decreased by the same value (µ) in every iteration, as it was stated in the Eq. 13.
Obtained results for various update pheromone methods strongly depend on values set for the following parameters used in ACO: p init -initial value of pheromone amount in each edge, µ -amount of pheromone evaporated in each iteration, δ -amount of pheromone left in edges by ants, p min -minimal value of pheromone set for resource in edge.
In the proposed approach, three strategies of setting pheromones have been researched: ALL (Liang et al., 2004), ELIT E (Merkle et al., 2002) and DIF F .The last of the proposed ones is the new one, proposed by the authors of this paper.

Update pheromone -ALL
In this approach, every ant can leave the pheromone value in the edge for selected resource (Liang et al., 2004).The better the solution is, the more pheromone could be left by the ant in given edge.The best ant leaves the pheromone in the amount equal to δ.All next ants leave the amount of pheromone equal to δ divided by the ant's position (pos) in the list ordered ascending by the evaluation function value.
The main advantage of this approach is the method's resistance to being stuck in local optima.On the other hand, this approach raises a risk of missing the best solutions because of the more exploratory than exploitationbased character of search process.

Update pheromone -ELITE
In this approach, only elite ants are allowed to leave the pheromone on given edges.The set of elite ants always contains two ants: the one with the best solution found in the current iteration (A b ) and the global best one (A g ) (Merkle et al., 2002) -with the best solution found from the beginning of search process.For both ants, the same pheromone amount update method is set: As this approach is more local-optimum oriented, it could lead to getting stuck in local optima.However, the convergence to the optimum of this approach is faster than in ALL method.

Update pheromone -DIFF
In this approach, the ant with the worst or best found solution in given iteration is selected.Updating the pheromone value by the worst allows to explore the search space in other than potentially the best directions and, consequently, escape from local optima.The same like in ELITE approach, only two ants are able to leave the pheromone: the best (A b ) / worst (A w ) in iteration and global best (A g ) / global worst (A v ) found so far.The decision which ant (best or worst) should leave pheromone is made on the basis of satisfaction of the following condition: Where π is regarded as an ant population variety and is computed as follows: Where f b and f w are the evaluation function values of the best and worst solutions contained by given ants in specified iteration.The right-sided variable ψ could be regarded as an ant population variety threshold and is set as an ACO parameter.
If condition in Eq. 16 is satisfied, ELITE update pheromone method is used.With every iteration in which condition from Eq. 16 is satisfied, the counter (κ) of possible worst pheromone update strategy use is incremented.If the variety computed in Eq. 16 is not satisfied, it means ants are concentrated near some local optima.Then, to avoid being stuck, the worst update method is launched.It means that not the best but worst ants leave pheromone on their path.Meanwhile the counter κ is decremented.The worst ant can leave pheromone as long as the ant population variety is smaller than ψ or the κ is not negative.Initial κ value is also set as an ACO parameter.
The value of pheromone left by the global ant is defined in Eq. 18.For the global best (or worst) ant the pheromone amount update value is defined as follows: Where γ is a number of iterations from the last found new global best.
For the best / worst ant in iteration, the pheromone amount update value is stated as follows: In update pheromone amount method for global ant (Eq.18) the pheromone amount is reduced, while the pheromone amount for the best local ant is increased (Eq.19).It enhances the possibility of finding new global optimum, reducing the probability of losing the best solution found so far at the same time.

Conflict fixing
A conflict appears when more than one task is assigned to the same resource in overlapping periods.In that case, it should be fixed by the following procedure.
It is performed by shifting one of conflicting task's start date.Consequently, the finish date of that task also has to be shifted in order to keep the task duration.The decision which of conflicting tasks should be shifted depends on which of them starts earlier.If they are set to start at exactly the same time, task to be shifted is selected by the way, which was firstly added to project definition.Furthermore, we do not investigate the velocity of resources.Therefore, job duration is constant regardless of assigned resource and skills it owns.
Conflict fixing procedure illustration has been presented in the Fig. 2.
Tasks T 4 and T 5 have been assigned to the Resource R2 in overlapping period.As a conflict fixing result, a new schedule has been presented, where T 5 starts

Experiments and results
The goal of the conducted experiments was to investigate the following issues: robustness of ACO approach for MS-RCPSP based on given dataset, robustness of various update pheromone methods, comparing HAntCO to classical ACO approach and other (meta-)heuristics.
Therefore, we have compared the results obtained for different update pheromone methods and results for hybrids and classical ACO approach.Furthermore, the results for simple heuristic scheduling have been provided to get a reference for the ACO-based mechanism.
The obtained results (project schedules) are described by duration time ([days]) and performance cost ([currency units]).Those project schedule properties have been used to compare the investigated methods.

iMOPSE dataset
Due to evaluating not only the project schedule duration, but also the cost of the schedule, we cannot use the standard PSPLIB benchmark dataset (Kolisch et al., 1996) that does not contain any information about the task performance cost.What is more, PSPLIB dataset instances do not reflect the MS-RCPSP.Hence, lack of benchmark data has encouraged us to prepare the iMOPSE dataset, containing 36 project instances, that have been artificially created1 , on the basis of realworld instances, obtained from an international enterprise.We recommend other scientists using iMOPSE dataset as a benchmark for investigating their approaches in solving MS-RCPSP as defined.
Instances of the dataset have been created according to the analysis made in cooperation with experienced project manager from Volvo IT.We were not allowed to get real project data because of their sensitive character for the enterprise.However, we made a statistical analysis of real projects.Then we prepared artificial dataset instances according to the analysis result, regarding the most common project characteristics, like a number of tasks, a number of resources, various skill types in enterprise and the structure of critical chain (a number of tasks involved by precedence relations), etc.The iMOPSE dataset summary has been presented in the Table 1.There are two groups of created project instances: one contains 100 tasks and the second -200 tasks.Within each group, project instances are varied by a number of available resources and the precedence relationship complexity.The number of resources for instances from both groups were chosen in a way to preserve constant average resource load and average task relations ratio for given instances.Hence for project instances with 200 tasks the number of possible resources and precedence relations is twice bigger than for project instances containing 100 tasks.The skill variety has been set up to 9 or 15 different skill types for each project instance, while any resource can dispose of exactly six different skill types.Because of the different resources and relations number, the scheduling complexity for each project is varied.
This dataset stands as an extension of dataset presented in (Skowroński et al., 2013a), (Myszkowski et al., 2013), (Skowroński et al., 2013b) and that is the reason some instances are named with suffix Dx.This suffix refers to dataset instances that have been previously created and presented in those papers.Because of the extension of the dataset, the need of introducing more clear namesystem has arisen.Suffix has been added to a reference of previously created files, keeping the naming convention applied after dataset extension.

Experiments' set-up
The experiments have been divided into investigating the influence of ACO parameters' configurations for project duration and performance cost in three various components' weights in evaluation function: duration optimization (DO: w τ = 1, see.Eq. 10), balanced optimization (BO: w τ = 0.5) and cost optimization (CO: w τ = 0).Because of the stochastic nature of ACObased methods, each experiment for given parameter configuration has been repeated ten times.For K-S test and t-test each experiment has been repeated 50 times (see Tab.9 and Tab.10).On the other hand, deterministic character of heuristics allowed us to obtain results for those methods in only one iteration for every parameters' configuration.
The further step of the conducted experiments was to compare results obtained for random initial solution with boosting initial solution by using described hybrids.Initial solution has been previously obtained by using the above-mentioned heuristics and then set them as input for ACO and made those results as more favourable in local search by enhancing the pheromone value left in this path representing initial solution.We decided to use SLS(D) (Skowroński et al., 2013b) for DO mode and RS(A) (Skowroński et al., 2013b) for CO mode optimization within HAntCO hybrid.Because of some code refactoring, we were able to tune our heuristics and obtain a better solution than the found ones in (Skowroński et al., 2013b).That is the reason why the results of those heuristics in this paper are slightly better than the results in (Skowroński et al., 2013b) for given dataset instances.To present averaged results in detail (see Tab. 4), a standard deviation measure (σ) has been introduced and applied to each average value, presented as a percentage value in relation to the average.We have also added information about the best found solution for a given method (see Tab. 2) that have been compared with the results obtained by most promising heuristics, described in (Skowroński et al., 2013b).
Both for the best and averaged results, pheromone update methods have been compared and the one that provided best results (shortest duration in DO, smallest cost in CO and smallest evaluation function value in BO) has been presented in Tab. 4 and Tab. 2. The notation for methods used in tables with obtained results is as the following: E -update ELITE pheromone method, A -update ALL, D -update DIFF.If more than one pheromone update methods turned out to be the best and gave the same results, they have been presented both separated by "/" (e.g.: E/D -both update DIFF and update ELITE methods gave the same, best results).In Tab. 2 a sign * has been also introduced to indicate a situation where all three methods provided the same, regarded as the best, result.

Experiments' performance
The processing time was varied in relation to the used update method.For ALL method that could be regarded as the simplest, the processing time was relatively small (from 7 to 90 seconds, depending on processed dataset instance).However, for ELITE and DIFF methods, that are regarded as more complex because of the need of sorting ants and choosing best / worst, the processing time varied from 30 to 270 seconds per one execution in one CPU for given parameter configuration2 .

The best found results
The best results obtained by ACO for CO and DO modes have been compared with the results obtained by using heuristics proposed in (Skowroński et al., 2013b).In Tab. 2 this comparison is presented.For each dataset instance and optimization mode, the best results have been chosen from various pheromone update methods.Indication which method provided the best results is stored in columns named M for every optimization mode.
The obtained best results have been compared with the heuristic results.We decided to omit the name of heuristic if possible to reduce the space covered by the table.For heuristic results in CO, SA heuristic name has been omitted without losing any important information, as the parameter configuration for that method has been written in the table.To give a more detailed view about those methods, please refer to (Skowroński et al., 2013b).

Better values from comparison optimization modes between ACO and heuristics have been written in bold.
If key values (duration for DO or cost for CO) were equal for ACO and heuristic approaches, the smallest value of the second aspect has been taken into account to choose a better solution.If both project schedule properties turned out to be the same, both solutions were written in bold.To determine the best obtained result for BO mode, neither duration nor cost has been investigated.Instead of those aspects, the evaluation function value has been taken into account.Furthermore, we were not able to compare strictly the results of BO for ACO with corresponding ones for heuristics, as no evaluation function has been used to evaluate results of heuristics.
A similar analysis has been made for the best found results within investigated hybrid.The best HAntCO results have been presented in Tab. 3. The most significant difference for HAntCO best results table in comparison with table of best results for classical ACO is that there is no BO mode included.It is because hybrid is activated only for DO or CO mode -depending on selected heuristic for initialization.
Taking into account the results gathered in Tab. 3, we can assume that the ELITE strategy mode for HAntCO generally provides better results than DIFF in DO mode.It provided better results in 26 cases (72%).However, in CO we noticed that the DIFF strategy turned out to be more suitable than the ELITE, provided better results in 9 cases (25%), while the ELITE became better in only one case (less than 3%).In remaining cases, both strategies gave the same best results.An interesting fact is that for DO no equal best results for both strategies have been found.
Also comparing HAntCO best results (see Tab. 3) to single heuristics results (see.Tab 2) we can see that hybrid ACO with heuristics is more effective for DO than CO mode.In most instances (89%) HAntCO found a better solution than simple heuristic or ACO.

Averaged results
Averaged results obtained for various pheromone update methods have been presented in Tab. 4 in a similar way as the ones in Tab. 2 respectively.We also provided in Tab. 4 the notation for the method that provided best results (A, D, E, D/E).In opposition to Tab. 2 no comparison to averaged heuristic results has been introduced, because heuristics are deterministic methods for which result can be obtained in only one iteration.On the other hand, in Tab. 4 a standard deviation measure (σ) has been introduced, to indicate the level of variability of the obtained results.It is presented as a percentage value of an average.
For DO and CO modes the smallest averaged values of project duration or project cost respectively have been taken into account to determine the best pheromone update method.If values of given aspect are equal, the smallest value of the second aspect is taken into account.If there is still no possibility to determine which pheromone update method provides better solutions, the standard deviation of more important aspect is taken into account (duration for DO and cost for CO respectively) and the method with smaller standard deviation value is regarded as better.
We have also provided averaged results for HAntCO approach, presented in Tab. 5. Analogously to best HAntCO approach results, averaged ones regard only DO and CO modes.Averaged values are supported by standard deviation values that reflect the variability of the obtained results.We have also decided to count how many times one strategy became better than another also in averaged results.For DO, ELITE strategy became better in 25 cases (69%), while DIFF became better in the remaining ones.For CO, DIFF strategy provided better results in 14 cases (39%), while only in one case ELITE strategy became better.For the remaining ones, the obtained averaged results became the same.It leads to conclusion that HAntCO searches space in CO mode in very directed way, being unable to explore other parts of the solution space.Independent character of searching is, in many cases, regardless of applied pheromone update strategy.
To investigate the level of stability of HAntCO in comparison with classical ACO, we have checked how many times 0-equal standard deviation value has been obtained in the conducted experiments.Those results are presented in Tab. 6.The results gathered in this table prove that the proposed hybrid approach is more directed and thus, the proposed approach found the same solution in many more cases than classical ACO which stochastic nature allows to explore the search space more widely.The most interesting results found in Tab.6 concern CO mode.For that mode, HAntCO found the same cost solutions 21 (58%) times for ELITE and 16 times (44%) for DIFF strategies, while the same duration solutions have been found 24 (67%) and 25 (69%) times respectively.

Computational complexity
Our research has been extended by investigating the level of complexity of compared methods.The complexity has been estimated as a number of potential assignments of resources to a given task as dominant operations.As this value is constant regardless of the optimization process and depends only on initial skill constraints, we can compute the level of complexity as a factor of an average number of iterations and a number of possible assignments.The results of those computations are presented in Tab. 7.
As we decided to set a constant number of iterations in most methods like TS, EA S and EA C, the complexity level for those methods was easy to compute.For ACO and HAntCO we decided to get an av-erage number of iterations from all optimization modes (DO, BO, CO) and update pheromone methods (ALL, ELITE, DIFF) as the value that should be multiplied by a number of possible assignments.Based on the results gathered in Tab.7 we can notice that ACO and HAntCO are most processing-complex methods.However, the level of complexity for HAntCO is lower than for classical ACO.It is because the number of iterations for hybrids is generally smaller, as searching is started from more directed place in the solution space.
Complexity level of heuristics has been computed as multiplication of a number of possible assignments by 1, as there is only one iteration in heuristic scheduling process.What is more, heuristic are deterministic approaches.Therefore, we always get the same results that are obtained in only one iteration.Hence, heuristic could be used as a powerful tool to get the first glance of optimization possibilities for given dataset instance.

Results' discussion
Both for the best and averaged results for classical ACO, ELITE update pheromone method turned out to be the best for DO mode, while DIFF update pheromone method became the most suitable for BO mode.However it is not possible to get such straightforward conclusions for CO mode, because DIFF method became the most suitable choice for the best obtained results while both DIFF and ELITE methods provided equally good results for average obtained optimization results.
We have also compared pheromone update methods in hybrids performance.For that approach ELITE mode turned out to be the most suitable for DO, while any ( * ) proposed pheromone update method became equally good for CO mode for most project instances.No difference between pheromone update method has been also observed in 15/36 (42%) cases in CO.It could lead to conclusion that pheromone update method is not as crucial as for classical ACO.It is because initial solution is preferred -hybrid is more exploitation-than exploration-oriented.
We have also compared how many times heuristics provided better results than the best ones obtained from an application of ACO approaches (see Tab. 2).For DO, SLS heuristic became better 9 times (25%), while for CO SA or RS heuristics became better 18 times (50%).It shows that classical ACO approach, proposed in this paper cannot be fully regarded as better in comparison with heuristic methods.However, combining it with heuristic in hybrid (HAntCO) approach turned out to give usually much better results than any other investigated methods, especially for CO mode.
An interesting fact is that DO mode is generally more stable than other based on the provided results.It has been deducted by counting number of bigger than 10% σ values in Tab. 4. For DO there were no such values, while, for BO, there were 3 over 10% values (2 for duration aspect and 1 for a cost aspect).Finally, for CO, there were 7 over 10% values of standard deviation -all for a duration aspect.
An interesting conclusion that could be made regardless of the best or averaged results is that a DIFF strategy provided better solutions in DO mode but mostly for dataset instances containing 200 tasks.The best results obtained by a DIFF strategy were better than obtained by an ELITE in 9 cases for 200 task-project instances (50%), while ELITE strategy provided only one better solution than a DIFF (5%).Averaged results obtained in a DIFF mode were better in 12 cases (67%), while ELITE strategy still provided only one better solution in comparison with a DIFF.
Comparing the best results obtained by ACO and HAnt-CO it can be noticed that HAntCO outclasses classical ACO, whichever pheromone update method would be used.For DO, classical ACO approach has been better than HAntCO in only 5 from 36 cases, while for CO HAntCO became better than ACO for every project instance.Analysing averaged results, there are only 3 cases with ACO results better than HAntCO ones.Still only for DO.For CO, ACO has never been better than HAntCO.It proves the legitimacy of using hybrids that become robust way of boosting optimization process.
To get bigger awareness of classical ACO and HAnt-CO approaches' robustness, we decided to compare the obtained best results for ACO with best results obtained using other methods, as EA (Skowroński et al., 2013a) and TS (Myszkowski et al., 2013).However, we needed to distinguish the best results obtained for DO and CO modes from BO mode, because no heuristic scheduling method has been proposed for BO.Comparison of DO, BO and CO modes has been presented in Tab. 8.This comparison has been made only for project instances D1-D6, because only those have been investigated in (Skowroński et al., 2013a), (Myszkowski et al., 2013) and (Skowroński et al., 2013b).The compared methods are Taboo Search (TS), specialized Evolutionary Algorithms (EA S), classic EA (EA C), classical ACO, HAntCO and heuristics (H).
The results presented in Tab. 8 show that both HAnt-CO and TS outclassed other methods in DO mode, obtaining best cost results for half of investigated project instances for each method (D1, D2, D5, D6 for HAntCO and D3, D4 for TS).For CO mode, classical ACO became the best approach for D2 and D3 instances, while HAntCO obtained the best results for the same instances plus D1.However, the most successful approach for these instances is a heuristic one that allowed to get best results in 5/6 cases.
The averaged results of investigated methods are presented in Tab.9.It differs slightly from the results in Tab.8, as methods are non-deterministic.However, conclusions are very similar: HAntCO outperforms other methods in almost every case or results are comparable.We developed extra statistical analysis to prove a quality of presented method.We have provided the Kolmogorov-Smirnov (K-S test) to investigate the normality of the distribution of gained results.The K-S test proved that results of used methods are normally distributed and t-test can be used.Moreover, a sample size around 50 allows the normality assumptions conducive for performing the unpaired t-tests (Flury, 1997).We used two tailed t-test with 95% confidence interval (see results in Tab.10) for the best and the second best performing methods applied in D1-D6 instances for DO and CO modes.
We found that HAntCO results are the best in most cases.Very interesting results are noticed for EA S, especially for D5 instance (in DO and CO mode) where EA S gives the best (average) solutions.Only in one case (D3 instance DO mode) ACO gives better average solution.The results are significant in statistical meaning.The statistical significance of results for HAntCO in CO mode comes mostly from the fact that HAntCO is a method directed by a heuristic that finds the best cost-oriented solution (algorithm).Hence, the statistical significance of this method should be mostly investigated in DO mode.In this mode, the results obtained by HAntCO are statistically significant in 3 cases (D1, D2, D6), while DO-oriented results obtained by ACO are statistically significant in only one case (D3).It additionally proves the legitimacy of using proposed hybrid rather than classical ACO approach.We have also investigated results for several methods in BO mode.In this case, classical ACO approach outclassed the rest of examined methods and became the best choice in 5/6 cases.However, it caused enlarging the project schedule duration of analysed instances and make them mostly the longest from all obtained with various methods.EA with specialized genetic operators gave the smallest project cost for BO mode.It was the best in 5/6 cases.An interesting fact is that the results obtained for ACO are completely different from the results from other methods like TS or EA.The conclusion could be that ACO searches the solution space totally different from the above-mentioned methods.Hence combining those approaches into one could be possibly effective and potentially give promising results.

Conclusions and further work
In this paper we have presented hybrid approach for solving Multi-Skill Resource-Constrained Project Scheduling Problem.MS-RCPSP is an extension of classical RCPSP with skills and cost domain.Our approach bases on classical ACO metaheuristics for discrete combinatorial problems.However, it has been enhanced by modified pheromone update methods.Furthermore, we have proposed a hybridization of ACO approach (HAnt-CO) by using simple heuristics based on priority rules to find an initial solution in optimization process.
What is more, we have prepared and published iMOP-SE dataset instances to allow others to investigate their approaches for such defined MS-RCPSP.The dataset consists of 36 instances containing 100 or 200 tasks.All instances are varied by the number of resources, precedence relations and skill types what makes them more or less difficult to be scheduled.
We have also defined evaluation methods for the proposed approaches not only in case of their robustness (how good the final solution is) but also their effectiveness.To evaluate method's quality, we rate not only the project schedule duration, as in classical RCPSP, but also the project schedule performance cost, regarding the MS-RCPSP as multi-objective optimization problem.The method's effectiveness is rated by the number of dominant operations that need to be performed during the optimization process.
Finally, we have compared the results obtained by HAnt-CO and ACO with the ones received with the use of other methods as simple heuristics, Taboo Search and Evolutionary Algorithms with classic and specialized genetic operators that have been published before.The provided results have been also supported by the statistical significance tests.The obtained results lead to the conclusion that ACO-based approaches stand suitable ones for solving MS-RCPSP as they provide mostly the best results from all investigated methods.

Future work
After observation that pheromone update method in ACO has an impact on the obtained results depending on selected optimization mode (aspect), we are encouraged to use this outcome and propose an approach more dedicated to multi-objective optimization using Pareto front from various ant populations performing in different pheromone update methods.It could provide us a mechanism to find very good solutions leaving the need of setting optimization mode.It could give us good solutions in DO and BO in the same run of ACO-based run.
Pareto-based approach could be implemented in the investigated methods to distinguish non-dominated solutions.By non-dominated solution a one with the smallest value of given criterion is taken while remaining criterion values are equal.In MS-RCPSP non-dominated solution is regarded as the one that has smallest cost or duration from subset of solutions with the same duration or cost respectively.It could make the optimization process more robust and effective, as good enough results could be found in a smaller number of iterations within the examined method.
As cost oriented optimization in ACO and HAntCO has not provided significantly better results than other methods investigated in this paper, we discuss a potential application of dedicated neighbourhood definition for ants to make them more oriented to search solutions cheaper.
Investigating the comparison of the results obtained for CO and DO modes could lead to conclusion that ACO is a powerful tool for solving MS-RCPSP, especially if it was boosted by initial solution obtained by heuristic (HAntCO).It leads to conclusion that other hybrids should be investigated using the proposed heuristics.Hence we would examine and compare the results obtained for EA, TS or SA approaches to check, whether boosting initial solution by heuristic provides better results for other metaheuristics.
According to the experiences with ACO of other researchers, ACO can be extended by additional heuristic (Dorigo, 1997) to enhance the potential of optimization.We plan to find suitable, problem specific heuristic that could be used and investigate whether it could make our approach better in solving MS-RCPSP.

Fig. 2
Fig. 2 Example of conflict resolving

Table 2
Comparison of the best obtained results for DO, BO and CO modes in classical ACO and selected heuristics from(Skowroński et al., 2013b)

Table 3
Best results obtained for HAntCO with various pheromone update methods in DO and CO optimization mode

Table 4
Averaged results obtained for classical ACO in various optimization modes

Table 5
Averaged results obtained for HAntCO with various pheromone update methods in DO and CO optimization mode

Table 6
Number of 0-equal standard deviation measures for given pheromone update strategies and optimization modes.

Table 7
Average number of dominant operations (divided by 10 3 ) during optimization process using investigated methods for given parameters' configurations

Table 8
Comparison of best obtained results for investigated methods in DO, BO and CO modes

Table 9
Comparison of averaged obtained results for investigated methods in DO and CO modes

Table 10
Results of the unpaired t-test between the best and the second best performing methods (for each instances D1-D6) based on Tab.9 (heuristic H(Skowroński et al., 2013b)results not included as a part of HAntCO)