Effective heuristics and meta-heuristics for the quadratic assignment problem with tuned parameters and analytical comparisons

Abstract

Quadratic assignment problem (QAP) is a well-known problem in the facility location and layout. It belongs to the NP-complete class. There are many heuristic and meta-heuristic methods, which are presented for QAP in the literature. In this paper, we applied 2-opt, greedy 2-opt, 3-opt, greedy 3-opt, and VNZ as heuristic methods and tabu search (TS), simulated annealing, and particle swarm optimization as meta-heuristic methods for the QAP. This research is dedicated to compare the relative percentage deviation of these solution qualities from the best known solution which is introduced in QAPLIB. Furthermore, a tuning method is applied for meta-heuristic parameters. Results indicate that TS is the best in 31%of QAPs, and the IFLS method, which is in the literature, is the best in 58 % of QAPs; these two methods are the same in 11 % of test problems. Also, TS has a better computational time among heuristic and meta-heuristic methods.

Background

Quadratic assignment problem (QAP) is a well-known problem which delineates assigning number of economic facility to the same number of location. The objective function of this problem needs to be minimized. This problem was originally initiated by Koopmans and Beckmann ([1957]). QAP is used in combinational optimization, participating with traveling salesman problem (TSP) and graph problem. QAP is a kind of location problem and, additionally, a kind of layout problem. A lot of research presented exact solutions for QAP in the literature. Figure 1 shows the number of research, which applies the exact methods for QAP until 2005 (Loiola et al.. [2007]). These methods include branch and bound, dynamic programming, cutting plan, and branch and cut.

Figure 1

Exact methods for QAP.

Sahni and Gonzalez ([1976]) proved that QAP belongs to the NP-complete class. In this class of the problem, computational time for solving problems, which have large instances, is too much and grows exponentially, so heuristic approaches and meta-heuristic methods are recommended for solving this problem. Figure 2 shows the literature about solving QAP with heuristic and meta-heuristic methods. This figure reveals that the meta-heuristic methods have been applied more than the heuristic approaches.

Figure 2

Heuristic and meta-heuristic methods for QAP.

For the first time, a simulated annealing (SA) approach in solving QAP was proposed by Burkard and Rendl ([1984]). Wilhelm and Ward ([1987]) then presented the new equilibrium components for solving QAP with SA, and after them, research suggested better SA solution methods. Skorin-Kapov ([1990]) executed the tabu search (TS) technique for QAP. After that, Taillard ([1991]) demonstrated robust TS for QAP. TS was also proposed for QAP in later research. Particle swarm optimization (PSO) is a new approach to solve QAP that Kennedy and Eberhart ([1997]), Shi et al. ([2004]), and Hongbo et al. presented (Hongbo and Ajith [2007]; Hongbo et al. [2007]). Figure 3 shows the application of meta-heuristic methods for QAP until 2005 (Loiola et al. [2007]).

Figure 3

Meta-heuristic approach for QAP.

Figure 3 shows that many research used hybrid algorithm for QAP in recent years. Table 1 shows some of the recent research that applied TS, SA, and PSO for QAP from 2007 to 2009.

Table 1 TS, SA, and PSO approach for QAP from 2007 to 2009

This paper is organized as follows: in the next section, QAP formulation is displayed; afterwards, the heuristic methods are described. After that, the meta-heuristic methods such as SA, TS, and PSO are presented; the tuning method is described next. Computational analyses and comparison results are mentioned in the ‘Results and discussion’ section, and the last section is the ‘Conclusions’ section.

Methods

QAP formulation

This section is dedicated to introduce the classical formulation of QAP considered in this research. This formulation is as follows:

$$min\sum _{i=1}^n\sum _{j=1}^n\sum _{k=1}^n\sum _{l=1}^n{c}_{\mathit{ijkl}}{x}_{\mathit{ik}}{x}_{\mathit{jl}}\text{,}$$

(1)

$$\sum _{i=1}^n{x}_{\mathit{ij}},\forall j=\text{1…}n\text{,}$$

(2)

$$\sum _{i=1}^n{x}_{\mathit{ij}},\forall i=\text{1…}n\text{,}$$

(3)

$$x_{\mathit{ij}}=\{0,1\},\forall i,j=\text{1…}n\text{,}$$

(4)

where c _ijkl is the cost of assigning facility i in location k and simultaneously facility j in location l, and x _ik = 1 if location k is assigned to facility i; otherwise, x _ik = 0. Also, x _jl = 1 if location l is assigned to facility j; otherwise, x _jl = 0. The objective function (Equation 1) of this model must be minimized. Each location must be assigned just to one facility, as Equation 2 shows. Equation 3 displays that each facility must be assigned just in one location. The number of facility and location is the same and is equal to n. The variable in this model is binary.

Heuristic methods

Heuristic algorithms do not provide an assurance for optimization of the problem. These methods are an approximation. They have an additional property that worst-case solutions are known. In this section, some heuristic methods as procedures to search the better solution that contains 2-opt, greedy 2-opt, 3-opt, greedy 3-opt, and Vollman, Nugent, Zartler (VNZ) are contemplated.

2-Opt algorithm

Among simple local search algorithms, the most famous ones are 2-opt and 3-opt. The 2-opt algorithm was first proposed by Croes ([1958]) for TSP. If there are four locations and four facilities, the transposition of facility location in 2-opt method is like that in Figure 4. This figure illustrates that for the first transposition, the facility in location one can be changed with the facility in location two, and for the second transposition, the facility in location one can be changed with the facility in location three, so that if the number of location and facility is shown by n, the number of transposition in each iteration will be n(n − 1)/2.

Figure 4

Transposition of facility in the 2-opt method.

Initially, the algorithm considers the transposition of facilities 1 and 2. If the resulting solution's objective function value (OFV) is smaller than that of the initial solution, then it is stored as a candidate for future consideration. Otherwise, it is discarded, and the algorithm considers the transposing of facilities 1 and 3. If this exchange generates a better solution, then it is stored as a candidate for future consideration; otherwise, it is discarded, and so on. Thus, whenever a better solution is found, the algorithm discards the previous best solution. This procedure continues until all the pair-wise exchanges are considered.

For n location in the QAP problem, the 2-opt algorithm consists of three steps:

Step 1. Let S be the initial feasible solution and Z its objective function value; then, set S* = S, Z* = Z, i = 1 and j = i + 1 = 2.

Step 2. Consider the exchange results in a solution S′ that has OFV Z′ < Z*, set Z* = Z′ and S* = S′. If j < n, then repeat step 2; otherwise, set i = i + 1 and j = i + 1. If i < n, repeat step 2; otherwise, proceed to step 3.

Step 3. If S ≠ S*, set S = S*, Z = Z*, i = 1, j = i + 1 = 2 and go to step 2. Otherwise, output S* is selected as the best solution, and the process is terminated.

Greedy 2-opt algorithm

The greedy 2-opt algorithm is a variant of the 2-opt algorithm. The difference between this method and 2-opt is in selecting the best transposition. This method transposes the facility location if the OFV is better than the known OFV and stabilizes this assignment; it then goes to transpose the facility location from the start. It also consists of three steps. Like the 2-opt algorithm, greedy 2-opt also considers pair-wise exchanges. Initially, it considers transposing facilities 1 and 2. If the resulting OFV is less than the previous one, two facilities are immediately transposed. Otherwise, the algorithm will go on to facility 3 and evaluate the exchange, and so on until improvement is found. If facilities 1 and 2 are transposed, then the algorithm will take it as an initial solution and will repeat the algorithm until it is impossible to improve the solution any further. Greedy 2-opt algorithm makes the exchange permanent whenever an improvement is found and thus consumes less computational time than the 2-opt algorithm. On the other hand, greedy 2-opt algorithm produces slightly worse solutions than the 2-opt algorithm.

3-Opt algorithm

The 3-opt algorithm is similar to the 2-opt algorithm except that it considers transposing two facilities at a time. This algorithm is originally applied for TSP by Bock ([1958]). For example, if there are three facilities in the same location, two types of transposition can be used with the 3-opt algorithm. These types are shown in Figure 5. Type (2) is applied in this research.

Figure 5

Types of transposition in 3-opt algorithm.

Greedy 3-opt algorithm

Greedy 3-opt algorithm is also similar to the greedy 2-opt algorithm, but it makes the three facility exchange permanent whenever its resulting OFV is better than the current OFV and repeats the algorithm with the new transposition as the initial solution. The transposition in this method is similar to that in 3-opt.

VNZ method

The VNZ method was introduced by Vollman et al. ([1968]). This method is using less storage space than 2-opt.

There is not any randomization, and also, these methods cannot orient the current solution to the optimum solution in a limited time. However, the meta-heuristic methods contain a good search approach with a reasonable time. These methods are considered in the next subsection.

Meta-heuristic methods

In the original definition, meta-heuristics are solution methods that manage an interaction between the local improvement procedures and higher level strategies to create a process capable of escaping from local optimum solution and performing a good search of solution space. These methods have also come to include any procedures that employ strategies for overcoming the trap of local optimality in complex solution spaces, especially those procedures that take advantage of one or more neighborhood structures as a means of defining acceptable moves to transformation from one solution to another. In this research, TS, SA, and PSO are applied for the QAP, and their comparison has been done for the selected data sets.

Tabu search

Classical methods often face great difficulty when confronted with hard optimization problems present in real situations. Tabu search (TS), for the first time, was proposed by Glover ([1989, 1990]). This meta-heuristic approach is, in a theatrical manner, changing the ability of solving problems of practical significance. The pseudo code of TS, which is applied in this research, is as follows:

Step 1. Let S be the initial feasible solution and Z its objective function value; then, set S* = S, Z* = Z, max short-term memory (STM) = 5, and max iteration = 1,000; iter = 1. Best O value = O value.

Step 2. Random (i, j) = rand/Long-term memory (LTM) (i, j), (n 1, n 2) = the indices of maximum value in random.

Step 3. If there is none (n 1, n 2) in STM matrix, change n 1 and n 2 locations; otherwise, repeat step 2.

Step 4. Insert n 1 and n 2 in STM and release the last indices from STM (e.g., m 1, m 2); and LTM(m 1, m 2) = LTM(m 1, m 2) + 1.

Step 5. Calculate the objective function value (Z) of the new permutation.

Step 6. If Z ≤ Z*, then Z* = Z, S* = S, and iter = iter + 1.

Step 7. If iter ≤ max iteration, then repeat step 2; otherwise, print Z* and S*.

Simulated annealing

Simulated annealing is a famous and popular local search meta-heuristic applied to address discrete and continuous optimization problems. This method, like the other meta-heuristic methods, can be escaping from the local solution. Simulated annealing is so named because of its similarity to the process of physical annealing with solids, in which a crystalline solid is heated and then allowed to cool very slowly until it achieves its most regular possible crystal lattice configuration and thus is free of crystal defects. If the cooling schedule is sufficiently slow, the final configuration results in a solid with such superior structural integrity. Equation 5 shows the Metropolis acceptance criteria for each move in the cooling process (Metropolis et al. [1953]).

$${}_P=exp\left(\frac{({\text{OFV}}_B-\text{OFV})}{T}\right)\text{,}$$

(5)

where OFV and OFV _B are the objective function values for this iteration and are the best computed one until this iteration. T is the temperature of the algorithm in the iteration, and P is the probability of acceptance for each move in the annealing process. The proposed SA pseudo code for QAP is as follows:

Step 1. Let S be the initial feasible solution and Z its objective function value; then, set S* = S, Z* = Z. T = 100, T₀ = 0.1, r = 0.95, n limit max = 5 and n over max = 10.

Step 2. n limit = 0 and n over = 0.

Step 3. Transpose two facilities in the current layout randomly and calculate the objective function value (Z).

Step 4. If Z ≤ Z*, then accept the transposition; it means that Z* = Z, S* = S, and then n limit = n limit + 1, n over = n over + 1; if n over = n over max or n limit = n limit max, then proceed to step 6.Otherwise, repeat step3; if Z > Z*, then proceed to step 5.

Step 5. Calculate Equation 3; if P ≥ rand (0, 1), then Z* = Z, S* = S, and then n limit = n limit + 1, n over = n over + 1; if n over = n over max or n limit = n limit max, then proceed to step 6; otherwise, repeat step 3.If P < rand(0, 1), then n over = n over + 1; if n over = n over max, then proceed to step 6; otherwise, repeat step 3.

Step 6. T = r × T, where r is the rate of cooling. If T ≤ T₀, then proceed to step 7; otherwise, repeat step 2.

Step 7. Print S* and Z*.

Partial swarm optimization

The particle swarm optimization (PSO) is a population-based search algorithm founded on the simulation of the social behavior of bees, birds, or a school of fish. This method is a significant member of the swarm intelligence. It was proposed by Eberhart and Kennedy as an optimization method (Eberhart and Kennedy [1995]; Kennedy and Eberhart [1995]). Each individual within the swarm is represented by a vector in multidimensional search space. This vector has one assigned vector that determines the next movement of the particle and is called the velocity vector. This technique also determines how to update the velocity of a particle. Each particle updates its velocity based on the current velocity and the best position (p best) it has explored so far, as well as based on the global best position (g best) explored by a swarm. Movement of each particle is shown in Figure 6, and it is based on Equations 6 and 7. Equation 6 illustrates that the velocity vector is updated by the global best position, personal best position, and current position of each particle. Equation 7 shows that each particle moves by its velocity.

$$v_i(t+1)=w.v_i\left(t\right)+b_1.\text{rand}(p\text{best}-x_i(t\left)\right)+b_2.\text{rand}(g\text{best}-x_i(t\left)\right)\text{,}$$

(6)

$$x_i(t+1)=x_i\left(t\right)+v_i\left(t\right)\text{,}$$

(7)

where i is the index of the particle, t is the index of an iteration, v _i is the vector of velocity, x _i is the position, w is the weight of current velocity, b ₁ is the weight of difference between personal best and current positions, b ₂ is the weight of difference between global best and current positions, and rand is used for randomization. The PSO pseudo code, which is presented for QAP in this research, is as follows:

Step 1. max iteration = 100, number of particle = 15, and w = n − 1, b₁ = n/2 and b₂ = (n/2) + 2, where n is the dimension of the problem. Make 15 permutations as the initial solutions and Z* = min (OFV), S* = S, and iter = 1.

Step 2. For i = 1 to w, transpose two facility. Do this step for each particle.

Step 3. Calculate objective function value (Z) for each new particle; find the personal best OFV for each particle (p best), and find the global best OFV (g best).

Step 4. Generate a random discrete number between 0 and b₁, and for i = 1 to this random number, simulate each particle to p best.

Step 5. Generate a random discrete number between 0 and b₂, and for i = 1 to this random number, simulate each particle to g best.

Step 6. iter = iter + 1; if iter < max iteration, then repeat step 2;otherwise, proceed to step 7.

Step 7. Print Z* = g best S* = g best permutation (S).

Figure 6

Each particle movement.

Tuning method

The parameters in meta-heuristic methods which are introduced, such as max STM and max iteration, were tuned by carrying out the general factorial design. To achieving this purpose, some levels were defined initially for each parameter. These levels are determined to be focused on the logic of meta-heuristics. For instance, when the number of iteration in PSO is increased, then the time of search is intensified. This is just one characteristic of this meta-heuristic approach. After defining the levels, an experimental design is determined by general factorial design. The meta-heuristic for these experiments is then run, and their results are analyzed. Thus, the best parameter level can be found. Figure 7 illustrates this tuning method.

Figure 7

The procedure of tuning the parameters.

For example, for ‘Nug’ instances taken from the Quadratic Assignment Problem Library (QAPLIB), the analysis is done for ‘Nug27’ for the TS method. The proposed levels for max STM are 5, 7, 9, and 11, and for max iteration, they are 1,000, 1,500, and 2,000. We consider two replicates for each combination of factor levels; hence, for this instance, 24 treatment combinations are arranged. Figure 8 shows the interaction plot for OFV, and it also shows the best combination of these factors. The best max STM is 5, and the best max iteration is 1,000. Hence, we tune the TS parameter based on this experiment. Moreover, the parameters like n over max, n limit max, T, T ₀ , and others were set by analyzing the general factorial design similar to TS.

Figure 8

Interaction plot for OFV of ‘Nug27’.

Results and discussion

Computational analyses

In this research, an analytical comparison between heuristics and meta-heuristics, which are described as the Nug and ‘Bur’ instances selected from QAPLIB, are applied. The computer which carried out the experiments is equipped with a 2 53-GHz processor with programming coded in MATLAB7.8 (MathWorks, Natick, MA, USA). Table 2 shows the computational results for heuristic methods. Results show that 3-opt as a heuristic method is the best solution method for QAP among the presented heuristic methods in this research. In this table, sizes of the problems are determined by n.

Table 2 Heuristic computational results for the test problems

Tables 34, and 5 reveal the computational results for meta-heuristic methods. In this computational analysis, the test problems are classified in small-, medium-, and large-size cases. These groups are taken from the study of Ramkumar et al. ([2008]). Our experiment results are compared with iterated fast local search (IFLS) when the order crossover with random pair-wise interchange mutation (OXPM) is used (Ramkumar et al [2008]). The criterion considered for evaluating the performance is the relative percentage deviation (RPD) of the solution quality from the best known solution. The number of location is n.

Table 3 Solution qualities with CPU time of small-size test problems

Table 4 Solution qualities with CPU time of medium-size test problems

Table 5 Solution qualities with CPU time of large size test problems

The superior methods in CPU time and OFV are illustrated in the boldface. The average of gaps for heuristic methods and meta-heuristic methods introduced in this research with the exact solution are shown in Figure 9. These results are related to the Nug and Bur instances. This figure shows that TS is the best in our methods. In addition, it can be concluded that meta-heuristics have better solution quality; hence, we compare meta-heuristic methods with IFLS. Furthermore, Figure 9 shows that the greedy-2opt is not a good method because the average gap for this method is about 12 % in selected data sets.

Figure 9

Average gaps for heuristics.

The performances of our proposed meta-heuristic approaches and IFLS with local search are studied by analysis of variance (ANOVA) test.

Figure 10 summarizes the results of our investigation where the critical value for this analysis is considered as 0.05. As we can observe from Figure 10, in small-size test problems, the means of IFLS and PSO are significantly different, and no meta-heuristics have means significantly different from IFLS in the medium- and large-test problems. It is clear from this figure that TS and IFLS have the best solution quality in small-, medium-, and large-size instances, respectively. Hence, it is not necessary that the p values of these analyses be declared.

Figure 10

ANOVA results give 95 % confidence interval for the difference among meta-heuristics and IFLS.

Conclusions

This research considers the heuristic and meta-heuristic solution methods for QAP, and the comparison among them has been done. In addition, a tuning method is declared. The comparison has been executed for the selected data set which was extracted from QAPLIB. The results show that 3-opt has better results than the other heuristic methods. Moreover, our meta-heuristic methods are better than the heuristic methods in solution quality. In this paper, some small, medium, and large test problems are used for comparing our meta-heuristic methods to a method from literature (i.e., IFLS with local search). ANOVA test is run for the results, and it showed that our methods are not considerably different from IFLS. TS is the most excellent method in computational time. Comparisons between the selected solution methods for more instances from the QAPLIB and comparisons of these algorithms with other meta-heuristics and hybrid algorithms can be conducted in future research.