SDP-based bounds for graph partition via extended ADMM

We study two NP-complete graph partition problems, k-equipartition problems and graph partition problems with knapsack constraints (GPKC). We introduce tight SDP relaxations with nonnegativity constraints to get lower bounds, the SDP relaxations are solved by an extended alternating direction method of multipliers (ADMM). In this way, we obtain high quality lower bounds for k-equipartition on large instances up to n=1000\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n =1000$$\end{document} vertices within as few as 5 min and for GPKC problems up to n=500\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n=500$$\end{document} vertices within as little as 1 h. On the other hand, interior point methods fail to solve instances from n=300\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n=300$$\end{document} due to memory requirements. We also design heuristics to generate upper bounds from the SDP solutions, giving us tighter upper bounds than other methods proposed in the literature with low computational expense.


Introduction
Graph partition problems have gained importance recently due to their applications in the area of engineering and computer science such as telecommunication [16] and parallel computing [12].The solution of a graph partition problem would serve to partition the vertices of a graph G(V, E) into several groups under certain constraints for capacity or cardinality in each group.The optimal solution is expected to have the smallest total weight of cut edges.This problem is NP-complete [8].Previous studies worked on improving quadratic programming or linear programming formulations to reduce the computational expense with commercial solvers [6].Relaxations are also used to approximate this problem.Garey et al. [8] were the first to use eigenvalue and eigenvector information to get relaxations for graph partition.Ghaddar et al. [9] have recently used a branch-and-cut algorithm based on SDP relaxations to compute global optimal solutions of k-partition problems.
The k-equipartition problem, which is to find a partition with the minimal total weight of cut edges and the vertex set V is equally partitioned into k groups, is one of the most popular graph partition problems.Another problem that interests us is the graph partition problem under knapsack constraints (GPKC).In the GPKC, each vertex of the graph network is assigned a weight and the knapsack constraint needs to be satisfied in each group.
Lisser and Rendl [16] compared various semidefinite programming (SDP) relaxations and linear programming (LP) relaxations for k-equipartition problems.They showed that the nonnegativity constraints become dominating in the SDP relaxation when k increases.However, the application of this formulation is limited by the computing power of SDP solvers, because adding all the sign constraints for a symmetric matrix of dimension n causes O(n 2 ) new constraints and entails a huge computational burden especially for large instances.Nguyen [19] proposed a tight LP relaxation for GPKC problems and a heuristic to build upper bounds as well.Semidefinite programming has shown advantages in generating tight lower bounds for quadratic problems with knapsack constraints [11] and k-equipartition problem, but so far there have been no attempts to apply SDP relaxations to GPKC.
Algorithms for solving SDPs have been intensively studied in the previous years.Malick et al. [18] designed the boundary point method to solve SDP problems with equations.It can solve instances with a huge number of constraints that interior point methods (IPMs) fail to solve.This method falls into the class of alternating direction method of multipliers (ADMM).ADMM has been studied in the area of convex optimization and been proved of linear convergence when one of the objective terms is strongly convex [20].In recent years, there have been studies focusing on generalizing this idea on solving convex optimizations with more blocks of variables.Chen et al. [3], for example, have proved the convergence of 3-block ADMM on certain scenarios, but the question as to whether the direct extension 3-block ADMM on SDP problems is convergent is still open.
There have also been varied ideas about combining other approaches with ADMM for solving SDP problems.De Santis et al. [4] added the dual factorization in the ADMM update scheme, while Sun et al. [23] combined ADMM with Newton's methods.Both attempts have improved the performance of the algorithms.

Main results and outline
In this paper, we will introduce an extended ADMM algorithm and apply it to the tight SDP relaxations for graph partition problems with nonnegativity constraints.We will also introduce heuristics to obtain a feasible partition from the solution of the SDP relaxation.
This paper is structured as follows.In Section 2, we will introduce two graph partition problems, the k-equipartition problem and the graph partition problems with knapsack constraints (GPKC).We will discuss different SDP relaxations for both problems.In Section 3, we will design an extended ADMM and illustrate its advantages in solving large SDP problems with nonnegativity constraints.In Section 4, we will introduce two post-processing methods used to generate lower bounds using the output from the extended ADMM.In Section 5, we will design heuristics to build a tight upper bound from the SDP solution to address the original problem.Numerical results of experiments carried out on graphs with different sizes and densities will be presented in Section 6. Section 7 concludes the paper.

Notation
We define by e n the vector of all ones of length n, by 0 n the vector of all zeros of length n and by 0 n×n the square matrix of all zeros of dimension n.We omit the subscript in case the dimension is clear from the context.The notation [n] stands for the set of integers {1, . . ., n}.Let S n denote the set of all n × n real symmetric matrices.We denote by M 0 that the matrix M is positive semidefinite and let S n + be the set of all positive semidefinite matrices of order n × n.We denote by •, • the trace inner product.That is, for any M, N ∈ R n×n , we define M, N := trace(M N ).Its associated norm is the Frobenius norm, denoted by M F := trace(M M ).We denote by diag(M ) the operation of getting the diagonal entries of matrix M as a vector.The projection on the cone of positive semidefinite matrices is denoted by P 0 (•).The projection onto the interval [L, U ] is denoted by P [L,U ] .We denote by λ(•) the eigenvalues.That is, for any M ∈ R n×n , we define λ(M ) the set of all eigenvalues of M .Also, we denote λ max (•) the largest eigenvalue.We denote by x ∼ U (0, 1) a variable x from uniform distribution between 0 and 1.We define by argmaxk(•, s) the index set of the s largest elements.

k-equipartition problem
For a graph G(V, E), the k-equipartition problem is the problem of finding an equipartition of the vertices in V with k groups that has the minimal total weight of edges cut by this partition.The problem can be described with binary variables, where L is the Laplacian matrix for G, variable Y ∈ R n×k indicates which group each vertex is assigned to and e n (resp.e k ) is the all-one vector of dimension n (resp.k).
This problem is NP-hard and Lisser and Rendl [16] proposed the SDP relaxation min where X ∈ S n , and e ∈ R n is the all-one vector.
To tighten this SDP relaxation, we can add more inequalities to problem (2).Here, we introduce two common inequalities for SDP relaxations derived from 0/1 problems.
The process of relaxing Y Y to X implies that X is a nonnegative matrix, hence the first group of inequalities we consider is X ≥ 0 and the corresponding new SDP relaxation is This kind of SDP is also called a doubly nonnegative program (DNN) since the matrix variable is both, positive semidefinite and elementwise nonnegative.Another observation is the following.For any vertices triple (i, j, k), if vertices i and j are in the same group, and vertices j and k are in the same group, then vertices i and k must be in the same group.This can be modeled by the transitivity constraints [16] given as follows The set formed by these inequalities is the so-called metric polytop.Adding the transitivity constraints to the SDP relaxation (3) gives min 1 2 L, X

Graph partition problem under knapsack constraints (GPKC)
Given a graph G(V, E) with nonnegative weights on the vertices and a capacity bound W , the GPKC asks to partition the vertices such that the total weight of cut edges is minimized and the total weight of vertices in each group does not exceed the capacity bound W .A mathematical programming formulation is given as where Y ∈ R n×n , a ∈ R n is the vertex weight vector and W is the capacity bound.We assume , otherwise the problem is infeasible.Again, we can derive the SDP relaxation min 1 2 L, X Similar as the k-equipartition problem, we can tighten the relaxation by imposing sign constraints, i.e., min 1 2 L, X and additionally by imposing the transitivity constraints which gives min 1 2 L, X

Extended ADMM
The SDP relaxations introduced in Section 2 have a huge number of constraints, even for medium-sized graphs.The total number of sign constraints for X is O(n 2 ) and adding the constraint X ∈ MET in the SDP relaxations causes 3 n 3 extra constraints.Therefore, solving these tight relaxations is out of reach for state-of-the-art algorithms like interior point methods (IPMs).However, finding high quality lower bounds by tight SDP relaxations for graph partition problems motivates us to develop an efficient algorithm that can deal with SDP problems with inequalities and sign constraints on large-scale instances.Since the 2-block alternating direction method of multiplier (ADMM) has shown efficiency in solving large-scale instances that interior point methods fail to solve, we are encouraged to extend this algorithm for SDP problems with inequalities in the form min C, X where We have the slack variable s ∈ R q to form the inequality constraints, and l and u can be set to −∞ and +∞ respectively.Also, L ∈ S n and U ∈ S n can be symmetric matrices filled with all elements as −∞ and +∞ respectively.That makes formulation (9) able to represent SDP problems with any equality and inequality constraints.This formulation is inspired by the work of Sun et al. [23].All semidefinite programs given above fit into this formulation.E.g., in (3) operator A includes the diagonal-constraint and the constraint Xe = me, the operator B as well as the variables s are not present, L is the matrix of all zeros and U the matrix having +∞ everywhere.Following the ideas for the 2-block ADMM [18], we form the update scheme in Algorithm 1 to solve the dual of problem (9).Lemma 3.1.The dual problem for (9) is given as where Proof.We derive this dual problem by rewriting the primal SDP problem (9) in a more explicit way, namely min C, X Then, the dual of ( 11) is The following equivalences hold for each entry of the dual variables S L and S U in ( 12) If we let S := S L − S U , then for each entry of Thus, in the dual objective function, we have Combining the observations above, we end up with Similarly, let v := v l − v u , then we have Hence, problem (10) is equivalent to (12) and it is the dual of (9).
We now form the augmented Lagrangian function corresponding to (10).
The saddle point of this augmented Lagrangian function is L(y, ȳ, Z, S, v; X, s), (18) which is also an optimal solution for the primal and dual problems.If both, the primal and the dual problem, have strictly feasible points, then a point (X, s, y, ȳ, Z, S, v) is optimal if and only if Remark 3.1.(19d) and (19e) is derived from the optimality conditions for (11), namely from With S = S L − S U and v = v l − v u , we obtain (19d) and (19e).
We solve problem (18) coordinatewise, i.e., we optimize only over a block of variables at a time while keeping all other variables fixed.The procedure is outlined in Algorithm 1.
In Step 1, the minimization over (y, ȳ), we force the first order optimality conditions to hold, i.e., we set the gradient with respect to (y, ȳ) to zero and thereby obtain the explicit expression Note that the size of y k is the number of equality constraints and the size of ȳk the number of inequality constraints.By abuse of notation we write AA * for the matrix product formed by the system matrix underlying the operator A(•).Similarly for BA * , BB * .Algorithm 1: Extended ADMM for problem (10) Tune stepsize and obtain σ k+1 ; In practice, we solve (21) in the following way.First, we apply the Cholesky decomposition Since A and B are row independent, the Cholesky decomposition exists.Moreover, the Cholesky decomposition only needs to be computed once since matrix Q remains the same in all iterations.Then, we update (y, ỹ) as by solving two systems of equations subsequently, i.e., Rx = rhs and then solve the system R y = x and thereby having solved RR y = rhs.
In Step 2, the minimization amounts to a projection onto the non-negative orthant. where Similarly, in Step 3 for v k+1 we have In Step 3, for Z k+1 0, the minimizer is found via a projection onto the cone of positive semidefinite matrices where Finally, by substituting Z k+1 and v k+1 into Step 4 we obtain Remark 3.2.Throughout Algorithm 1, the complementary slackness condition for (X k+1 , Z k+1 ) holds.This is since 1 σ k X k+1 is the projection onto the positive semidefinite cone of matrix N k+1 , while −Z k+1 is a projection onto the negative semidefinite cone of the same matrix N k+1 .

Stepsize adjustment
Previous numerical results showed that the practical performance of an ADMM is strongly influenced by the stepsize σ.The most common way is to adjust σ to balance primal and dual infeasibilities: if Lorenz and Tran-Dinh [17] derived an adaptive stepsize for the Douglas-Rachford Splitting (DRS) scheme.In the setting of the 2-block ADMM, this translates to the ratio between norms of the primal and dual variables X k Z k in the k-th iteration.In general, for the 2-block ADMM this update rule yields a better performance than the former one.
In this paper, we use either of these update rules, depending on the type of problems we solve.For SDP problems with equations and nonnegativity constraints only, we apply the adaptive stepsize method from Lorenz and Tran-Dinh [17] since it works very well in practice.However, the situation is different for SDP problems with inequalities different than nonnegativity constraints.In this case, we use the classic method to adjust the stepsize σ according to the ratio between the primal and dual infeasibilities.

Lower bound post-processing algorithms
We relax the original graph partition problem to an SDP problem, thereby generating a lower bound on the original problem.However, when solving the SDP by a first-order method, it is hard to reach a solution to high precision in reasonable computational time.Therefore, we stop the ADMM already when a medium precision is reached.In this way, however, the solution obtained by the ADMM is not always a safe underestimate for the optimal solution of the SDP problem.Hence, we need a post-processing algorithm that produces a safe underestimate for the SDP relaxation, which is then also a lower bound for the graph partition problem.
Theorem 4.1 leads to the first post-processing algorithm.Before stating it, we rewrite Lemma 3.1 from [13] in our context.Lemma 4.1.Let X, Z ∈ S n , and 0 ≤ λ(X) ≤ x, where λ(•) indicates the operation of getting the eigenvalues of the respective matrix.Then

and let X ∈ S n
+ be an optimal solution for (9) and x ≥ λ max (X), then we have a safe lower bound for the optimal value p * lb : Proof.We recall the alternative formulation of (9), min C, X and the corresponding dual problem Given an optimal solution X * from ( 11) and the free variable ỹ and nonnegative variables (ṽ l , ṽu , SL , SU ), we define where inequality (30c) holds because ṽl , ṽu , SU and SL are nonnegative.This gives us a lower bound for the problem (11) as On substituting S := S L − S U and v := v l − v u into the objective function we have Consequently, we can rewrite (31) as For specifically structured SDP problems, a value of x might be known.Otherwise, without any information about an upper bound x in (29) for the maximal eigenvalue λ max (X), we approximate x as λ max ( X) where the output from the extended ADMM is ( X, ỹ, ṽ, S).Then, we scale it with µ > 1, e.g., µ = 1.1, to have a safe bound µx.Note that this requires that the solution of the extended ADMM, i.e., Algorithm 1, is satisfied with reasonable accuracy, say ε = 10 −5 .
The complete post-processing algorithm is summarized in Algorithm 2.
As for the k-equipartition problem (3), we have X m • I for any feasible solution X. Hence, we let x = m when applying post-processing Algorithm 2 for k-equipartition problems.As for the GPKC, we have no value x at hand.
Another way to get a safe lower bound for ( 9) is to tune the output results and get a feasible solution for its dual problem (10).This is outlined as Algorithm 3. The brief idea is to build a feasible solution (y new , v new , Z new , S new ) from an approximate solution (ỹ, ṽ, Z, S).To guarantee feasibility of (y new , v new , Z new , S new ), we first get a Z new by projecting Z on the cone of positive semidefinite matrices.We then keep Z new fixed and hence have a linear problem.The final step is to find the optimal solution for this linear programming problem.
In Algorithm 1, the condition Z 0 is guaranteed by the projection operation onto the cone of positive semidefinite matrices.Hence, we can skip Step 1 in Algorithm 3.
We would like to remark that the linear program can be infeasible, but this algorithm works well when the input solution has a good precision.The comparisons of numerical results of these two post processing algorithms are given in Section 6.2.

Building upper bounds from the SDP solutions
Computing upper bounds of a minimization problem is typically done via finding feasible solutions of the original problem by heuristics.
Algorithm 3: Adjusted lower bound for (11) Input: Data P = (A, B, b, l, u, L, U, C), approximate primal and dual optimal solution Z for P .Output: Lower bound d * 1 Update Z: Z → P 0 ( Z) ; A k-equipartition problem can be transformed into a quadratic assignment problem (QAP), and we can find feasible solutions for a QAP by simulated annealing (SA), see, e.g., [22].However, this method comes with a high computational expense for large graphs.Moreover, it cannot be generalized to GPKC problems.
Here we consider building upper bounds from the optimizer of the SDP relaxations.We apply different rounding strategies to the solution X of the SDP relaxations presented in Section 2.

Randomized algorithm for k-equipartition
The first heuristic is a hyperplane rounding algorithm that is inspired by the Goemans and Williamson algorithm for the max-cut problem [10] and Frieze and Jerrum [7]'s improved randomized rounding algorithm for k-cut problems.
Note that the Goemans and Williamson algorithm as well as the Frieze and Jerrum algorithm are designed for cut-problems formed as models on variables in {−1/(k −1), 1} n , while our graph partition problems are formed on {0, 1} n .Therefore, we need to transform the SDP solutions of problems (3) and ( 7) before applying the hyperplane rounding procedure.Our hyperplane rounding algorithm for k-equipartition is given in Algorithm 4.

Vector clustering algorithm for k-equipartition
We next propose a heuristic via the idea of vector clustering.Given a feasible solution X of (3), we can get V ∈ R n×n with V V = X.Let v i be the i-th row of V and associate it with vertex i in the graph.The problem of building a feasible solution from X can Algorithm 4: Hyperplane rounding algorithm (Hyp) for k-equipartition problem Data: number of partitions k, cluster cardinality m, number of sampling M , objective matrix C; Input: Optimal solution X ∈ S n from the SDP relaxation; Output: end end where argmaxk i∈I (a i , s) returns the index set of s largest elements in a i , ∀i ∈ I.
then be interpreted as the problem of clustering vectors v 1 , . . ., v n into k groups.This can be done heuristically as follows.
1. Form a new group with an unassigned vector.
2. Select its m − 1 closest unassigned neighbors and add them in the same group.
3. Update the status of those vectors as assigned.
This process is repeated k − 1 times until all vectors are assigned in a group, yielding a k-equipartition for the vertices in V .The details are given in Algorithm 5.
Algorithm 5: Vector clustering algorithm (Vc) for k-equipartition problem Data: number of partitions k, cluster cardinality m, maximum iteration number M ; Input: SDP relaxation optimal solution X ∈ S n ; Output: X * , partition end end where argmaxk i∈I (a i , s) returns the index set of s largest elements in a i , ∀i ∈ I.

Measure closeness between vertices
We explain in this section how we determine the closest neighbor for a vector.The idea of vector clustering is to have vectors with more similarities in the same group.In our setting, we need a measure to define the similarity between two vectors according to the SDP solution.
For a pair of unit vectors v i , v j , using the relationship cos (v i , v j ) = v i v j one can measure the angle between v i and v j .
By the setting V V = X, we have for any i ∈ [n] where x i is the i-th row vector in X.
Hence, x i consists of the cosines of the angle between v i and other vectors.We define sim(v i , v j ) := n k=1 cos (v i , v k ) cos (v j , v k ) = x i x j and use this as a measure in Algorithm 5.In other words, we measures the closeness between v i and v j by their geometric relationships with other vectors.
In Algorithm 5, we choose a vector as the center of its group and then find vectors surrounding it and assign them to this group.
In each iteration we randomly choose one vector to be the center.

Vector clustering algorithms for GPKC
Using similar ideas as in Algorithm 5, we construct a rounding algorithm (see Algorithm 6) for GPKC as follows.
1.In each iteration, randomly choose an unassigned vector v i to start with.
2. Add vectors in the group of v i in the order according to sim until the capacity constraint is violated.
3. If no more vector fits into the group, then this group is completed and we start forming a new group.

2-opt for graph partition problems
2-opt heuristics are used to boost solution qualities for various combinatorial problems, e.g., TSP [15].We apply this method after running our rounding algorithms for the graph partition problems to improve the upper bounds.According to the rounding method we choose, the hybrid strategies are named as Hyperplane+2opt (also short as Hyp+2opt) and Vc+2opt for Algorithms 4 and 5, respectively.The 2-opt heuristic for bisection problems is outlined in Algorithm 7. Given a partition with more than two groups, we apply 2-opt on a pair of groups (P s , P t ), which is randomly chosen from all groups in the partition, and repeat it on a different pair of groups until no more improvement can be found.
For GPKC, some adjustments are needed because of the capacity constraints.We only traverse among swaps of vertices that still give feasible solutions to find the best swap that improves the objective function value.

Algorithm 6: Vector clustering algorithm (Vc) for GPKC problem
Data: vertex weight a, knapsack bound W , maximum iteration number M ; Input: SDP relaxation optimal solution X; Output: X * ∈ {0, 1} n×n , partition for i ∈ P t do

Numerical results
We implemented all the algorithms in MATLAB and run the numerical experiments on a ThinkPad-X1-Carbon-6th with 8 Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz.The maximum iterations for extended ADMM is set to be 20 000 and the stopping tolerance ε tol is set to be 10 −5 by default.The code can be downloaded from https://github.com/shudianzhao/ADMM-GP.

Instances
In order to evaluate the performance of our algorithms, we run numerical experiments on several classes of instances.All instances can be downloaded from https://github.com/shudianzhao/ADMM-GP.The first set of instances for the k-equipartition problem are described in [16], the construction is as follows.

Choose the partition numbers as divisors of the graph size n.
We name those three groups of instances rand20, rand50 and rand80, respectively.Furthermore, we consider instances that have been used in [2].These are constructed in the following way.
• U |V |,|V | πd 2 : For a graph G(V, E), first choose 2|V | independent numbers uniformly from the interval (0, 1) and view them as coordinates of |V | nodes on the unit square.Then, an edge is inserted between two vertices if and only if their Euclidian distance is less or equal to some pre-specified value d [14].Here |V | ∈ {500, 1000} and |V | πd 2 ∈ {5, 10, 20, 40}.
• mesh: Instances from finite element meshes; all edge weights are equal to one [5].
For GPKC we generate instances as described in [19].This is done by the following steps.
2. Determine a feasible solution for this instance for a k-equipartition problem by some heuristic method.
3. Produce 1000 permutations of the vertices in this k-equipartition.
4. Calculate the capacity bound for each instance and select the one such that only 10% of instances are feasible.
We name those three groups of instances GPKCrand20, GPKCrand50 and GPKCrand80, respectively.

Comparison of Post-processing Methods
Our first numerical comparisons evaluate the different post-processing methods used to produce safe lower bounds for the graph partition problems.Recall that in Section 4, we introduced Algorithms 2 and 3.
Figure 1 shows how the lower bounds from the post-processing methods evolve as the number of iterations of the extended ADMM increases.We used the DNN relaxation on an instance of the k-equipartition problem of size n = 100 and k = 2.There are three lines: EB_eADMM represents the lower bounds obtained by the rigorous lower bound method given in Algorithm 3, LpB_eADMM represents the linear programming bound given in Algorithm 2 and dualOfv_eADMM displays the approximate dual objective function value obtained by our extended ADMM. Figure 1a shows that the rigorous error bound method gives tighter bounds in general, while the linear programming bound method is more stable and less affected by the quality of the dual objective function value.The other figures indicate that for small k, the rigorous error bound method gives tighter bounds (see Figure 1b), but as k increases, the linear programming bound method dominates (see Figure 1c and 1d).Remark 6.1.We choose Algorithm 2 in all following experiments as post-processing for k-equipartition problems because this method is more stable for varying k.For GPKC we use Algorithm 3 for the post-processing since we have no information on the eigenvalue of an optimal solution.

Comparison of the Lower Bounds using SDP, DNN and Transitivity Constraints
In this section we want to highlight the improvement of the bounds obtained from the relaxations introduced in Section 2. Note that the timings for computing these bounds are discussed later in Section 6.3.2.
In practice, adding all the transitivity constraints is computationally too expensive, we run a DNN-based loop instead.The idea is as follows.
1. Solve the DNN (3) to obtain the solution X DN N .
2. Add m met transitivity constraints that are most violated by X DN N to the relaxation.

Solve the resulting relaxation and repeat adding newly violated constraints until
the maximum number of iterations is reached or no more violated constraints are found.
Tables 1, 2 and 3 compare the lower bounds obtained from the relaxations for the k-equipartition problem.The improvements are calculated as (d DN N − d SDP )/d SDP and (d DN N +M ET − d SDP )/d SDP , respectively; a '−' indicates that no transitivity constraints violated by the SDP solution of problem (3) have been found.In [21] it has been observed that the violation of the transitivity constraints is small and the nonnegativity constraints X ≥ 0 are more important than X ∈ MET when the partition number k increases.In our experiments we also observe that the improvement due to the nonnegativity constraints gets even better as k increases.

Comparisons between extended ADMM and Interior Point Methods (IPMs) on k-equipartition
In this section we want to demonstrate the advantage of our extended ADMM over interior point methods.For our comparisons we use Mosek [1], one of the currently best performing interior point solvers.Note that computing an equipartition for these graphs using commercial solvers is out of reach.For instance, Gurobi obtains for a graph with n = 100 vertices and k ∈ {2, 4, 5, 10, 20, 25} after 120 seconds a gap of at least 80 %, whereas we obtain gap up to at most 7 %.
We list the results for solving the SDP (2) using an ADMM, and the results when solving the DNN relaxation (3) by our extended ADMM and by Mosek.We run the experiments on randomly generated graphs with 80% density, the results are given in Table 4. Table 4 shows that the convergence behavior of the extended ADMM is not worse than the 2-block ADMM for SDP problems, and we can get a tighter lower bound by forcing nonnegativity constraints in the model without higher computational expense.
The results for Mosek solving problem (3) clearly show that these problems are out of reach for interior point solvers.A "−" indicates that Mosek failed to solve this instance due to memory requirements.

Heuristics on k-equipartition Problems
We now compare the heuristics introduced in Section 5 to get upper bounds for the graph partition problems.
We use the solutions obtained from the DNN relaxation to build upper bounds for the k-equipartition problem since the experimental results in Section 6.4.1 showed that the DNN relaxation has a good tradeoff between quality of the bound and solution time.
We compare to the best know primal solution given in [2]; we set the time limit for our heuristics to 5 seconds.The gaps between the upper bounds by Vc+2opt (resp.Hyp+2opt) and the best know solution are shown in Table 5.The primal bounds that are proved to be optimal are marked with " * ".Table 5 shows that on small instances our heuristics can find upper bounds not worse than the best known upper bounds.For large instances, Vc+2opt performs better than Hyp+2opt.The corresponding upper bounds are less than 10 % away from the best known upper bounds, some of them computed using 5 hours.
We next compare the upper bounds for the instances rand80, rand50, and rand20.Figure 2 shows that, for small instances (i.e., n = 100), our hybrid methods (eg.Vc+2opt and Hyp+2opt) can find tight upper bounds quickly while simulated annealing (SA) needs a longer burning down time to achieve an upper bound of good quality.Figure 3 shows how the heuristics behave for large-scale instances (i.e, n = 1000).The time limit is set to 5 seconds.Compared to Figure 2, Vc+2opt and Hyp+2opt take more time to generate the first upper bounds but these upper bounds are much tighter than the one found by SA.Also, when the time limit is reached, the upper bounds found by Vc+2-opt and Hyp+2opt are much tighter than those from SA.
Tables 6, 7 and 8 give a detailed comparison of the upper bounds for the instances rand80, rand50, and rand20, respectively.We display the gap between the lower bounds obtained from the DNN relaxation (3) and the upper bounds built by varied heuristics.The time limit for the heuristics is set to 1 second for n ∈ {100, 200} and 3 seconds for n ∈ {900, 1000} for rand80.For rand50 and rand20 we set the limit to 5 seconds.The best upper bounds are typeset in bold.
The numbers confirm that Vc+2opt and Hyp+2opt can build tighter upper bounds than SA, in particular for the dense graphs rand80.Overall, Vc+2opt has the best performance.
Comparing lower and upper bounds, the numerical results show that our methods perform very well on dense graphs; for rand80 the largest gap is less than 4%, for rand50 the largest gap is less than 6%.As the randomly generated graph gets sparser, the gap  Comparing Tables 6, 7 and 8, it can be observed that the gaps get larger as the graph gets sparser.We conjecture that this is due to less tightness of the lower bound which is supported by the following experiment.
We regard the best upper bounds obtained from all three heuristics within an increased time limit of 10 seconds.In this way, we should have an upper bound that approximates the optimal solution well enough for all densities.As an example, in Table 9 we report for a graph on 100 vertices and three different densities the lower and these upper bounds.We can clearly see that when the graph gets sparser, adding the nonnegativity constraints to the SDP relaxation (2) gains more improvement.However, the gap between lower and upper bound gets worse as the graph gets sparser.We now turn our attention to the GPKC problem.We run experiments similar to those presented in Section 6.3.1, i.e., we solve DNN (7) to obtain the solution X DN N .Table 10 shows the lower bounds for GPKC problems on the randomly generated graphs rand80.The improvements are calculated in the same way as in the previous section.The experimental results on GPKCrand50 and GPKCrand20 are omitted since they have a similar behavior.The lower bounds obtained from different SDP relaxations show that when the capacity bound W decreases (and thus the number of groups increases), the improvement of the nonnegativity constraints gets more significant.This is in line with the results for k-equipartition.And, also similar to k-equipartition, for GPKC the improvement due to the transitivity constraints is only minor.

Comparisons between extended ADMM and IPMs for GPKC
Table 11 compares the computation times when solving the DNN relaxations for the GPKC ( 7) by the extended ADMM and Mosek, respectively.A "-" indicates for extended ADMM that the maximum number of iterations is reached, and for Mosek that the instance could not be solved due to memory requirements.The results of the SDP relaxation (6) in Table 10 are computed using Mosek, hence we omit these timings in Table 11.
In the thesis [19], numerical results on the GPKC are presented using an LP relaxation.However, the method therein is capable of getting bounds either for very sparse graphs of density at most 6 % (up to 2000 vertices) or for graphs with up to 140 vertices and density of at most 50 %.We clearly outperform these results in terms of the density of the graphs that can be considered.While for instances of size n = 100, the timings of the extended ADMM and Mosek are comparable, the picture rapidly changes as n increases.For n ≥ 300, Mosek cannot solve any instance while the extended ADMM manages to obtain bounds for instances with n = 500 within one hour.

Heuristics on GPKC problems
As mentioned in Section 5, the simulated annealing heuristic for the QAP cannot be applied to the GPKC, because there is no equivalence between the GPKC and the QAP.Therefore, we compare the upper bounds for the GPKC from the heuristic introduced in Section 5.3 with the lower bounds given by the DNN relaxation (7).We set a time limit of 5 seconds.
Also, we set the maximum number of iterations to be 50 000 for the sparse graph GP-KCrand20, while the maximum numbers of iterations for GPKCrand50 and GPKCrand80 are 20 000.In Table 12, 13 and 14, a * indicates for the extended ADMM that the maximum number of iterations is reached.
Table 12 shows that the gaps between the lower and upper bounds are less than 3% for GPKCrand80, they are less than 7% for GPKCrand50, see Table 13, and for GPKCrand20, the gaps are less than 15%, see Table 14.Similar to the k-equipartition problem, we note that computing the lower bound on the sparse instances is harder.The maximum number of iterations is reached for rand20 much more often than for rand80 or rand50.with bound constraints, the extended ADMM deals with them separately from inequalities and equations, thereby solving the problems more efficiently.Mosek fails to solve the DNN relaxations of problems with n ≥ 300 due to memory requirements while the extended ADMM can solve the DNN relaxations for k-equipartition problems on large instances up to n = 1000 within as few as 5 minutes and for GPKC problems up to n = 500 within as little as 1 hour.We run numerical tests on instances from the literature and on randomly generated graphs with different densities.The results show that SDP relaxations can produce tighter bounds for dense graphs than sparse graphs.In general, the results show that nonnegativity constraints give more improvement when k increases.
We compare our heuristics with a simulated annealing method in the generation of upper bounds for k-equipartition problems.Our heuristics obtain upper bounds displaying better quality within a short time limit, especially for large instances.Our methods show better performance on dense graphs, where the final gaps are less than 4% for graphs with 80% density, while the gaps between lower and upper bounds for sparse graphs with 20% density are bounded by 12%.This is mainly due to the tighter lower bounds for dense graphs.

20 Figure 1 :
Figure 1: Lower bounds obtained with post processing

Figure 2 :
Figure 2: Upper bounds for k-equipartition problems on rand80 with n = 100

Table 4 :
Computation times for k-equipartitioning problems

Table 5 :
[2]sible solutions for the graphs from[2](the time limit is 5 seconds, optimal solutions are indicated by a " * ")

Table 7 :
Feasible solutions for randomly generated graphs rand50 (time limit 5 seconds)

Table 8 :
Feasible solutions for randomly generated graphs rand20 (time limit 5 seconds)

Table 9 :
Feasible solutions for the randomly generated graphs rand80, rand50, rand20 (n = 100 k = 5, with an increased time limit for heuristics of 10 seconds)

Table 11 :
Computation times for GPKC problems

Table 12 :
Feasible solutions for randomly generated graphs on GPKC problems GPKCrand80 (the maximum number of iterations for eADMM is 20 000, a * indicates that the maximum number of iterations is reached)

Table 13 :
Feasible solutions for randomly generated graphs on GPKC problems GPKCrand50 (the maximum number of iterations for eADMM is 20 000, a * indicates that the maximum number of iterations is reached)