## Abstract

We derive an explicit formula, as well as an efficient procedure, for constructing a generalized Jacobian for the projector of a given square matrix onto the Birkhoff polytope, i.e., the set of doubly stochastic matrices. To guarantee the high efficiency of our procedure, a semismooth Newton method for solving the dual of the projection problem is proposed and efficiently implemented. Extensive numerical experiments are presented to demonstrate the merits and effectiveness of our method by comparing its performance against other powerful solvers such as the commercial software Gurobi and the academic code PPROJ (Hager and Zhang in SIAM J Optim 26:1773–1798, 2016). In particular, our algorithm is able to solve the projection problem with over one billion variables and nonnegative constraints to a very high accuracy in less than 15 min on a modest desktop computer. More importantly, based on our efficient computation of the projections and their generalized Jacobians, we can design a highly efficient augmented Lagrangian method (ALM) for solving a class of convex quadratic programming (QP) problems constrained by the Birkhoff polytope. The resulted ALM is demonstrated to be much more efficient than Gurobi in solving a collection of QP problems arising from the relaxation of quadratic assignment problems.

### Similar content being viewed by others

## References

Anstreicher, K.M., Brixius, N.W.: A new bound for the quadratic assignment problem based on convex quadratic programming. Math. Program.

**89**, 341–357 (2001)Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci.

**2**, 183–202 (2009)Birkhoff, G.: Three observations on linear algebra. Universidad Nacional de Tucumán, Revista, Serie A

**5**, 147–151 (1946)Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)

Burkard, R.E., Karisch, S.E., Rendl, F.: QAPLIB—a quadratic assignment problem library. J. Glob. Optim.

**10**, 391–403 (1997)Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol.

**2**, 27:1–27:27 (2011)Chiche, A., Gilbert, JCh.: How the augmented Lagrangian algorithm can deal with an infeasible convex quadratic optimization problem. J. Convex Anal.

**23**, 425–459 (2016)Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

Cui, Y., Sun, D.F., Toh, K.-C.: On the asymptotic superlinear convergence of the augmented Lagrangian method for semidefinite programming with multiple solutions (2016). arXiv:1610.00875

Delbos, F., Gilbert, JCh.: Global linear convergence of an augmented Lagrangian algorithm to solve convex quadratic optimization problems. J. Convex Anal.

**12**, 45–69 (2005)Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc.

**78**, 837–842 (1983)Fischer, A., Kanzow, C.: On finite termination of an iterative method for linear complementarity problems. Math. Program.

**74**, 279–292 (1996)Fogel, F., Jenatton, R., Bach, F., d’Aspremont, A.: Convex relaxations for permutation problems. In: Advances in Neural Information Processing Systems, pp. 1016–1024 (2013)

Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl.

**2**, 17–40 (1976)Glowinski, R., Marroco, A.: Sur approximation, par elements finis dordre un, et la resolution, par penalisation-dualite, dune classe de problemes de Dirichlet non lineares. Revue Francaise dAutomatique, Informatique et Recherche Operationelle

**9**(R–2), 41–76 (1975)Optimization, I. Gurobi: Gurobi Optimizer Reference Manual (2016)

Han, J.Y., Sun, D.F.: Newton and quasi-Newton methods for normal maps with polyhedral sets. J. Optim. Theory Appl.

**94**, 659–676 (1997)Hager, W.W., Zhang, H.: Projection onto a polyhedron that exploits sparsity. SIAM J. Optim.

**26**, 1773–1798 (2016)Higham, N.: Computing the nearest symmetric correlation matrix-a problem from finance. IMA J. Numer. Anal.

**22**, 329–343 (2002)Haraux, A.: How to differentiate the projection on a convex set in Hilbert space. Some applications to variational inequalities. J. Math. Soc. Jpn.

**29**, 615–631 (1977)Hiriart-Urruty, J.-B., Strodiot, J.-J., Nguyen, V.H.: Generalized Hessian matrix and second-order optimality conditions for problems with \({C}^{1,1}\) data. Appl. Math. Optim.

**11**, 43–56 (1984)Jiang, B., Liu, Y.F., Wen, Z.W.: \(L_p\)-norm regularization algorithms for optimization over permutation matrices. SIAM J. Optim.

**26**, 2284–2313 (2016)Li, X.D., Sun, D.F., Toh, K.-C.: QSDPNAL: a two-phase augmented Lagrangian method for convex quadratic semidefinite programming. Math. Program. Comput.

**10**, 703–743 (2018)Li, X.D., Sun, D.F., Toh, K.-C.: A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems. SIAM J. Optim.

**28**, 433–458 (2018)Lim, C.H., Wright, S.J.:

*Beyond the Birkhoff polytope: convex relaxations for vector permutation problems*. In: Advances in Neural Information Processing Systems, pp. 2168–2176 (2014)Malick, J.: A dual approach to semidefinite least-squares problems. SIAM J. Matrix Anal. Appl.

**26**, 272–284 (2004)Luque, F.J.: Asymptotic convergence analysis of the proximal point algorithm. SIAM J. Control Optim.

**22**, 277–293 (1984)Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim.

**15**, 959–972 (1977)Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(O(1/k^2)\). Sov. Math. Dokl.

**27**, 372–376 (1983)Pang, J.-S.: Newton’s method for B-differentiable equations. Math. Oper. Res.

**15**, 311–341 (1990)Pang, J.-S., Ralph, D.: Piecewise smoothness, local invertibility, and parametric analysis of normal maps. Math. Oper. Res.

**21**, 401–426 (1996)Qi, H., Sun, D.F.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl.

**28**, 360–385 (2006)Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program.

**58**, 353–367 (1993)Robinson, S.M.: Some continuity properties of polyhedral multifunctions. In: Mathematical Programming at Oberwolfach, vol. 14 of Mathematical Programming Studies, pp. 206–214 . Springer, Berlin Heidelberg (1981)

Robinson, S.M.: Implicit B-differentiability in generalized equations. Technical Report #2854, Mathematics Research Center, University of Wisconsin, Madison (1985)

Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res.

**1**, 97–116 (1976)Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, New York (1998)

Sun, D.F., Han, J.Y., Zhao, Y.: On the finite termination of the damped-Newton algorithm for the linear complementarity problem. Acta Math. Appl. Sin.

**21**, 148–154 (1998)Sun, J.:

*On Monotropic Piecewise Quadratic Programming*. Ph.D. thesis, Department of Mathematics, University of Washington (1986)Trefethen, L.N., Bau III, D.: Numerical Linear Algebra. SIAM, Philadelphia (1997)

Von Neumann, J.: A certain zero-sum two-person game equivalent to an optimal assignment problem. Ann. Math. Stud.

**28**, 5–12 (1953)Wang, F., Li, P., Konig, A.C.:

*Learning a bi-stochastic data similarity matrix*. In: 2010 IEEE 10th International Conference on Data Mining (ICDM), pp 551–560Wright, S.J., Nowak, R.D., Figueiredo, M.A.T.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process.

**57**, 2479–2493 (2009)Zhao, X., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim.

**20**, 1737–1765 (2010)

## Acknowledgements

We would like to thank Professor Jong-Shi Pang at University of Southern California for his helpful comments on an early version of this paper and the referees for helpful suggestions to improve the quality of this paper.

## Author information

### Authors and Affiliations

### Corresponding author

## Additional information

The research of Defeng Sun was supported in part by a start-up research grant from the Hong Kong Polytechnic University. The research of Kim-Chuan Toh was supported in part by the Ministry of Education, Singapore, Academic Research Fund under Grant R-146-000-257-112.

## Rights and permissions

## About this article

### Cite this article

Li, X., Sun, D. & Toh, KC. On the efficient computation of a generalized Jacobian of the projector over the Birkhoff polytope.
*Math. Program.* **179**, 419–446 (2020). https://doi.org/10.1007/s10107-018-1342-9

Received:

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s10107-018-1342-9