Mathematical Programming

, Volume 130, Issue 1, pp 125–151 | Cite as

Moment inequalities for sums of random matrices and their applications in optimization

  • Anthony Man-Cho So
Full Length Paper Series A


In this paper, we consider various moment inequalities for sums of random matrices—which are well-studied in the functional analysis and probability theory literature—and demonstrate how they can be used to obtain the best known performance guarantees for several problems in optimization. First, we show that the validity of a recent conjecture of Nemirovski is actually a direct consequence of the so-called non-commutative Khintchine’s inequality in functional analysis. Using this result, we show that an SDP-based algorithm of Nemirovski, which is developed for solving a class of quadratic optimization problems with orthogonality constraints, has a logarithmic approximation guarantee. This improves upon the polynomial approximation guarantee established earlier by Nemirovski. Furthermore, we obtain improved safe tractable approximations of a certain class of chance constrained linear matrix inequalities. Secondly, we consider a recent result of Delage and Ye on the so-called data-driven distributionally robust stochastic programming problem. One of the assumptions in the Delage–Ye result is that the underlying probability distribution has bounded support. However, using a suitable moment inequality, we show that the result in fact holds for a much larger class of probability distributions. Given the close connection between the behavior of sums of random matrices and the theoretical properties of various optimization problems, we expect that the moment inequalities discussed in this paper will find further applications in optimization.


Non-commutative Khintchine’s inequality Semidefinite programming Approximation algorithms Stochastic programming 

Mathematics Subject Classification (2000)

60B20 60F10 68W20 68W25 68W40 90C15 90C22 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ai W., Zhang S.: Strong duality for the CDT subproblem: a necessary and sufficient condition. SIAM J. Optim. 19(4), 1735–1756 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Alon N., Makarychev K., Makarychev Y., Naor A.: Quadratic forms on graphs. Invent. Math. 163(3), 499–522 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Anstreicher K., Chen X., Wolkowicz H., Yuan Y.X.: Strong duality for a trust-region type relaxation of the quadratic assignment problem. Linear Algebra Appl. 301(1–3), 121–136 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Anstreicher K., Wolkowicz H.: On Lagrangian relaxation of quadratic matrix constraints. SIAM J. Matrix Anal. Appl. 22(1), 41–55 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Arora S., Lee J.R., Naor A.: Euclidean distortion and the parsest cut. J. Am. Math. Soc. 21(1), 1–21 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Arora, S., Rao, S., Vazirani, U.: Expander flows, geometric embeddings and graph partitioning. J. ACM 56(2): Article 5 (2009)Google Scholar
  7. 7.
    Barvinok A.I.: Problems of distance geometry and convex properties of quadratic maps. Discret. Comput. Geom. 13, 189–202 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Beck A., Eldar Y.C.: Strong duality in nonconvex quadratic optimization with two quadratic constraints. SIAM J. Optim. 17(3), 844–860 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Ben-Tal A., Nemirovski A.: On safe tractable approximations of chance-constrained linear matrix inequalities. Math. Oper. Res. 34(1), 1–25 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Buchholz A.: Operator Khintchine inequality in non-commutative probability. Math. Ann. 319, 1–16 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Delage, E., Ye, Y.: Distributionally robust optimization under moment uncertainty with application to data-driven problems. To appear in Oper. Res. (2009)Google Scholar
  12. 12.
    Dupačová, J.: Stochastic programming: minimax approach. In: Floudas, C.A., Pardalos, P.M. Encyclopedia of Optimization, 2nd edn, Springer Science+Business Media, LLC, New York (2009)Google Scholar
  13. 13.
    Goemans M.X.: Semidefinite programming in combinatorial optimization. Math. Program. 79, 143–161 (1997)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Goemans M.X., Williamson D.P.: Improved approximation algorithms for Maximum Cut and satisfiability problems using semidefinite programming. J. ACM 42(6), 1115–1145 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Gower J.C., Dijksterhuis G.B.: Procrustes Problems, Oxford Statistical Science Series, vol. 30. Oxford University Press, New York (2004)Google Scholar
  16. 16.
    Grötschel M., Lovász L., Schrijver A.: Geometric Algorithms and Combinatorial Optimization, Algorithms and Combinatorics, vol. 2, 2nd corrected edn. Springer, Berlin (1993)Google Scholar
  17. 17.
    Horn R.A., Johnson C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)zbMATHGoogle Scholar
  18. 18.
    Karger D., Motwani R., Sudan M.: Approximate graph coloring by semidefinite programming. J. ACM 45(2), 246–265 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Khintchine A.: Über dyadische brüche. Math. Zeit. 23, 109–116 (1923)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Koopmans T.C., Beckmann M.: Assignment problems and the location of economic activities. Econometrica 25(1), 53–76 (1957)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Ledoux M., Talagrand M.: Probability in Banach Spaces: Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge/A Series of Modern Surveys in Mathematics, vol. 23. Springer, Berlin (1991)Google Scholar
  22. 22.
    Li, W.L., Zhang, Y.J., So, A.M.C., Win, M.Z.: Slow adaptive OFDMA through chance constrained programming. Preprint (2009)Google Scholar
  23. 23.
    Luo Z.Q., Sidiropoulos N.D., Tseng P., Zhang S.: Approximation bounds for quadratic optimization with homogeneous quadratic constraints. SIAM J. Optim. 18(1), 1–28 (2007)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Lust-Piquard F.: Inégalités de Khintchine dans C p (1 < p < ∞). Comptes Rendus de l’Académie des Sciences de Paris, Série I 303(7), 289–292 (1986)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Nemirovski A.: Sums of random symmetric matrices and quadratic optimization under orthogonality constraints. Math. Program. Ser. B 109(2–3), 283–317 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Nemirovski A., Roos C., Terlaky T.: On maximization of quadratic form over intersection of ellipsoids with common center. Math. Program. Ser. A 86, 463–473 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Nemirovski A., Shapiro A.: Convex approximations of chance constrained programs. SIAM J. Optim. 17(4), 969–996 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Nemirovski, A., Shapiro, A.: Scenario approximations of chance constraints. In: Calafiore, G., Dabbene, F. Probabilistic and Randomized Methods for Design under Uncertainty, pp. 3–47. Springer, London (2006)Google Scholar
  29. 29.
    Nesterov, Yu.: Quality of semidefinite relaxation for nonconvex quadratic optimization. CORE Discussion Paper 9719, Université Catholique de Louvain, Belgium (1997)Google Scholar
  30. 30.
    Pardalos, P.M., Wolkowicz, H. (eds.): Quadratic Assignment and Related Problems, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 16. American Mathematical Society, Providence, Rhode Island (1994)Google Scholar
  31. 31.
    Pataki G.: On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues. Math. Oper. Res. 23(2), 339–358 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Peshkir G., Shiryaev A.N.: The Khintchine inequalities and martingale expanding sphere of their action. Russ. Math. Surv. 50(5), 849–904 (1995)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Pisier, G.: Non-commutative vector valued L p-spaces and completely p-summing maps. Astérisque, 247 (1998)Google Scholar
  34. 34.
    Quine M.P.: A calculus-based proof of a Stirling formula for the gamma function. Int. J. Math. Educ. Sci. Technol. 28(6), 914–917 (1997)MathSciNetGoogle Scholar
  35. 35.
    Scarf, H.: A min-max solution of an inventory problem. In: Arrow, K.J., Karlin, S., Scarf, H. Studies in the Mathematical Theory of Inventory and Production, pp. 201–209. Stanford University Press, Stanford (1958)Google Scholar
  36. 36.
    Shapiro A.: Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis. Psychometrika 47(2), 187–199 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  37. 37.
    Shenouda, M.B., Davidson, T.N.: Outage-based designs for multi-user transceivers. In: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 2389–2392 (2009)Google Scholar
  38. 38.
    So, A.M.C.: On the performance of semidefinite relaxation MIMO detectors for QAM constellations. In: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 2449–2452 (2009)Google Scholar
  39. 39.
    So, A.M.C.: Probabilistic analysis of the semidefinite relaxation detector in digital communications. To appear in Proceedings of the 21st Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2010)Google Scholar
  40. 40.
    So A.M.C., Ye Y., Zhang J.: A unified theorem on SDP rank reduction. Math. Oper. Res. 33(4), 910–920 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    So A.M.C., Zhang J., Ye Y.: On approximating complex quadratic optimization problems via semidefinite programming relaxations. Math. Program. Ser. B 110(1), 93–110 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Tomczak-Jaegermann N.: The moduli of smoothness and convexity and the Rademacher averages of trace classes S p (1 ≤ p < ∞). Stud. Math. 50, 163–182 (1974)MathSciNetzbMATHGoogle Scholar
  43. 43.
    Tropp J.A.: The random paving property for uniformly bounded matrices. Stud. Math. 185(1), 67–82 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Ye Y.: Approximating global quadratic optimization with convex quadratic constraints. J. Global Optim. 15(1), 1–17 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Ye Y., Zhang S.: New results on quadratic minimization. SIAM J. Optim. 14(1), 245–267 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Zhao Q., Karisch S.E., Rendl F., Wolkowicz H.: Semidefinite programming relaxations for the quadratic assignment problem. J. Comb. Optim. 2(1), 71–109 (1998)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer and Mathematical Programming Society 2009

Authors and Affiliations

  1. 1.Department of Systems Engineering and Engineering ManagementThe Chinese University of Hong KongShatin, N. T.Hong Kong

Personalised recommendations