An Approximate ADMM for Solving Linearly Constrained Nonsmooth Optimization Problems with Two Blocks of Variables

  • Adil M. BagirovEmail author
  • Sona Taheri
  • Fusheng Bai
  • Zhiyou Wu
Part of the International Series of Numerical Mathematics book series (ISNM, volume 170)


Nonsmooth convex optimization problems with two blocks of variables subject to linear constraints are considered. A new version of the alternating direction method of multipliers is developed for solving these problems. In this method the subproblems are solved approximately. The convergence of the method is studied. New test problems are designed and used to verify the efficiency of the proposed method and to compare it with two versions of the proximal bundle method.


Nonsmooth optimization Constrained optimization Convex programming Subgradient methods 

Mathematics Subject Classification (2000)

Primary 90C25; Secondary 49J52 



The authors would like to thank the anonymous referee for valuable comments that helped to improve the quality of this paper.

This research was started when Dr. A.M. Bagirov visited Chongqing Normal University and the visit was supported by this university. The research by A.M. Bagirov and S.Taheri was also supported by Australian Research Council’s Discovery Projects funding scheme (Project No. DP190100580).


  1. 1.
    A. Bagirov, N. Karmitsa, and M. Makela. Introduction to Nonsmooth Optimization: Theory, Practice and Software. Springer, Cham, 2014.CrossRefGoogle Scholar
  2. 2.
    A. Bnouhachem, M.H. Xu, M. Khalfaoui, and Sh. Zhaohana. A new alternating direction method for solving variational inequalities. Computers and Mathematics with Applications, 62 (2011), 626–634.MathSciNetCrossRefGoogle Scholar
  3. 3.
    S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1) (2011), 1–122.CrossRefGoogle Scholar
  4. 4.
    Sh. Cao, Y. Xiao, and H. Zhu. Linearized alternating directions method for l 1 -norm inequality constrained l 1 -norm minimization. Applied Numerical Mathematics, 85 (2014), 142–153.MathSciNetCrossRefGoogle Scholar
  5. 5.
    F.H. Clarke. Optimization and Nonsmooth Analysis. Canadian Mathematical Society series of monographs and advanced texts. Wiley-Interscience, 1983.zbMATHGoogle Scholar
  6. 6.
    E.D. Dolan and J.J. Moré. Benchmarking optimization software with performance profiles. Mathematical Programming, 91(2) (2002), 201–213.MathSciNetCrossRefGoogle Scholar
  7. 7.
    J. Eckstein and W. Yao. Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results. In RUTCOR Research Reports, 32. 2012.Google Scholar
  8. 8.
    M. El Anbari, S. Alam, and H. Bensmail. COFADMM: A computational features selection with alternating direction method of multipliers. Procedia Computer Science, 29 (2014), 821–830.Google Scholar
  9. 9.
    M. Fukushima. Application of the alternating direction method of multipliers to separable convex programming problems. Computational Optimization and Applications, 1(1) (1992), 93–111.MathSciNetCrossRefGoogle Scholar
  10. 10.
    D. Gabay and B. Mercier. A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Computers and Mathematics with Applications, 2 (1976) 17–40.CrossRefGoogle Scholar
  11. 11.
    R. Glowinski and A. Marrocco. Sur l approximation par éléments finis d ordre 1 et la résolution par pénalisation-dualité d une classe de problémes de dirichlet. RAIRO, 2 (1975), 41–76.Google Scholar
  12. 12.
    D. Han and H.K. Lo. A new stepsize rule in He and Zhou’s alternating direction method. Applied Mathematics Letters, 15 (2002), 181–185.MathSciNetCrossRefGoogle Scholar
  13. 13.
    B. He and J. Zhou. A modified alternating direction method for convex minimization problems. Applied Mathematics Letters, 13 (2000), 123–130.MathSciNetCrossRefGoogle Scholar
  14. 14.
    M.R. Hestenes. Multiplier and gradient methods. Journal of Optimization Theory and Applications, 4 (1969) 302–320.MathSciNetzbMATHGoogle Scholar
  15. 15.
    K.C. Kiwiel. An alternating linearization bundle method for convex optimization and nonlinear multicommodity flow problems. Mathematical Programming, Ser. A, 130 (2011), 59–84.Google Scholar
  16. 16.
    X. Li, L. Mo, X. Yuan, and J. Zhang. Linearized alternating direction method of multipliers for sparse group and fused lasso models. Computational Statistics and Data Analysis, 79 (2014), 203–221.MathSciNetCrossRefGoogle Scholar
  17. 17.
    L. Luks̈an and J. Vlc̈ek. Algorithm 811: NDA: algorithms for nondifferentiable optimization. ACM Transactions on Mathematical Software, 27(2) (2001), 193–213.Google Scholar
  18. 18.
    M.J.D. Powell. A method for nonlinear constraints in minimization problems. In R. Fletcher, editor, Optimization, pages 283–298. Academic Press, NY, 1969.Google Scholar
  19. 19.
    A. Rakotomamonjy. Applying alternating direction method of multipliers for constrained dictionary learning. Neurocomputing, 106 (2013), 126–136.CrossRefGoogle Scholar
  20. 20.
    Y. Shen and M.H. Xu. On the O(1∕t) convergence rate of Ye–Yuan’s modified alternating direction method of multipliers. Applied Mathematics and Computation, 226 (2014), 367–373.MathSciNetCrossRefGoogle Scholar
  21. 21.
    J. Sun and S. Zhang. A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. European Journal of Operational Research, 207 (2010), 1210–1220.MathSciNetCrossRefGoogle Scholar
  22. 22.
    K. Zhao and G. Yao. Application of the alternating direction method for an inverse monic quadratic eigenvalue problem. Applied Mathematics and Computation, 244 (2014), 32–41.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Adil M. Bagirov
    • 1
    Email author
  • Sona Taheri
    • 1
  • Fusheng Bai
    • 2
  • Zhiyou Wu
    • 2
  1. 1.School of Science, Engineering and Information TechnologyFederation University AustraliaMount Helen BallaratAustralia
  2. 2.School of Mathematical SciencesChongqing Normal UniversityChongqingChina

Personalised recommendations