Skip to main content
SpringerLink
Log in

A generalized projection-based scheme for solving convex constrained optimization problems

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper we present a new algorithmic realization of a projection-based scheme for general convex constrained optimization problem. The general idea is to transform the original optimization problem to a sequence of feasibility problems by iteratively constraining the objective function from above until the feasibility problem is inconsistent. For each of the feasibility problems one may apply any of the existing projection methods for solving it. In particular, the scheme allows the use of subgradient projections and does not require exact projections onto the constraints sets as in existing similar methods. We also apply the newly introduced concept of superiorization to optimization formulation and compare its performance to our scheme. We provide some numerical results for convex quadratic test problems as well as for real-life optimization problems coming from medical treatment planning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Agmon, S.: The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amelunxen, D.: Geometric analysis of the condition of the convex feasibility problem. Ph. D. Dissertation, University of Paderborn (2011)

  3. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, Berlin (2017)

    Book  MATH  Google Scholar 

  5. Bauschke, H.H., Koch, V.R.: Projection methods: Swiss army knives for solving feasibility and best approximation problems with halfspaces. Contemp. Math. 636, 1–40 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 3rd edn. Wiley, Hoboken (2006)

    Book  MATH  Google Scholar 

  7. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  8. Blatt, D., Hero III, A.O.: Energy based sensor network source localization via projection onto convex sets (POCS). IEEE Trans. Signal Process. 54, 3614–3619 (2006)

    Article  MATH  Google Scholar 

  9. Boyd, S.P., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  10. Boyd, S., Vandenberghe, L.: Localization and Cutting-Plane Methods, Lecture Topics and Notes (2008). https://see.stanford.edu/materials/lsocoee364b/05-localization_methods_notes.pdf

  11. Boyd, S., Park, J.: Subgradient Methods, Notes for EE364b, Stanford University, (2014). https://stanford.edu/class/ee364b/lectures/subgrad_method_notes.pdf

  12. Butnariu, D., Censor, Y., Gurfil, P., Hadar, E.: On the behavior of subgradient projections methods for convex feasibility problems in Euclidean spaces. SIAM J. Optim. 19, 786–807 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Butnariu, D., Davidi, R., Herman, G.T., Kazantsev, I.G.: Stable convergence behavior under summable perturbations of a class of projection methods for convex feasibility and optimization problems. IEEE J. Sel. Top. Signal Process. 1, 540–547 (2007)

    Article  Google Scholar 

  14. Byrne, C.: Iterative projection onto convex sets using multiple Bregman distances. Inverse Probl. 15, 1295–1313 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  15. Byrne, C.: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 20, 103–120 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cegielski, A.: A method of projection onto an acute cone with level control in convex minimization. Math. Program. 85, 469–490 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  17. Cegielski, A., Censor, Y.: Projection methods: an annotated bibliography of books and reviews. Optimization 64, 2343–2358 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  18. Cegielski, A., Dylewski, R.: Selection strategies in a projection method for convex minimization problems. Discuss. Math. Differ. Incl. Control Optim. 22, 97–123 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Cegielski, A., Dylewski, R.: Residual selection in a projection method for convex minimization problems. Optimization 52, 211–220 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  20. Censor, Y.: Weak and strong superiorization: Between feasibility-seeking and minimization. Analele Stiintifice ale Universitatii Ovidius Constanta-Seria Matematica 23, 41–54 (2015)

    MathSciNet  MATH  Google Scholar 

  21. Censor, Y.: Superiorization and Perturbation Resilience of Algorithms: A Bibliography compiled and continuously updated by Yair Censor, http://math.haifa.ac.il/yair/bib-superiorization-censor.html. 2 March 2017. arXiv:1506.04219 [math.OC] ([v2], 9 Mar 2017)

  22. Censor, Y., Altschuler, M.D., Powlis, W.D.: On the use of Cimmino’s simultaneous projections method for computing a solution of the inverse problem in radiation therapy treatment planning. Inverse Probl. 4, 607–623 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  23. Censor, Y., Chen, W., Combettes, P.L., Davidi, R., Herman, G.T.: On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints. Comput. Optim. Appl. 51, 1065–1088 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  24. Censor, Y., Chen, W., Pajoohesh, H.: Finite convergence of a subgradient projections method with expanding controls. Appl. Math. Optim. 64, 273–285 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  25. Censor, Y., Davidi, R., Herman, G.T.: Perturbation resilience and superiorization of iterative algorithms. Inverse Probl. 26, 065008 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Censor, Y., Elfving, T., Kopf, N., Bortfeld, T.: The multiple-sets split feasibility problem and its applications for inverse problems. Inverse Probl. 21, 2071–2084 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  27. Censor, Y., Herman, G.T., Jiang, M.: Special issue on superiorization: theory and applications. Inverse Probl. 33 (2017)

  28. Censor, Y., Lent, A.: Cyclic subgradient projections, Mathematics Publication Series, Report No 35. University of Haifa, Department of Mathematics (1981)

  29. Censor, Y., Lent, A.: Cyclic subgradient projections. Math. Program. 24, 233–235 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  30. Censor, Y., Reem, D.: Zero-convex functions, perturbation resilience, and subgradient projections for feasibility-seeking methods. Math. Program. (Ser. A) 152, 339–380 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  31. Censor, Y., Tom, E.: Convergence of string-averaging projection schemes for inconsistent convex feasibility problems. Optim. Methods Softw. 18, 543–554 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  32. Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms and Applications. Oxford University Press, New York (1997)

    MATH  Google Scholar 

  33. Censor, Y., Zenios, S.A.: Interval-constrained matrix balancing. Linear Algebra Appl. 150, 393–421 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  34. Censor, Y., Zur, Y.: Linear superiorization for infeasible linear programming. In: Kochetov, Y., Khachay, M., Beresnev, V., Nurminski, E., Pardalos P. (eds.) Discrete Optimization and Operations Research. Lecture Notes in Computer Science (LNCS), vol. 9869, Springer International Publishing, pp. 15–24 (2016)

  35. Cetin, A.E., Ozaktas, H., Ozaktas, H.M.: Resolution enhancement of low resolution wavefields with POCS algorithm. Electron. Lett. 39, 1808–1810 (2003)

    Article  Google Scholar 

  36. Cimmino, G.: Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. La Ricerca Scientifica (Roma) 1, 326–333 (1938)

    MATH  Google Scholar 

  37. Chen, W., Craft, D., Madden, T.M., Zhang, K., Kooy, H.M., Herman, G.T.: A fast optimization algorithm for multi-criteria intensity modulated proton therapy planning. Med. Phys. 37, 4938–4945 (2010)

    Article  Google Scholar 

  38. Chinneck, J.W.: Feasibility and Infeasibility in Optimization: Algorithms and Computational Methods. International Series in Operations Research and Management Science, 118. Springer, New York (2008)

    MATH  Google Scholar 

  39. CoinUtils package availble from https://projects.coin-or.org/CoinUtils, 22 April 2013

  40. Combettes, P.L., Luo, J.: An adaptive level set method for nondifferentiable constrained image recovery. IEEE Trans. Image Process. 11, 1295–1304 (2002)

    Article  MathSciNet  Google Scholar 

  41. De Pierro, A.R., Iusem, A.N.: A finitely convergent “row-action” method for the convex feasibility problem. Appl. Math. Optim. 17, 225–235 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  42. De Pierro, A.R., Helou Neto, E.S.: From convex feasibility to convex constrained optimization using block action projection methods and underrelaxation. Int. Trans. Oper. Res. 16, 495–504 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  43. Elzinga, J., Moore, T.G.: A central cutting plane algorithm for the convex programming problem. Math. Program. Stud. 8, 134–145 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  44. Fukushima, M.: A finitely convergent algorithm for convex inequalities. IEEE Trans. Autom. Control 27, 1126–1127 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  45. Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. J. Theor. Biol. 29, 471–481 (1970)

    Article  Google Scholar 

  46. Gubin, L.G., Polyak, B.T., Raik, E.V.: The method of projections for finding a common point of convex sets. Comput. Math. Math. Phys. 7, 1–24 (1967)

    Article  Google Scholar 

  47. Goffin, J.-L., Kiwiel, K.C.: Convergence of a simple subgradient level method. Math. Program. 85, 207–211 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  48. Goffin, J.-L., Luo, Z.-Q., Ye, Y.: Complexity analysis of an interior cutting plane method for convex feasibility problems. SIAM J. Optim. 6, 638–652 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  49. Herman, G.T.: Superiorization for image analysis. In: Combinatorial Image Analysis, Lecture Notes in Computer Science, vol. 8466, pp. 1–7. Springer (2014)

  50. Herman, G.T., Chen, W.: A fast algorithm for solving a linear feasibility problem with application to intensity-modulated radiation therapy. Linear Algebra Appl. 428, 1207–1217 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  51. Herman, G.T., Garduňo, E., Davidi, R., Censor, Y.: Superiorization: an optimization heuristic for medical physics. Med. Phys. 39, 5532–5546 (2012)

    Article  Google Scholar 

  52. Herman, G.T., Meyer, L.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging 12, 600–609 (1993)

    Article  Google Scholar 

  53. Hiriart-Urruty, J.-B., Lemaréchal, C.: Fundamentals of Convex Analysis. Springer, Berlin (2001)

    Book  MATH  Google Scholar 

  54. Iusem, A.N., Moledo, L.: On finitely convergent iterative methods for the convex feasibility problem. Bull. Braz. Math. Soc. 18, 11–18 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  55. Kaczmarz, S.: Angenöherte Auflösung von Systemen linearer Gleichungen. Bulletin de l’Académie Polonaise des Sciences at Lettres A 35, 355–357 (1937)

    MATH  Google Scholar 

  56. Kelley, J.E.: The cutting-plane method for solving convex programs. J. Soc. Ind. Appl. Math. 8, 703–712 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  57. Khabibullin, R.F.: On a method to find a point of a convex set. J. Sov. Math. 39, 2958–2963 (1987). Translated from Issledovaniya po Prikladnoi Matematike 4, 23–30 (1977)

  58. Khabibullin, R.F.: Generalized descent method for minimization of functionals. J. Sov. Math. 39, 2963–2968 (1987). Translated from Issledovaniya po Prikladnoi Matematike 4, 23–30 (1977)

  59. Kim, S., Ahn, H., Cho, S.-C.: Variable target value subgradient method. Math. Program. 49, 359–369 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  60. Kiwiel, K.C.: The efficiency of subgradient projection methods for convex optimization, part I: general level methods and part II: implementations and extensions. SIAM J. Control Optim. 34, 660–697 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  61. Konnov, I.V.: A combined relaxation method for variational inequalities with nonlinear constraints. Math. Program. 80, 239–252 (1998)

    MathSciNet  MATH  Google Scholar 

  62. Küfer, K.-H., Monz, M., Scherrer, A., Süss, P., Alonso, F.V., Azizi Sultan, A.S., Bortfeld, T., Thieke, C.: Multicriteria optimization in intensity modulated radiotherapy planning. In: Handbook of Optimization in Medicine (Springer Optimization and Its Applications), vol. 26, pp. 123–167 (2009)

  63. Kulikov, A.N., Fazylov, V.R.: A finite method to find a point in a set defined by a convex differentiable functional. J. Sov. Math. 45, 1273–1277 (1989)

    Article  MATH  Google Scholar 

  64. Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  65. Maros, I., Mészáros, Cs: A repository of convex quadratic programming problems. Optim. Methods Softw. 11, 671–681 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  66. Mayne, D.Q., Polak, E., Heunis, A.J.: Solving nonlinear inequalities in a finite number of iterations. J. Optim. Theory Appl. 33, 207–221 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  67. Motzkin, T.S., Schoenberg, I.J.: The relaxation method for linear inequalities. Can. J. Math. 6, 393–404 (1954)

    Article  MathSciNet  MATH  Google Scholar 

  68. Pang, C.H.J.: Set intersection problems: supporting hyperplanes and quadratic programming. Math. Program. (Ser. A) 149, 329–359 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  69. Pang, C.H.J.: SHDQP: an algorithm for convex set intersection problems based on supporting hyperplanes and dual quadratic programming (2013). arXiv:1307.0053

  70. Polak, E., Mayne, D.Q.: On the finite solution of nonlinear inequalities. IEEE Trans. Autom. Control 24, 443–445 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  71. Reem, D., De Pierro, A.: A new convergence analysis and perturbation resilience of some accelerated proximal forward-backward algorithms with errors. Inverse Probl. 33, 044001 (2017). arXiv:1508.05631 [math.OC], 2015 (current version: [v3], 29 Jun 2016)

  72. Scherrer, A., Yaneva, F., Grebe, T., Kfer, K.-H.: A new mathematical approach for handling DVH criteria in IMRT planning. J. Glob. Optim. 61, 407–428 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  73. van Tiel, J.: Convex Analysis: An Introductory Text. Wiley, Belfast (1984)

    MATH  Google Scholar 

  74. van Wyk, B.J., van Wyk, M.A.: A POCS-based graph matching algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1526–1530 (2004)

    Article  Google Scholar 

  75. Yamagishi, M., Yamada, I.: A deep monotone approximation operator based on the best quadratic lower bound of convex functions. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E91–A, 1858–1866 (2008)

    Article  Google Scholar 

  76. Zenios, S.A., Censor, Y.: Massively parallel row-action algorithms for some nonlinear transportation problems. SIAM J. Optim. 1, 373–400 (1991)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We wish to thank the referees for their thorough analysis and review, all their comments and suggestions helped tremendously in improving the quality of this paper and made it suitable for publication. In addition we thank the Associate Editor for his time and effort invested in handling our paper and providing useful remarks. Last but not least, we wish to thank Prof. Yair Censor for his helpful comments and providing useful references. This work was supported by the Federal Ministry of Education and Research of Germany (BMBF), Grant No. 01IB13001 (SPARTA). The first author’s work was also supported by ORT Braude College and the Galilee Research Center for Applied Mathematics, ORT Braude College.

Author information

Authors and Affiliations

  1. Department of Mathematics, ORT Braude College, P.O. Box 78, 2161002, Karmiel, Israel

    Aviv Gibali

  2. Optimization Department, Fraunhofer - ITWM, 67663, Kaiserslautern, Germany

    Karl-Heinz Küfer & Philipp Süss

  3. Technion - Israel Institute of Technology, 3200003, Haifa, Israel

    Daniel Reem

Authors
  1. Aviv Gibali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karl-Heinz Küfer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Reem
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philipp Süss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aviv Gibali.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gibali, A., Küfer, KH., Reem, D. et al. A generalized projection-based scheme for solving convex constrained optimization problems. Comput Optim Appl 70, 737–762 (2018). https://doi.org/10.1007/s10589-018-9991-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-9991-4

Keywords

Mathematics Subject Classification

Search

Navigation