Skip to main content
SpringerLink
Log in

Column-oriented algebraic iterative methods for nonnegative constrained least squares problems

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

This paper considers different versions of block-column iterative (BCI) methods for solving nonnegative constrained linear least squares problems. We present the convergence analysis for a family of stationary BCI methods with nonnegativity constraints (BCI-NC), which is applicable to linear complementarity problems (LCP). We also consider the flagging idea for BCI methods, which allows saving computational work by skipping small updates. Also, we combine the BCI-NC algorithm and the flagging version of a nonstationary BCI method with nonnegativity constraints to derive a convergence analysis for the resulting method (BCI-NF). The performance of our algorithms is shown on ill-posed inverse problems taken from tomographic imaging. We compare the BCI-NF and BCI-NC algorithms with three recent algorithms: the inner-outer modulus method (Modulus-CG method), the modulus-based iterative method to Tikhonov regularization with nonnegativity constraint (Mod-TRN method), and nonnegative flexible CGLS (NN-FCGLS) method. Our algorithms are able to produce more stable results than the mentioned methods with competitive computational times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Andersen, A.H., Kak, A.C.: Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm. Ultrason. Imaging 6(1), 81–94 (1984)

    Article  Google Scholar 

  2. Bai, Z.Z.: Modulus-based matrix splitting iteration methods for linear complementarity problems. Numer. Linear Algebra Appl. 17(6), 917–933 (2010)

    Article  MathSciNet  Google Scholar 

  3. Bai, Z.Z., Buccini, A., Hayami, K., Reichel, L., Yin, J.F., Zheng, N.: Modulus-based iterative methods for constrained tikhonov regularization. J. Comput. Appl. Math. 319, 1–13 (2017)

    Article  MathSciNet  Google Scholar 

  4. Bai, Z.Z., Jin, C.H.: Column-decomposed relaxation methods for the overdetermined systems of linear equations. Int. J. Appl. Math. 13(1), 71–82 (2003)

    MathSciNet  MATH  Google Scholar 

  5. Beck, A., Tetruashvili, L.: On the convergence of block coordinate descent type methods. SIAM J. Opt. 23(4), 2037–2060 (2013)

    Article  MathSciNet  Google Scholar 

  6. Bierlaire, M., Toint, P.L., Tuyttens, D.: On iterative algorithms for linear least squares problems with bound constraints. Linear Algebra Appl. 143, 111–143 (1991)

    Article  MathSciNet  Google Scholar 

  7. Björck, Å.: Numerical methods for least squares problems, vol. 51, SIAM (1996)

  8. Björck, Å., Elfving, T.: Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations. BIT Numer. Math. 19(2), 145–163 (1979)

    Article  MathSciNet  Google Scholar 

  9. Censor, Y.: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23(4), 444–466 (1981)

    Article  MathSciNet  Google Scholar 

  10. Censor, Y., Elfving, T.: Block-iterative algorithms with diagonally scaled oblique projections for the linear feasibility problem. SIAM J. Matrix Anal. Appl. 24 (1), 40–58 (2002)

    Article  MathSciNet  Google Scholar 

  11. Censor, Y., Elfving, T., Herman, G.T., Nikazad, T.: On diagonally relaxed orthogonal projection methods. SIAM J. Sci. Comput. 30(1), 473–504 (2008)

    Article  MathSciNet  Google Scholar 

  12. Censor, Y., Gordon, D., Gordon, R.: BICAV: a block-iterative parallel algorithm for sparse systems with pixel-related weighting. IEEE Trans. Med. Imaging 20(10), 1050–1060 (2001)

    Article  Google Scholar 

  13. Censor, Y., Gordon, D., Gordon, R.: Component averaging: an efficient iterative parallel algorithm for large and sparse unstructured problems. Parallel Comput. 27(6), 777–808 (2001)

    Article  MathSciNet  Google Scholar 

  14. Censor, Y., Zenios, S.A.: Parallel optimization: theory, algorithms, and applications. Oxford University Press on Demand (1997)

  15. Cimmino, G.: Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari Istituto per le applicazioni del calcolo (1938)

  16. Cottle, R.W., Pang, J.S., Stone, R.E.: The linear complementarity problem. SIAM (2009)

  17. Dong, J.L., Gao, J., Ju, F., Shen, J.: Modulus methods for nonnegatively constrained image restoration. SIAM J. Imaging Sci. 9(3), 1226–1246 (2016)

    Article  MathSciNet  Google Scholar 

  18. Dong, J.L., Jiang, M.Q.: A modified modulus method for symmetric positive-definite linear complementarity problems. Numer. Linear Algebr. Appl. 16 (2), 129–143 (2009)

    Article  MathSciNet  Google Scholar 

  19. Elfving, T.: Block-iterative methods for consistent and inconsistent linear equations. Numer. Math. 35(1), 1–12 (1980)

    Article  MathSciNet  Google Scholar 

  20. Elfving, T., Hansen, P.C., Nikazad, T.: Semiconvergence and relaxation parameters for projected SIRT algorithms. SIAM J. Sci. Comput. 34(4), A2000–A2017 (2012)

    Article  MathSciNet  Google Scholar 

  21. Elfving, T., Hansen, P.C., Nikazad, T.: Semi-convergence properties of Kaczmarz’s method. Inverse Probl. 30(5), 055,007 (2014)

    Article  MathSciNet  Google Scholar 

  22. Elfving, T., Hansen, P.C., Nikazad, T.: Convergence analysis for column-action methods in image reconstruction. Numer. Algorithms 74(3), 905–924 (2017)

    Article  MathSciNet  Google Scholar 

  23. Elfving, T., Nikazad, T.: Properties of a class of block-iterative methods. Inverse Probl. 25(11), 115,011 (2009)

    Article  MathSciNet  Google Scholar 

  24. Elfving, T., Nikazad, T., Hansen, P.C.: Semi-convergence and relaxation parameters for a class of SIRT algorithms. Electron. Trans. Numer. Anal. 37, 321–336 (2010)

    MathSciNet  MATH  Google Scholar 

  25. Escalante, R., Raydan, M.: Alternating projection methods. SIAM (2011)

  26. Garduño, E., Herman, G.T., Davidi, R.: Reconstruction from a few projections by ℓ1-minimization of the Haar transform. Inverse Probl. 27(5), 055,006 (2011)

    Article  MathSciNet  Google Scholar 

  27. Gazzola, S., Hansen, P.C., Nagy, J.G.: IR Tools: a MATLAB package of iterative regularization methods and large-scale test problems. Numerical Algorithm, pp. 1–39. https://doi.org/10.1007/s11075-018-0570-7 (2018)

  28. Gazzola, S., Wiaux, Y.: Fast nonnegative least squares through flexible Krylov subspaces. SIAM J. Sci. Comput. 39(2), A655–A679 (2017)

    Article  MathSciNet  Google Scholar 

  29. Golub, G., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis 2(2), 205–224 (1965)

    Article  MathSciNet  Google Scholar 

  30. Haltmeier, M.: Convergence analysis of a block iterative version of the loping landweber–kaczmarz iteration. Nonlinear Anal. Theory Methods Appl. 71(12), e2912–e2919 (2009)

    Article  MathSciNet  Google Scholar 

  31. Hansen, P.C., Jørgensen, J.S.: AIR Tools II: algebraic iterative reconstruction methods, improved implementation. Numer. Algorithms 79(1), 107–137 (2018)

    Article  MathSciNet  Google Scholar 

  32. Herman, G.T.: Fundamentals of computerized tomography: image reconstruction from projections. Springer Science & Business Media (2009)

  33. Herman, G.T., Davidi, R.: Image reconstruction from a small number of projections. Inverse Probl. 24(4), 045,011 (2008)

    Article  MathSciNet  Google Scholar 

  34. Jiang, M., Wang, G.: Convergence studies on iterative algorithms for image reconstruction. IEEE Trans. Med. Imaging 22(5), 569–579 (2003)

    Article  Google Scholar 

  35. Kaczmarz, S.: Angenäherte auflösung von systemen linearer gleichungen. Bulletin International de l’ Académie Polonaise des Sciences et des Lettres 35, 355–357 (1937)

    MATH  Google Scholar 

  36. Klarbring, A.: Quadratic programs in frictionless contact problems. Int. J. Eng. Sci. 24(7), 1207–1217 (1986)

    Article  MathSciNet  Google Scholar 

  37. Nagy, J., Strakos, Z.: Enforcing nonnegativity in image reconstruction algorithms. Inverse Problems Estimation, and Imaging 4121, 182–190 (2000)

    Google Scholar 

  38. Natterer, F.: The Mathematics of Computerized Tomography. Wiley, New York (1986)

    MATH  Google Scholar 

  39. Nikazad, T., Abbasi, M.: A unified treatment of some perturbed fixed point iterative methods with an infinite pool of operators. Inverse Probl. 33(4), 044,002 (2017)

    Article  MathSciNet  Google Scholar 

  40. Nikazad, T., Abbasi, M., Elfving, T.: Error minimizing relaxation strategies in Landweber and Kaczmarz type iterations. J. Inverse Ill-Posed Probl. https://doi.org/10.1515/jiip-2015-0082 (2015)

  41. Nikazad, T., Karimpour, M.: Controlling noise error in block iterative methods. Numerical Algorithms, pp. 1–19 (2016)

  42. Watt, D.W.: Column-relaxed algebraic reconstruction algorithm for tomography with noisy data. Appl. Opt. 33(20), 4420–4427 (1994)

    Article  Google Scholar 

  43. Wright, S.J.: Coordinate descent algorithms. Math. Program. 151(1), 3–34 (2015)

    Article  MathSciNet  Google Scholar 

  44. Zheng, N., Hayami, K., Yin, J.F.: Modulus-type inner outer iteration methods for nonnegative constrained least squares problems. SIAM Journal on Matrix Analysis and Applications 37(3), 1250–1278 (2016)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, Iran University of Science and Technology, 16846-13114, Tehran, Iran

    T. Nikazad & M. Karimpour

Authors
  1. T. Nikazad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Karimpour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Nikazad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nikazad, T., Karimpour, M. Column-oriented algebraic iterative methods for nonnegative constrained least squares problems. Numer Algor 86, 1265–1284 (2021). https://doi.org/10.1007/s11075-020-00932-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-020-00932-7

Keywords

Mathematics Subject Classification (2010)

Search

Navigation