Journal of Optimization Theory and Applications

, Volume 54, Issue 3, pp 471–477 | Cite as

Local convergence analysis of a grouped variable version of coordinate descent

  • J. C. Bezdek
  • R. J. Hathaway
  • R. E. Howard
  • C. A. Wilson
  • M. P. Windham
Contributed Papers


LetF(x,y) be a function of the vector variablesxR n andyR m . One possible scheme for minimizingF(x,y) is to successively alternate minimizations in one vector variable while holding the other fixed. Local convergence analysis is done for this vector (grouped variable) version of coordinate descent, and assuming certain regularity conditions, it is shown that such an approach is locally convergent to a minimizer and that the rate of convergence in each vector variable is linear. Examples where the algorithm is useful in clustering and mixture density decomposition are given, and global convergence properties are briefly discussed.

Key Words

Coordinate descent local linear convergence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zangwill, W.,Nonlinear Programming: A Unified Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1969.Google Scholar
  2. 2.
    Stewart, G. W.,Introduction to Matrix Computations, Academic Press, Orlando, Florida, 1973.Google Scholar
  3. 3.
    Bezdek, J. C.,Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, New York, 1981.Google Scholar
  4. 4.
    Hathaway, R. J.,Another Interpretation of the EM Algorithm, Statistics and Probability Letters, Vol. 4, pp. 53–56, 1986.Google Scholar
  5. 5.
    Redner, R. A., andWalker, H. F.,Mixture Densities, Maximum Likelihood, and the EM Algorithm, SIAM Review, Vol. 26, pp. 195–239, 1984.Google Scholar
  6. 6.
    Dennis, J. E., Jr., andSteihaug, T.,On the Successive Projections Approach to Least-Squares Problems, Technical Report No. 83-18, Department of Mathematical Sciences, Rice University, Houston, Texas, 1983.Google Scholar

Copyright information

© Plenum Publishing Corporation 1987

Authors and Affiliations

  • J. C. Bezdek
    • 1
  • R. J. Hathaway
    • 2
  • R. E. Howard
    • 3
  • C. A. Wilson
    • 4
  • M. P. Windham
    • 5
  1. 1.Department of Computer ScienceUniversity of South CarolinaColumbia
  2. 2.Department of StatisticsUniversity of South CarolinaColumbia
  3. 3.Department of MathematicsUniversity of South CarolinaColumbia
  4. 4.Department of MathematicsWinthrop CollegeRock Hill
  5. 5.Department of MathematicsUtah State UniversityLogan

Personalised recommendations