Conjugate Gradient Methods

Abstract

In the following, A ∈ ℝ^{I x I} and b ∈ ℝ^I are real. We consider a system

$$ Ax\, = \,b $$

(9.1.1)

and assume that

$$ A\,is\,positive\,definite. $$

(9.1.2)

System (1) is associated with the function

$$ F\left( x \right): = \,\frac{1}{2}\left\langle {Ax,\,x} \right\rangle \, - \,\left\langle {b,\,x} \right\rangle . $$

(9.1.3)

The derivative (gradient) of F is $F'\left( x \right): = \,\frac{1}{2}\left( {A\, + \,{A^T}} \right)x\, - \,b$. Since A = A^T by assumption (2), the derivative equals

$$ F{\text{'}}'\left( x \right)\, = \,grad\,F\left( x \right)\, = \,Ax\, - \,b. $$

(9.1.4)