Linear Algebra Review
Linear algebra is a broad topic in mathematics with a wide range of applications in engineering. In particular, it is the main prerequisite of optimization. To satisfy this requirement, this chapter provides an intensive review of linear algebra with the aim of covering mathematical fundamentals in the next chapters. The chapter begins with an introduction to vector and matrix spaces. It then employs this introduction to analyze the solution of a set of linear equations which often appear in constraints of optimization problems. The chapter then discusses eigenvalues, eigenvectors, matrix diagonalization, and quadratic forms and introduces positive and negative definite properties accordingly. Some calculus concepts such as level set, gradient, Hessian, and directional derivative with a broad range of applications in optimization are finally presented.