Skip to main content

Linear Algebra

  • 1147 Accesses

Abstract

Linear algebra is the basis of logic constructions in any science. In this chapter, we learn about inverse matrices, determinants, linear independence, vector spaces and their dimensions, eigenvalues and eigenvectors, orthonormal bases and orthogonal matrices, and diagonalizing symmetric matrices. In this book, to understand the essence concisely, we define ranks and determinants based on the notion of Gaussian elimination and consider linear spaces and their inner products within the range of the Euclidean space and the standard inner product. By reading this chapter, the readers should solve the reasons why.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-981-15-7568-6_1
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   39.99
Price excludes VAT (USA)
  • ISBN: 978-981-15-7568-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   49.99
Price excludes VAT (USA)

Notes

  1. 1.

    In general, any subset V that satisfies (1.2) \(\mathbb R\) is said to be a vector space with scalars in \(\mathbb R\).

  2. 2.

    In general, for vector spaces V and W, we say that \(f: V\rightarrow W\) is a linear map if \(f(x+y)=f(x)+f(y)\), where \(\ x,y\in V,\ f(ax)=af(x),\ a\in {\mathbb R}\) and \(\ x\in V\).

  3. 3.

    In general, we say that the map \((\cdot ,\cdot )\) is an inner product of V if \((u+u',v)=(u,v)+(u',v)\), \((cu,v)=c(u,v)\), \((u,v)=(u',v)\), and \(u\not =0 \Longrightarrow (u,u)>0\) for \(u,v\in V\), where \(c\in {\mathbb R}\).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Joe Suzuki .

Appendix: Proof of Propositions

Appendix: Proof of Propositions

Proposition 2

For square matrices A and B of the same size, we have \(\det (AB)=\det (A)\det (B)\) and \(\det (A^T)=\det (A)\).

Proof

For steps 1, 2, and 3, we multiply the following matrices from left:  

\(V_i(\alpha )\)::

a unit matrix where the (ii)th element has been replaced with \(\alpha \)

\(U_{i,j}\)::

a unit matrix where the (ii), (jj)th and (ij), (ji)th elements have been replaced by zero and one, respectively.

\(W_{i,j}(\beta )\)::

a unit matrix where the (ij)th zero (\(i\not =j\)) has been replaced by \(-\beta \).

  Then, for \(B\in {\mathbb R}^{n\times n}\),

$$\begin{aligned} \det (V_i(\alpha )B)=\alpha \det (B) ,\ \det (U_{i,j}B)=-\det (B) ,\ \det (W_{i,j}(\beta )B)=\det (B). \end{aligned}$$
(1.4)

Since

$$\begin{aligned} \det (V_i(\alpha ))=\alpha ,\ \det (U_{i,j})=-1 ,\ \det (W_{i,j}(\beta ))=1 \end{aligned}$$
(1.5)

holds, if we write matrix A as the product \(E_1,\ldots ,E_r\) of matrices of the three types, then we have

$$\det (A)=\det (E_1)\ldots \det (E_r)\ .$$
$$\begin{aligned} \det (AB)= & {} \det (E_1\cdot E_2\ldots E_rB)=\det (E_1)\det (E_2\ldots E_rB)=\ldots \\= & {} \det (E_1)\ldots \det (E_r)\det (B)=\det (A)\det (B). \end{aligned}$$

On the other hand, since matrices \(V_i(\alpha )\) and \(U_{i,j}\) are symmetric and \(W_{i,j}(\beta )^T=W_{j,i}(\beta )\), we have a similar equation to (1.4) and (1.5). Hence, we have

$$\det (A^T)=\det (E_r^T\ldots E_1^T)=\det (E_r^T)\ldots \det (E_1^T)=\det (E_1)\ldots \det (E_r)=\det (A)\ .$$

Proposition 4

Let V and W be subspaces of \({\mathbb R}^n\) and \({\mathbb R}^m\), respectively. The image and kernel of the linear map \(V\rightarrow W\) w.r.t. \(A\in {\mathbb R}^{m\times n}\) are subspaces of W and V, respectively, and the sum of the dimensions is n. The dimension of the image coincides with the rank of A.

Proof

Let r and \(x_1,\ldots ,x_r\in V\) be the dimension and basis of the kernel, respectively. We add \(x_{r+1},\ldots ,x_n\), which are linearly independent of them, so that \(x_1,\ldots ,x_r,x_{r+1},\ldots ,x_n\) are the bases of V. It is sufficient to show that \(Ax_{r+1},\ldots ,Ax_n\) are the bases of the image.

First, since \(x_1,\ldots ,x_r\) are vectors in the kernel, we have \(Ax_1=\cdots =Ax_r=0\). For an arbitrary \(x=\sum _{j=1}^n b_jx_j\) with \(b_{r+1},\ldots ,b_n\in {\mathbb R}\), the image can be expressed as follows: \(Ax=\sum _{j=r+1}^nb_jAx_j\), which is a linear combination of \(Ax_{r+1},\ldots ,Ax_{n}\). Then, our goal is to show that

$$\begin{aligned} \sum _{i=r+1}^n b_{i}Ax_{i}=0 \Longrightarrow b_{r+1},\ldots ,b_n=0. \end{aligned}$$
(1.6)

If \(A\sum _{i=r+1}^n b_{i}x_{i}=0\), then \(\sum _{i=r+1}^nb_{i}x_{i}\) is in the kernel. Therefore, there exist \(b_1,\ldots ,b_r\) such that \(\sum _{i=r+1}^n b_ix_i=-\sum _{i=1}^rb_ix_i\), which means that \(\sum _{i=1}^nb_ix_i=0\). However, we assumed that \(x_1,\ldots ,x_n\) are linearly independent, which means that \(b_1=\ldots =b_n=0\) and Proposition (1.6) is obtained.

Proposition 7

For any square matrix A, we can obtain an upper-triangular matrix \(P^{-1}AP\) by multiplying an orthogonal matrix P and its inverse \(P^-1\) from right and left, respectively.

Proof: We prove the proposition by induction. For \(n=1\), since the matrix is scalar, the claim holds. From the assumption of induction, for an arbitrary \(\tilde{B}\in {\mathbb R}^{(n-1)\times (n-1)}\), there exists an orthogonal matrix \(\tilde{Q}\) such that

$$\tilde{Q}^{-1}\tilde{B}\tilde{Q}= \left[ \begin{array}{c@{\quad }c@{\quad }c} \tilde{\lambda }_2&{}&{}*\\ &{}\ddots &{}\\ 0&{}&{}\tilde{\lambda }_n\\ \end{array} \right] \ , $$

where \(*\) represents the nonzero elements and \(\tilde{\lambda }_2,\ldots ,\tilde{\lambda }_{n}\) are the eigenvalues of \(\tilde{B}\).

For a nonsingular matrix \(A\in {\mathbb R}^{n\times n}\) with eigenvalues \(\lambda _1,\ldots ,\lambda _n\), allowing multiplicity, let \(u_1\) be an eigenvector of eigenvalue \(\lambda _1\), and R an orthogonal matrix such that the first column is \(u_1\). Then, we have \(Re_1=u_1\) and \(Au_1=\lambda _1 u_1\), where \(e_1:=[1,0,\ldots ,0]^T\in {\mathbb R}^n\). Hence, we have

$$R^{-1}ARe_1=R^{-1}Au_1=\lambda _1R^{-1}u_1=\lambda _1R^{-1}Re_1=\lambda _1e_1$$

and we may express

$$R^{-1}AR= \left[ \begin{array}{c@{\quad }c} \lambda _1&{}b\\ 0&{}B \end{array} \right] \ , $$

where \(b\in {\mathbb R}^{1\times (n-1)}\) and \(0\in {\mathbb R}^{(n-1)\times 1}\). Note that R and A are nonsingular, so is B.

We claim that \(P=R \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q \end{array} \right] \) is an orthogonal matrix, where Q is a orthogonal matrix that diagonalizes \(B\in {\mathbb R}^{(n-1)\times (n-1)}\). In fact, \(Q^TQ\) is a unit matrix, so is \(P^TP= \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q \end{array} \right] R^TR \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q \end{array} \right] \). Note that the eigenvalues of B are \(\lambda _2,\ldots ,\lambda _n\) of A:

$$\prod _{i=1}^n(\lambda _i-\lambda )=\det (A-\lambda I_n)=\det (R^{-1}AR-\lambda I_n)=(\lambda _1-\lambda )\det (B-\lambda I_{n-1})\ ,$$

where \(I_n\) is a unit matrix of size n.

Finally, we claim that A is diagonalized by multiplying \(P^{-1}\) and P from left and right, respectively.

$$\begin{aligned} P^{-1}AP= & {} \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q^{-1} \end{array} \right] R^{-1}AR \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q \end{array} \right] = \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q^{-1} \end{array} \right] \left[ \begin{array}{c@{\quad }c} \lambda _1&{}b\\ 0&{}B \end{array} \right] \left[ \begin{array}{c@{\quad }c} 1&{}0\\ 0&{}Q \end{array} \right] \\= & {} \left[ \begin{array}{c@{\quad }c} \lambda _1&{}bQ\\ 0&{}Q^{-1}BQ \end{array} \right] = \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} \lambda _1&{}&{}&{}*\\ &{}\lambda _2&{}&{}\\ &{}&{}\ddots &{}\\ &{}&{}&{}\lambda _n\\ \end{array} \right] \, \end{aligned}$$

which completes the proof.

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Suzuki, J. (2020). Linear Algebra. In: Statistical Learning with Math and R. Springer, Singapore. https://doi.org/10.1007/978-981-15-7568-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-7568-6_1

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-7567-9

  • Online ISBN: 978-981-15-7568-6

  • eBook Packages: Computer ScienceComputer Science (R0)