Abstract
This is a book about statistical analyses: how to conduct them and how to interpret them. Not all of the analyses involve linear functions and few of them have an exact solution, so it might seem odd to begin by learning to solve a consistent system of linear equations. Yet there are good reasons to start this way. For one thing, linear systems form the backbone of most statistical analyses and many nonlinear functions have a linear component. Moreover, many of the methods that are used to solve linear equations were developed long ago by some of the world’s most famous mathematicians. A familiarity with these techniques is therefore of historical interest.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
One way to frame the issue is to ask “what is the smallest quantity that can be added to 1 that will be judged by a computer to be greater than 1? The answer on most computers is 2−52 = 2.220446e − 16. You can prove this with \( \mathcal{R} \): The statement, 1+2^−52 == 1 returns FALSE, but the statement 1+2^−53 == 1 returns TRUE.
- 2.
\( \mathcal{R} \) code for performing Gauss-Jordan elimination is provided in Sect. 1.2.4.
- 3.
Crout’s method provides another factorization.
- 4.
The solution can also be found using matrix multiplication [x = U−1(P′L)b], but using forward and backward substitution is more efficient and sometimes more accurate.
- 5.
Other methods of assessing positive definiteness will be discussed later in this chapter and in Chap. 5.
- 6.
Many statistical packages, including \( \mathcal{R} \), return an upper triangular matrix R when performing the Cholesky decomposition (R = L′). In this case, the decomposition is A = R′R.
- 7.
The determinant of a symmetric, positive definite matrix can be found from the product of the squared diagonal elements of the Cholesky decomposition, and a matrix is positive definite if the determinants of its upper-left k × k sub-matrices are all > 0.
- 8.
More sophisticated ways of assessing the singularity of a matrix will be discussed in Chap. 5.
- 9.
Only a square matrix can be singular, but any rectangular matrix can be rank-deficient.
- 10.
- 11.
The efficiency of a computer algorithm is judged, in part, by counting how many floating point operations (addition, subtraction, multiplication and division) are needed to obtain a solution. Using this metric, the matrix inverse method is approximately three times less efficient for solving linear equations than is the LU decomposition (Golub & van Loan, 2013).
- 12.
An alternative measure of accuracy, used often when the true solution is not known, is to compute a residual norm that quantifies how well the obtained solution reproduces the constants.
$$ {\left\Vert \mathbf{b}-\mathbf{A}\ \widehat{\mathbf{x}}\right\Vert}_p $$This norm will be discussed in greater detail in Chap. 2.
- 13.
Matrices that are nearly singular (determinant ∼ 0) tend to be ill-conditioned, but the association is not guaranteed. Consequently, the determinant should not be used to gauge the condition of a matrix.
- 14.
The reported condition number uses the Frobenius norm. The 2-norm, which will be discussed in Chap. 5, is usually used instead.
- 15.
The condition number also provides information about the accuracy of the solution, with log10(κ) indicating the number of decimal places that are lost to round-off error using matrix inversion.
- 16.
A third approach, the Successive Over-Relaxation Method, can also be used. In this case, we weight the values before inserting them (see Golub & Van Loan, 2013).
- 17.
Convergence can occur with matrices that are not positive definite or diagonally dominant, but it isn’t guaranteed. In this sense, we can think of these features as sufficient but not necessary for convergence.
References
Brown, J. D. (2014). Linear models in matrix form: A hands-on approach for the behavioral sciences. New York: Springer.
Golub, G. H., & van Loan, C. F. (2013). Matrix computations (4th ed.). Baltimore: John Hopkins.
Higham, N. (2002). Accuracy and stability of numerical algorithms (2nd ed.). Philadelphia: Society for Industrial and Applied Mathematics.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Brown, J.D. (2018). Linear Equations. In: Advanced Statistics for the Behavioral Sciences. Springer, Cham. https://doi.org/10.1007/978-3-319-93549-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-93549-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93547-8
Online ISBN: 978-3-319-93549-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)