Principal Components Analysis: An Alternative Way for Removing Natural Growth Trends

Abstract

In this article, we establish a new approach for removing natural growth trends from tree-ring samples, also called detrending. We demonstrate this approach using Ocotea porosa (Nees & Mart) Barroso trees. Nondestructive samples were collected in General Carneiro city, located in the Brazilian southern region (Paraná state). To remove natural tree growth trends, principal components analysis (PCA) was applied on the tree-ring series as a new detrending method. From this, we obtained the tree-ring indices by reconstructing the tree-ring series without the first principal component (PC), which we expect to represent the natural growth trend. The performance of this PCA method was then compared to other detrending methods commonly used in dendrochronology, such as the cubic spline method, negative exponential or linear regression curve, and the regional curve standardization method. A comparison of these methods showed that the PCA detrending method can be used as an alternative to traditional methods since (1) it preserves the low-frequency variance in the 566-year chronology and (2) represents an automatic way to remove the natural growth trends of all individual measurement series at the same time. Moreover, when implemented using the alternating least squares (ALS) method, the PCA can deal with tree-ring series of different lengths.

Appendix A: Principal Components Analysis

Let $X \in {\mathbb {R}}^{n \times m}$ be the data matrix whose m columns represent the variables $x_1, x_2, \ldots , x_m$, zero-centered by hypothesis, and the n rows represent the observations of each of them, as described by

((1))

where $\mathrm {X}_i \in {\mathbb {R}}^{m}$ is a row vector that corresponds to the ith observation, with $1 \le i \le n$. The idea in the standard derivation of PCA is to find a direction determined by a unit vector $q \in {\mathbb {R}}^{m}$ such that the variance of projections $y_{i}$ associated to observations $\mathrm {X}_{i}$ is maximum, as illustrated in Fig. 14.

Fig. 14

Representation of some observations, restricted to variables $x_{1}, x_{2}$ and $x_{3}$, and the direction q that maximize the variance of the projections $y_{i}$

The projection of $\mathrm {X}_i$ onto the unit vector q is given by $y_{i} = \mathrm {X}_iq$, so that the variance of all projections is defined by

$$\begin{aligned} V&= \dfrac{1}{n - 1}\sum \limits _{i = 1}^{n}y_{i}^2 = q^T\underbrace{\left[ \dfrac{1}{n - 1}\sum \limits _{i = 1}^{n} \mathrm {X}_i^T\mathrm {X}_i\right] }_{C_X}q = q^TC_Xq, \end{aligned}$$

(2)

where $C_{X} \in {\mathbb {R}}^{m \times m}$ is the covariance matrix associated to the original set of variables and can be rewritten in a more convenient way as follows:

$$\begin{aligned} C_X = \dfrac{1}{n - 1}\sum \limits _{i = 1}^{n} \mathrm {X}_i^T\mathrm {X}_i = \dfrac{1}{n - 1}X^TX. \end{aligned}$$

(3)

The covariance matrix $C_{X}$ has some properties that form the basis to the understanding of PCA, and, in order to visualize them, it is necessary to expand $C_{X}$ in Eq. (3) considering that the observations are of the form $\mathrm {X}_{i} = \begin{bmatrix} x_{i1}&x_{i2}&\cdots&x_{im}\end{bmatrix}$, resulting in

$$\begin{aligned} \begin{aligned} C_X&= \begin{bmatrix} \text {var}(x_{1}) &{} \text {cov}(x_{1},x_{2}) &{} \dots &{} \text {cov}(x_{1}, x_{m})\\ \text {cov}(x_{2}, x_{1}) &{} \text {var}(x_{2}) &{} \dots &{} \text {cov}(x_{2}, x_{m})\\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ \text {cov}(x_{m}, x_{1}) &{} \text {cov}(x_{m}, x_{2}) &{} \dots &{} \text {var}(x_{m}) \end{bmatrix}. \end{aligned} \end{aligned}$$

(4)

Equation (4) shows that $C_{X}$ is a symmetric matrix due to the symmetry of covariance, and therefore, it has a orthonormal set of m eigenvectors, and it is orthogonally diagonalizable, so that there is an orthogonal matrix $Q \in {\mathbb {R}}^{m\times m}$ such that

$$\begin{aligned} D_{X} = Q^TC_{X}Q \iff C_{X} = QD_{X}Q^{T}, \end{aligned}$$

(5)

where $D_{X} = \text {diag}(\lambda _{1}, \lambda _{2}, \ldots , \lambda _{m})$ is a diagonal matrix with the eigenvalues of $C_{X}$ ordered in such a way that $\lambda _{1} \ge \lambda _{2} \ge \cdots \ge \lambda _{m}$ and $Q = \begin{bmatrix}q_{1}&q_{2}&\ldots&q_{m}\end{bmatrix}$, whose column $q_{i} \in {\mathbb {R}}^{m}$ is the respective eigenvector of $\lambda _{i}$. Besides that, $C_{X}$ is a positive semidefinite matrix such that if $y \in {\mathbb {R}}^{m}$ is a nonzero vector, then

$$\begin{aligned} y^{T}C_{X}y = \dfrac{1}{n - 1}\sum \limits _{i = 1}^{n}(\mathrm {X}_{i}y)^2 \ge 0, \end{aligned}$$

where equality can only be obtained when $\mathrm {X}_{i} = 0$ since $y \ne 0$. As a consequence, the eigenvalues of $C_{X}$ are nonnegative, which is consistent because they represent the variance of each principal component, as will be seen later. Finally, note that the diagonal elements of the matrix correspond to the variances of each variable $x_{1}, x_{2}, \ldots , x_{m}$, whereas the off-diagonal elements correspond to all possibilities of covariance between these variables.

In order to maximize the variance in Eq. (2), it is necessary to solve an optimization problem involving the quadratic form with one constraint, as stated in

$$\begin{aligned} \begin{array}{llll} &{}\max _{q} &{}\quad &{} V = q^TC_{X}q\\ &{}\text {subject to} &{} &{} q^Tq = 1 \end{array}. \end{aligned}$$

(6)

The standard way to approach such a problem is by the Lagrange multiplier method that allows one to combine an objective function with a sequence of constraints, producing the Lagrange function, which in this case, is given by

$$\begin{aligned} {\mathcal {L}}(q, \lambda ) = q^TC_{X}q - \lambda (q^Tq - 1). \end{aligned}$$

(7)

By differentiating (7) and equating the result to zero, considering that $\frac{\partial q^Tq}{\partial q} = 2q^T$ and $\frac{\partial q^TC_{X}q}{\partial q} = 2q^TC_{X}$, one gets

$$\begin{aligned} 2q^TC_{X} - \lambda 2q^T = 0&\iff C_{X}q = \lambda q. \end{aligned}$$

(8)

Equation (8) is a major result since it indicates that the direction q that maximizes the variance V is given by the eigenvectors of the covariance matrix $C_{X}$, and as a consequence, the numerical value of V is given by the respective eigenvalue $\lambda$, as indicated by

$$\begin{aligned} V = q^TC_Xq = q^T\lambda q = \lambda \underbrace{q^Tq}_{1} = \lambda . \end{aligned}$$

(9)

Nevertheless, since the matrix $C_{X}$ has a set of m orthonormal eigenvectors, which one maximizes V? This question can be answered by observing Eq. (9), where one can conclude that the eigenvector with the largest eigenvalue is the one that maximizes it. Since the eigenvalues are ordered, one can make the following statements:

• (a) $q_{1}$ is the direction of largest variance $\lambda _{1}$ of the original data set;

• (b) $q_{2}$ is the direction of the second-largest variance $\lambda _{2}$ and so on.

An important point that deserves attention is the perpendicular distances $d_{i}$ of observations $\mathrm {X}_{i}$ relative to line $\ell$, as illustrated in Fig. 14. Mathematically, $d_{i}$ is given by

$$\begin{aligned} d_{i} = \left\Vert \mathrm {X}_{i}^T - y_{i}q\right\Vert _{2} = \left\Vert \mathrm {X}_{i}^T - (\mathrm {X}_{i}q)q\right\Vert _{2}. \end{aligned}$$

(10)

The goal is to verify that the variance D of these distances is minimized when the variance of the projections V is maximized. To do this, it is necessary to square Eq. (10), allowing us to express $d_{i}^2$ in matrix form as follows:

$$\begin{aligned} d_{i}^2&= \left\Vert \mathrm {X}_i^T - (\mathrm {X}_iq)q\right\Vert ^2_2 = \mathrm {X}_i\mathrm {X}_i^T - (\mathrm {X}_iq)^2. \end{aligned}$$

(11)

From Eq. (11) one can obtain a closed expression for variance D, as in

$$\begin{aligned} D&= \dfrac{1}{n - 1} \sum \limits _{i = 1}^{n}d_{i}^2 = \dfrac{1}{n - 1} \sum \limits _{i = 1}^{n}\mathrm {X}_i\mathrm {X}_i^T - V. \end{aligned}$$

(12)

Note that Eq. (12) indicates that when V is maximum, D is minimum due to the minus sign, since the other term is constant. This explains why the eigenvectors seem to fit the cloud of observations when they overlap in the same plot.

With the largest variance directions determined by the eigenvalues of $C_{X}$, all that remains is to project the observations $\mathrm {X}_{i}$ in the eigenspace of $C_{X}$. Since the projection $\mathrm {X}_{i}$ in the direction of $q_{j}$ is given by $y_{ij} = \mathrm {X}_{i}q_{j}$, by doing this for all eigenvectors indexed by $1\le j \le m$, there is a new representation of $\mathrm {X}_{i}$ given by the row vector $Y_{i} = \begin{bmatrix}\mathrm {X}_{i}q_{1}&\mathrm {X}_{i}q_{2}&\dots&\mathrm {X}_{i}q_{m} \end{bmatrix}$. Thus, for all observations indexed by $1 \le i \le n$, one can get a new representation for the original data set, denoted by the matrix $Y \in {\mathbb {R}}^{n\times m}$, as given by

((13))

where the new variables $y_{1}, y_{2}, \ldots , y_{m}$ are named principal components. It should be noted that the entry $y_{ij} = \mathrm {X}_{i}q_{j}$ corresponds to the score of the ith observation in the jth principal component, whereas the eigenvector $q_{j}$ is the so-called coefficient vector of this component. This nomenclature comes from the fact that the entry $y_{ij}$ is written as a linear combination whose coefficients are the entries of $q_{j}$, as follows:

$$\begin{aligned} y_{ij} = q_{1j}x_{i1} + q_{2j}x_{i2} + \cdots + q_{mj}x_{im} =\sum \limits _{r = 1}^m q_{rj}x_{ir}. \end{aligned}$$

(14)

In general, the new representation of the original data set as given by the Eq. (13) is not made considering all the columns of matrix Q. The idea is to reduce the dimension of the original data set disregarding the columns that represent the principal components with the smallest variances, resulting in a new set of variables that represents the original ones with a certain degree of accuracy. For example, suppose that only the first $k<< m$ principal components are used to represent a data set in an m-dimensional space. The accuracy of this representation is defined in terms of how much the total variance remains in the k-dimensional space and is given as a percentage as follows:

$$\begin{aligned} \dfrac{100}{\text {tr}(C_{X})}\cdot \sum \limits _{j = 1}^{k}\lambda _{j} = 100\cdot \dfrac{\lambda _{1} + \lambda _{2} + \cdots + \lambda _{k}}{\lambda _{1} + \lambda _{2} + \cdots + \lambda _{m}}, \end{aligned}$$

(15)

where it is quite common to get high values with only two principal components. Note that with Eq. (13), it becomes possible to obtain the original data matrix X by multiplying both sides on the right by $Q^T$, as indicated by

$$\begin{aligned} YQ^T = X\underbrace{QQ^T}_{I} \iff X = YQ^{T}. \end{aligned}$$

(16)

One of the main results of PCA can be noticed when calculating the covariance matrix $C_{Y}$ associated to the principal components, as given by

$$\begin{aligned} C_Y = \dfrac{1}{n - 1} Y^TY = Q^TC_XQ. \end{aligned}$$

(17)

Substituting Eq. (5) into Eq. (17), it turns out that $C_{Y}$ is a diagonal matrix,

$$\begin{aligned} C_{Y} = D_{X} = \begin{bmatrix} \lambda _{1} &{} 0 &{} \dots &{} 0\\ 0 &{} \lambda _{2} &{} \dots &{} 0 \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ 0 &{} 0 &{} \dots &{} \lambda _{m} \end{bmatrix}, \end{aligned}$$

(18)

whereby matrix equality follows that $\text {var}(y_{j}) = \lambda _{j}$, that is, the eigenvalues of $C_{X}$ are the variances of the principal components. Moreover, note that there is no covariance between them, indicating that they are unrelated.

1.1 A.1 Alternating Least Squares

The alternating least squares (ALS) is a matrix factorization algorithm that plays an important role in the context of collaborative filtering (CF) in recommendation systems. Specifically, CF is a technique used to make predictions about the interests of a user based on their preferences over a set of available items (Zhou et al., 2008), which can be movies and/or songs in streaming platforms, for example. In this sense, to illustrate the ALS, suppose that a video streaming platform has $n_{u}$ users and $n_{m}$ movies available which can be rated. In that scenario, it is convenient to define a user-movie matrix $R \in {\mathbb {R}}^{n_{u}\times n_{m}}$ whose entry $r_{ij}$ represents the rating score of movie j by user i. Unfortunately, the matrix R has a lot of missing entries since a user only rates a subset of the available movies. The purpose of a recommendation system is to estimate these missing entries based on the previous ratings of the users.

The idea is to perform a low-rank approximation of the user-movie matrix R by introducing a user feature matrix $U \in {\mathbb {R}}^{n_{f}\times n_{u}}$ and a movie feature matrix $M \in {\mathbb {R}}^{n_{f}\times n_{m}}$ in such a way that $r_{ij} \approx u_{i}^Tm_{j} \forall i \in \{1,\ldots ,n_{u}\}, \forall j \in \{1,\ldots ,n_{m}\}$ or $R \approx U^TM$, where $u_{i} \in {\mathbb {R}}^{n_{f}}$ and $m_{j} \in {\mathbb {R}}^{n_{f}}$ are the user and movie feature vectors, respectively (Zhou et al., 2008). The $n_{f}$ is the dimension of the feature space. In this sense, the matrices U and M are determined by minimizing the least squares error of the known ratings, i.e.,

$$\begin{aligned}&\min _{U,M} f(U, M)= \min _{U,M} \sum _{(i, j) \in I}\left( r_{i j}-u_{i}^{T} m_{j}\right) ^{2} \nonumber \\&\quad +\lambda \left( \sum _{i} n_{u_{i}}\left\| u_{i}\right\| ^{2}+\sum _{j} n_{m_{j}}\left\| m_{j}\right\| ^{2}\right) , \end{aligned}$$

(19)

where I denotes the index set of known ratings, $n_{u_{i}}$ the number of ratings of user i, and $n_{m_{j}}$ the number of ratings of movie j. The term involving $\lambda$ in Eq. (19) is a regularization term to prevent overfitting in ALS (Zhou et al., 2008). It is noteworthy that the objective function in Eq. (19) is non-convex due to the term $u_{i}^{T} m_{j}$, but if one of the matrices is fixed, U for example, then it becomes a convex function of M and vice versa. In this way, the approach used to solve this minimization problem is to fix U and optimize M, then fix M and optimize U until convergence is established by some stopping criterion. Actually, this fact explains the word alternating in the ALS name. Specifically, to do the optimization, it suffices to perform a matrix differentiation in Eq. (19) considering that one of the matrices is held constant and equates the result to zero. In this sense, suppose that M is given, then the columns of matrix U will be

$$\begin{aligned} u_{i} = (M_{I_{i}}M_{I_{i}}^T + \lambda n_{u_{i}}E)^{-1}M_{I_{i}}R^T(i, I_{i}), \quad i \in \{1,\ldots , n_{u}\}, \end{aligned}$$

(20)

where E is the $n_{f}\times n_{f}$ identity matrix, $M_{I_{i}} = [m_{j}]_{j \in I_{i}}$, and $R(i, I_{i})$ is the ith row vector of R considering only the columns with indexes in $I_{i}$ (Zhou et al., 2008). Similarly, by carrying out the same procedure, but considering that U is given, it results

$$\begin{aligned} m_{j} = (U_{I_{j}}U_{I_{j}}^T + \lambda n_{m_{j}}E)^{-1}U_{I_{j}}R(I_{j}, j), \quad j \in \{1,\ldots , n_{m}\}, \end{aligned}$$

(21)

where $U_{I_{j}} = [u_{i}]_{i\in I_{j}}$, and $R(I_{j}, j)$ is a column vector of R considering only the rows with indexes in $I_{j}$, which represents the set of users who rated movie j.

By using Eqs. (20) and (21) in the ALS algorithm, one can find the matrix $\tilde{R} = U^TM$ that fits the known entries of matrix R and fills the missing ones.