Fast DD-classification of functional data

Abstract

A fast nonparametric procedure for classifying functional data is introduced. It consists of a two-step transformation of the original data plus a classifier operating on a low-dimensional space. The functional data are first mapped into a finite-dimensional location-slope space and then transformed by a multivariate depth function into the DD-plot, which is a subset of the unit square. This transformation yields a new notion of depth for functional data. Three alternative depth functions are employed for this, as well as two rules for the final classification in $[0,1]^2$. The resulting classifier has to be cross-validated over a small range of parameters only, which is restricted by a Vapnik–Chervonenkis bound. The entire methodology does not involve smoothing techniques, is completely nonparametric and allows to achieve Bayes optimality under standard distributional settings. It is robust, efficiently computable, and has been implemented in an R environment. Applicability of the new approach is demonstrated by simulations as well as by a benchmark study.

Appendices

Appendix 1: Implementation details

In calculating the depths, $\mu _Y$ and $\Sigma _Y$ for the Mahalanobis depth have been determined by the usual moment estimates and similarly, $\Sigma _Y$ for the spatial depth. The projection depth has been approximated by drawing 1000 directions from the uniform distribution on the unit sphere. Clearly, the number of directions needed for satisfactory approximation depends on the dimension of the space. Observe that for higher-dimensional problems 1000 directions are not enough, which becomes apparent from the analysis of Model 2 in Sect. 7.2. There the location-slope spaces chosen have dimension eight and higher; see also Tables 4 and 8 in Appendix 2. On the other hand, calculating the projection depth even in one dimension costs something. Computing 1 000 directions to approximate the projection depth takes substantially more time than computing the exact Mahalanobis or spatial depths (see Tables 2 and 14 in Appendix 2).

LDA and QDA are used with classical moment estimates, and priors are estimated by the class portions in the training set. The kNN-classifier is applied to location-slope data in its affine invariant form, based on the covariance matrix of the pooled classes. For time reasons, its parameter k is determined by leave-one-out cross-validation over a reduced range, viz. $k\in \{1, \dots , \max \{\min \{10(m+n)^{1/d}+1,m+n-1\},2\}\}$. The $\alpha$-procedure separating the DD-plot uses polynomial space extensions with maximum degree three; the latter is selected by cross-validation. To keep the training speed of the depth-based kNN-classifier comparable with that of the $DD\alpha$-classifier, we also determine k by leave-one-out cross-validation on a reduced range of $k\in \{1, \dots , \max \{\min \{10\sqrt{m+n}+1,(m+n)/2\},2\}\}$.

Due to linear interpolation, the levels are integrated as piecewise-linear functions, and the derivatives as piecewise constant ones. If the dimension of the location-slope space is too large (in particular for inverting the covariance matrix, as it can be the case in Model 2), PCA is used to reduce the dimension. Then $\epsilon _{max}$ is estimated and all further computations are performed in the subspace of principal components having positive loadings.

To construct the location-slope space, firstly all pairs (L, S) satisfying $2\le L+S\le M/2$ are considered. (M / 2 amounts to 26 for the synthetic and to 16 for the real data sets.) For each (L, S) the data are transformed to $\mathbb {R}^{L+S}$, and the Vapnik–Chervonenkis bound $\epsilon _{max}$ is calculated. Then those five pairs are selected that have smallest $\epsilon _{max}$. Here, tied values of $\epsilon _{max}$ are taken into account as well, with the consequence that on an average slightly more than five pairs are selected; see the growth data in Table 2 and both synthetic models in Table 14 of Appendix 2. Finally, among these the best (L, S)-pair is chosen by means of cross-validation. Note that the goal of this cross-validation is not to actually choose a best location-slope dimension but rather to get rid of obviously misleading (L, S)-pairs, which may yield relatively small values of $\epsilon _{max}$. This is seen from Figs. 4 and 5. When determining an optimal (L, S)-pair by crossLS, the same set of (L, S)-pairs is considered as with VCcrossLS.

In implementing the componentwise method of finite-dimensional space synthesis (crossDHB) we have followed Delaigle et al. (2012) with slight modifications. The original approach of Delaigle et al. (2012) is combined with the sequential approach of Ferraty et al. (2010). Initially, a grid of equally ($\Delta t$) distanced discretization points is built. Then a sequence of finite-dimensional spaces is synthesized by adding points of the grid step by step. We start with all pairs of discretization points that have at least distance $2\Delta t$. [Note that Delaigle et al. (2012) start with single points instead of pairs.] The best of them is chosen by cross-validation. Then step by step features are added. In each step, that point that has best discrimination power (again, in the sense of cross-validation) when added to the already constructed set is chosen as a new feature. The resulting set of points is used to construct a neighborhood of combinations to be further considered. As a neighborhood we use twenty $2\Delta t$-distanced points in the second step, and ten in the third; from the fourth step on the sequential approach is applied only.

All our cross-validations are tenfold, except the leave-one-out cross-validations in determining k with both kNN-classifiers. Of course, partitioning the sample into ten parts only may depreciate our approach against a more comprehensive leave-one-out cross-validation. We have chosen it to keep computation times of the crossDHB approach (Delaigle et al. 2012) in practical limits and also to make the comparison of approaches equitable throughout our study.

The calculations have been implemented in an R-environment, based on the R-package “ddalpha” (Mozharovskyi et al. 2015), with speed critical parts written in C++. The R-code implementing our methodology as well as that performing the experiments can be obtained upon request from the authors. In all experiments, one kernel of the processor Core i7-2600 (3.4 GHz) having enough physical memory has been used. Thus, regarding the methodology of Delaigle et al. (2012) our implementation differs from their original one and, due to its module-based structure, may result in larger computation times. For this reason we provide the number of cross-validations performed; see Tables 2 and 14 of Appendix 2. The comparison appears to be fair, as we always use ten-fold cross-validation together with an identical set of classification rules in the finite-dimensional spaces.

Appendix 2: Additional tables

See Tables 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, and 14

Table 3 Frequency (in %) of selected location-slope dimensions using the Vapnik–Chervonenkis bound; Model 1

Table 4 Frequency (in %) of selected location-slope dimensions using the Vapnik–Chervonenkis bound; Model 2

Table 5 Frequency (in %) of location-slope dimensions chosen using the Vapnik–Chervonenkis bound; growth data

Table 6 Frequency (in %) of location-slope dimensions chosen using the Vapnik–Chervonenkis bound; medflies data

Table 7 Frequency (in %) of selected location-slope dimensions using cross-validation; Model 1

Table 8 Frequency (in %) of selected location-slope dimensions using cross-validation; Model 2

Table 9 Frequency (in %) of location-slope dimensions chosen using cross-validation; growth data

Table 10 Frequency (in %) of location-slope dimensions chosen using cross-validation; medflies data

Table 11 Frequency (in %) of selected dimensions using componentwise method; Model 1

Table 12 Frequency (in %) of selected dimensions using componentwise method; Model 2

Table 13 Frequency (in %) of selected dimensions using componentwise method; growth data

Table 14 Average (median for componentwise classification = crossDHB) training and classification (in parentheses) times (in s), and numbers of cross-validations performed (in square brackets), over 100 tries