Variable selection for multivariate functional data via conditional correlation learning

Abstract

Variable selection involves selecting truly important predictors from p-dimensional multivariate functional predictors in functional predictive models. In this paper, a variable selection method is designed for scalar-on-function predictions entangled with nonlinear joint associations among scalar response and multiple functional predictors. First, a nonparametric functional nonlinear conditional correlation coefficient, namely, the FunNCC coefficient, is proposed to measure complex dependencies, including the nonmonotonic marginal dependence, along with the conditional associations of redundancy, complement, and interaction. Then, a model-free feature ordering and selection method is designed, where the FunNCC is utilized to rank relevance, enabling the selection of a subset of predictors with the strongest joint dependence. Since this method allows for quantitatively evaluating the contributions of predictors in explaining responses, it achieves moderate model interpretability. Finally, extensive simulation studies and two real-data cases involving air pollution regression and hand gesture recognition are conducted to evaluate the finite sample performance of the proposed method, and the results show that the proposed FunNCC and variable selection methods outperform state-of-the-art baselines.

Appendices

Appendix A: FOFCD-Back and FOFCD-Step

Pseudocodes of FOFCD-Back and FOFCD-Step are shown in Algorithms A1 and A2, respectively.

Algorithm A1

Pseudocode of FOFCD-Back.

FOFCD-Back starts with the full set of input predictors to be selected. All the elements in the selected set $j\in {\hat{\mathcal{S}}}$ are subsequently assessed by $\text{RelS}( j,{\textbf{Y}}\!\mid \!({\hat{\mathcal{S}}}\backslash \{j\}))$; i.e., their conditional dependencies on the response given other elements in the selected set. Predictors with lower-than-$\delta$ RelS are removed from ${\hat{\mathcal{S}}}$. The process is iterated until the stopping criterion is satisfied. The stopping criterion of FOFCD-Back is that the cardinality of the selected subset $\left| \hat{S} \right|$ reaches the predetermined lower bound $d_{\min }$ or $\text{RelS}( j,{\textbf{Y}}\!\mid \!({\hat{\mathcal{S}}}\backslash \{j\}))$ of all $j\in {\hat{\mathcal{S}}}$ are no less than the relevant threshold $\delta$.

Algorithm A2

Pseudocode of FOFCD-Step.

FOFCD-Step starts with an empty selected subset and a candidate set that equals the index set of inputs. In each iteration, the most relevant predictor is selected from ${\mathcal{V}}$, and its index is put into ${\hat{\mathcal{S}}}$ to form a new selected subset. Every time ${\hat{\mathcal{S}}}$ is updated, all the elements in the selected set $j\in {\hat{\mathcal{S}}}$ are assessed, and the conditionally independent predictor with the smallest RelS is returned to the candidate set. The process is iterated until the stopping criterion is satisfied, and the stopping criterion of FOFCD-Step is the same as that of FOFCD-FW.

Appendix B: Supplementary information on Ninapro DB1

1.1 Appendix B.1: Sensor positions of Ninapro DB1

The sensor positions are shown in Fig. 5. sEMG signals reveal muscle activity using 10 surface electromyography electrodes. Eight electrodes were evenly placed around the forearm at a constant distance from the radiohumeral joint below the elbow joint, and the other two electrodes were placed on the flexor major and extensor muscles of the forearm. Kinematic signals were collected using CyberGlove II with 22 sensors, which measure angular changes between pairs of hand joints. Every sample is linked to an accurate timestamp. All the data streams were linearly interpolated to the maximum recorded frequency of 100 Hz to eliminate the differences in the sampling rates of the sEMG and kinematic signals.

Fig. 5

Positions of sensors used to record sEMG and kinematic signals. a Positions of surface electromyography electrodes; b Positions of sensors on CyberGlove II

1.2 Appendix B.2: Variable selection results

The variable selection results for FOFCD and fLARS are listed in Table 13, including the results for the three subclasses and the whole Ninapro DB1. For FOFCD, a subset of signals from Gloves 1–22 was selected for all datasets. All sEMG signals were retained by FOFCD, except a few evenly placed sEMG signals that were removed on DB1_E2. Gloves 2, 3, 4 and 11, and most sEMG signals were important predictors in all the datasets.

Table 13 Variable selection result on Ninapro DB1

1.3 Appendix B.3: Post hoc interpretation

Feix et al. (2016) classified 20 daily movements into three oppositions, namely palm, pad, and side, according to differences in force directions between the hand and objects, as shown in Fig. 6a, b, respectively. In these figures, virtual fingers (VF) were defined as several fingers that work together as a unit for certain tasks. In Fig. 6a, VF 1 of the palm was the palm, and the third or forth fingers acted as VF 2. For both pad and side, VF 1 were thumb, but VF 2 at each position varied greatly. In the example shown in Fig. 6b, the forefinger opposed the thumb and acted as VF 2.

Fig. 6

Comparison of the selected predictors in different fine categories. a Diagram of palm (Feix et al. 2016); b Diagram of pad; c Diagram of side; d Selected predictors of palm; e Selected predictors of pad; and f Selected predictors of side

A visual analysis of the variable selection results on these oppositions was as follows.

• 1. Movements of the palm mainly involve holding objects by squeezing all powerful fingers perpendicularly toward the palm, e.g., by grabbing a hammer or screwdriver. Predictors selected by FOFCD were mostly located at the tips of middle finger and ring finger (Gloves 10 and 14), as well as the palm (Gloves 4, 5, 12, 15, and 16), and only Glove 3 on thumb was selected, as is shown in Fig. 6d.

• 2. Pad usually involves the thumb and other fingers, with contact near or at the fingertips and sometimes include contact with palmar surfaces, e.g., by holding a needle or a small ball. The predictors selected by FOFCD were mostly located at the tips of the forefinger and middle finger and at the wrist. All signals in the thumb were also retained, as shown in Fig. 6e.

• 3. Side mainly involves the tip of the thumb and transverse sides of the other four fingers, e.g., holding a key between the thumb and the radial side of the remaining fingers or holding a cigarette between the fingers. In FOFCD, predictors were mostly located at the five fingers and less often at the palm than the palm opposition, as shown in Fig. 6f.