Improving estimated sufficient summary plots in dimension reduction using minimization criteria based on initial estimates

Abstract

In this paper we show that estimated sufficient summary plots can be greatly improved when the dimension reduction estimates are adjusted according to minimization of an objective function. The dimension reduction methods primarily considered are ordinary least squares, sliced inverse regression, sliced average variance Estimates and principal Hessian directions. Some consideration to minimum average variance estimation is also given. Simulations support the usefulness of the approach and three data sets are considered with an emphasis on two- and three-dimensional estimated sufficient summary plots.

Appendix: Computing details

In this section we detail the use of R which was used to compute all results. The analyses that have been carried out in the previous sections using the freely available statistical software package R version 2.13.1 (R Core Team 2014, see R: A Language and Environment for Statistical Computing by). The tolerance level for exiting the iterative process was set at $\text {tol}=1\times 10^{-5}$ although a maximum iterations of 100 was also enforced.

Initial SIR, SAVE and pHD estimates were obtained using the dr package (Weisberg 2002). For SIR and SAVE we did not specify the slicing parameter H and instead used the package default of $H=\max (8,p + 3)$. When robust estimates were obtained in Steps 0.2 and i.2, the function rlm from the ‘MASS’ package (Venables and Ripley 2002) with the Huber weight function as shown in (7) and where the default choice of $c=1.345\sigma$ was used where $\sigma$ is replaced with a robust estimate of the residual standard deviation (the median absolute deviation estimate). Otherwise ordinary least squares was utilized. For $\rho (r)=r^2$ (non-robust), in Step i.1 non-linear least squares was carried out using the function nls with the Gauss-Newton algorithm. For the robust equivalent, robust non-linear least squares was employed via the nlrob function from the ‘robustbase’ package (Rousseeuw et al. 2011) and again with the Huber weight function. Polynomial transformation of the $\widehat{\mathbf {{B}}}^\top {\mathbf {x}}_i$’s in R is routine using the poly function and setting the optional argument raw to TRUE to ensure that the dimension reduced regressors are transformed such that they are original polynomial terms and not orthogonal terms.

With respect to the spline fitting in Sect. 2.3, for the minimization we used the nlminb function available through the R stats package (available as standard in the R base distribution) whose arguments were the initial estimates based on either OLS, SIR, SAVE of pHd and the objective function to be minimized. A cubic smooth spline was fitted using the R function smooth.spline.

For MAVE, we used functionality provided in support of work from Li et al. (2010) found at http://www4.stat.ncsu.edu/~li/software/GroupDR.R. This functionality uses SIR to initialize the estimates and we used the default tolerance and kernel settings. In the final example where the method PFC was employed, we used the R package ldr (Adragni and Raim 2014) which includes functionality for several likelihood-based dimension reduction methods. We used the same arguments in the call to the PFC functionality for this example as was found in the paper.