SURE-Optimal Bandwidth Selection in Nonparametric Regression

Abstract

An important problem in nonparametric regression is to control the amount of smoothing in the regression, which in turn is specified by the so-called bandwidth parameter. The problem has received much attention in the statistics literature, leading to a variety of methods for bandwidth selection. Most methods arrive at an empirical risk estimate by partitioning the data into two sets: training samples and testing samples. The training samples are used to compute a family of regression functions indexed by their bandwidths. The optimal bandwidth is chosen as the value for which the regression function minimizes the quadratic empirical risk on the training sample. In this paper, we propose to select the bandwidth within the framework of minimum mean-squared-error (MMSE) estimation. Since the oracle MSE cannot be computed in practice due to non-availability of the ground truth, we propose to employ an unbiased estimator of the MSE. Specifically, we rely on the Stein’s unbiased risk estimator (SURE) to optimize for the bandwidth parameter. We address the problems of kernel regression and local polynomial regression (LPR) within a common framework and show that SURE provides a good approximation to the oracle MSE, especially with larger data lengths, thus helping us in arriving at the MMSE solution. As a specific application, we address the problem of signal reconstruction from noisy nonuniform samples in both one and two dimensions. For a smooth progression of the exposition, we also provide certain results in the uniform sampling case, before proceeding with the case of nonuniform sampling.