Near Optimal Solutions to Least-Squares Problems with Stochastic Uncertainty

Summary

In this chapter, we consider least-squares problems where the regression data is affected by stochastic uncertainty. In this setting, we study the problem of minimizing the expected value with respect to the uncertainty of the least-squares residual. For general nonlinear dependence of the data on the uncertain parameters, determining an exact solution to this problem is known to be computationally prohibitive. Here, we follow a probabilistic approach, and determine a probable near optimal solution by minimizing the empirical mean of the residual. Finite sample convergence of the proposed method is assessed using statistical learning methods. In particular, we prove that, if one constructs the empirical approximation of the mean using a finite number N of samples, then the minimizer of this empirical approximation is, with high probability, an ∈-suboptimal solution for the original problem. Moreover, this approximate solution can be efficiently determined numerically by a standard recursive algorithm. Comparisons with gradient algorithms for stochastic optimization are also discussed in this contribution and some numerical examples illustrate the proposed methodology.