Localized Generalization Error Model

Abstract

The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machines, radial basis function neural networks and multilayer Perceptron neural networks are local learning machines for solving problems and thus treat unseen samples near the training samples as more important. In this chapter, we describe a localized generalization error model which bounds the generalization error from above within a neighborhood of the training samples using a stochastic sensitivity measure (Yeung et al., 2007 and Ng et al., 2007). This model is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold.