Uncertainty, Information, and Learning Mechanisms

Abstract

This chapter introduces and formalizes the most important forms of uncertainty that digital embedded systems have to deal with and how it propagates within a computational flow. We found uncertainty associated with data representation of input variables (e.g., introduced by the truncation and rounding operators) and that propagated along the computational flow. Moreover, we have uncertainty associated with models learned from data and, finally, that introduced at the application design phase. All these sources of uncertainty combine in a nonlinear way and influence the result of the computation in execution on the embedded system, which becomes approximated. The main elements of the statistical theory of learning are then presented. It is shown how information can be extracted from noisy data and how the limited number of available training data, the effectiveness of the learning mechanism, and the uncertainty intrinsic with the problem affect the performance of the learned model.