Iteration Rules with Weak Memory

Abstract

In this chapter we are concerned with constraints under which a broad class of machine learning procedures, governed by iteration rules with memory, converge. More distinctly, we assume iteration rules with weak (or just finite) memory with respect to previous data and previously proposed estimates. Procedures of this sort offer rich possibilities to find convergent learning algorithms with powerful processing steps, and reduce in this way the time necessary for learning. We prove, in what follows, relations of interest in devising such sort of iteration rules. Use of these results, e.g., for modifying convergent learning algorithms heuristically, while retaining their convergence, and other related topics are subjects we consider in Chapter 5.