Maximum Likelihood Hebbian Learning

Abstract

Throughout previous chapters, we have used the residuals after feedback as a means of enabling self-organisation to filters of the data sets which have been optimal for the particular effect for which we were striving. In this chapter, the focus is on the residuals themselves in that we consider how the network will learn in response to various types of residuals with different probability density functions (pdfs). We are, in effect, attempting to find optimal learning rules which will enable us to create residuals whose pdfs match pdfs which we state we will find interesting a priori. This use of the word “interesting” may evoke a memory of our use of the term in Chapter 6, and so it should. We will, in the second half of this chapter, sphere the data and then, by deciding in advance that it will be interesting to leave particular residuals after feedback, create filters which are rather similar to those which we created using the Exploratory Projection Pursuit network of Chapter 6. We will compare these two sets of rules and then combine them in a joint learning rule which attempts to get the best effects from both.