Variational Information Maximization for Neural Coding
Mutual Information (MI) is a long studied measure of coding efficiency, and many attempts to apply it to population coding have been made. However, this is a computationally intractable task, and most previous studies redefine the criterion in forms of approximations. Recently we described properties of a simple lower bound on MI . Here we describe the bound optimization procedure for learning of population codes in a simple point neural model. We compare our approach with other techniques maximizing approximations of MI, focusing on a comparison with the Fisher Information criterion.
Unable to display preview. Download preview PDF.
- 1.Agakov, F.V., Barber, D.: Variational Information Maximization and Fisher Information. Technical report, UoE (2004)Google Scholar
- 2.Barber, D., Agakov, F.V.: The IM Algorithm: A Variational Approach to Information Maximization. In: NIPS (2003)Google Scholar
- 7.Linsker, R.: An Application of the Principle of Maximum Information to Linear Systems. In: NIPS (1989)Google Scholar