Overcomplete and Non-negative Models

Abstract

In this chapter, we discuss two generalizations of the basic ICA and sparse coding models. These do not reject the assumption of independence of the components but change some of the other assumptions in the model. Although the generative models are linear, the computation of the features is non-linear. In the overcomplete basis model, the number of independent components is larger than the number of pixels. In the non-negative model, the components, as well as the feature vectors, are constrained to be non-negative.