Skip to main content
Log in

Generative binary codes

  • Published:
Formal Pattern Analysis & Applications Aims and scope Submit manuscript

An Erratum to this article was published on 16 June 2004


A generative or latent variable model corresponds to a Bayesian network where arcs point from (presumed) hidden sources to observed variables. In this paper we introduce a particular generative model with binary valued hidden sources (i. e. each source can be either ‘on’ or ‘off’) but continuous observable variables. The purpose of this model is to learn a binary, distributed code for continuous data, for example to learn a bit code for gray value image data. For inference we rely on a mean field approximation. A novel and surprisingly simple derivation of general mean field equations is given. The structure of our model is chosen such that it is optimally suited for the structure of mean field inference. Hence, the mean field equations can be solved efficiently even with a few hundred hidden nodes, thus allowing one to learn highly distributed codes. For learning the parameters in the generative model from data, an appropriate EM-procedure is derived. In the second part of the paper we employ our approach to learning a sparse representation of natural images which is applied to code and to denoise images. The image compression rate is comparable to JPEG-coding and image denoising clearly outperforms standard methods such as Wiener filtering. In the outlook we present potential further directions of research in particular with respect to a more complex hidden topography.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Author information

Authors and Affiliations


Corresponding author

Correspondence to M. Haft.

Additional information

An erratum to this article can be found at

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haft, M., Hofman, R. & Tresp, V. Generative binary codes. Formal Pattern Analysis & Applications 6, 269–284 (2004).

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: