Abstract
A Bayesian blind source separation (BSS) algorithm is proposed in this paper to recover independent sources from observed multivariate spatial patterns. As a widely used mechanism, Gaussian mixture model is adopted to represent the sources for statistical description and machine learning. In the context of linear latent variable BSS model, some conjugate priors are incorporated into the hyperparameters estimation of mixing matrix. The proposed algorithm then approximates the full posteriors over model structure and source parameters in an analytical manner based on variational Bayesian treatment. Experimental studies demonstrate that this Bayesian source separation algorithm is appropriate for systematic spatial pattern analysis by modeling arbitrary sources and identify their effects on high dimensional measurement data. The identified patterns will serve as diagnosis aids for gaining insight into the nature of physical process for the potential use of statistical quality control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Apley D.W. and Shi J. 2001. A factor analysis method for diagnosing variability in multivariate manufacturing processes. Technometrics 43: 84–95.
Apley D.W. and Lee H.Y. 2003. Identifying spatial variation patterns in multivariate manufacturing processes: A blind separation approach. Technometrics 45: 220–235.
Attias H. 2000. A variational bayesian framework for graphical models. Advances in Neural Information Processing Systems (NIPS) 12: 49–52.
Bartholomew D.J. and Knott M. 1999. Latent Variable Models and Factor Analysis. Oxford University Press, New York.
Bell A. and Sejnowski T. 1995. An information-maximization approach to blind separation and blind deconvolution. Neural Computation 7(6): 1129–1159.
Cardoso J.F. 1999. High-order contrasts for independent component analysis. Neural Computation 11: 157–192.
Ceglarek D. and Shi J. 1996. Fixture failure diagnosis for autobody assembly using pattern recognition. ASME Journal of Engineering for Industry 118: 55–66.
Comon P. 1994. Independent component analysis: A new concept? Signal Processing 36(3): 287–314.
Corduneanu A. and Bishop C.M. 2001. Variational Bayesian model selection for mixture distributions. In: Jaakkola T. and Richardson T. (Eds.), Artificial Intelligence and Statistics, Morgan Kaufmann, pp. 27–34.
Dempster A.P. et al. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society B 39: 1–38.
Figueiredo M.A. and Jain A.K. 2002. Unsupervised learning of finite mixture models. IEEE Trans. on Pattern Analysis and Machine Intelligence 24: 381–396.
Gelman A. et al. 2000. Bayesian Data Analysis, Chapman & Hall, London.
Haykin S. 2000. Unsupervised Adaptive Filtering, vol. 1, New York: Wiley.
Hyvarinen A. 1999. Survey on independent component analysis. Neural Computing Surveys 2: 94–128.
Hyvarinen A. and Oja E. 2000. Independent component analysis: Algorithms and applications. Neural Networks 13: 411–430.
Jackson J.E. 1980. Principal components and factor analysis: Part I—principal components. Journal of Quality Technology 12: 201–213.
Jackson J.E. 1981. Principal components and factor analysis: Part II—additional topics related to principal components. Journal of Quality Technology 13: 46–58.
Johnson R.A. and Wichern D.W. 2002. Applied Multivariate Statistical Analysis, 5th edn., Prentice Hall, Upper Saddle River, NJ.
Jordan M.I. and Jacobs R.A. 1994. Hierarchical mixtures of experts and the EM algorithm. Neural Computation 6: 181–214.
Jutten C. and Herault J. 1991. Blind separation of sources, Part I: An adaptive algorithm based on a neuromimetic architecture. Signal Processing 24: 1–10.
Knuth K.H. 1998. Bayesian source separation and localization. SPIE'98 Proceedings: Bayesian Inference for Inverse Problems 3459: 147–158, San Diego.
Lee T.W. et al. 2000. ICA mixture models for unsupervised classification of non-gaussian classes and automatic context switching in blind signal separation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22: 078–1089.
MacKay D.C. 1996. Maximum likelihood and covariant algorithms for independent component analysis, Technical report, Cavendish Laboratory, Cambridge University.
McLachlan G. and Peel D. 2000. Finite Mixture Models, John Wiley & Sons, New York.
Moulines E. et al. 1997. Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models. Proceedings of IEEE Conference on Acoustics, Speech, and Signal Processing 5: 3617–3620.
Pearlmutter B.A. and Parra L.C. 1997. Maximum likelihood blind source separation: A context-sensitive generalization of ICA. Advances in Neural Information Processing Systems 9: 613–619, MIT Press, Cambridge, MA.
Rowe D.B. 2002. A bayesian approach to blind source separation. Journal of Interdisciplinary Mathematics 5: 49–76.
Roweis S. and Ghahramani Z. 1999. A unifying review of linear gaussian models. Neural Computation 11(2): 305–345.
Tipping M.E. and Bishop C.M. 1999. Probabilistic principal component analysis. Journal of Royal Statistical Society B 61: 611–622.
Tong L. et al. 1990. AMUSE: A new blind identification algorithm. IEEE International Symposium on Circuits and Systems 3: 1784–1787.
Wax M. and Sheinvald J. 1997. A least-squares approach to joint diagonalization. IEEE Signal Processing Letters 4: 52–53.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, F., Mallick, B. & Weng, Z. A Bayesian method for identifying independent sources of non-random spatial patterns. Stat Comput 15, 329–339 (2005). https://doi.org/10.1007/s11222-005-4075-6
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11222-005-4075-6