Multilinear Analysis of Image Ensembles: TensorFaces
Natural images are the composite consequence of multiple factors related to scene structure, illumination, and imaging. Multilinear algebra, the algebra of higher-order tensors, offers a potent mathematical framework for analyzing the multifactor structure of image ensembles and for addressing the difficult problem of disentangling the constituent factors or modes. Our multilinear modeling technique employs a tensor extension of the conventional matrix singular value decomposition (SVD), known as the N-mode SVD. As a concrete example, we consider the multilinear analysis of ensembles of facial images that combine several modes, including different facial geometries (people), expressions, head poses, and lighting conditions. Our resulting “TensorFaces” representation has several advantages over conventional eigenfaces. More generally, multilinear analysis shows promise as a unifying framework for a variety of computer vision problems.
KeywordsSingular Value Decomposition Independent Component Analysis Facial Image Independent Component Analysis Multilinear Algebra
Unable to display preview. Download preview PDF.
- 1.P.N. Belhumeur, J. Hespanha, and D.J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. In Proceedings of the European Conference on Computer Vision, pages 45–58, 1996.Google Scholar
- 2.R. Chellappa, C.L. Wilson, and S. Sirohey. Human and machine recognition of faces: A survey. Proceedings of the IEEE, 83(5):705–740, May 1995.Google Scholar
- 3.L. de Lathauwer. Signal Processing Based on Multilinear Algebra. PhD thesis, Katholieke Univ. Leuven, Belgium, 1997.Google Scholar
- 4.W. Freeman and J. Tenenbaum. Learing bilinear models for two-factor problems in vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 554–560, 1997.Google Scholar
- 5.A. Kapteyn, H. Neudecker, and T. Wansbeek. An approach to n-mode component analysis. Psychometrika, 51(2):269–275, June 1986.Google Scholar
- 7.J. B. Kruskal. Rank, decomposition, and uniqueness for 3-way and n-way array. In R. Coppi and S. Bolasco, editors, Multiway Data Analysis, pages 7–18, Amsterdam, 1989. North Holland.Google Scholar
- 9.D.H. Marimont and B.A. Wandell. Linear models of surface and illuminance spectra. J. Optical Society of America, A., 9:1905–1913, 1992.Google Scholar
- 11.A. Pentland and B. Moghaddam. View-based and modular eigenspaces for face recognition. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1994.Google Scholar
- 12.A. Shashua and A. Levin. Linear image coding for regression and classification using the tensor-rank principle. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, page in press, Hawai, 2001.Google Scholar
- 14.J.B. Tenenbaum and W.T. Freeman. Separating style and content. In M. Moser, M. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, pages 662–668. MIT Press, 1997.Google Scholar
- 18.M.A.O. Vasilescu. An algorithm for extracting human motion signatures. In IEEE Conference on Computer Vision and Pattern Recognition, Hawai, 2001. in press.Google Scholar
- 19.M. A. O. Vasilescu. Human motion signatures for character animation. In ACM SIGGRAPH 2001 Conf. Abstracts and Applications, page 200, Los Angeles, August 2001.Google Scholar