Skip to main content
SpringerLink
Log in

Model-Driven Domain Adaptation on Product Manifolds for Unconstrained Face Recognition

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Many classification algorithms see a reduction in performance when tested on data with properties different from that used for training. This problem arises very naturally in face recognition where images corresponding to the source domain (gallery, training data) and the target domain (probe, testing data) are acquired under varying degree of factors such as illumination, expression, blur and alignment. In this paper, we account for the domain shift by deriving a latent subspace or domain, which jointly characterizes the multifactor variations using appropriate image formation models for each factor. We formulate the latent domain as a product of Grassmann manifolds based on the underlying geometry of the tensor space, and perform recognition across domain shift using statistics consistent with the tensor geometry. More specifically, given a face image from the source or target domain, we first synthesize multiple images of that subject under different illuminations, blur conditions and 2D perturbations to form a tensor representation of the face. The orthogonal matrices obtained from the decomposition of this tensor, where each matrix corresponds to a factor variation, are used to characterize the subject as a point on a product of Grassmann manifolds. For cases with only one image per subject in the source domain, the identity of target domain faces is estimated using the geodesic distance on product manifolds. When multiple images per subject are available, an extension of kernel discriminant analysis is developed using a novel kernel based on the projection metric on product spaces. Furthermore, a probabilistic approach to the problem of classifying image sets on product manifolds is introduced. We demonstrate the effectiveness of our approach through comprehensive evaluations on constrained and unconstrained face datasets, including still images and videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Arandjelović, O. (2009). Unfolding a face: From singular to manifold. In Proc. ACCV (pp. 203–213).

  • Arandjelović, O., Shakhnarovich, G., Fisher, J., Cipolla, R., & Darrell, T. (2005). Face recognition with image sets using manifold density divergence. In Proc. CVPR (pp. 581–588).

  • Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.

    Article  Google Scholar 

  • Begelfor, E., & Werman, M. (2006). Affine invariance revisited. In Proc. CVPR (pp 2087–2094).

  • Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010). A theory of learning from different domains. Machine Learning, 79(1), 151–175.

    Article  MathSciNet  Google Scholar 

  • Biswas, S., Aggarwal, G., & Chellappa, R. (2009). Robust estimation of albedo for illumination-invariant matching and shape recovery. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 884–899.

    Article  Google Scholar 

  • Björck, A., & Golub, G. H. (1973). Numerical methods for computing angles between linear subspaces. Mathematics of Computations, 27, 579–594.

    Article  MATH  Google Scholar 

  • Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3D faces. In SIGGRAPH (pp. 187–194).

  • Brooks, M. J., & Horn, B. K. P. (1985). Shape and source from shading. In Proc. IJAI (pp. 932–936).

  • Cevikalp, H., & Triggs, B. (2010). Face recognition based on image sets. In Proc. CVPR (pp. 2567–2573).

  • Chen, Y., Patel, V. M., Phillips, P. J., & Chellappa, R. (2012). Dictionary-based face recognition from video. In Proc. ECCV (pp. 766–779).

  • Daumé, H, I. I. I., & Marcu, D. (2006). Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26(1), 101–126.

    MATH  MathSciNet  Google Scholar 

  • Duan, L., Tsang, I., Xu, D., & Chua, T.-S. (2009). Domain adaptation from multiple sources via auxiliary classifiers. In Proc. ICML (pp. 289–296).

  • Duan, L., Tsang, I., Xu, D., & Chua, T.-S. (2012). Domain transfer multiple Kernel learning. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3), 465–479.

    Article  Google Scholar 

  • Edelman, A., Arias, T. A., & Smith, S. T. (1999). The geometry of algorithms with orthogonality constraints. The SIAM Journal on Matrix Analysis and Applications, 20, 303–353.

    Article  MathSciNet  Google Scholar 

  • Gong, B., Shi, Y., Sha, F., & Grauman, K. (2012). Geodesic flow Kernel for unsupervised domain adaptation. In Proc. CVPR (pp. 2066–2073).

  • Gopalan, R., Li, R., & Chellappa, R. (2011). Domain adaptation for object recognition: An unsupervised approach. In Proc. ICCV (pp. 999–1006).

  • Gopalan, R., Taheri, S., Turaga, P., & Chellappa, R. (2012). A blur-robust descriptor with applications to face recognition. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1220–1226.

    Article  Google Scholar 

  • Hamm, J., & Lee, D. (2008). Grassmann discriminant analysis: A unifying view on subspace-based learning. In Proc. ICML (pp. 376–383).

  • Hoffman, J., Kulis, B., Darrell, T., & Saenko, K. (2012). Discovering latent domains for multisource domain adaptation. In Proc. ECCV (pp. 702–715).

  • Hu, Y., Mian, A. S., & Owens, R. (2011). Sparse approximated nearest points for image set classification. In Proc. CVPR (pp. 27–40).

  • Huang, G. B., Jain, V., & Learned-Miller, E. (2007). Unsupervised joint alignment of complex images. In Proc. ICCV.

  • Huang, G. B., Ramesh, M., Berg, T., & Learned-Miller, E. (2007). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07–49, University of Massachusetts, Amherst.

  • Jhuo, I.-H., Liu, D., Lee, D., & Chang, S.-F. (2012). Robust visual domain adaptation with low-rank reconstruction. In Proc. CVPR (pp. 2168–2175).

  • Jia, H., & Martinez, A. M. (2008). Face recognition with occlusions in the training and testing sets. In Proc. FG (pp. 1–6).

  • Jia, H., & Martinez, A. M. (2009). Support vector machines in face recognition with occlusions. In Proc. CVPR (pp. 136–141).

  • Joliffe, I. T. (1986). Principal component analysis. Berlin: Springer.

    Book  Google Scholar 

  • Kim, T. K., Kittler, J., & Cipolla, R. (2007). Discriminative learning and recognition of image set classes using canonical correlations. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 1005–1018.

    Article  Google Scholar 

  • Kulis, B., Saenko, K., & Darrell, T. (2011). What you saw is not what you get: Domain adaptation using asymmetric Kernel transforms. In Proc. CVPR (pp. 1785–1792).

  • Lee, J. M. (2010). Introduction to topological manifolds (2nd ed.). Berlin: Springer.

    Google Scholar 

  • Lee, K. C., Ho, J., & Kriegman, D. J. (2005). Acquiring linear subspaces for face recognition under variable lighting. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 684–698.

    Article  Google Scholar 

  • Lee, K. C., Ho, J., Yang, M. H., & Kriegman, D. J. (2005). Visual tracking and recognition using probabilistic appearance manifolds. CVIU, 99(3), 303–331.

    Google Scholar 

  • Li, H., Hua, G., Lin, Z., Brandt, J., & Yang, J. (2013). Probabilistic elastic matching for pose invariant face recognition. In Proc. CVPR.

  • Li, Y., Du, Y., & Lin, X. (2005). Kernel-based multifactor analysis for image synthesis and recognition. In Proc. ICCV (pp. 114–119).

  • Liu, J., Chen, S., Zhou, Z., & Tan, X. (2007). Single image subspace for face recognition. In Proc. AMFG (pp. 205–219).

  • Lowe, D. J. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60(2), 91–110.

    Article  Google Scholar 

  • Lu, J., Tan, Y.P., & Wang, G. (2011). Discriminative multi-manifold analysis for face recognition from a single training sample per person. In Proc. ICCV (pp. 1943–1950).

  • Lui, Y. M. (2012). Advances in matrix manifolds for computer vision. Image and Vision Computing, 30, 380–388.

    Article  Google Scholar 

  • Lui, Y. M., & Beveridge, J. R. (2008). Grassmann registration manifolds for face recognition. In Proc. ECCV (vol. 2, pp. 44–57).

  • Lui, Y. M., Beveridge, J. R., & Kirby, M. (2010). Action classifications on product manifolds. In Proc. CVPR (pp. 833–839).

  • Martinez, A. M. (2009). Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6), 748–763.

    Article  Google Scholar 

  • Martinez, A. M., & Benavente, R. (1998). The AR Face Database. CVC Technical Report, 24.

  • Ni, J., Qiu, Q., & Chellappa, R. (2013). Subspace interpolation via dictionary learning for unsupervised domain adaptation. In Proc. CVPR.

  • Nishiyama, M., Hadid, A., Takeshima, H., Shotton, J., Kozakaya, T., & Yamaguchi, O. (2011). Facial deblur inference using subspace analysis for recognition of blurred faces. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 838–845.

    Article  Google Scholar 

  • Nowak, E., & Jurie, F. (2007). Learning visual similarities measures for comparing never seen objects. In Proc. CVPR.

  • Ojala, T., Pietikäinen, M., & Mäenpää, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.

    Article  Google Scholar 

  • Ojansivu, V., & Heikkilä, J. (2008). Blur insensitive texture classification using local phase quantization. In Proc. ICISP (pp. 236–243).

  • Park, S. W., & Savvides, M. (2010). An extension of multifactor analysis for face recognition based on submanifold learning. In Proc. CVPR (pp. 2645–2652).

  • Park, S. W., & Savvides, M. (2011a). Multifactor analysis based on factor-dependent geometry. In Proc. CVPR (pp. 2817–2824).

  • Park, S. W., & Savvides, M. (2011b). The multifactor extension of grassmann manifolds for face recognition. In Proc. FG (pp. 464–469).

  • Pinto, N., Dicarlo, J. J., & Cox, D. D. (2009). How far can you get with a modern face recognition test set using only simple features. In Proc. CVPR.

  • Qiu, Q., Patel, V., Turaga, P., & Chellappa, R. (2012). Domain adaptive dictionary learning. In Proc. ECCV (pp. 631–645).

  • Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.

    Article  Google Scholar 

  • Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010). Adapting visual category models to new domains. In Proc. ECCV (pp. 213–226).

  • Sanderson, C., & Lovell, B. C. (2009). Multi-region probabilistic histograms for robust and scalable identity inference. In Proc. ICB (pp. 199–208).

  • Shakhnarovich, G., Fisher, J. W., & Darell, T. (2002). Face recognition from long term observations. In Proc. ECCV (pp. 851–868).

  • Shekhar, S., Patel, V., Nguyen, H., & Chellappa, R. (2013). Generalized domain-adaptive dictionaries. In Proc. CVPR.

  • Shi, Y., & Sha, F. (2012). Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. In Proc. ICML.

  • Sim, T., Baker, S., & Bsat, M. (2003). The CMU pose, illumination, and expression database. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1615–1618.

    Article  Google Scholar 

  • Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for non-linear dimensionality reduction. Science, 290, 2319–2323.

    Article  Google Scholar 

  • Vageeswaran, P., Mitra, K., & Chellappa, R. (2013). Blur and illumination robust face recognition via set-theoretic characterization. IEEE Transactions on Image Processing, 22(4), 1362–1372.

    Article  MathSciNet  Google Scholar 

  • Vapnik, V. N. (1998). Statistical Learning Theory. New York: Wiley.

    MATH  Google Scholar 

  • Vasilescu, M. A. O., & Terzopoulos, D. (2002). Multilinear analysis of image ensembles. In Proc. ECCV (pp. 447–460).

  • Vasilescu, M. A. O., & Terzopoulos, D. (2007). Multilinear projection for appearance-based recognition in the tensor framework. In Proc. ICCV (pp. 1–8).

  • Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proc. CVPR (pp. 511–518).

  • Wang, R., & Chen, X. (2009). Manifold discriminant analysis. In Proc. CVPR (pp. 429–436).

  • Wolf, L., Hassner, T., & Taigman, Y. (2008). Descriptor based methods in the wild. In Faces in real-life images workshop in ECCV.

  • Yang, J., Yan, R., & Hauptmann, A. (2007). Cross-domain video concept detection using adaptive SVMs. In Proc. ACM MM (pp. 188–197).

  • Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Computing Surveys, 35(4), 399–458.

    Article  Google Scholar 

  • Zheng, J., Liu, M.-Y., Chellappa, R., & Phillips, J. (2012). A grassmann manifold-based domain adaptation approach. In Proc. ICPR (pp. 2095–2099).

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering and the Center for Automation Research, UMIACS, University of Maryland, College Park, MD , 20742, USA

    Huy Tho Ho

  2. Department of Video and Multimedia Technologies Research, AT&T Labs - Research, Middletown, NJ , 07748, USA

    Raghuraman Gopalan

Authors
  1. Huy Tho Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raghuraman Gopalan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huy Tho Ho.

Additional information

Communicated by Hal Daumé

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ho, H.T., Gopalan, R. Model-Driven Domain Adaptation on Product Manifolds for Unconstrained Face Recognition. Int J Comput Vis 109, 110–125 (2014). https://doi.org/10.1007/s11263-014-0720-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-014-0720-x

Keywords

Search

Navigation