International Journal of Computer Vision

, Volume 109, Issue 1–2, pp 110–125 | Cite as

Model-Driven Domain Adaptation on Product Manifolds for Unconstrained Face Recognition

Article
First Online:
Received:
Accepted:

Abstract

Many classification algorithms see a reduction in performance when tested on data with properties different from that used for training. This problem arises very naturally in face recognition where images corresponding to the source domain (gallery, training data) and the target domain (probe, testing data) are acquired under varying degree of factors such as illumination, expression, blur and alignment. In this paper, we account for the domain shift by deriving a latent subspace or domain, which jointly characterizes the multifactor variations using appropriate image formation models for each factor. We formulate the latent domain as a product of Grassmann manifolds based on the underlying geometry of the tensor space, and perform recognition across domain shift using statistics consistent with the tensor geometry. More specifically, given a face image from the source or target domain, we first synthesize multiple images of that subject under different illuminations, blur conditions and 2D perturbations to form a tensor representation of the face. The orthogonal matrices obtained from the decomposition of this tensor, where each matrix corresponds to a factor variation, are used to characterize the subject as a point on a product of Grassmann manifolds. For cases with only one image per subject in the source domain, the identity of target domain faces is estimated using the geodesic distance on product manifolds. When multiple images per subject are available, an extension of kernel discriminant analysis is developed using a novel kernel based on the projection metric on product spaces. Furthermore, a probabilistic approach to the problem of classifying image sets on product manifolds is introduced. We demonstrate the effectiveness of our approach through comprehensive evaluations on constrained and unconstrained face datasets, including still images and videos.

Keywords

Domain adaptation Unconstrained face recognition Manifold learning Tensor computation 
This is a preview of subscription content, log in to check access.

References

  1. Arandjelović, O. (2009). Unfolding a face: From singular to manifold. In Proc. ACCV (pp. 203–213).Google Scholar
  2. Arandjelović, O., Shakhnarovich, G., Fisher, J., Cipolla, R., & Darrell, T. (2005). Face recognition with image sets using manifold density divergence. In Proc. CVPR (pp. 581–588).Google Scholar
  3. Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.CrossRefGoogle Scholar
  4. Begelfor, E., & Werman, M. (2006). Affine invariance revisited. In Proc. CVPR (pp 2087–2094).Google Scholar
  5. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010). A theory of learning from different domains. Machine Learning, 79(1), 151–175.CrossRefMathSciNetGoogle Scholar
  6. Biswas, S., Aggarwal, G., & Chellappa, R. (2009). Robust estimation of albedo for illumination-invariant matching and shape recovery. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 884–899.CrossRefGoogle Scholar
  7. Björck, A., & Golub, G. H. (1973). Numerical methods for computing angles between linear subspaces. Mathematics of Computations, 27, 579–594.CrossRefMATHGoogle Scholar
  8. Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3D faces. In SIGGRAPH (pp. 187–194).Google Scholar
  9. Brooks, M. J., & Horn, B. K. P. (1985). Shape and source from shading. In Proc. IJAI (pp. 932–936).Google Scholar
  10. Cevikalp, H., & Triggs, B. (2010). Face recognition based on image sets. In Proc. CVPR (pp. 2567–2573).Google Scholar
  11. Chen, Y., Patel, V. M., Phillips, P. J., & Chellappa, R. (2012). Dictionary-based face recognition from video. In Proc. ECCV (pp. 766–779).Google Scholar
  12. Daumé, H, I. I. I., & Marcu, D. (2006). Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26(1), 101–126.MATHMathSciNetGoogle Scholar
  13. Duan, L., Tsang, I., Xu, D., & Chua, T.-S. (2009). Domain adaptation from multiple sources via auxiliary classifiers. In Proc. ICML (pp. 289–296).Google Scholar
  14. Duan, L., Tsang, I., Xu, D., & Chua, T.-S. (2012). Domain transfer multiple Kernel learning. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3), 465–479.CrossRefGoogle Scholar
  15. Edelman, A., Arias, T. A., & Smith, S. T. (1999). The geometry of algorithms with orthogonality constraints. The SIAM Journal on Matrix Analysis and Applications, 20, 303–353.CrossRefMathSciNetGoogle Scholar
  16. Gong, B., Shi, Y., Sha, F., & Grauman, K. (2012). Geodesic flow Kernel for unsupervised domain adaptation. In Proc. CVPR (pp. 2066–2073).Google Scholar
  17. Gopalan, R., Li, R., & Chellappa, R. (2011). Domain adaptation for object recognition: An unsupervised approach. In Proc. ICCV (pp. 999–1006).Google Scholar
  18. Gopalan, R., Taheri, S., Turaga, P., & Chellappa, R. (2012). A blur-robust descriptor with applications to face recognition. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1220–1226.CrossRefGoogle Scholar
  19. Hamm, J., & Lee, D. (2008). Grassmann discriminant analysis: A unifying view on subspace-based learning. In Proc. ICML (pp. 376–383).Google Scholar
  20. Hoffman, J., Kulis, B., Darrell, T., & Saenko, K. (2012). Discovering latent domains for multisource domain adaptation. In Proc. ECCV (pp. 702–715).Google Scholar
  21. Hu, Y., Mian, A. S., & Owens, R. (2011). Sparse approximated nearest points for image set classification. In Proc. CVPR (pp. 27–40).Google Scholar
  22. Huang, G. B., Jain, V., & Learned-Miller, E. (2007). Unsupervised joint alignment of complex images. In Proc. ICCV.Google Scholar
  23. Huang, G. B., Ramesh, M., Berg, T., & Learned-Miller, E. (2007). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07–49, University of Massachusetts, Amherst.Google Scholar
  24. Jhuo, I.-H., Liu, D., Lee, D., & Chang, S.-F. (2012). Robust visual domain adaptation with low-rank reconstruction. In Proc. CVPR (pp. 2168–2175).Google Scholar
  25. Jia, H., & Martinez, A. M. (2008). Face recognition with occlusions in the training and testing sets. In Proc. FG (pp. 1–6).Google Scholar
  26. Jia, H., & Martinez, A. M. (2009). Support vector machines in face recognition with occlusions. In Proc. CVPR (pp. 136–141).Google Scholar
  27. Joliffe, I. T. (1986). Principal component analysis. Berlin: Springer.CrossRefGoogle Scholar
  28. Kim, T. K., Kittler, J., & Cipolla, R. (2007). Discriminative learning and recognition of image set classes using canonical correlations. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 1005–1018.CrossRefGoogle Scholar
  29. Kulis, B., Saenko, K., & Darrell, T. (2011). What you saw is not what you get: Domain adaptation using asymmetric Kernel transforms. In Proc. CVPR (pp. 1785–1792).Google Scholar
  30. Lee, J. M. (2010). Introduction to topological manifolds (2nd ed.). Berlin: Springer.Google Scholar
  31. Lee, K. C., Ho, J., & Kriegman, D. J. (2005). Acquiring linear subspaces for face recognition under variable lighting. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 684–698.CrossRefGoogle Scholar
  32. Lee, K. C., Ho, J., Yang, M. H., & Kriegman, D. J. (2005). Visual tracking and recognition using probabilistic appearance manifolds. CVIU, 99(3), 303–331.Google Scholar
  33. Li, H., Hua, G., Lin, Z., Brandt, J., & Yang, J. (2013). Probabilistic elastic matching for pose invariant face recognition. In Proc. CVPR.Google Scholar
  34. Li, Y., Du, Y., & Lin, X. (2005). Kernel-based multifactor analysis for image synthesis and recognition. In Proc. ICCV (pp. 114–119).Google Scholar
  35. Liu, J., Chen, S., Zhou, Z., & Tan, X. (2007). Single image subspace for face recognition. In Proc. AMFG (pp. 205–219).Google Scholar
  36. Lowe, D. J. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60(2), 91–110.CrossRefGoogle Scholar
  37. Lu, J., Tan, Y.P., & Wang, G. (2011). Discriminative multi-manifold analysis for face recognition from a single training sample per person. In Proc. ICCV (pp. 1943–1950).Google Scholar
  38. Lui, Y. M. (2012). Advances in matrix manifolds for computer vision. Image and Vision Computing, 30, 380–388.CrossRefGoogle Scholar
  39. Lui, Y. M., & Beveridge, J. R. (2008). Grassmann registration manifolds for face recognition. In Proc. ECCV (vol. 2, pp. 44–57).Google Scholar
  40. Lui, Y. M., Beveridge, J. R., & Kirby, M. (2010). Action classifications on product manifolds. In Proc. CVPR (pp. 833–839).Google Scholar
  41. Martinez, A. M. (2009). Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6), 748–763.CrossRefGoogle Scholar
  42. Martinez, A. M., & Benavente, R. (1998). The AR Face Database. CVC Technical Report, 24.Google Scholar
  43. Ni, J., Qiu, Q., & Chellappa, R. (2013). Subspace interpolation via dictionary learning for unsupervised domain adaptation. In Proc. CVPR.Google Scholar
  44. Nishiyama, M., Hadid, A., Takeshima, H., Shotton, J., Kozakaya, T., & Yamaguchi, O. (2011). Facial deblur inference using subspace analysis for recognition of blurred faces. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 838–845.CrossRefGoogle Scholar
  45. Nowak, E., & Jurie, F. (2007). Learning visual similarities measures for comparing never seen objects. In Proc. CVPR.Google Scholar
  46. Ojala, T., Pietikäinen, M., & Mäenpää, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.CrossRefGoogle Scholar
  47. Ojansivu, V., & Heikkilä, J. (2008). Blur insensitive texture classification using local phase quantization. In Proc. ICISP (pp. 236–243).Google Scholar
  48. Park, S. W., & Savvides, M. (2010). An extension of multifactor analysis for face recognition based on submanifold learning. In Proc. CVPR (pp. 2645–2652).Google Scholar
  49. Park, S. W., & Savvides, M. (2011a). Multifactor analysis based on factor-dependent geometry. In Proc. CVPR (pp. 2817–2824).Google Scholar
  50. Park, S. W., & Savvides, M. (2011b). The multifactor extension of grassmann manifolds for face recognition. In Proc. FG (pp. 464–469). Google Scholar
  51. Pinto, N., Dicarlo, J. J., & Cox, D. D. (2009). How far can you get with a modern face recognition test set using only simple features. In Proc. CVPR.Google Scholar
  52. Qiu, Q., Patel, V., Turaga, P., & Chellappa, R. (2012). Domain adaptive dictionary learning. In Proc. ECCV (pp. 631–645).Google Scholar
  53. Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.CrossRefGoogle Scholar
  54. Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010). Adapting visual category models to new domains. In Proc. ECCV (pp. 213–226).Google Scholar
  55. Sanderson, C., & Lovell, B. C. (2009). Multi-region probabilistic histograms for robust and scalable identity inference. In Proc. ICB (pp. 199–208).Google Scholar
  56. Shakhnarovich, G., Fisher, J. W., & Darell, T. (2002). Face recognition from long term observations. In Proc. ECCV (pp. 851–868).Google Scholar
  57. Shekhar, S., Patel, V., Nguyen, H., & Chellappa, R. (2013). Generalized domain-adaptive dictionaries. In Proc. CVPR.Google Scholar
  58. Shi, Y., & Sha, F. (2012). Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. In Proc. ICML.Google Scholar
  59. Sim, T., Baker, S., & Bsat, M. (2003). The CMU pose, illumination, and expression database. The IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1615–1618.CrossRefGoogle Scholar
  60. Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for non-linear dimensionality reduction. Science, 290, 2319–2323.CrossRefGoogle Scholar
  61. Vageeswaran, P., Mitra, K., & Chellappa, R. (2013). Blur and illumination robust face recognition via set-theoretic characterization. IEEE Transactions on Image Processing, 22(4), 1362–1372.CrossRefMathSciNetGoogle Scholar
  62. Vapnik, V. N. (1998). Statistical Learning Theory. New York: Wiley.MATHGoogle Scholar
  63. Vasilescu, M. A. O., & Terzopoulos, D. (2002). Multilinear analysis of image ensembles. In Proc. ECCV (pp. 447–460).Google Scholar
  64. Vasilescu, M. A. O., & Terzopoulos, D. (2007). Multilinear projection for appearance-based recognition in the tensor framework. In Proc. ICCV (pp. 1–8).Google Scholar
  65. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proc. CVPR (pp. 511–518).Google Scholar
  66. Wang, R., & Chen, X. (2009). Manifold discriminant analysis. In Proc. CVPR (pp. 429–436).Google Scholar
  67. Wolf, L., Hassner, T., & Taigman, Y. (2008). Descriptor based methods in the wild. In Faces in real-life images workshop in ECCV.Google Scholar
  68. Yang, J., Yan, R., & Hauptmann, A. (2007). Cross-domain video concept detection using adaptive SVMs. In Proc. ACM MM (pp. 188–197).Google Scholar
  69. Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Computing Surveys, 35(4), 399–458.CrossRefGoogle Scholar
  70. Zheng, J., Liu, M.-Y., Chellappa, R., & Phillips, J. (2012). A grassmann manifold-based domain adaptation approach. In Proc. ICPR (pp. 2095–2099).Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Electrical and Computer Engineering and the Center for Automation Research, UMIACSUniversity of MarylandCollege ParkUSA
  2. 2.Department of Video and Multimedia Technologies ResearchAT&T Labs - ResearchMiddletownUSA

Personalised recommendations