Learning Kernels for Unsupervised Domain Adaptation with Applications to Visual Object Recognition

  • Boqing GongEmail author
  • Kristen Grauman
  • Fei Sha


Domain adaptation aims to correct the mismatch in statistical properties between the source domain on which a classifier is trained and the target domain to which the classifier is to be applied. In this paper, we address the challenging scenario of unsupervised domain adaptation, where the target domain does not provide any annotated data to assist in adapting the classifier. Our strategy is to learn robust features which are resilient to the mismatch across domains and then use them to construct classifiers that will perform well on the target domain. To this end, we propose novel kernel learning approaches to infer such features for adaptation. Concretely, we explore two closely related directions. In the first direction, we propose unsupervised learning of a geodesic flow kernel (GFK). The GFK summarizes the inner products in an infinite sequence of feature subspaces that smoothly interpolates between the source and target domains. In the second direction, we propose supervised learning of a kernel that discriminatively combines multiple base GFKs. Those base kernels model the source and the target domains at fine-grained granularities. In particular, each base kernel pivots on a different set of landmarks—the most useful data instances that reveal the similarity between the source and the target domains, thus bridging them to achieve adaptation. Our approaches are computationally convenient, automatically infer important hyper-parameters, and are capable of learning features and classifiers discriminatively without demanding labeled data from the target domain. In extensive empirical studies on standard benchmark recognition datasets, our appraches yield state-of-the-art results compared to a variety of competing methods.


Domain adaptation Kernels  Object recognition  Cross-dataset bias 



This work is partially supported by DARPA D11-AP00278 and NSF IIS-1065243 (B.G. and F.S.), and ONR ATL #N00014-11-1-0105 (K.G.). We thank the anonymous reviewers for their constructive comments and suggestions. The Flickr images in Fig. 1 are under a CC Attribution 2.0 Generic license, courtesy of berzowska,, warrantedarrest, HerryLawford, yuichi.sakuraba, zimaowuyu, GrahamAndDairne, Bernt Rostad, Keith Roper, flavorrelish, and deflam.


  1. Ando, R., & Zhang, T. (2005). A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR, 6, 1817–1853.zbMATHMathSciNetGoogle Scholar
  2. Bay, H., Tuytelaars, T., & Van Gool, L. (2006). SURF: Speeded up robust features. In ECCV.Google Scholar
  3. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Wortman, J. (2010). A theory of learning from different domains. Machine Learning, 79, 151–175.CrossRefMathSciNetGoogle Scholar
  4. Ben-David, S., Blitzer, J., Crammer, K., & Pereira, F. (2007). Analysis of representations for domain adaptation. In NIPS.Google Scholar
  5. Bergamo, A., & Torresani, L. (2010). Exploiting weakly-labeled web images to improve object classification: a domain adaptation approach. In NIPS.Google Scholar
  6. Blitzer, J., Dredze, M., & Pereira, F. (2007). Biographies, Bolly-wood, boomboxes and blenders: Domain adaptation for sentiment classification. In ACL.Google Scholar
  7. Blitzer, J., Foster, D., & Kakade, S. (2011). Domain adaptation with coupled subspaces. In AISTATS.Google Scholar
  8. Blitzer, J., McDonald, R., & Pereira, F. (2006). Domain adaptation with structural correspondence learning. In EMNLP.Google Scholar
  9. Bruzzone, L., & Marconcini, M. (2010). Domain adaptation problems: A DASVM classification technique and a circular validation strategy. IEEE PAMI, 32(5), 770–787.CrossRefGoogle Scholar
  10. Chen, M., Weinberger, K., & Blitzer, J. (2011). Co-training for domain adaptation. In NIPS.Google Scholar
  11. Daumé, H., III. (2007). Frustratingly easy domain adaptation. In ACL.Google Scholar
  12. Daumé, H., Kumar, A., & Saha, A. (2010). Co-regularization based semi-supervised domain adaptation. In NIPS.Google Scholar
  13. Daumé, H, I. I. I., & Marcu, D. (2006). Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26(1), 101–126.zbMATHMathSciNetGoogle Scholar
  14. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In CVPR.Google Scholar
  15. Dollár, P., Wojek, C., Schiele, B., & Perona, P. (2009). Pedestrian detection: A benchmark. In CVPR.Google Scholar
  16. Dredze, M., & Crammer, K. (2008). Online methods for multi-domain learning and adaptation. In Proceedings of the conference on empirical methods in natural language processing (EMNLP ’08) (pp. 689–697).Google Scholar
  17. Duan, L., Tsang, I., Xu, D., & Maybank, S. (2009). Domain transfer SVM for video concept detection. In CVPR.Google Scholar
  18. Duan, L., Xu, D., & Tsang, I. (2012). Domain adaptation from multiple sources: A domain-dependent regularization approach. IEEE Transactions on Neural Networks and Learning Systems, 23(3), 504–518.CrossRefGoogle Scholar
  19. Duan, L., Xu, D., Tsang, I., & Luo, J. (2010) Visual event recognition in videos by learning from web data. In CVPR.Google Scholar
  20. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., Zisserman, A. (2007). The PASCAL visual object classes, challenge 2007.Google Scholar
  21. Fei-Fei, L., Fergus, R., & Perona, P. (2007). Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Comp Vis & Img Under, 106(1), 59–70.CrossRefGoogle Scholar
  22. Gong, B., Grauman, K., & Sha, F. (2013a). Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In ICML.Google Scholar
  23. Gong, B., Grauman, K., & Sha, F. (2013b). Reshaping visual datasets for domain adaptation. In NIPS.Google Scholar
  24. Gong, B., Shi, Y., Sha, F., & Grauman, K. (2012). Geodesic flow kernel for unsupervised domain adaptation. In CVPR.Google Scholar
  25. Gopalan, R. (2013). Learning cross-domain information transfer for location recognition and clustering. In CVPR.Google Scholar
  26. Gopalan, R., Li, R., & Chellappa, R. (2011). Domain adaptation for object recognition: An unsupervised approach. In ICCV.Google Scholar
  27. Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A. (2006). A kernel method for the two-sample-problem. In NIPS.Google Scholar
  28. Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., & Scholkopf, B. (2009). Covariate shift by kernel mean matching. In J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, & N. Lawrence (Eds.), Dataset shift in machine learning. Cambridge: MIT Press.Google Scholar
  29. Griffin, G., Holub, A., & Perona, P. (2007). Caltech-256 object category dataset. Tech. rep., Caltech.Google Scholar
  30. Ham, J., Lee, D. D., Mika, S., Schölkopf, B. (2004). A kernel view of the dimensionality reduction of manifolds. In ICML.Google Scholar
  31. Hamm, J., & Lee, D. (2008). Grassmann discriminant analysis: A unifying view on subspace-based learning. In ICML.Google Scholar
  32. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning. Berlin: Springer.CrossRefzbMATHGoogle Scholar
  33. Huang, J., Smola, A., Gretton, A., Borgwardt, K., & Scholkopf, B. (2006). Correcting sample selection bias by unlabeled data. In NIPS.Google Scholar
  34. Jain, V., & Learned-Miller, E. (2011). Online domain adaptation of a pre-trained cascade of classifiers. In CVPR.Google Scholar
  35. Kulis, B,, Saenko, K., & Darrell, T. (2011). What you saw is not what you get: Domain adaptation using asymmetric kernel transforms. In CVPR.Google Scholar
  36. Lanckriet, G. R. G., Cristianini, N., Bartlett, P., Ghaoui, L. E., & Jordan, M. (2004). Learning the kernel matrix with semidefinite programming. JMLR, 5, 27–72.zbMATHGoogle Scholar
  37. Leggetter, C., & Woodland, P. (1995). Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models. Computer Speech and Language, 9(2), 171–185.CrossRefGoogle Scholar
  38. Li, R., & Zickler, T. (2012). Discriminative virtual views for cross-view action recognition. In CVPR.Google Scholar
  39. Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009a). Domain adaptation: Learning bounds and algorithms. Arxiv, preprint arXiv:09023430.Google Scholar
  40. Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009b). Multiple source adaptation and the rényi divergence. In UAI.Google Scholar
  41. Pan, S., Tsang, I., Kwok, J., & Yang, Q. (2009). Domain adaptation via transfer component analysis. IEEE Trans Neural Nets, 99, 1–12.Google Scholar
  42. Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Trans Knowledge and Data Engineering, 22(10), 1345–1359.Google Scholar
  43. Perronnin, F., Sánchez, J., & Liu, Y. (2010). Large-scale image categorization with explicit data embedding. In CVPR. Google Scholar
  44. Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2323–2326.CrossRefGoogle Scholar
  45. Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008). LabelMe: A database and web-based tool for image annotation. IJCV, 77, 157–173.CrossRefGoogle Scholar
  46. Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010). Adapting visual category models to new domains. In ECCV.Google Scholar
  47. Shi, Y,, & Sha, F. (2012). Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. In ICML.Google Scholar
  48. Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90(2), 227–244.CrossRefzbMATHMathSciNetGoogle Scholar
  49. Torralba, A., & Efros, A. (2011). Unbiased look at dataset bias. In CVPR.Google Scholar
  50. Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009). Multiple kernels for object detection. In ICCV.Google Scholar
  51. Wang, M., & Wang, X. (2011). Automatic adaptation of a generic pedestrian detector to a specific traffic scene. In CVPR.Google Scholar
  52. Weinberger, K. Q., & Saul, L. K. (2006). Unsupervised learning of image manifolds by semidefinite programming. International Journal of Computer Vision, 70(1), 77–90.CrossRefGoogle Scholar
  53. Zheng, J., Liu, M. Y., Chellappa, R., & Phillips, P. J. (2012) A grassmann manifold-based domain adaptation approach. In ICPR.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.Department of Computer ScienceUniversity of Texas at AustinAustinUSA

Personalised recommendations