ECCV 2016: Computer Vision – ECCV 2016 pp 597-613 | Cite as

Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation

  • Muhammad Ghifary
  • W. Bastiaan Kleijn
  • Mengjie Zhang
  • David Balduzzi
  • Wen Li
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9908)

Abstract

In this paper, we propose a novel unsupervised domain adaptation algorithm based on deep learning for visual object recognition. Specifically, we design a new model called Deep Reconstruction-Classification Network (DRCN), which jointly learns a shared encoding representation for two tasks: (i) supervised classification of labeled source data, and (ii) unsupervised reconstruction of unlabeled target data. In this way, the learnt representation not only preserves discriminability, but also encodes useful information from the target domain. Our new DRCN model can be optimized by using backpropagation similarly as the standard neural networks.

We evaluate the performance of \( DRCN \) on a series of cross-domain object recognition tasks, where \( DRCN \) provides a considerable improvement (up to \(\sim \)8\(\%\) in accuracy) over the prior state-of-the-art algorithms. Interestingly, we also observe that the reconstruction pipeline of \( DRCN \) transforms images from the source domain into images whose appearance resembles the target dataset. This suggests that \( DRCN \)’s performance is due to constructing a single composite representation that encodes information about both the structure of target images and the classification of source images. Finally, we provide a formal analysis to justify the algorithm’s objective in domain adaptation context.

Keywords

Domain adaptation Object recognition Deep learning Convolutional networks Transfer learning 
This is a preview of subscription content, log in to check access.

Supplementary material

419976_1_En_36_MOESM1_ESM.pdf (2.6 mb)
Supplementary material 1 (pdf 2657 KB)

References

  1. 1.
    Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR, pp. 1521–1528 (2011)Google Scholar
  2. 2.
    Hsu, H.J., Chen, K.T.: Face recognition on drones: issues and limitations. In: Proceedings of ACM DroNet 2015 (2015)Google Scholar
  3. 3.
    Patel, V.M., Gopalan, R., Li, R., Chellapa, R.: Visual domain adaptation: a survey of recent advances. IEEE Signal Process. Mag. 32(3), 53–69 (2015)CrossRefGoogle Scholar
  4. 4.
    Aljundi, R., Emonet, R., Muselet, D., Sebban, M.: Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In: CVPR (2015)Google Scholar
  5. 5.
    Baktashmotlagh, M., Harandi, M.T., Lovell, B.C., Salzmann, M.: Unsupervised domain adaptation by domain invariant projection. In: ICCV, pp. 769–776 (2013)Google Scholar
  6. 6.
    Bruzzone, L., Marconcini, M.: Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE TPAMI 32(5), 770–787 (2010)CrossRefGoogle Scholar
  7. 7.
    Gong, B., Grauman, K., Sha, F.: Connecting the dots with landmarks: discriminatively learning domain-invariant features for unsupervised domain adaptation. In: ICML (2013)Google Scholar
  8. 8.
    Long, M., Ding, G., Wang, J., Sun, J., Guo, Y., Yu, P.S.: Transfer sparse coding for robust image representation. In: CVPR, pp. 404–414 (2013)Google Scholar
  9. 9.
    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S.: Transfer joint matching for unsupervised domain adaptation. In: CVPR, pp. 1410–1417 (2014)Google Scholar
  10. 10.
    Pan, S.J., Tsang, I.W.H., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  11. 11.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 213–226. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_16 CrossRefGoogle Scholar
  12. 12.
    Caruana, R.: Multitask learning. Mach. Learn. 28, 41–75 (1997)CrossRefGoogle Scholar
  13. 13.
    Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 41–48. MIT Press, Cambridge (2006)Google Scholar
  14. 14.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Classification with deep convolutional neural networks. In: NIPS, vol. 25, pp. 1106–1114 (2012)Google Scholar
  15. 15.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  16. 16.
    Hinton, G.E., Osindero, S.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy Layer-Wise training of deep networks. In: NIPS, vol. 19, pp. 153–160 (2007)Google Scholar
  18. 18.
    Ganin, Y., Lempitsky, V.S.: Unsupervised domain adaptation by backpropagation. In: ICML, pp. 1180–1189 (2015)Google Scholar
  19. 19.
    Cohen, I., Cozman, F.G.: Risks of semi-supervised learning: how unlabeled data can degrade performance of generative classifiers. In: Semi-Supervised Learning. MIT Press (2006)Google Scholar
  20. 20.
    Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–450 (2002)MATHGoogle Scholar
  21. 21.
    Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Plann. Infer. 90(2), 227–244 (2000)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Zadrozny, B.: Learning and evaluating classifiers under sample selection bias. In: Proceedings of the 21th Annual International Conference on Machine Learning, pp. 114–121 (2004)Google Scholar
  23. 23.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  24. 24.
    Blitzer, J., McDonald, R., Pereira, F.: Domain adaptation with structural correspondence learning. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pp. 120–128 (2006)Google Scholar
  25. 25.
    Daumé-III, H.: Frustratingly easy domain adaptation. In: Proceedings of ACL (2007)Google Scholar
  26. 26.
    Margolis, A.: A literature review of domain adaptation with unlabeled data. Technical report, University of Washington (2011)Google Scholar
  27. 27.
    Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: ICCV, pp. 2960–2967 (2013)Google Scholar
  28. 28.
    Ghifary, M., Balduzzi, D., Kleijn, W.B., Zhang, M.: Scatter component analysis: a unified framework for domain adaptation and domain generalization. CoRR abs/1510.04373 (2015)Google Scholar
  29. 29.
    Gopalan, R., Li, R., Chellapa, R.: Domain adaptation for object recognition: an unsupervised approach. In: ICCV, pp. 999–1006 (2011)Google Scholar
  30. 30.
    Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow Kernel for unsupervised domain adaptation. In: CVPR, pp. 2066–2073 (2012)Google Scholar
  31. 31.
    Bay, H., Tuytelaars, T., Gool, L.V.: SURF: speeded up robust features. CVIU 110(3), 346–359 (2008)Google Scholar
  32. 32.
    Glorot, X., Bordes, A., Bengio, Y.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: ICML, pp. 513–520 (2011)Google Scholar
  33. 33.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)MathSciNetMATHGoogle Scholar
  34. 34.
    Ghifary, M., Kleijn, W.B., Zhang, M.: Domain adaptive neural networks for object recognition. In: Pham, D.-N., Park, S.-B. (eds.) PRICAI 2014. LNCS (LNAI), vol. 8862, pp. 898–904. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-13560-1_76 Google Scholar
  35. 35.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  36. 36.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)Google Scholar
  37. 37.
    Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: ICML (2014)Google Scholar
  38. 38.
    Hoffman, J., Tzeng, E., Donahue, J., Jia, Y., Saenko, K., Darrell, T.: One-Shot Adaptation of Supervised Deep Convolutional Models (2013). CoRR abs/1312.6204Google Scholar
  39. 39.
    Chopra, S., Balakrishnan, S., Gopalan, R.: DLID: deep learning for domain adaptation by interpolating between domains. In: ICML Workshop on Challenges in Representation Learning (2013)Google Scholar
  40. 40.
    Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. In: ICML (2015)Google Scholar
  41. 41.
    Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: maximizing for domain invariance (2014). CoRR abs/1412.3474Google Scholar
  42. 42.
    Tzeng, E., Hoffman, J., Darrell, T., Saenko, K.: Simultaneous deep transfer across domains and tasks. In: ICCV (2015)Google Scholar
  43. 43.
    Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.P., Schölkopf, B., Smola, A.J.: Integrating structured biological data by Kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)CrossRefGoogle Scholar
  44. 44.
    Masci, J., Meier, U., Cireşan, D., Schmidhuber, J.: Stacked convolutional auto-encoders for hierarchical feature extraction. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 52–59. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-21735-7_7 CrossRefGoogle Scholar
  45. 45.
    Zeiler, M.D., Krishnan, D., Taylor, G.W., Fergus, R.: Deconvolutional networks. In: CVPR, pp. 2528–2535 (2010)Google Scholar
  46. 46.
    Tieleman, T., Hinton, G.: Lecture 6.5–RmsProp: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. (2012)Google Scholar
  47. 47.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. JMLR 15, 1929–1958 (2014)MathSciNetMATHGoogle Scholar
  48. 48.
    Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I.J., Bergeron, A., Bouchard, N., Bengio, Y.: Theano: new features and speed improvements. In: Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop (2012)Google Scholar
  49. 49.
    Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, vol. 2, pp. 958–962 (2003)Google Scholar
  50. 50.
    Hull, J.J.: A database for handwritten text recognition research. IEEE TPAMI 16(5), 550–554 (1994)CrossRefGoogle Scholar
  51. 51.
    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011)Google Scholar
  52. 52.
    Krizhevsky, A.: Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto, April 2009Google Scholar
  53. 53.
    Coates, A., Lee, H., Ng, A.Y.: An analysis of single-layer networks in unsupervised feature learning. In: AISTATS, pp. 215–223 (2011)Google Scholar
  54. 54.
    Sermanet, P., Chintala, S., LeCun, Y.: Convolutional neural networks applied to house number digit classification. In: ICPR, pp. 3288–3291 (2012)Google Scholar
  55. 55.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML (2010)Google Scholar
  56. 56.
    White, H.: Maximum likelihood estimation of misspecified models. Econometrica 50(1), 1–25 (1982)MathSciNetCrossRefMATHGoogle Scholar
  57. 57.
    Bengio, Y., Yao, L., Guillaume, A., Vincent, P.: Generalized denoising auto-encoders as generative models. In: NIPS, pp. 899–907 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Muhammad Ghifary
    • 1
    • 3
  • W. Bastiaan Kleijn
    • 1
  • Mengjie Zhang
    • 1
  • David Balduzzi
    • 1
  • Wen Li
    • 2
  1. 1.Victoria University of WellingtonWellingtonNew Zealand
  2. 2.ETH ZürichZürichSwitzerland
  3. 3.Weta DigitalWellingtonNew Zealand

Personalised recommendations