Advertisement

Cognitive Computation

, Volume 10, Issue 4, pp 577–590 | Cite as

A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine

  • Boyang Zhang
  • Yingyi Liu
  • Haiwen Yuan
  • Lingjie Sun
  • Zhao Ma
Article

Abstract

Extreme learning machine (ELM) is a well-known cognitive model, that has been extended to cross-domain tasks. Nonetheless, most existing paradigms that are based on ELM either concern about a case in which a specific number of instances are labelled in the target domain or a learner is trained without sufficient capacity to eliminate the gap between domains. To cope with the scenario in which there are no target labels and to acquire a better adaptive learner, we propose a joint unsupervised cross-domain model via scalable discriminative ELM, which is abbreviated as JUC-SDELM. Within the framework, the scalable factor is integrated into discriminative ELM (DELM) to adjust the output margin, which strengthens the discriminative capacity of the ELM classifier. In addition, we follow the basic strategy of joint distribution adaptation (JDA) to align the subspaces generated by JUC-SDELM in terms of their statistics. The discrepancy across domains is alleviated after a few iterations. Moreover, a metric on the outputs of ELM is utilized to filter unreliable pseudo labels in the target domain, with the aim of eliminating the negative transfer effect. Results are obtained by comparing JUC-SDELM with state-of-the-art baseline methods on 16 cross-domain benchmarks that were constructed based on three combined datasets. Likewise, the outcomes in terms of key parameters are also examined. According to the experiments, our proposed model achieves competitive overall performance.

Keywords

Extreme learning machine (ELM) Cross-domain adaptation Joint distribution Scalable factor 

Notes

Funding

This work was funded by SGCC (State Grid Corporation of China) Thousand Talents program special support project (EPRIPDKJ (2014)2863), and partially funded by the National Natural Science Foundation of China (61273165).

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

No studies with human participants or animals are involved in this paper.

References

  1. 1.
    Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–1359.CrossRefGoogle Scholar
  2. 2.
    Gao S, Li H. A cross-domain adaptation method for sentiment classification using probabilistic latent analysis. Proceedings of the 20th ACM international conference on Information and knowledge management; 2011. p. 1047–1052.Google Scholar
  3. 3.
    Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw. 2011;22(2):199–210.CrossRefPubMedGoogle Scholar
  4. 4.
    Patel VM, Gopalan R, Li R, Chellappa R. Visual domain adaptation: a survey of recent advances. IEEE Signal Proc Mag. 2015;32(3):53–69.CrossRefGoogle Scholar
  5. 5.
    Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. IEEE conference on computer vision and pattern recognition (CVPR); 2012. p. 2066–2073.Google Scholar
  6. 6.
    Yu J, Hong C, Rui Y, Tao D. Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron. 2017;65(6):5060–5068.CrossRefGoogle Scholar
  7. 7.
    Ando S, Suzuki E. Unsupervised cross-domain learning by interaction information co-clustering. Eighth IEEE international conference on data mining data mining; 2008. p. 13–22.Google Scholar
  8. 8.
    Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans Image Process. 2016;25(10):4959–4973.CrossRefGoogle Scholar
  9. 9.
    Zhang L, Zhang D. Domain adaptation extreme learning machines for drift compensation in E-nose systems. IEEE Trans Instrum Meas. 2015;64(7):1790–1801.CrossRefGoogle Scholar
  10. 10.
    Pan SJ, Kwok JT, Yang Q. Transfer learning via dimensionality reduction. Proceedings of the Twenty-Third AAAI conference on artificial intelligence; 2008. p. 677–682.Google Scholar
  11. 11.
    Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Schölkopf B, Smola AJ. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics. 2006; 22(14):e49–e57.Google Scholar
  12. 12.
    Jiang M, Huang W, Huang Z, Yen GG. Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE Trans Cybern. 2017;47(1):38–51.CrossRefPubMedGoogle Scholar
  13. 13.
    Long M, Cao Y, Wang J, Jordan MI. Learning transferable features with deep adaptation networks. International conference on machine learning; 2015. p. 97–105.Google Scholar
  14. 14.
    Fernando B, Habrard A, Sebban M, Tuytelaars T. Unsupervised visual domain adaptation using subspace alignment. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2960–2967.Google Scholar
  15. 15.
    Jolliffe IT. Principal component analysis, 2nd ed. Berlin: Springer; 2002. http://www.springer.com/us/book/9780387954424.Google Scholar
  16. 16.
    Hong C, Zeng Z, Xie R, Zhuang W, Wang X. Domain adaptation with low-rank alignment for weakly supervised hand pose recovery. Signal Process. 2018;142:223–230.CrossRefGoogle Scholar
  17. 17.
    Long M, Wang J, Ding G, Sun J, Yu PS. Transfer feature learning with joint distribution adaptation. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2200–2207.Google Scholar
  18. 18.
    Tahmoresnezhad J, Hashemi S. Visual domain adaptation via transfer feature learning. Knowl Inf Syst 2017; 50(2):585–605.CrossRefGoogle Scholar
  19. 19.
    Luo L, Wang X, Hu S, Chen L. Robust data geometric structure aligned close yet discriminative domain adaptation. 2017. arXiv:1705.08620v1.
  20. 20.
    Long M, Wang J, Ding G, Pan SJ, Yu PS. Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng. 2014;26(5):1076–1089.CrossRefGoogle Scholar
  21. 21.
    Gheisari M, Baghshah MS. Joint predictive model and representation learning for visual domain adaptation. Eng Appl Artif Intel 2017;58:157–170.CrossRefGoogle Scholar
  22. 22.
    Wang X, Ma Y, Cheng Y. 2017. Domain adaptation network based on hypergraph regularized denoising autoencoder. Artif Intell Rev.  https://doi.org/10.1007/s10462-017-9576-0.
  23. 23.
    Lu B, Chellappa R, Nasrabadi NM. Incremental dictionary learning for unsupervised domain adaptation. British machine vision conference (BMVC); 2015. p. 108.1–108.12.Google Scholar
  24. 24.
    Hou CA, Yeh YR, Wang YCF. An unsupervised domain adaptation approach for cross-domain visual classification. IEEE international conference on advanced video and signal based surveillance (AVSS); 2015. p. 1–6.Google Scholar
  25. 25.
    Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006; 70(1):489–501.CrossRefGoogle Scholar
  26. 26.
    Huang GB, Zhou H, Ding X, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern. 2012;42(2):513–529.CrossRefPubMedGoogle Scholar
  27. 27.
    Huang GB. What are extreme learning machines? Filling the gap between Frank Rosenblatts dream and John von Neumanns puzzle. Cogn Comput 2015;7(3):263–278.CrossRefGoogle Scholar
  28. 28.
    Guo T, Zhang L, Tan X. Neuron pruning-based discriminative extreme learning machine for pattern classification. Cogn Comput. 2017;9(4):581–595.CrossRefGoogle Scholar
  29. 29.
    Liu Y, Zhang L, Deng P, He Z. Common subspace learning via cross-domain extreme learning machine. Cogn Comput. 2017;9(4):555–563.CrossRefGoogle Scholar
  30. 30.
    Peng Y, Wang S, Long X, Lu B. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing. 2015;149:340–353.CrossRefGoogle Scholar
  31. 31.
    Huang G, Song S, Gupta JND, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 2014;44(12):2405–2417.CrossRefPubMedGoogle Scholar
  32. 32.
    Iosifidis A, Tefas A, Pitas I. Graph embedded extreme learning machine. IEEE Trans Cybern 2016;46 (1):311–324.CrossRefPubMedGoogle Scholar
  33. 33.
    Yan D, Chu Y, Zhang H, Liu D. 2016. Information discriminative extreme learning machine. Soft Comput.  https://doi.org/10.1007/s00500-016-2372-y.
  34. 34.
    Iosifidis A, Tefas A, Pitas I. Semi-supervised classification of human actions based on neural networks. 22nd international conference on pattern recognition (ICPR), IEEE; 2014. p. 1336–1341.Google Scholar
  35. 35.
    Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst. 2012;23(11):1738–1754.CrossRefPubMedGoogle Scholar
  36. 36.
    Xu Y, Fang X, Wu J, Li X, Zhang D. Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process. 2016;25(2):850–863.CrossRefPubMedGoogle Scholar
  37. 37.
    Lei B, Yang P, Wang T, Chen S, Ni D. Relational-Regularized Discriminative sparse learning for alzheimers disease diagnosis. IEEE Trans Cybern. 2017;47(4):1102–1113.CrossRefPubMedGoogle Scholar
  38. 38.
    Luo M, Zhang K. A hybrid approach combining extreme learning machine and sparse representation for image classification. Eng Appl Artif Intel 2014;27:228–235.CrossRefGoogle Scholar
  39. 39.
    Li X, Mao W, Jiang W. Extreme learning machine based transfer learning for data classification. Neurocomputing. 2016;174:203–210.CrossRefGoogle Scholar
  40. 40.
    Zhang L, He Z, Liu Y. Deep object recognition across domains based on adaptive extreme learning machine. Neurocomputing. 2017;239:194–203.CrossRefGoogle Scholar
  41. 41.
    Uzair M, Mian A. Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern. 2017;47(3):651–660.CrossRefGoogle Scholar
  42. 42.
    Wang L, Zhang XY, Pan C. MSDLSR Margin Scalable discriminative least squares regression for multicategory classification. IEEE Trans Neural Netw Learn Syst. 2016;27(12):2711–2717.CrossRefPubMedGoogle Scholar
  43. 43.
    Pan J, Wang X, Cheng Y, Cao G. Multi-source transfer ELM-based Q learning. Neurocomputing 2014; 137:57–64.CrossRefGoogle Scholar
  44. 44.
    Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–297.Google Scholar
  45. 45.
    Lu Y, Lai Z, Fan Z, Cui J, Zhu Q. Manifold discriminant regression learning for image classification. Neurocomputing 2015;166:475–486.CrossRefGoogle Scholar
  46. 46.
    Al-Shedivat M, Wang JJY, Alzahrani M, Huang JZ, Gao X. Supervised transfer sparse coding. Proceedings of the Twenty-Eighth AAAI conference on artificial intelligence; 2014. p. 1665–1672.Google Scholar
  47. 47.
    Iosifidis A, Mygdalis V, Tefas A, Pitas I. One-class classification based on extreme learning and geometric class information. Neural Process Lett. 2017;45(2):577–592.CrossRefGoogle Scholar
  48. 48.
    Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. European conference on computer vision (ECCV); 2010. p. 213–226.Google Scholar
  49. 49.
    Griffin G, Holub AD, Perona P. Caltech-256 object category dataset. Caltech Technical Report. 2007.Google Scholar
  50. 50.
    LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86(11):2278–2324.CrossRefGoogle Scholar
  51. 51.
    Hull JJ. A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 1994;16 (5):550–554.CrossRefGoogle Scholar
  52. 52.
    Nene SA, Nayar SK, Murase H. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96.Google Scholar
  53. 53.
    Chang CC, Lin CJ. LIBSVM: A library for support vector machines. ACM TIST. 2011;2(3):27.Google Scholar
  54. 54.
    Ghifary M, Kleijn WB, Zhang M. Domain adaptive neural networks for object recognition. Pacific Rim international conference on artificial intelligence; 2014. p. 898–904.Google Scholar
  55. 55.
    Yu Y, Sun Z. Sparse coding extreme learning machine for classification. Neurocomputing 2017;261:50–56.CrossRefGoogle Scholar
  56. 56.
    Niu J, Shi Y, Cai M, Cao Z, Wang D, Zhang Z, et al. 2017. Detection of sputum by interpreting the time-frequency distribution of respiratory sound signal using image processing techniques. Bioinformatics.  https://doi.org/10.1093/bioinformatics/btx652.
  57. 57.
    Cao J, Zhang K, Luo M, Yin C, Lai X. Extreme learning machine and adaptive sparse representation for image classification. Neural Networks. 2016;81(C):91–102.CrossRefPubMedGoogle Scholar
  58. 58.
    Chen Y, Song S. Domain transfer extreme learning machine and its application on domain adaptation problems. China Sciencepaper. 2017;12(14):1565–1569 + 1609.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Automation Science and Electrical EngineeringBeihang UniversityBeijingChina
  2. 2.China Electric Power Research InstituteBeijingChina

Personalised recommendations