Abstract
Extreme learning machine (ELM) is a well-known cognitive model, that has been extended to cross-domain tasks. Nonetheless, most existing paradigms that are based on ELM either concern about a case in which a specific number of instances are labelled in the target domain or a learner is trained without sufficient capacity to eliminate the gap between domains. To cope with the scenario in which there are no target labels and to acquire a better adaptive learner, we propose a joint unsupervised cross-domain model via scalable discriminative ELM, which is abbreviated as JUC-SDELM. Within the framework, the scalable factor is integrated into discriminative ELM (DELM) to adjust the output margin, which strengthens the discriminative capacity of the ELM classifier. In addition, we follow the basic strategy of joint distribution adaptation (JDA) to align the subspaces generated by JUC-SDELM in terms of their statistics. The discrepancy across domains is alleviated after a few iterations. Moreover, a metric on the outputs of ELM is utilized to filter unreliable pseudo labels in the target domain, with the aim of eliminating the negative transfer effect. Results are obtained by comparing JUC-SDELM with state-of-the-art baseline methods on 16 cross-domain benchmarks that were constructed based on three combined datasets. Likewise, the outcomes in terms of key parameters are also examined. According to the experiments, our proposed model achieves competitive overall performance.
Similar content being viewed by others
Notes
References
Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–1359.
Gao S, Li H. A cross-domain adaptation method for sentiment classification using probabilistic latent analysis. Proceedings of the 20th ACM international conference on Information and knowledge management; 2011. p. 1047–1052.
Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw. 2011;22(2):199–210.
Patel VM, Gopalan R, Li R, Chellappa R. Visual domain adaptation: a survey of recent advances. IEEE Signal Proc Mag. 2015;32(3):53–69.
Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. IEEE conference on computer vision and pattern recognition (CVPR); 2012. p. 2066–2073.
Yu J, Hong C, Rui Y, Tao D. Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron. 2017;65(6):5060–5068.
Ando S, Suzuki E. Unsupervised cross-domain learning by interaction information co-clustering. Eighth IEEE international conference on data mining data mining; 2008. p. 13–22.
Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans Image Process. 2016;25(10):4959–4973.
Zhang L, Zhang D. Domain adaptation extreme learning machines for drift compensation in E-nose systems. IEEE Trans Instrum Meas. 2015;64(7):1790–1801.
Pan SJ, Kwok JT, Yang Q. Transfer learning via dimensionality reduction. Proceedings of the Twenty-Third AAAI conference on artificial intelligence; 2008. p. 677–682.
Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Schölkopf B, Smola AJ. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics. 2006; 22(14):e49–e57.
Jiang M, Huang W, Huang Z, Yen GG. Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE Trans Cybern. 2017;47(1):38–51.
Long M, Cao Y, Wang J, Jordan MI. Learning transferable features with deep adaptation networks. International conference on machine learning; 2015. p. 97–105.
Fernando B, Habrard A, Sebban M, Tuytelaars T. Unsupervised visual domain adaptation using subspace alignment. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2960–2967.
Jolliffe IT. Principal component analysis, 2nd ed. Berlin: Springer; 2002. http://www.springer.com/us/book/9780387954424.
Hong C, Zeng Z, Xie R, Zhuang W, Wang X. Domain adaptation with low-rank alignment for weakly supervised hand pose recovery. Signal Process. 2018;142:223–230.
Long M, Wang J, Ding G, Sun J, Yu PS. Transfer feature learning with joint distribution adaptation. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2200–2207.
Tahmoresnezhad J, Hashemi S. Visual domain adaptation via transfer feature learning. Knowl Inf Syst 2017; 50(2):585–605.
Luo L, Wang X, Hu S, Chen L. Robust data geometric structure aligned close yet discriminative domain adaptation. 2017. arXiv:1705.08620v1.
Long M, Wang J, Ding G, Pan SJ, Yu PS. Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng. 2014;26(5):1076–1089.
Gheisari M, Baghshah MS. Joint predictive model and representation learning for visual domain adaptation. Eng Appl Artif Intel 2017;58:157–170.
Wang X, Ma Y, Cheng Y. 2017. Domain adaptation network based on hypergraph regularized denoising autoencoder. Artif Intell Rev. https://doi.org/10.1007/s10462-017-9576-0.
Lu B, Chellappa R, Nasrabadi NM. Incremental dictionary learning for unsupervised domain adaptation. British machine vision conference (BMVC); 2015. p. 108.1–108.12.
Hou CA, Yeh YR, Wang YCF. An unsupervised domain adaptation approach for cross-domain visual classification. IEEE international conference on advanced video and signal based surveillance (AVSS); 2015. p. 1–6.
Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006; 70(1):489–501.
Huang GB, Zhou H, Ding X, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern. 2012;42(2):513–529.
Huang GB. What are extreme learning machines? Filling the gap between Frank Rosenblatts dream and John von Neumanns puzzle. Cogn Comput 2015;7(3):263–278.
Guo T, Zhang L, Tan X. Neuron pruning-based discriminative extreme learning machine for pattern classification. Cogn Comput. 2017;9(4):581–595.
Liu Y, Zhang L, Deng P, He Z. Common subspace learning via cross-domain extreme learning machine. Cogn Comput. 2017;9(4):555–563.
Peng Y, Wang S, Long X, Lu B. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing. 2015;149:340–353.
Huang G, Song S, Gupta JND, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 2014;44(12):2405–2417.
Iosifidis A, Tefas A, Pitas I. Graph embedded extreme learning machine. IEEE Trans Cybern 2016;46 (1):311–324.
Yan D, Chu Y, Zhang H, Liu D. 2016. Information discriminative extreme learning machine. Soft Comput. https://doi.org/10.1007/s00500-016-2372-y.
Iosifidis A, Tefas A, Pitas I. Semi-supervised classification of human actions based on neural networks. 22nd international conference on pattern recognition (ICPR), IEEE; 2014. p. 1336–1341.
Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst. 2012;23(11):1738–1754.
Xu Y, Fang X, Wu J, Li X, Zhang D. Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process. 2016;25(2):850–863.
Lei B, Yang P, Wang T, Chen S, Ni D. Relational-Regularized Discriminative sparse learning for alzheimers disease diagnosis. IEEE Trans Cybern. 2017;47(4):1102–1113.
Luo M, Zhang K. A hybrid approach combining extreme learning machine and sparse representation for image classification. Eng Appl Artif Intel 2014;27:228–235.
Li X, Mao W, Jiang W. Extreme learning machine based transfer learning for data classification. Neurocomputing. 2016;174:203–210.
Zhang L, He Z, Liu Y. Deep object recognition across domains based on adaptive extreme learning machine. Neurocomputing. 2017;239:194–203.
Uzair M, Mian A. Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern. 2017;47(3):651–660.
Wang L, Zhang XY, Pan C. MSDLSR Margin Scalable discriminative least squares regression for multicategory classification. IEEE Trans Neural Netw Learn Syst. 2016;27(12):2711–2717.
Pan J, Wang X, Cheng Y, Cao G. Multi-source transfer ELM-based Q learning. Neurocomputing 2014; 137:57–64.
Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–297.
Lu Y, Lai Z, Fan Z, Cui J, Zhu Q. Manifold discriminant regression learning for image classification. Neurocomputing 2015;166:475–486.
Al-Shedivat M, Wang JJY, Alzahrani M, Huang JZ, Gao X. Supervised transfer sparse coding. Proceedings of the Twenty-Eighth AAAI conference on artificial intelligence; 2014. p. 1665–1672.
Iosifidis A, Mygdalis V, Tefas A, Pitas I. One-class classification based on extreme learning and geometric class information. Neural Process Lett. 2017;45(2):577–592.
Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. European conference on computer vision (ECCV); 2010. p. 213–226.
Griffin G, Holub AD, Perona P. Caltech-256 object category dataset. Caltech Technical Report. 2007.
LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86(11):2278–2324.
Hull JJ. A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 1994;16 (5):550–554.
Nene SA, Nayar SK, Murase H. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96.
Chang CC, Lin CJ. LIBSVM: A library for support vector machines. ACM TIST. 2011;2(3):27.
Ghifary M, Kleijn WB, Zhang M. Domain adaptive neural networks for object recognition. Pacific Rim international conference on artificial intelligence; 2014. p. 898–904.
Yu Y, Sun Z. Sparse coding extreme learning machine for classification. Neurocomputing 2017;261:50–56.
Niu J, Shi Y, Cai M, Cao Z, Wang D, Zhang Z, et al. 2017. Detection of sputum by interpreting the time-frequency distribution of respiratory sound signal using image processing techniques. Bioinformatics. https://doi.org/10.1093/bioinformatics/btx652.
Cao J, Zhang K, Luo M, Yin C, Lai X. Extreme learning machine and adaptive sparse representation for image classification. Neural Networks. 2016;81(C):91–102.
Chen Y, Song S. Domain transfer extreme learning machine and its application on domain adaptation problems. China Sciencepaper. 2017;12(14):1565–1569 + 1609.
Funding
This work was funded by SGCC (State Grid Corporation of China) Thousand Talents program special support project (EPRIPDKJ (2014)2863), and partially funded by the National Natural Science Foundation of China (61273165).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical Approval
No studies with human participants or animals are involved in this paper.
Rights and permissions
About this article
Cite this article
Zhang, B., Liu, Y., Yuan, H. et al. A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine. Cogn Comput 10, 577–590 (2018). https://doi.org/10.1007/s12559-018-9555-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-018-9555-z