Skip to main content
SpringerLink
Log in

A novel active multi-source transfer learning algorithm for time series forecasting

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In Time Series Forecasting (TSF), researchers usually assume that there is enough training data can be obtained, with the old a`nd new data satisfying the same distribution. However, time series data always produces some time-varying characteristics over time, which will lead to relatively large differences between old and new data. As we all know, single-source TSF Transfer Learning (TL) faces the problem of negative transfer. Addressing this issue, this paper proposes a new Multi-Source TL algorithm, abbreviated as the MultiSrcTL algorithm, and a novel Active Multi-Source Transfer Learning, abbreviated as the AcMultiSrcTL algorithm, with the latter one integrating Multi-Source TL with Active Learning (AL), and taking the former one as its sub-algorithm. We introduce domain adaptation theory into this work, and analyze the expected target risk of TSF under the multi-source setting, accordingly. For the development of MultiSrcTL, we make full use of source similarity and domain dependability, using the Maximum Mean Discrepancy statistical indicator to measure the similarity between domains, so as to promote better transfer. A domain relation matrix is constructed to describe the relationship between source domains, so that the source-source and source-target relations are adequately considered. In the design of AcMultiSrcTL, Kullback-Leibler divergence is used to measure the similarity of related indicators to select the appropriate source domain. The uncertainty sampling method and the distribution match weighting technique are integrated, obtaining a new sample selection scheme. The empirical results on six benchmark datasets demonstrate the applicability and effectiveness of the two proposed algorithms for multi-source TSF TL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Corberán-Vallet A, Bermúdez JD, Vercher E (2011) Forecasting correlated time series with exponential smoothing models. Int J Forecast 27(2):252–265

    Article  Google Scholar 

  2. Cortez P, Rocha M, Neves J (2004) Evolving time series forecasting ARMA models. J Heuristics 10(4):415–429

    Article  Google Scholar 

  3. Lee YS, Tong LI (2011) Forecasting time series using a methodology based on autoregressive integrated moving average and genetic programming. Knowl-Based Syst 24(1):66–72

    Article  Google Scholar 

  4. Liang LZ, Shao F (2010) The study on short-time wind speed prediction based on time-series neural network algorithm Asia-Pacific Power Energy Eng Conf 2010

  5. Mager J, Paasche U, Sick B (2009) "Forecasting financial time series with support vector machines based on dynamic kernels," 2008 IEEE Conference on Soft Computing in Industrial Applications, pp 252-257

  6. Yang HM, Pan ZS, Tao Q (2017) Robust and adaptive online time series prediction with long short-term memory. Comput Intell Neurosci 1–9

  7. Zhao R, Wang D, Yan R, Mao K, Shen F, Wang J (2018) Machine health monitoring using local feature-based gated recurrent unit networks. IEEE Trans Ind Electron 65(2):1539–1548

    Article  Google Scholar 

  8. Ye R, Dai Q (2018) A novel transfer learning framework for time series forecasting. Knowl-Based Systems 156:74–99

    Article  Google Scholar 

  9. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  10. Shao L, Zhu F, Li XL (2015) Transfer learning for visual categorization: a survey. IEEE Trans Neural Netw Learn Syst 26(5):1019–1034

    Article  MathSciNet  Google Scholar 

  11. Yang Q (2009) Transfer learning beyond text classification. Asian Conf Mach Learn, Springer 5828:10–22

    Google Scholar 

  12. Wang XY, Han M (2014) Online sequential extreme learning machine with kernels for nonstationary time series prediction. Neurocomputing 145:90–97

    Article  Google Scholar 

  13. Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2015) Online sequential extreme learning machine with kernels. IEEE Trans Neural Netw Learn Syst 26(9):2214–2220

    Article  MathSciNet  Google Scholar 

  14. Fang M, Guo Y, Zhang XS, Li X (2015) Multi-source transfer learning based on label shared subspace. Pattern Recogn Lett 51:101–106

    Article  Google Scholar 

  15. Eaton E, Desjardins M (2011) "Selective transfer between learning tasks using task-based boosting," Proceedings of the 25th AAAI Conference on Artificial Intelligence, pp. 337–342

  16. Yao Y, Doretto G (2010) "Boosting for transfer learning with multiple sources," Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, pp. 1855–1862

  17. Al-Stouhi S, Reddy CK (2011) "Adaptive boosting for transfer learning using dynamic updates," Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 60–75

  18. Eaton E, Desjardins M (2009) "Set-Based Boosting for Instance-level Transfer," Proceedings of the International Conference on Data Mining Workshops, pp. 422–428

  19. Gao J, Fan W, Sun Y, Han J (2009) "Heterogeneous Source Consensus Learning via Decision Propagation and Negotiation," Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 339–348

  20. Wang Z, Carbonell J (2018) "Towards more reliable transfer learning," Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, pp. 794–810

  21. Ardalani-Farsa M, Zolfaghari S (2010) Chaotic time series prediction with residual analysis method using hybrid Elman-NARX neural networks. Neurocomputing 73(13–15):2540–2553

    Article  Google Scholar 

  22. Time Series Data Library. Available: Time Series Data Library - Data provider — DataMarket

  23. Yahoo Finance[EB/OL]. Available: http://finance.yahoo.com/

  24. Ferreira TAE, Vasconcelos GC, Adeodato PJL (2008) A new intelligent system methodology for time series forecasting with artificial neural networks. Neural Process Lett 28(2):113–129

    Article  Google Scholar 

  25. Fakhr MW (2015) "Sparse locally linear and leighbor embedding for nonlinear time series prediction," 2015 Tenth International Conference on Computer Engineering & Systems, pp. 371–377

  26. Ciz R, Rudajev V (2007) Linear and nonlinear attributes of ultrasonic time series recorded from experimentally loaded rock samples and total failure prediction. Int J Rock Mech Min Sci 44(33):457–467

    Article  Google Scholar 

  27. Chen YH, Yang B, Dong JW (2006) Time-series prediction using a local linear wavelet neural network. Neurocomputing 69(4–6):449–465

    Article  Google Scholar 

  28. Gromov VA, Shulga AN (2012) Chaotic time series prediction with employment of ant colony optimization. Expert Syst Appl 39(9):8474–8478

    Article  Google Scholar 

  29. Khashei M, Bijari M (2011) A novel hybridization of artificial neural networks and ARIMA models for time series forecasting. Appl Soft Comput 11(2):2664–2675

    Article  Google Scholar 

  30. Kuremoto T, Kimura S, Kobayashi K, Obayashi M (2014) Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing 137:47–56

    Article  Google Scholar 

  31. Shi ZW, Han M (2007) Support vector echo-state machine for chaotic time-series prediction. IEEE Trans Neural Netw 18(2):359–372

    Article  Google Scholar 

  32. Zhu JY, Ren B, Zhang HX, Deng ZT (2002) Time series prediction via new support vector machines. Int Conf Mach Learn Cybernet 1-4:364–366

    Google Scholar 

  33. Cao LJ, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw Learn Syst 14(6):1506–1518

    Article  Google Scholar 

  34. Jiang W, Zavesky E, Chang SF, Loui A (2008) "Cross-Domain Learning Methods for High-Level Visual Concept Classification," 2008 15th IEEE International Conference on Image Processing, vols 1–5, pp. 161–164

  35. Lee CW, Fang W, Yeh CK, Wang YCF (2018) "Multi-label zero-shot learning with structured knowledge graphs," IEEE Conference on Computer Vision and Pattern Recognition, pp. 1576–1585

  36. Song J, Shen CC, Yang YZ, Liu Y, Song ML (2018) "Transductive unbiased embedding for zero-shot learning," Proc IEEE Conf Comput Vis Pattern Recognit, pp. 1024–1033

  37. Konyushkova K, Raphael S, Fua P (2017) Learning Active Learning from Data. Neural Inf Process Syst Conf 30

  38. Xiang JP (2012) Active learning for person re-identification. 2012 International Conference on Machine Learning and Cybernetics. IEEE 1: 336–340

  39. Bruzzone L, Persello C (2009) Active Learning for Classification of Remote Sensing Images. IEEE Int Geosci Remote Sens Symp 1-5:1995–1998

    Google Scholar 

  40. Zhou ZW, Shin J, Zhang L, Gurudu S, Gotway M, Liang JM (2017) "Fine-tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally," 30th IEEE Conference on Computer Vision and Pattern Recognition, pp. 4761–4772

  41. Kale D, Liu Y (2013) "Accelerating Active Learning with Transfer Learning," 2013 IEEE 13th International Conference on Data Mining, pp. 1085–1090

  42. Yan Y, Subramanian R, Lanz O, Sebe N (2012) "Active Transfer Learning for Multi-view Head-pose Classification," Proceedings of the 21st International Conference on Pattern Recognition. IEEE, pp. 1168–1171

  43. Blitzer J, Crammer K, Kulesza A, Pereira F, Wortman J (2008) "Learning bounds for domain adaptation," Conference on Neural Information Processing Systems, pp. 129–136

  44. Ben-David S, Blitzer J, Crammer K, Pereira F (2006) Analysis of representations for domain adaptation. Int Conf Neural Inf Process Syst, pp. 137–144

  45. Nguyen HT, Smeulders A (2004) Active Learning Using Pre-clustering. Proceedings of the 21st International Conference on Machine learning. ACM, pp. 336–340

  46. Huang J, Smola AJ, Gretton A, Borgwardt KM, Schölkopf B (2006) Correcting sample selection Bias by unlabeled data. Int Conf Neural Inf Process Syst, pp. 601–608

  47. Gretton A, Borgwardt K, Rasch MJ, Scholkopf B, Smola AJ (2008) A kernel method for the two-sample problem. Adv Neural Inf Proces Syst, pp. 513–520

  48. Chandra R, Zhang M (2012) Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction. Neurocomputing 86(12):116–123

    Article  Google Scholar 

  49. Rong HJ, Sundararajan N, Huang GB, Saratchandran P (2006) Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275

    Article  MathSciNet  Google Scholar 

  50. Nan-Ying L, Guang-Bin H, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  51. Jiao X, Liu Z, Yong G, Pan Z (2016) Time Series Prediction Based on Online Sequential Improved Error Minimized Extreme Learning Machine. Proc ELM-2015 1:193–209

    Google Scholar 

  52. Ma J, Theiler J, Perkins S (2003) Accurate on-line support vector regression. Neural Comput 15:2683–2703

    Article  Google Scholar 

  53. van Heeswijk M et al. (2009) Adaptive Ensemble Models of Extreme Learning Machines for Time Series Prediction. Proc 19th Int Conf Artif Neural Netw 5769, pp. 305–314

Download references

Acknowledgments

This work is supported by the National Key R&D Program of China (Grant Nos. 2018YFC2001600, 2018YFC2001602), and the National Natural Science Foundation of China under Grant no. 61473150.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

    Qitao Gu & Qun Dai

Authors
  1. Qitao Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qun Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qun Dai.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, Q., Dai, Q. A novel active multi-source transfer learning algorithm for time series forecasting. Appl Intell 51, 1326–1350 (2021). https://doi.org/10.1007/s10489-020-01871-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01871-5

Keywords

Search

Navigation