Soft Computing

, Volume 21, Issue 23, pp 7039–7052 | Cite as

Temporal sampling forest (\(\varvec{\textit{TS-F}}\)): an ensemble temporal learner

  • Shih Yin Ooi
  • Shing Chiang Tan
  • Wooi Ping Cheah
Methodologies and Application

Abstract

Ensemble learning is in favour of machine learning community due to its tolerance in handling divergence and biasness issues faced by a single learner. In this work, an ensemble temporal learner, namely temporal sampling forest (TS-F), is proposed. Building on the random forest, we consider its limitations in handling temporal classification tasks. Temporal data classification is an important area of machine learning and data mining, where it fills the gap of ordinary data classification when the observed datasets are temporally related across sequential and time domains. TS-F incorporated the temporal sampling (bagging) and temporal randomization procedures in the classical random forest, hence extending its ability to handle temporal data . TS-F was tested on 11 public sequential and temporal datasets from different domains . Experiments demonstrate that TS-F could provide promising results with average classification accuracy of 98 %, substantiating its ability to escalate the random forest performance in the application of temporal classification.

Keywords

Ensemble learner Temporal classification Random forest Temporal application 

References

  1. Altun K, Barshan B, Tunçel O (2010) Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognit 43(10):3605–3620. doi:10.1016/j.patcog.2010.04.019 CrossRefMATHGoogle Scholar
  2. Anacleto R, Figueiredo L, Almeida A, Novais P (2014) Localization system for pedestrians based on sensor and information fusion. IEEE 17th international conference on information fusion (FUSION), p 8. http://ieeexplore.ieee.org.ezproxy.auckland.ac.nz/stamp/stamp.jsp?tp=&arnumber=6916127&isnumber=6915967
  3. Anacleto R, Figueiredo L, Almeida A, Novais P, Meireles A (2015) Step characterization using sensor information fusion and machine learning. Int J Interact Multimed Artif Intell 3(5):53–60. doi:10.9781/ijimai.2015.357 Google Scholar
  4. Bache K, Lichman M (2013) UCI machine learning repository. School of Information and Computer Science, University of California, Irvin. [http://archive.ics.uci.edu/ml]
  5. Bernard S, Adam S, Heutte L (2012) Dynamic random forests. Pattern Recognit Lett 33(12):1580–1586. doi:10.1016/j.patrec.2012.04.003 CrossRefGoogle Scholar
  6. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140. doi:10.1007/BF00058655 MATHGoogle Scholar
  7. Breiman L (2001) Random forest. Mach Learn 45(1):5–32Google Scholar
  8. Casale P, Pujol O, Radeva P (2012) Personalization and user verification in wearable systems using biometric walking patterns. Pers Ubiquitous Comput 16(5):563–580. doi:10.1007/s00779-011-0415-z CrossRefGoogle Scholar
  9. Chen R, Deng Z, Song Z (2015) The prediction of malignant middle cerebral artery infarction: a predicting approach using random forest. J Stroke Cerebrovasc Dis 24(5):958–964. doi:10.1016/j.jstrokecerebrovasdis.2014.12.016 CrossRefGoogle Scholar
  10. Cohen J (1960) A coefficient of agreement for nominal scale. Educ Psychol Meas 20(1):37–46. doi:10.1177/001316446002000104 CrossRefGoogle Scholar
  11. Corcoran J, Frank W, Maloney M (1974) CORST. 1. pdf. J Symb Logic 39(4):625–637CrossRefGoogle Scholar
  12. Deng H, Runger G, Tuv E, Vladimir M (2013) A time series forest for classification and feature extraction. Inf Sci 239:142–153. doi:10.1016/j.ins.2013.02.030 CrossRefMATHMathSciNetGoogle Scholar
  13. Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874. doi:10.1016/j.patrec.2005.10.010 CrossRefMathSciNetGoogle Scholar
  14. Firmino PRA, de Mattos Neto PSG, Ferreira TAE (2014) Correcting and combining time series forecasters. Neural Netw 50:1–11. doi:10.1016/j.neunet.2013.10.008 CrossRefMATHGoogle Scholar
  15. Freund Y, Schapire R (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139. doi:10.1006/jcss.1997.1504 CrossRefMATHMathSciNetGoogle Scholar
  16. Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Proceedings of the 13th international conference on machine learning, Bari, Italy, 3–6 July 1996, pp 148–156Google Scholar
  17. Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232CrossRefMATHMathSciNetGoogle Scholar
  18. García-Díaz V, Pascual-Espada J, Pelayo G-Bustelo C, Cueva-Lovelle JM (2015) Towards a standard-based domain-specific platform to solve machine learning-based problems. Int J Interact Multimed Artif Intell 3(5):6–12. doi:10.9781/ijimai.2015.351 Google Scholar
  19. Geurts P, Ernst D, Wehenkel L (2006) Extremely Randomized Trees. Mach Learn 63(1):3–42. doi:10.1007/s10994-006-6226-1 CrossRefMATHGoogle Scholar
  20. González Crespo R, Escobar RF, Joyanes Aguilar L, Velazco S, Castillo Sanz AG (2013) Use of ARIMA mathematical analysis to model the implementation of expert system courses by means of free software OpenSim and Sloodle platforms in virtual university campuses. Expert Syst Appl 40(18):7381–7390. doi:10.1016/j.eswa.2013.06.054 CrossRefGoogle Scholar
  21. Heo J, Yang JY (2014) AdaBoost based bankruptcy forecasting of Korean construction companies. Appl Soft Comput 24:494–499. doi:10.1016/j.asoc.2014.08.009 CrossRefGoogle Scholar
  22. Ho TK (1998) The random subspace method for constructing decision forest. IEEE Trans Pattern Anal Mach Intell 20(8):832–844. doi:10.1109/34.709601 CrossRefGoogle Scholar
  23. Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial. Stat Sci 14(4):382–417. doi:10.2307/2676803 CrossRefMATHMathSciNetGoogle Scholar
  24. Hong S, Khim S, Rhee PK (2014) Efficient facial landmark localization using spatial-contextual AdaBoost algorithm. J Vis Commun Image Represent 25(6):1366–1377. doi:10.1016/j.jvcir.2014.05.001 CrossRefGoogle Scholar
  25. Kim M-J, Kang D-K, Kim HB (2015) Geometric mean based boosting algorithm with over-sampling to resolve data imbalance problem for bankruptcy prediction. Expert Syst Appl 42(3):1074–1082. doi:10.1016/j.eswa.2014.08.025
  26. Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint conference on artificial intelligence (IJCAI), vol 5. Morgan Kaufmann, San Mateo, pp 1137–1143Google Scholar
  27. Lebedev AV, Westman E, Van Westen GJP, Kramberger MG, Lundervold A, Aarsland D, Simmons A (2014) Random forest ensembles for detection and prediction of Alzheimer’s disease with a good between-cohort robustness. NeuroImage Clin 6:115–125. doi:10.1016/j.nicl.2014.08.023 CrossRefGoogle Scholar
  28. Liu B, Ma Y, Wong CK, Yu PS (2003) Scoring the data using association rules. Appl Intell 18(2):119–135CrossRefMATHGoogle Scholar
  29. Liu S, Xu J, Zhao J, Xie X, Zhang W (2014) Efficiency enhancement of a process-based rainfall-runoff model using a new modified AdaBoost.RT technique. Appl Soft Comput 23:521–529. doi:10.1016/j.asoc.2014.05.033 CrossRefGoogle Scholar
  30. Louzada F, Ara A (2012) Bagging k-dependence probabilistic networks: an alternative powerful fraud detection tool. Expert Syst Appl 39(14):11583–11592. doi:10.1016/j.eswa.2012.04.024 CrossRefGoogle Scholar
  31. Mitsa T (2010) Temporal data mining, 1st edn. Chapman & Hall/CRC. http://dl.acm.org/citation.cfm?id=1809755
  32. Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106Google Scholar
  33. Radicioni DP, Esposito R (2010) BREVE?: an HMPerceptron-based chord recognition system. Adv Music Inf Retr Stud Comput Intell 274:143–164Google Scholar
  34. Revesz P, Triplet T (2011) Temporal data classification using linear classifiers. Inf Syst 36(1):30–41. doi:10.1016/j.is.2010.06.006 CrossRefGoogle Scholar
  35. Rodríguez JJ, Kuncheva LI, Alonso CJ (2006) Rotation forest: a new classifier ensemble method. IEEE Trans Pattern Anal Mach Intell 28(10):1619–1630. doi:10.1109/TPAMI.2006.211 CrossRefGoogle Scholar
  36. Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227. doi:10.1007/BF00116037 Google Scholar
  37. Seewald A, Fürnkranz J (2001) An evaluation of grading classifiers. Advances in intelligent data analysis. Lecture notes in computer science, 2189, pp 115–124. http://link.springer.com/chapter/10.1007/3-540-44816-0_12
  38. Ting KM, Witten IH (1997) Stacking bagged and dagged models. In: Proceedings of the fourteenth international conference on machine learning, pp 367–375Google Scholar
  39. Tripoliti EE, Fotiadis DI, Manis G (2013) Modifications of the construction and voting mechanisms of the random forests algorithm. Data Knowl Eng 87:41–65. doi:10.1016/j.datak.2013.07.002 CrossRefGoogle Scholar
  40. Tseng VS, Lee CH (2009) Effective temporal data classification by integrating sequential pattern mining and probabilistic induction. Expert Syst Appl 36(5):9524–9532. doi:10.1016/j.eswa.2008.10.077 CrossRefGoogle Scholar
  41. Webb GI (2000) MultiBoosting: a technique for combining boosting and wagging. Mach Learn 40(2):159–196. doi:10.1023/A:1007659514849 CrossRefGoogle Scholar
  42. Witten IH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques. Elsevier, Amsterdam. doi:10.1016/B978-0-12-374856-0.00014-6 Google Scholar
  43. Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241–259. doi:10.1016/S0893-6080(05)80023-1 CrossRefGoogle Scholar
  44. Yang Y, Jiang J (2014) HMM-based hybrid meta-clustering ensemble for temporal data. Knowl Based Syst 56:299–310. doi:10.1016/j.knosys.2013.12.004 CrossRefGoogle Scholar
  45. Zhang Y, Haghani A (2015) A gradient boosting method to improve travel time prediction. Trans Res Part C Emerg Technol. doi:10.1016/j.trc.2015.02.019 Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Shih Yin Ooi
    • 1
  • Shing Chiang Tan
    • 1
  • Wooi Ping Cheah
    • 1
  1. 1.Faculty of Information Science and TechnologyMultimedia UniversityMelakaMalaysia

Personalised recommendations