# Novel hybrid SVM-TLBO forecasting model incorporating dimensionality reduction techniques

- 390 Downloads
- 3 Citations

## Abstract

In this paper, we present a highly accurate forecasting method that supports improved investment decisions. The proposed method extends the novel hybrid SVM-TLBO model consisting of a support vector machine (SVM) and a teaching-learning-based optimization (TLBO) method that determines the optimal SVM parameters, by combining it with dimensional reduction techniques (DR-SVM-TLBO). The dimension reduction techniques (feature extraction approach) extract critical, non-collinear, relevant, and de-noised information from the input variables (features), and reduce the time complexity. We investigated three different feature extraction techniques: principal component analysis, kernel principal component analysis, and independent component analysis. The feasibility and effectiveness of this proposed ensemble model were examined using a case study, predicting the daily closing prices of the COMDEX commodity futures index traded in the Multi Commodity Exchange of India Limited. In this study, we assessed the performance of the new ensemble model with the three feature extraction techniques, using different performance metrics and statistical measures. We compared our results with results from a standard SVM model and an SVM-TLBO hybrid model. Our experimental results show that the new ensemble model is viable and effective, and provides better predictions. This proposed model can provide technical support for better financial investment decisions and can be used as an alternative model for forecasting tasks that require more accurate predictions.

## Keywords

Support Vector Machine (SVM) Teaching-Learning-Based Optimization (TLBO) Dimensional reduction Commodity futures contract Financial time series## Notes

### Acknowledgments

We would like to express our gratitude to the National Institute of Science and Technology (NIST), for the facilities and resources provided at the Data Science Laboratory at NIST for the development of this study.

### Compliance with ethical standards

### Conflict of interests

The authors declare that there are no conflict of interests (either financial or non-financial) regarding the publication of the paper.

## References

- 1.Cai LJ, Zhang JQ, Zongwu CAI, Kian Guan LIM (2006) An empirical study of dimensionality reduction in support vector machine. Neural Network World 16(3):177–192Google Scholar
- 2.Cao LJ (2003) Support vector machines experts for time series forecasting. Neorocomputing 51:321–339CrossRefGoogle Scholar
- 3.Cao LJ, Chua KS, Chong WK, Lee HP, Gu QM (2003) A comparison of PCA, KPCA and ICA for dimensional reduction in support vector machines. Neurocomputing 55(1):321–336Google Scholar
- 4.Cao LJ, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw 14(6):1506–1518CrossRefGoogle Scholar
- 5.Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27Google Scholar
- 6.Chang PC, Wu JL (2015) A critical feature extraction by kernel PCA in stock trading model. Soft Comput 19(5):1393–1408CrossRefGoogle Scholar
- 7.Chen WH, Shih JY, Wu S (2006) Comparison of support-vector machines and back propagation neural networks in forecasting the six major Asian stock markets. International Journal of Electronic Finance 1(1):49–67CrossRefGoogle Scholar
- 8.Das SP, Padhy S (2015) A novel hybrid model using teaching–learning-based optimization and a support vector machine for commodity futures index forecasting. Int J Mach Learn Cyber:1–15. doi: 10.1007/s13042-015-0359-0
- 9.Diebold FX, Mariano RS (1995) Comparing predictive accuracy. J Bus Econ Stat 13(3):253–263MathSciNetGoogle Scholar
- 10.Ekenel HK Sankur B (2004) Feature selection in the independent component subspace for face recognition. Pattern Recogn Lett 25(12):377–1388Google Scholar
- 11.Haykin S (2010) Neural Networks and Learning Machines. 3rd Edition, PHI Learning Private LimitedGoogle Scholar
- 12.Hsu CM (2013) A hybrid procedure with feature selection for resolving stock/futures price forecasting problems. Neural Comput Applic 22(3–4):651–671. doi: 10.1007/s00521-011-07214 CrossRefGoogle Scholar
- 13.Huang CL, Tsai CY (2009) A hybrid SOFM-SVR with a filter based feature selection for stock market forecasting. Expert Syst Appl 36(2):1529–1539. doi: 10.1016/j.eswa.2007.11.062 MathSciNetCrossRefGoogle Scholar
- 14.Hyvarinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, New YorkCrossRefGoogle Scholar
- 15.Hyvarinen A, Oja E (1997) A fast fixed-point algorithm for independent component analysis. Neural Comput 9(7):1483– 1492CrossRefGoogle Scholar
- 16.Hyvarinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural networks 13(4):411–430CrossRefGoogle Scholar
- 17.Ince H, Trafalis TB (2007) Kernel principal component analysis and support vector machines for stock price prediction. IIE Trans 39(6):629–637CrossRefGoogle Scholar
- 18.Ince H, Trafalis TB (2008) Short term forecasting with support vector machines and application to stock price prediction. Int J Gen Syst 37(6):77–687. doi: 10.1080/03081070601068595 MathSciNetCrossRefzbMATHGoogle Scholar
- 19.Jiang M, Jiang S, Zhu L, Wang Y, Huang W, Zhang H (2013) Study on parameter optimization for support vector regression in solving the inverse ECG problem. Comput Math Methods Med Article ID 158059. doi: 10.1155/2013/158056 MathSciNetzbMATHGoogle Scholar
- 20.Jolliffe IT (2002) Principle components analysis 2
^{nd}Edition. Springer, New YorkGoogle Scholar - 21.Kim KJ (2003) Financial time series forecasting using support vector machines. Neurocomputing 55(1):307–319CrossRefGoogle Scholar
- 22.Kim KJ, Han I (2000) Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Syst Appl 19(2):125–132MathSciNetCrossRefGoogle Scholar
- 23.Kim KJ, Lee WB (2004) Stock market prediction using artificial neural networks with optimal feature transformation. Neural Comput Applic 13(3):255–260. doi: 10.1007/s00521-004-0428-x CrossRefGoogle Scholar
- 24.Kuang F, Zhang S, Jin Z, Xu W (2015) A novel SVM by combining kernel principal component analysis and improved chaotic particle swarm optimization for intrusion detection. Soft Comput 19:1187–1199. doi: 10.1007/s00500-014-1332-7 CrossRefGoogle Scholar
- 25.Lai RK, Fan CY, Huang WH, Chang PC (2009) Evolving and clustering fuzzy decision tree for financial time series data forecasting. Expert Syst Appl 36(2):3761–3773. doi: 10.1016/j.eswa.2008.02.025 CrossRefGoogle Scholar
- 26.Leung MT, Daouk H, Chen AS (2000) Forecasting stock indices: a comparison of classification and level estimation models. Int J Forecast 16(2):173–190CrossRefGoogle Scholar
- 27.Liang X, Zhang H, Xiao J, Chen Y (2009) Improving option price forecasts with neural networks and support vector regressions. Neurocomputing 72(13):3055–3065. doi: 10.1016/j.neucom.2009.03.015 CrossRefGoogle Scholar
- 28.Lin HT, Lin CJ (2003) A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods Technical report, University of National Taiwan Department of Computer Science and Information Engineering, March 1–32Google Scholar
- 29.Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817– 1824CrossRefGoogle Scholar
- 30.Liu S, Tian L, Huang Y (2014) A comparative study on prediction of throughput in coal ports among three models. Int J Mach Learn Cybern 5(1):125–133. doi: 10.1007/s13042-013-0201-5 CrossRefGoogle Scholar
- 31.Lu CJ (2013) Hybridizing nonlinear independent component analysis and support vector regression with particle swarm optimization for stock index forecasting. Neural Comput Applic 23(7–8):2417–2427. doi: 10.1007/s00521-012-1198-5 CrossRefGoogle Scholar
- 32.Lu CJ, Lee TS, Chiu CC (2009) Financial time series forecasting using independent component analysis and support vector regression. Decis Support Syst 47(2):115–125CrossRefGoogle Scholar
- 33.Musa AB (2014) A comparison of 1-regularizion, PCA, KPCA and ICA for dimensionality reduction in logistic regression. Int J Mach Learn Cybern 5(6):861–873. doi: 10.1007/s13042-013-0171-7 CrossRefGoogle Scholar
- 34.Pawar PV, Rao RV (2013) Parameter optimization of machining using teaching-learning-based optimization algorithm. Int J Adv Manuf Technol 67:995–1006CrossRefGoogle Scholar
- 35.Porikli F, Haga T (2004) Event detection by eigenvector decomposition using object and frame features. IEEE Conference In Computer Vision and Pattern Recognition Workshop 2004(CVPRW’04):114–114Google Scholar
- 36.Rao RV, Patel V (2014) A multi-objective improved teaching-learning based optimization algorithm for unconstrained and constrained optimization problems. Int J Ind Eng Comput 5(1):1–22. doi: 10.5267/j.ijiec.2013.09.007 Google Scholar
- 37.Rao RV, Savsani VJ, Vakharia DP (2011) Teaching-learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303– 315CrossRefGoogle Scholar
- 38.Sapankevych NI, Sankar R (2009) Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 4(2):24–38. doi: 10.1109/MCI.2009.932254 CrossRefGoogle Scholar
- 39.Tay FE, Cao LJ (2002) Modified support vector machines in financial time series forecasting. Neurocomputing 48(1):847– 861CrossRefzbMATHGoogle Scholar
- 40.Tsai CF, Hsiao YC (2010) Combining multiple feature selection methods for stock prediction: Union, intersection, and multi-intersection approaches. Decis Support Syst 50(1):258–269. doi: 10.1007/s00500-014-1350-5 CrossRefGoogle Scholar
- 41.Tsang PM, Kwok P, Choy SO, Kwan R, Ng SC, Mak J, Tsang J, Koong K, Wong TL (2007) Design and implementation of NN5 for Hong Kong stock price forecasting. Eng Appl Artif Intell 20 (4):453–461. doi: 10.1016/j.engappai.2006.10.002 CrossRefGoogle Scholar
- 42.Twining CJ, Taylor CJ (2003) The use of kernel principal component analysis to model data distributions. Pattern Recogn 36(1):217–227CrossRefzbMATHGoogle Scholar
- 43.Van Gestel T, Suykens JA, Baestaens DE, Lambrechts A, Lanckriet G, Vandaele B, Vandewalle J (2001) Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Trans Neural Netw 12(4):809–821CrossRefGoogle Scholar
- 44.Vapnik V (1995) The nature of statistical learning theory. Springer, NYCrossRefzbMATHGoogle Scholar
- 45.Wang J, Wang J (2015) Forecasting stock market indexes using principle component analysis and stochastic time effective neural networks. Neurocomputing 156:68–78CrossRefGoogle Scholar
- 46.Wang S, Meng B (2011) Parameter selection algorithm for support vector machine. Prog Environ Sci 11:538–544. doi: 10.1016/j.proenv.2011.12.085 CrossRefGoogle Scholar
- 47.Wu CH, Tzeng GH, Lin RH (2009) A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36(3):4725–4735. doi: 10.1016/j.eswa.2008.06.046 CrossRefGoogle Scholar
- 48.Zhai G, Chen J, Wang S, Li K, Zhang L (2015) Material identification of loose particles in sealed electronics devices using PCA and SVM. Neurocomputing 148:222–228. doi: 10.1016/j.neucom.2013.10.043 CrossRefGoogle Scholar