Abstract
For modeling of multivariate time series, input variable selection is a key problem. Feature selection is to select a relevant subset to reduce the dimensionality of the problem without significant loss of information. This paper presents the estimation of mutual information and its application in feature selection problem. Mutual information is one of the most common strategies borrowed from information theory for feature selection. However, the calculation of probability density function (PDF) according to the definition of mutual information is difficult, especially for high dimensional variables. A k-nearest neighbor (k-NN) method based estimator is widely used to estimate the mutual information between two variables directly from the data set. Nevertheless, this estimator depends on smoothing parameter. There is no theoretically method to choose the parameter. This paper purposes to solve two problems: one is to employ resampling methods to help the mutual information estimator to improve feature selection and the other is to apply these methods to a wind power prediction problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Keynia, F.: A new feature selection algorithm and composite neural network for electricity price forecasting. Eng. Appl. Artif. Intell. 25(8), 1687–1697 (2012)
Wu, C.L., Chau, K.W.: Prediction of rainfall time series using modular soft computing methods. Eng. Appl. Artif. Intell. 26(3), 997–1007 (2013)
Du Preez, J., Witt, S.F.: Univariate versus multivariate time series forecasting: An application to international tourism demand. Int. J. Forecast. 19(3), 435–451 (2003)
Franois, D., Rossi, F., Wertz, V.: Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing 70, 1276–1288 (2007)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neual. Comput. 4(1), 1–58 (1992)
Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948)
Frnay, B., Doquire, G., Verleysen, M.: Is mutual information adequate for feature selection in regression? Neural Netw. 48, 1–7 (2013)
Han, M., Ren, W.J., Liu, X.X.: Joint mutual information-based input variable selection for multivariate time series modeling. Eng. Appl. Rtif. Intel. 37, 250–257 (2015)
Lin, Y., Hu, Q., Liu, J., Chen, J., Duan, J.: Multi-label feature selection based on neighborhood mutual information. Appl. Soft. Comput. 38, 244–256 (2016)
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5, 537–550 (1994)
Van Dijck, G., Van Hulle, M.M.: Speeding up the wrapper feature subset selection in regression by mutual information relevance and redundancy analysis. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) Artificial Neural Networks–ICANN 2006. LNCS, vol. 4131, pp. 31–40. Springer, Heidelberg (2006)
Fleuret, F.: Fast binary feature selection with conditional mutual information. J. Mach. Learn. Res. 5, 1531–1555 (2004)
Rossi, F., Franois, D., Wertz, V., Meurens, M., Verleysen, M.: Fast selection of spectral variables with b-spline compression. Chemometr. Intell. Lab. 86(2), 208–218 (2007)
Kraskov, A., Stgbauer, H., Grassberger, P.: Estimating mutual information. Phys. Rev. E 69, 066–138 (2004)
Verleysen, M., Rossi, F., François, D.: Advances in feature selection with mutual information. In: Biehl, M., Hammer, B., Verleysen, M., Villmann, T. (eds.) Similarity-Based Clustering. LNCS, vol. 5400, pp. 52–69. Springer, Heidelberg (2009)
Stogbauer, H., Kraskov, A., Astakhov, S.A., Grassberger, P.: Least dependent component analysis based on mutual information. Phys. Rev. E 70, 066–123 (2004)
Peng, H.C., Long, F.H., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern. Anal. Mach. Intell. 27(8), 1226–1238 (2005)
Acknowledgments
The authors gratefully acknowledge the financial support of this research by the National Natural Science Foundation of China (Grant No. 61374006), the Major Program of National Natural Science Foundation of China (Grant No. 11190015) and the Natural Science Foundation of Jiangsu (Grant No. BK20131300).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Liu, T., Wei, H., Zhang, C., Zhang, K. (2016). Mutual Information with Parameter Determination Approach for Feature Selection in Multivariate Time Series Prediction. In: Jayne, C., Iliadis, L. (eds) Engineering Applications of Neural Networks. EANN 2016. Communications in Computer and Information Science, vol 629. Springer, Cham. https://doi.org/10.1007/978-3-319-44188-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-44188-7_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44187-0
Online ISBN: 978-3-319-44188-7
eBook Packages: Computer ScienceComputer Science (R0)