Combining Unsupervised and Supervised Approaches to Feature Selection for Multivariate Signal Compression
A problem of learning from a database where each sample consists of several time series and a single response is considered. We are interested in maximum data reduction that preserves predictive power of the original time series, and at the same time allows reasonable reconstruction quality of the original signals. Each signal is decomposed into a set of wavelet features that are coded according to their importance consisting of two terms. The first depends on the influence of the feature on the expected signal reconstruction error, and the second is determined by feature importance for the response prediction. The latter is calculated by building series of boosted decision tree ensembles. We demonstrate that such combination maintains small signal distortion rates, and ensures no increase in the prediction error in contrast to the unsupervised compression with the same reduction ratio.
KeywordsFeature Selection Compression Rate Original Time Series Response Prediction Wavelet Feature
Unable to display preview. Download preview PDF.
- Borisov, A., Eruhimov, V., Tuv, E.: Dynamic soft feature selection for tree-based ensembles. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction, Foundations and Applications. Springer, New York (2005)Google Scholar
- Daubechies, I.: Ten lectures on wavelets. SIAM, Philadelphia, PA (1992)Google Scholar
- Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Technical report, Dept. of Statistics, Stanford University (1999)Google Scholar
- Friedman, J.H.: Stochastic gradient boosting. Technical report, Dept. of Statistics, Stanford University (1999)Google Scholar
- Gersho, A., Gray, R.M.: Vector Quantization and Signal Compression. Springer, Heidelberg (1991)Google Scholar
- MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003), available from http://www.inference.phy.cam.ac.uk/mackay/itila/
- Mallat, S.: A Wavelet Tour on Signal Processing. Academic Press, London (1999)Google Scholar
- Torkkola, K., Tuv, E.: Ensembles of regularized least squares classifiers for high-dimensional problems. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction, Foundations and Applications. Springer, Heidelberg (2005)Google Scholar
- Tuv, E.: Feature selection and ensemble learning. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction, Foundations and Applications. Springer, New York (2005)Google Scholar
- Tuv, E., Torkkola, K.: Feature filtering with ensembles using artificial contrasts. accepted for publication in IEEE Intelligent Systems Journal (2005)Google Scholar