Advertisement

Combining Unsupervised and Supervised Approaches to Feature Selection for Multivariate Signal Compression

  • Victor Eruhimov
  • Vladimir Martyanov
  • Peter Raulefs
  • Eugene Tuv
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4224)

Abstract

A problem of learning from a database where each sample consists of several time series and a single response is considered. We are interested in maximum data reduction that preserves predictive power of the original time series, and at the same time allows reasonable reconstruction quality of the original signals. Each signal is decomposed into a set of wavelet features that are coded according to their importance consisting of two terms. The first depends on the influence of the feature on the expected signal reconstruction error, and the second is determined by feature importance for the response prediction. The latter is calculated by building series of boosted decision tree ensembles. We demonstrate that such combination maintains small signal distortion rates, and ensures no increase in the prediction error in contrast to the unsupervised compression with the same reduction ratio.

Keywords

Feature Selection Compression Rate Original Time Series Response Prediction Wavelet Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Borisov, A., Eruhimov, V., Tuv, E.: Dynamic soft feature selection for tree-based ensembles. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction, Foundations and Applications. Springer, New York (2005)Google Scholar
  2. Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)MATHCrossRefGoogle Scholar
  3. Daubechies, I.: Ten lectures on wavelets. SIAM, Philadelphia, PA (1992)Google Scholar
  4. Donoho, D.L., Johnstone, I.M.: Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association 90(432), 1200–1224 (1995)MATHCrossRefMathSciNetGoogle Scholar
  5. Donoho, D.L.: Denoising via soft-thresholding. IEEE Trans. Infrom. Theory 41(3), 613–627 (1995)MATHCrossRefMathSciNetGoogle Scholar
  6. Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Technical report, Dept. of Statistics, Stanford University (1999)Google Scholar
  7. Friedman, J.H.: Stochastic gradient boosting. Technical report, Dept. of Statistics, Stanford University (1999)Google Scholar
  8. Gersho, A., Gray, R.M.: Vector Quantization and Signal Compression. Springer, Heidelberg (1991)Google Scholar
  9. Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  10. Jeong, M.K., Lu, J.-C., Huo, X., et al.: Wavelet-based data reduction techniques for process fault detection. Technometrics 48(1), 26–40 (2006)CrossRefMathSciNetGoogle Scholar
  11. Jin, J., Shi, J.: Automatic feature extraction of waveform signals for in-process diagnostic performance improvement. Journal of Intellifent Manufacturing 12, 257–268 (2001)CrossRefGoogle Scholar
  12. MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003), available from http://www.inference.phy.cam.ac.uk/mackay/itila/
  13. Mallat, S.: A Wavelet Tour on Signal Processing. Academic Press, London (1999)Google Scholar
  14. Torkkola, K., Tuv, E.: Ensembles of regularized least squares classifiers for high-dimensional problems. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction, Foundations and Applications. Springer, Heidelberg (2005)Google Scholar
  15. Tuv, E.: Feature selection and ensemble learning. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction, Foundations and Applications. Springer, New York (2005)Google Scholar
  16. Tuv, E., Torkkola, K.: Feature filtering with ensembles using artificial contrasts. accepted for publication in IEEE Intelligent Systems Journal (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Victor Eruhimov
    • 1
  • Vladimir Martyanov
    • 1
  • Peter Raulefs
    • 1
  • Eugene Tuv
    • 1
  1. 1.Intel, Analysis & Control Technology 

Personalised recommendations