Sequential Feature Selection for Classification

  • Thomas Rückstieß
  • Christian Osendorfer
  • Patrick van der Smagt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7106)


In most real-world information processing problems, data is not a free resource; its acquisition is rather time-consuming and/or expensive. We investigate how these two factors can be included in supervised classification tasks by deriving classification as a sequential decision process and making it accessible to Reinforcement Learning. Our method performs a sequential feature selection that learns which features are most informative at each timestep, choosing the next feature depending on the already selected features and the internal belief of the classifier. Experiments on a handwritten digits classification task show significant reduction in required data for correct classification, while a medical diabetes prediction task illustrates variable feature cost minimization as a further property of our algorithm.


reinforcement learning feature selection classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bazzani, L., de Freitas, N., Larochelle, H., Murino, V., Ting, J.A.: Learning attentional policies for tracking and recognition in video with deep networks. In: Proceedings of the 28th International Conference on Machine Learning (2011)Google Scholar
  2. 2.
    Deisenroth, M.P., Rasmussen, C.E., Peters, J.: Gaussian process dynamic programming. Neurocomputing 72(7-9), 1508–1524 (2009)CrossRefGoogle Scholar
  3. 3.
    Ernst, D., Geurts, P., Wehenkel, L.: Tree-based batch mode reinforcement learning. Journal of Machine Learning Research 6(1), 503 (2005)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, Irvine, CA (October 2011),
  5. 5.
    Gaudel, R., Sebag, M.: Feature selection as a one-player game. In: Proceedings of the 2nd NIPS Workshop on Optimization for Machine Learning (2009)Google Scholar
  6. 6.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  7. 7.
    Hüsken, M., Stagge, P.: Recurrent neural networks for time series classification. Neurocomputing 50, 223–235 (2003)CrossRefzbMATHGoogle Scholar
  8. 8.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  9. 9.
    Neumann, G., Pfeiffer, M., Hauser, H.: Batch reinforcement learning methods for point to point movements. Technical report, Graz University of Technology (2006)Google Scholar
  10. 10.
    Norouzi, E., Nili Ahmadabadi, M., Nadjar Araabi, B.: Attention control with reinforcement learning for face recognition under partial occlusion. Machine Vision and Applications, 1–12 (2010)Google Scholar
  11. 11.
    Perkins, S., Theiler, J.: Online feature selection using grafting. In: Proceedings of the 20th International Conference on Machine Learning (2003)Google Scholar
  12. 12.
    Riedmiller, M.: Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 317–328. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Saar-Tsechansky, M., Provost, F.: Handling missing values when applying classification models. Journal of Machine Learning Research 8(1625-1657), 9 (2007)zbMATHGoogle Scholar
  14. 14.
    Schmidhuber, J., Huber, R.: Learning to generate artificial fovea trajectories for target detection. International Journal of Neural Systems 2(1), 135–141 (1991)Google Scholar
  15. 15.
    Vijayakumar, S., Schaal, S.: Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional space. In: Proceedings of the Seventeenth International Conference on Machine Learning (2000)Google Scholar
  16. 16.
    Williams, R.J., Peng, J.: An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural Computation 2(4), 490–501 (1990)CrossRefGoogle Scholar
  17. 17.
    Wu, X., Yu, K., Wang, H., Ding, W.: Online streaming feature selection. In: Proceedings of the 27nd International Conference on Machine Learning (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Thomas Rückstieß
    • 1
  • Christian Osendorfer
    • 1
  • Patrick van der Smagt
    • 2
  1. 1.Technische Universität MünchenGarchingGermany
  2. 2.German Aerospace Center / DLRWesslingGermany

Personalised recommendations