Interactive Selection of Visual Features through Reinforcement Learning
We introduce a new class of Reinforcement Learning algorithms designed to operate in perceptual spaces containing images. They work by classifying the percepts using a computer vision algorithm specialized in image recognition, hence reducing the visual percepts to a symbolic class. This approach has the advantage of overcoming to some extent the curse of dimensionality by focusing the attention of the agent on distinctive and robust visual features.
The visual classes are learned automatically in a process that only relies on the reinforcement earned by the agent during its interaction with the environment. In this sense, the visual classes are learned interactively in a task-driven fashion, without an external supervisor. We also show how our algorithms can be extended to perceptual spaces, large or even continuous, upon which it is possible to define features.
Unable to display preview. Download preview PDF.
- R. Bellman. Dynamic Programming. Princeton University Press, 1957.Google Scholar
- L. Chrisman. Reinforcement learning with perceptual aliasing: The perceptual distinctions approach. In National Conference on Artificial Intelligence, pages 183–188, 1992.Google Scholar
- D. Ernst, P. Geurts, and L. Wehenkel. Tree-based batch mode reinforcement learning, 2004. Submitted for publication.Google Scholar
- V. Gouet and N. Boujemaa. Object-based queries using color points of interest. In IEEE Workshop on Content-Based Access of Image and Video Libraries, pages 30–36, Kauai, Hawaii, USA, 2001.Google Scholar
- M. Huber and R. Grupen. A control structure for learning locomotion gaits. In 7th Int. Symposium on Robotics and Applications, Anchorage, AK, May 1998. TSI Press.Google Scholar
- L.P. Kaelbling, M.L. Littman, and A. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–285, 1996.Google Scholar
- T.K. Leung, M.C. Burl, and P. Perona. Finding faces in cluttered scenes using random labeled graph matching. In Proc. of the Fifth International Conference on Computer Vision, page 637. IEEE Computer Society, 1995.Google Scholar
- R.A. McCallum. Reinforcement learning with selective perception and Hidden State. PhD thesis, University of Rochestor, Rochestor, New York, 1996.Google Scholar
- K. Mikolajczyk and C. Schmid. A performance evaluation of local descriptors. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, volume 2, pages 257–263, Madison, Wisconsin, June 2003.Google Scholar
- T.M. Mitchell. Machine Learning. McGraw Hill, 1997.Google Scholar
- S.A. Nene, S.K. Nayar, and H. Murase. Columbia object image library (COIL-100). Technical Report CUCS-006-96, Columbia University, New York, NY, February 1996.Google Scholar
- J.H. Piater. Visual Feature Learning. PhD thesis, Computer Science Department, University of Massachusetts, Amherst, MA, February 2001.Google Scholar
- L.D. Pyeatt and A.E. Howe. Decision tree function approximation in reinforcement learning. In Proc. of the Third International Symposium on Adaptive Systems, pages 70–77, Havana, Cuba, March 2001.Google Scholar
- J.R. Quinlan. The effect of noise on concept learning. In Machine Learning: An Artificial Intelligence Approach: Volume II, pages 149–166. Kaufmann, Los Altos, CA, 1986.Google Scholar
- R.S. Sutton. Integrated architectures for learning, planning and reacting based on approximating dynamic programming. In Proc. of 7th Int. Conference on Machine Learning, pages 216–224, San Mateo, CA, 1990.Google Scholar
- R.S. Sutton and A.G. Barto. Reinforcement Learning, an Introduction. MIT Press, 1998.Google Scholar
- S.D. Whitehead and D.H. Ballard. Learning to perceive and act by trial and error. Machine Learning, 7:45–83, 1991.Google Scholar