Applied Intelligence

, Volume 17, Issue 1, pp 49–60 | Cite as

Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning

  • Stefan Schaal
  • Christopher G. Atkeson
  • Sethu Vijayakumar

Abstract

Locally weighted learning (LWL) is a class of techniques from nonparametric statistics that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been tested successfully in real-time learning of complex robot tasks. We discuss two major classes of LWL, memory-based LWL and purely incremental LWL that does not need to remember any data explicitly. In contrast to the traditional belief that LWL methods cannot work well in high-dimensional spaces, we provide new algorithms that have been tested on up to 90 dimensional learning problems. The applicability of our LWL algorithms is demonstrated in various robot learning examples, including the learning of devil-sticking, pole-balancing by a humanoid robot arm, and inverse-dynamics learning for a seven and a 30 degree-of-freedom robot. In all these examples, the application of our statistical neural networks techniques allowed either faster or more accurate acquisition of motor control than classical control engineering.

nonparametric regression locally weighted learning motor control internal models incremental learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    S. Schaal, “Is imitation learning the route to humanoid robots?” Trends in Cognitive Sciences, vol. 3, pp. 233–242, 1999.Google Scholar
  2. 2.
    C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press: New York, 1995.Google Scholar
  3. 3.
    C.K.I. Williams and C.E. Rasmussen, “Gaussian processes for regression,” in Advances in Neural Information Processing Systems, vol. 8, edited by D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo, MIT Press: Cambridge, MA, pp. 514–520, 1996.Google Scholar
  4. 4.
    V.N. Vapnik, Estimation of Dependences Based on Empirical Data, Springer: Berlin, 1982.Google Scholar
  5. 5.
    C. Cortes and V. Vapnik, “Support vector networks,” Machine Learning, vol. 20, pp. 273–297, 1995.Google Scholar
  6. 6.
    V. Vapnik, S. Golowich, and A. Smola, “Support vector method for function approximation, regression estimation, and signal processing,” in Advances in Neural Information Processing Systems, vol. 9, edited by M. Mozer, M.I. Jordan, and T. Petsche, MIT Press: Cambridge, MA, pp. 281–287, 1996.Google Scholar
  7. 7.
    S. Schaal and C.G. Atkeson, “Constructive incremental learning from only local information,” Neural Comput, vol. 10, pp. 2047–2084, 1998.Google Scholar
  8. 8.
    C.G. Atkeson and S. Schaal, “Memory-based neural networks for robot learning,” Neurocomputing, vol. 9, pp. 1–27, 1995.Google Scholar
  9. 9.
    W.S. Cleveland and C. Loader, Smoothing by Local Regression: Principles and Methods, AT&T Bell Laboratories: Murray Hill, NY, 1995.Google Scholar
  10. 10.
    T.J. Hastie and R.J. Tibshirani, “Nonparametric regression and classification: Part I: Nonparametric regression,” in From Statistics to Neural Networks: Theory and Pattern Recognition Applications, edited by V. Cherkassky, J.H. Friedman, and H. Wechsler, ASI Proceedings, Subseries F, Computer and Systems Sciences, Springer: Berlin, pp. 120–143, 1994.Google Scholar
  11. 11.
    C.G. Atkeson, A.W. Moore, and S. Schaal, “Locally weighted learning,” Artificial Intelligence Review, vol. 11, pp. 11–73, 1997.Google Scholar
  12. 12.
    C.G. Atkeson, A.W. Moore, and S. Schaal, “Locally weighted learning for control,” Artificial Intelligence Review, vol. 11, pp. 75–113, 1997.Google Scholar
  13. 13.
    D. Aha, “Lazy learning,” Artificial Intelligence Review, vol. 11, no. 1–5, pp. 325–337, 1997.Google Scholar
  14. 14.
    L. Ljung and T. Söderström, Theory and Practice of Recursive Identification, MIT Press: Cambridge, MA, 1986.Google Scholar
  15. 15.
    W.S. Cleveland, “Robust locally weighted regression and smoothing scatterplots,” Journal of the American Statistical Association, vol. 74, pp. 829–836, 1979.Google Scholar
  16. 16.
    T. Hastie and C. Loader, “Local regression: Automatic kernel carpentry,” Statistical Science, vol. 8, pp. 120–143, 1993.Google Scholar
  17. 17.
    C.G. Atkeson, “Using local models to control movement,” in Advances in Neural Information Processing Systems, vol. 1, edited by D. Touretzky, Morgan Kaufmann: San Mateo, CA, pp. 157–183, 1989.Google Scholar
  18. 18.
    C.G. Atkeson, “Memory-based approaches to approximating continuous functions,” in Nonlinear Modeling and Forecasting, edited by M. Casdagli and S. Eubank, Addison Wesley: Redwood City, CA, pp. 503–521, 1992.Google Scholar
  19. 19.
    A.W. Moore, “Efficient memory-based learning for robot control,” Computer Laboratory, University of Cambridge, October 1990.Google Scholar
  20. 20.
    D.G. Lowe, “Similarity metric learning for a variable-kernel classifier,” Neural Comput, vol. 7, pp. 72–85, 1995.Google Scholar
  21. 21.
    H. Wold, “Soft modeling by latent variables: The nonlinear iterative partial least squares approach,” in Perspectives in Probability and Statistics, Papers in Honour of M.S. Bartlett, edited by J. Gani, Academic Press: London, pp. 520–540, 1975.Google Scholar
  22. 22.
    I.E. Frank and J.H. Friedman, “Astatistical viewof some chemometric regression tools,” Technometrics, vol. 35, pp. 109–135, 1993.Google Scholar
  23. 23.
    S. Schaal, S.Vijayakumar, and C.G. Atkeson, “Local dimensionality reduction,” in Advances in Neural Information Processing Systems, vol. 10, edited by M.I. Jordan, M.J. Kearns, and S.A. Solla, MIT Press: Cambridge, MA, pp. 633–639, 1998.Google Scholar
  24. 24.
    S. Vijayakumar and S. Schaal, “Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional spaces,” in Proceedings of the Seventeenth International Conference on Machine Learning 2000 (ICML 2000), Stanford, CA.Google Scholar
  25. 25.
    W.P. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C—“The Art of Scientific Computing,” Press Syndiacate University of Cambridge: Cambridge, MA, 1989.Google Scholar
  26. 26.
    M. Kawato, “Internal models for motor control and trajectory planning,” Curr. Opin. Neurobiol, vol. 9, pp. 718–727, 1999.Google Scholar
  27. 27.
    D.M. Wolpert, R.C. Miall, and M. Kawato, “Internal models in the cerebellum,” Trends in Cognitive Sciences, vol. 2, pp. 338–347, 1998.Google Scholar
  28. 28.
    S. Schaal and C.G. Atkeson, “Robot juggling: An implementation of memory-based learning,” Control Systems Magazine, vol. 14, pp. 57–71, 1994.Google Scholar
  29. 29.
    P. Dyer and S.R. McReynolds, The Computation and Theory of Optimal Control, Academic Press: New York, 1970.Google Scholar
  30. 30.
    J.J. Craig, Introduction to Robotics, Addison-Wesley: Reading, MA, 1986.Google Scholar
  31. 31.
    C.H. An, C.G. Atkeson, and J.M. Hollerbach, Model-Based Control of a Robot Manipulator, MIT Press: Cambridge, MA, 1988.Google Scholar
  32. 32.
    J. Baillieul and D.P. Martin, “Resolution of kinematic redundancy,” in Proceedings of Symposia in Applied Mathematics, American Mathematical Society, pp. 49–89, 1990.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Stefan Schaal
    • 1
    • 2
  • Christopher G. Atkeson
    • 3
    • 4
  • Sethu Vijayakumar
    • 1
    • 2
  1. 1.Computer Science and Neuroscience, HNB-103University of Southern CaliforniaLos AngelesUSA;
  2. 2.Kawato Dynamic Brain Project (ERATO/JST)KyotoJapan
  3. 3.College of ComputingGeorgia Institute of TechnologyAtlantaUSA;
  4. 4.ATR Human Information Processing LaboratoriesKyotoJapan

Personalised recommendations