Online Learning of Vision-Based Robot Control during Autonomous Operation

  • Kristoffer Öfjäll
  • Michael Felsberg
Part of the Cognitive Systems Monographs book series (COSMOS, volume 23)


Online learning of vision-based robot control requires appropriate activation strategies during operation. In this chapter we present such a learning approach with applications to two areas of vision-based robot control. In the first setting, selfevaluation is possible for the learning system and the system autonomously switches to learning mode for producing the necessary training data by exploration. The other application is in a setting where external information is required for determining the correctness of an action. Therefore, an operator provides training data when required, leading to an automatic mode switch to online learning from demonstration. In experiments for the first setting, the system is able to autonomously learn the inverse kinematics of a robotic arm. We propose improvements producing more informative training data compared to random exploration. This reduces training time and limits learning to regions where the learnt mapping is used. The learnt region is extended autonomously on demand. In experiments for the second setting, we present an autonomous driving system learning a mapping from visual input to control signals, which is trained by manually steering the robot. After the initial training period, the system seamlessly continues autonomously.Manual control can be taken back at any time for providing additional training.


Training Data Mobile Robot Online Learning Inverse Kinematic Autonomous Navigation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ruiz de Angulo, V., Torras, C.: Learning inverse kinematics via cross-point function decomposition. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 856–861. Springer, Heidelberg (2002), CrossRefGoogle Scholar
  2. 2.
    Atkeson, C., Moore, A., Schaal, S.: Locally weighted learning for control. Artificial Intelligence Review 11(1-5), 75–113 (1997), doi:10.1023/A:1006511328852CrossRefGoogle Scholar
  3. 3.
    Baranes, A., Oudeyer, P.Y.: Intrinsically motivated goal exploration for active motor learning in robots: A case study. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1766–1773 (2010), doi:10.1109/IROS.2010.5651385Google Scholar
  4. 4.
    Cohn, D.A., Ghahramani, Z., Jordan, M.I.: Active learning with statistical models. CoRRcs.AI/9603104 (1996)Google Scholar
  5. 5.
    de la Cruz, J.S., Kulić, D., Owen, W.: Online incremental learning of inverse dynamics incorporating prior knowledge. In: Kamel, M., Karray, F., Gueaieb, W., Khamis, A. (eds.) AIS 2011. LNCS (LNAI), vol. 6752, pp. 167–176. Springer, Heidelberg (2011), CrossRefGoogle Scholar
  6. 6.
    Dickmanns, E., Graefe, V.: Dynamic monocular vision. Machine Vision and Applications 1, 223–240 (1988)CrossRefGoogle Scholar
  7. 7.
    Douze, M., Jégou, H., Sandhwalia, H., Amsaleg, L., Schmid, C.: Evaluation of gist descriptors for web-scale image search. In: CIVR 2009: Proceedings of the ACM International Conference on Image and Video Retrieval (2009)Google Scholar
  8. 8.
    D’Souza, A., Vijayakumar, S., Schaal, S.: Learning inverse kinematics. In: Proceedings of the 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 298–303 (2001), doi:10.1109/IROS.2001.973374Google Scholar
  9. 9.
    Kastner, R., Schneider, F., Michalke, T., Fritsch, J., Goerick, C.: Image–based classification of driving scenes by a hierarchical principal component classification (HPCC). In: IEEE Intelligent Vehicles Symposium, pp. 341–346 (2009)Google Scholar
  10. 10.
    Larsson, F., Jonsson, E., Felsberg, M.: Simultaneously learning to recognize and control a low-cost robotic arm. Image Vision Computing 27(11), 1729–1739 (2009),, doi:10.1016/j.imavis.2009.04.003CrossRefGoogle Scholar
  11. 11.
    Lecun, Y., Muller, U., Ben, J., Cosatto, E., Flepp, B.: Off-Road Obstacle Avoidance through End-to-End Learning. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 739–746. MIT Press, Cambridge (2006)Google Scholar
  12. 12.
    Markelic, I.: Teaching a robot to drive: A skill-learning inspired approach. Ph.D. thesis, University of Göttingen (2010)Google Scholar
  13. 13.
    Mitrovic, D., Klanke, S., Vijayakumar, S.: Adaptive optimal feedback control with learned internal dynamics models. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 65–84. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Moré, J.J.: The Levenberg-Marquardt algorithm: Implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis, pp. 105–116. Springer, Berlin (1977)Google Scholar
  15. 15.
    Öfjäll, K.: Leap, a platform for evaluation of control algorithms. Master’s thesis, Linköping University (2010)Google Scholar
  16. 16.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. International Journal of Computer Vision 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  17. 17.
    Oyama, E., Agah, A., MacDorman, K.F., Maeda, T., Tachi, S.: A modular neural network architecture for inverse kinematics model learning. Neuro-computing 38, 797–805 (2001)Google Scholar
  18. 18.
    Oyama, E., Maeda, T., Gan, J., Rosales, E., MacDorman, K., Tachi, S., Agah, A.: Inverse kinematics learning for robotic arms with fewer degrees of freedom by modular neural network systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2005, pp. 1791–1798 (2005), doi:10.1109/IROS.2005.1545084Google Scholar
  19. 19.
    Pomerleau, D.: Alvinn: An autonomous land vehicle in a neural network. In: Proc. of NIPS (1989)Google Scholar
  20. 20.
    Pomerleau, D.: Efficient training of artificial neural networks for autonomous navigation. Neural Computation 3(1), 88–97 (1991)CrossRefGoogle Scholar
  21. 21.
    Pourboghrat, F., Shiao, J.C.: Neural networks for learning inverse kinematics of redundant manipulators. In: Seattle International Joint Conference on Neural Networks, IJCNN 1991, vol. 2, p. 1004 (1991), doi:10.1109/IJCNN.1991.155683Google Scholar
  22. 22.
    Pugeault, N., Bowden, R.: Learning pre-attentive driving behaviour from holistic visual features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 154–167. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  23. 23.
    Pugeault, N., Bowden, R.: Driving me around the bend: Learning to drive from visual gist. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1022–1029 (2011), doi:10.1109/ICCVW.2011.6130363Google Scholar
  24. 24.
    Renninger, L., Malik, J.: When is scene identification just texture recognition? Vision Research 44, 2301–2311 (2004)CrossRefGoogle Scholar
  25. 25.
    Saegusa, R., Metta, G., Sandini, G., Sakka, S.: Active motor babbling for sensorimotor learning. In: IEEE International Conference on Robotics and Biomimetics, ROBIO 2008, pp. 794–799 (2009), doi:10.1109/ROBIO.2009.4913101Google Scholar
  26. 26.
    Schaal, S., Atkeson, C.: Robot juggling: implementation of memory-based learning. IEEE Control Systems 14(1), 57–71 (1994), doi:10.1109/37.257895CrossRefGoogle Scholar
  27. 27.
    Schaal, S., Atkeson, C.: Learning control in robotics. IEEE Robotics Automation Magazine 17(2), 20–29 (2010), doi:10.1109/MRA.2010.936957CrossRefGoogle Scholar
  28. 28.
    Schmiterlöw, M.: Autonomous path following using convolutional networks. Master’s thesis, Linköping University (2012)Google Scholar
  29. 29.
    Settles, B.: Active learning literature survey. Tech. rep. (2009)Google Scholar
  30. 30.
    Siagian, C., Itti, L.: Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(2), 300–312 (2007)CrossRefGoogle Scholar
  31. 31.
    Siagian, C., Itti, L.: Biologically inspired mobile robot vision localization. IEEE Transactions on Robotics 25(4), 861–873 (2009)CrossRefGoogle Scholar
  32. 32.
    Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics, Modelling, Planning and Control (2009) ISBN 978-1-84628-642-1Google Scholar
  33. 33.
    Sugihara, T.: Solvability-unconcerned inverse kinematics based on levenberg-marquardt method with robust damping. In: 9th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2009, pp. 555–560 (2009), doi:10.1109/ICHR.2009.5379515Google Scholar
  34. 34.
    Thrun, S., Montemerlo, M., Dahlkamp, H., Stavens, D., Aron, A., Diebel, J., Fong, P., Gale, J., Halpenny, M., Hoffmann, G., Lau, K., Oakley, C., Palatucci, M., Pratt, V., Stang, P., Strohband, S., Dupont, C., Jendrossek, L.E., Koelen, C., Markey, C., Rummel, C., van Niekerk, J., Jensen, E., Alessandrini, P., Bradski, G., Davies, B., Ettinger, S., Kaehler, A., Nefian, A., Mahoney, P.: Stanley: The robot that won the DARPA Grand Challenge. Journal of Robotic Systems 23(9), 661–692 (2006)Google Scholar
  35. 35.
    Turk, M., Morgenthaler, D., Gremban, K., Marra, M.: VITS—a vision system for autonomous land vehicle navigation. IEEE Trans. in Pattern Analysis and Machine Intelligence 10(3), 342–361 (1988)CrossRefGoogle Scholar
  36. 36.
    Vijayakumar, S., D’souza, A., Schaal, S.: Incremental online learning in high dimensions. Neural Comput. 17, 2602–2634 (2005),, doi:10.1162/089976605774320557MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of Electrical EngineeringLinköping UniversityLinköpingSweden

Personalised recommendations