Autonomous Robots

, 27:465 | Cite as

Performance evaluation of pure-motion tasks for mobile robots with respect to world models

  • Daniele CalisiEmail author
  • Daniele Nardi


The evaluation of the performance of robot motion methods and systems is still an open challenge, although substantial progress has been made in the field over the years. On the one hand, these techniques cannot be evaluated off-line, on the other hand, they are deeply influenced by the task, the environment and the specific representation chosen for it. In this paper we concentrate on “pure-motion tasks”: tasks that require to move the robot from one configuration to another, either being an independent sub-task of a more complex plan or representing a goal by itself. After characterizing the goals and the tasks, we describe the commonly-used problem decomposition and different kinds of modeling that can be used, from accurate metric maps to minimalistic representations. The contribution of this paper is an evaluation framework that we adopt in a set of experiments showing how the performance of the motion system can be affected by the use of different kinds of environment representations.


Robot motion system Performance measurement World model 


  1. Amigoni, F., & Gasparini, S. (2008). Analysis of methods for reducing line segments in maps: towards a general approach. In Proc. of IEEE/RSJ int. conf. on robots and intelligent systems (IROS) (pp. 2896–2901). Google Scholar
  2. Beeson, P., Jong, N. K., & Kuipers, B. (2005). Towards autonomous topological place detection using the extended Voronoi graph. In Proc. of the IEEE int. conf. on robotics and automation (ICRA) (pp. 4373–4379). Barcelona, Spain. Google Scholar
  3. Bhattacharya, P., & Gavrilova, M. (1991). Roadmap-based path planning. In IEEE robotics and automation magazine (pp. 58–66). Google Scholar
  4. Borenstein, J. (1991). The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Transactions on Robotics and Automation, 7(3), 278–288. CrossRefGoogle Scholar
  5. Brock, O., & Khatib, O. (1999). High-speed navigation using the global dynamic window approach. In IEEE int. conf. on robotics and automation (ICRA) (pp. 341–346). Google Scholar
  6. Bruce, J., & Veloso, M. (2002). Real-time randomized path planning for robot navigation. In Proceedings of IROS-2002, Switzerland, October 2002. Google Scholar
  7. Calisi, D., Farinelli, A., Iocchi, L., & Nardi, D. (2005). Autonomous navigation and exploration in a rescue environment. In Proceedings of IEEE international workshop on safety, security and rescue robotics (SSRR) (pp. 54–59). Kobe, Japan. ISBN 0-7803-8946-8. Google Scholar
  8. Calisi, D., Iocchi, L., & Nardi, D. (2008). A unified benchmark framework for autonomous mobile robots and vehicles motion algorithms (MoVeMA benchmarks). In RSS workshop on experimental methodology and benchmarking, Zurich, Switzerland. Google Scholar
  9. Censi, A., Calisi, D., De Luca, A., & Oriolo, G. (2008). A Bayesian framework for optimal motion planning with uncertainty. In Proc. of the IEEE int. conference on robotics and automation (ICRA) (pp. 1798–1805). doi: 10.1109/ROBOT.2008.4543469.
  10. Choset, H., & Nagatani, K. (2001). Topological simultaneous localization and mapping (SLAM): toward exact localization without explicit localization. IEEE Transactions on Robotics and Automation, 17, 125–137. CrossRefGoogle Scholar
  11. Collet, T., MacDonald, B., & Gerkey, B. (2005). Player 2.0: Toward a practical robot programming framework. In Proc. of the Australasian conf. on robotics and automation (ACRA 2005). Google Scholar
  12. Coolidge, J. (1952). The unsatisfactory story of curvature. The American Mathematical Monthly, 59(6), 375–379. zbMATHCrossRefMathSciNetGoogle Scholar
  13. Fernández, J., Sanz, R., Benayas, J., & Diéguez, A. (2004). Improving collision avoidance for mobile robots in partially known environments: the beam curvature method. Robotics and Autonomous Systems, 46(4), 205–219. CrossRefGoogle Scholar
  14. Filliat, D. (2008). Interactive learning of visual topological navigation. In IEEE/RSJ international conference con intelligent robots and systems (IROS) (pp. 248–254). Nice, France. Google Scholar
  15. Fox, D., Burgard, W., & Thrun, S. (1997). The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine, 4(1), 23–33. doi: 10.1109/100.580977. URL: CrossRefGoogle Scholar
  16. Guo, Y., Qu, Z., & Wang, J. (2003). A new performance-based motion planner for nonholonomic mobile robots. In E. Messina & A. Meystel (Eds.), Proc. of int. workshop on performance metrics for intelligent systems workshop (PerMIS). Google Scholar
  17. Gupta, K. K. (1998). Overview and state of the art. In K. Gupta & A. del Pobil (Eds.), Practical motion planning in robotics: current approaches and future directions (pp. 3–8). New York: Wiley. Google Scholar
  18. Hwang, Y. K., & Ahuja, N. (1992). Gross motion planning: a survey. ACM Computing Surveys, 24(3), 219–291. ISSN 0360-0300. doi: 10.1145/136035.136037. CrossRefGoogle Scholar
  19. Koren, Y., & Borenstein, J. (1991). Potential field methods and their inherent limitations for mobile robot navigation. In Proceedings of the IEEE conference on robotics and automation (ICRA) (pp. 1398–1404). URL:
  20. Kostov, V., & Degtiariova-Kostova, E. (1995). The planar motion with bounded derivative of the curvature and its suboptimal paths. Acta Mathematica Universitatis Comeianae, 64, 185–226. URL: zbMATHMathSciNetGoogle Scholar
  21. Kuipers, B. (1985). The map-learning critter. Technical report, University of Texas at Austin. Google Scholar
  22. Kuipers, B., & Byun, Y.-T. (1991). A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Journal of Robotics and Autonomous Systems, 8, 47–63. URL: CrossRefGoogle Scholar
  23. Latombe, J. C. (1991). Robot motion planning. Dordrecht: Kluwer Academic. Google Scholar
  24. Lynch, K. (1960). The image of the city. Cambridge: MIT Press. Google Scholar
  25. Minguez, J., & Montano, L. (2004). Nearness diagram (nd) navigation: Collision avoidance in troublesome scenarios. IEEE Transactions on Robotics and Automations, 20(1), 45–59. CrossRefGoogle Scholar
  26. Minguez, J., Montano, L., Simeon, T., & Alami, R. (2001). Global nearness diagram (gnd) navigation. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (vol. 1, pp. 33–39). Google Scholar
  27. Minguez, J., Montano, L., & Santos-Victor, J. (2006). Abstracting vehicle shape and kinematic constraints from obstacle avoidance methods. Autonomous Robots, 20(1), 43–59. ISSN 0929-5593. CrossRefGoogle Scholar
  28. Minguez, J., Lamiraux, F., & Laumond, J.-P. (2008). Motion planning and obstacle avoidance. In B. Siciliano & O. Khatib (Eds.), Springer handbook of robotics. Berlin: Springer. doi: 10.1007/978-3-540-30301-5. Google Scholar
  29. Mjolsness, R. C., & Swartz, B. (1987). Some plane curvature approximations. Mathematics of Computation, 49(179), 215–230. zbMATHCrossRefMathSciNetGoogle Scholar
  30. Montemerlo, M., Thrun, S., Koller, D., & Wegbreit, B. (2002). FastSLAM: a factored solution to the simultaneous localization and mapping problem. In Proc. of the conf. American association for artificial intelligence (AAAI), Edmonton, Canada. Google Scholar
  31. Muñoz, N., Valencia, J., & Londoño, N. (2007). Evaluation of navigation of an autonomous mobile robot. In Proc. of int. workshop on performance metrics for intelligent systems workshop (PerMIS) (pp. 15–21). Google Scholar
  32. Nilsson, N. J. (1969). A mobile automaton: an application of artificial intelligence techniques. In Proc. of int. joint conf. on artificial intelligence (IJCAI) (pp. 509–520). Google Scholar
  33. Ozguner, U., Stiller, C., & Redmill, K. (2007). Systems for safety and autonomous behavior in cars: The DARPA Grand Challenge experience. Proceedings of the IEEE, 95(2), 397–412. ISSN 0018-9219. doi: 10.1109/JPROC.2006.888394. CrossRefGoogle Scholar
  34. Raño, I., & Minguez, J. (2006). Steps towards the automatic evaluation of robot obstacle avoidance algorithms. In Proc. of workshop of benchmarking in robotics, in the IEEE/RSJ int. conf. on intelligent robots and systems (IROS). Google Scholar
  35. Rawlinson, D., & Jarvis, R. (2008). Topologically-directed navigation. Robotica, 26(02), 189–203. doi: 10.1017/S026357470700375X. CrossRefGoogle Scholar
  36. Reif, J. H. (1979) Complexity of the mover’s problem and generalization. In Proceedings of the 20th IEEE symposium on foundations of computer sciences (FOCS) (pp. 421–427). Google Scholar
  37. Rosenblatt, J. (1997). DAMN: A distributed architecture for mobile navigation. Journal of Experimental and Theoretical Artificial Intelligence, 9(1), 339–360. CrossRefMathSciNetGoogle Scholar
  38. Sack, D., & Burgard, W. (2004). A comparison of methods for line extraction from range data. In Proc. of the 5th IFAC symposium on intelligent autonomous vehicles (IAV). Google Scholar
  39. Shmaglit, A., Rinat, K., Brand, Z., Fischler, A., & Velger, M. (2006). Autonomous vehicle control and obstacle avoidance concepts oriented to meet the challenging requirements of realistic missions. In International conference on control, automation, robotics and vision (ICARCV) (pp. 1–6). doi: 10.1109/ICARCV.2006.345219.
  40. Simmons, R. (1996). The curvature-velocity method for local obstacle avoidance. In Proceedings of IEEE international conference on robots and automation (vol. 4, pp. 3375–3382). Google Scholar
  41. Stachniss, C., & Burgard, W. (2002). An integrated approach to goal-directed obstacle avoidance under dynamic constraints for dynamic environments. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems (IROS). Google Scholar
  42. Taïx, M., Malti, A. C., & Lamiraux, F. (2008). Planning robust landmarks for sensor based motion. In H. Bruyninckx, L. Preucil, & M. Kulich (Eds.), Springer tracts in advanced robotics : Vol. 44. European robotics symposium (EUROS) (pp. 195–204). Berlin: Springer. ISBN 978-3-540-78315-2. URL: doi: 10.1007/978-3-540-78317-6_20. CrossRefGoogle Scholar
  43. Tomomi, K., Jun, O., Rie, K., Takahisa, M., Tamio, A., Tsuyoshi, U., & Tsuyoshi, N. (2003). Path planning for a mobile robot considering maximum curvature, maximum curvature derivative, and curvature continuity. Transactions of the Japan Society of Mechanical Engineers, 69(688), 3269–3276. Google Scholar
  44. Ulrich, I., & Borenstein, J. (1998). VFH+: Reliable obstacle avoidance for fast mobile robots. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 1572–1577). Google Scholar
  45. Ulrich, I., & Borenstein, J. (2000). VFH*: Local obstacle avoidance with look-ahead verification. In Proceedings of the 2000 IEEE international conference on robotics and automation (ICRA) (pp. 2505–2511). Google Scholar
  46. Werner, F., Gretton, C., Maire, F., & Sitte, J. (2008). Induction of topological environment maps from sequences of visited places. In IEEE/RSJ international conference con intelligent robots and systems (IROS) (pp. 2890–2895). Google Scholar
  47. Yun, J., & Miura, J. (2008). Quantitative measure for the navigability of a mobile robot using rough maps. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3458–3464). Nice, France. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Dipartimento Informatica e Sistemistica“Sapienza” University of RomeRomaItaly

Personalised recommendations