Advertisement

Benchmarks for Robotic Soccer Vision

  • Ricardo Dodds
  • Luca Iocchi
  • Pablo Guerrero
  • Javier Ruiz-del-Solar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7416)

Abstract

Robotic soccer vision has been a major research problem in RoboCup and, even though many progresses have been made so that, for example, games now can run without many constraints on the lighting conditions, the problem has not been completely solved and on-site camera calibration is always a major activity for RoboCup soccer teams. While different robotic soccer vision and object perception techniques continue to appear in the RoboCup Soccer League, there is a lack of quantitative evaluation of existing methods.

Since we believe that a quantitative evaluation of soccer vision algorithms will led to significant advances in the performance on perception and on the entire soccer task, in this paper we propose a benchmarking methodology for evaluating robotic soccer vision systems. We discuss the main issues of a successful benchmarking methodology: (i) a large and complete data base or data sets with ground truth; (ii) a public repository with data sets, algorithms and implementations that can be dynamically updated and (iii) evaluation metrics, error functions and comparison results.

Keywords

Benchmarking and Evaluation Color-based Object Recognition Robotic Soccer Vision 

References

  1. 1.
    Cameron, D., Barnes, N.: Knowledge-Based Autonomous Dynamic Colour Calibration. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 226–237. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Ceriani, S., Fontana, G., Giusti, A., Marzorati, D., Matteucci, M., Migliore, D., Rizzi, D., Sorrenti, D.G., Taddei, P.: Rawseeds ground truth collection systems for indoor self-localization and mapping. Autonomous Robots 27(4), 353–371 (2009)CrossRefGoogle Scholar
  3. 3.
    Dahm, I., Deutsch, S., Hebbel, M., Osterhues, A.: Robust color classification for robot soccer. RoboCup 2003: Robot World Cup VII, Lecture Notes in Artificial Intelligence (2004)Google Scholar
  4. 4.
    Gönner, C., Rous, M., Kraiss, K.-F.: Real-time Adaptive Colour Segmentation for the Robocup Middle Size League. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, pp. 402–409. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Grillo, E., Matteucci, M., Sorrenti, D.G.: Getting the Most from Your Color Camera in a Color-Coded World. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, pp. 221–235. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Guerrero, P.E., Ruiz-del-Solar, J., Fredes, J., Palma-Amestoy, R.: Automatic On-Line Color Calibration Using Class-Relative Color Spaces. In: Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F. (eds.) RoboCup 2007. LNCS (LNAI), vol. 5001, pp. 246–253. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Gunnarsson, K., Wiesel, F., Rojas, R.: The Color and the Shape: Automatic On-Line Color Calibration for Autonomous Robots. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS (LNAI), vol. 4020, pp. 347–358. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Iocchi, L.: Robust Color Segmentation Through Adaptive Color Distribution Transformation. In: Lakemeyer, G., Sklar, E., Sorrenti, D.G., Takahashi, T. (eds.) RoboCup 2006. LNCS (LNAI), vol. 4434, pp. 287–295. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Jamzad, M., Lamjiri, A.K.: An Efficient Need-Based Vision System in Variable Illumination Environment of Middle Size RoboCup. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 654–661. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Jüngel, M., Hoffmann, J., Lötzsch, M.: A Real-Time Auto-Adjusting Vision System for Robotic Soccer. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 214–225. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Jüngel, M.: Using Layered Color Precision for a Self-Calibrating Vision System. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, pp. 209–220. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Lovell, N.: Illumination Independent Object Recognition. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS (LNAI), vol. 4020, pp. 384–395. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Mayer, G., Utz, H., Kraetzschmar, G.K.: Playing Robot Soccer Under Natural Light: A Case Study. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 238–249. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  14. 14.
    Palma-Amestoy, R., Guerrero, P., Ruiz-del-Solar, J., Garretón, C.: Bayesian Spatiotemporal Context Integration Sources in Robot Vision Systems. In: Iocchi, L., Matsubara, H., Weitzenfeld, A., Zhou, C. (eds.) RoboCup 2008. LNCS (LNAI), vol. 5399, pp. 212–224. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Ponce, J., Berg, T.L., Everingham, M., Forsyth, D., Hebert, M., Lazebnik, S., Marszalek, M., Schmid, C., Russell, B.C., Torralba, A., Williams, C.K.I., Zhang, J., Zisserman, A.: Dataset Issues in Object Recognition. In: Ponce, J., Hebert, M., Schmid, C., Zisserman, A. (eds.) Toward Category-Level Object Recognition. LNCS, vol. 4170, pp. 29–48. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Röfer, T.: Region-Based Segmentation with Ambiguous Color Classes and 2-D Motion Compensation. In: Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F. (eds.) RoboCup 2007. LNCS (LNAI), vol. 5001, pp. 369–376. Springer, Heidelberg (2008)Google Scholar
  17. 17.
    Sridharan, M., Stone, P.: Towards Illumination Invariance in the Legged League. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, pp. 196–208. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  18. 18.
    Wilking, D., Röfer, T.: Realtime Object Recognition Using Decision Tree Learning. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, pp. 556–563. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ricardo Dodds
    • 1
  • Luca Iocchi
    • 1
  • Pablo Guerrero
    • 2
  • Javier Ruiz-del-Solar
    • 2
  1. 1.Dipartimento di Informatica e SistemisticaUniversita “La Sapienza”RomeItaly
  2. 2.Departament of Electrical EngineeringUniversidad de ChileChile

Personalised recommendations