The Cognitive Interaction Toolkit – Improving Reproducibility of Robotic Systems Experiments

  • Florian Lier
  • Johannes Wienke
  • Arne Nordmann
  • Sven Wachsmuth
  • Sebastian Wrede
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8810)


Research on robot systems either integrating a large number of capabilities in a single architecture or displaying outstanding performance in a single domain achieved considerable progress over the last years. Results are typically validated through experimental evaluation or demonstrated live, e.g., at robotics competitions. While common robot hardware, simulation and programming platforms yield an improved basis, many of the described experiments still cannot be reproduced easily by interested researchers to confirm the reported findings. We consider this a critical challenge for experimental robotics. Hence, we address this problem with a novel process which facilitates the reproduction of robotics experiments. We identify major obstacles to experiment replication and introduce an integrated approach that allows (i) aggregation and discovery of required research artifacts, (ii) automated software build and deployment, as well as (iii) experiment description, repeatable execution and evaluation.We explain the usage of the introduced process along an exemplary robotics experiment and discuss our approach in the context of current ecosystems for robot programming and simulation.


Software Engineering Experimental Robotics Development Process Semantic Web Continuous Integration Software Deployment 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Amigoni, F., Reggiani, M., Schiaffonati, V.: An insightful comparison between experiments in mobile robotics and in science. Autonomous Robots 27(4), 313–325 (2009)CrossRefGoogle Scholar
  3. 3.
    Amigoni, F., Schiaffonati, V., Verdicchio, M.: Good experimental methodologies for autonomous robotics: From theory to practice. In: Amigoni, F., Schiaffonati, V. (eds.) Methods and Experimental Techniques in Computer Engineering. Springer Briefs in Applied Sciences and Technology, pp. 37–53. Springer International Publishing (2014)Google Scholar
  4. 4.
    Bonarini, A., et al.: RAWSEEDS: Robotics advancement through web-publishing of sensorial and elaborated extensive data sets. In: IROS 2006 Workshop on Benchmarks in Robotics Research, vol. 6 (2006)Google Scholar
  5. 5.
    Bruyninckx, H.: Open robot control software: the orocos project. In: Proceedings of IEEE International Conference on Robotics and Automation, ICRA, vol. 3, pp. 2523–2528. IEEE (2001)Google Scholar
  6. 6.
    Cousins, S.: ROS on the PR2 [ROS Topics]. IEEE Robotics Automation Magazine 17(3), 23–25 (2010)CrossRefGoogle Scholar
  7. 7.
    Cousins, S., Gerkey, B., Conley, K.: Sharing software with ros [ROS Topics]. Robotics & Automation Magazine 17(2), 12–14 (2010)CrossRefGoogle Scholar
  8. 8.
    Diankov, R.: Automated Construction of Robotic Manipulation Programs. PhD thesis, Carnegie Mellon University, Robotics Institute (August 2010)Google Scholar
  9. 9.
    Gouaillier, D., et al.: Mechatronic design of NAO humanoid. In: Proc. Int. Conf. on Robotics and Automation, pp. 769–774 (2009)Google Scholar
  10. 10.
    Jang, C., et al.: OPRoS: A new component-based robot software platform. ETRI Journal 32(5), 646–656 (2010)CrossRefGoogle Scholar
  11. 11.
    Lier, F., et al.: Facilitating research cooperation through linking and sharing of heterogenous research artifacts. In: Proc. 8th Int. Conf. on Semantic Systems, pp. 157–164. ACM (2012)Google Scholar
  12. 12.
    Lier, F., Lütkebohle, I., Wachsmuth, S.: Towards automated execution and evaluation of simulated prototype HRI experiments. In: Proc. 2014 ACM/IEEE Int. Conf. on Human-robot Interaction, pp. 230–231. ACM (2014)Google Scholar
  13. 13.
    Metta, G., et al.: The iCub humanoid robot: An open platform for research in embodied cognition. In: Proc. 8th Workshop on Performance Metrics for Intelligent Systems, pp. 50–56. ACM, New York (2008)CrossRefGoogle Scholar
  14. 14.
    Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3 (2009)Google Scholar
  15. 15.
    Soetens, P.: A software framework for real-time and distributed robot and machine control. PhD thesis, Katholieke Universiteit Leuven, Faculteit Ingenieurswetenschappen, Departement Werktuigkunde (2006)Google Scholar
  16. 16.
    Sproewitz, A., et al.: Oncilla robot, a light-weight bio-inspired quadruped robot for fast locomotion in rough terrain. In: Symposium on Adaptive Motion of Animals and Machines, pp. 63–64 (2011)Google Scholar
  17. 17.
    Wienke, J., Wrede, S.: A middleware for collaborative research in experimental robotics. In: 2011 IEEE/SICE Int. Symposium on System Integration, Kyoto, Japan. IEEE (2011)Google Scholar
  18. 18.
    Wiljes, C., Jahn, N., Lier, F., Paul-Stueve, T., Vompras, J., Pietsch, C., Cimiano, P.: Towards linked research data: An institutional approach. In: 3rd Workshop on Semantic Publishing, vol. 994, pp. 27–38 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Florian Lier
    • 1
  • Johannes Wienke
    • 1
    • 2
  • Arne Nordmann
    • 1
    • 2
  • Sven Wachsmuth
    • 1
  • Sebastian Wrede
    • 1
    • 2
  1. 1.Cognitive Interaction Technology, Center of ExcellenceBielefeld UniversityBielefeldGermany
  2. 2.Research Institute for Cognition and Robotics, CoR-Lab.Bielefeld UniversityBielefeldGermany

Personalised recommendations