KittingBot: A Mobile Manipulation Robot for Collaborative Kitting in Automotive Logistics

  • Dmytro PavlichenkoEmail author
  • Germán Martín García
  • Seongyong Koo
  • Sven Behnke
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 867)


Individualized manufacturing of cars requires kitting: the collection of individual sets of part variants for each car. This challenging logistic task is frequently performed manually by warehouseman. We propose a mobile manipulation robotic system for autonomous kitting, building on the Kuka Miiwa platform which consists of an omnidirectional base, a 7 DoF collaborative iiwa manipulator, cameras, and distance sensors. Software modules for detection and pose estimation of transport boxes, part segmentation in these containers, recognition of part variants, grasp generation, and arm trajectory optimization have been developed and integrated. Our system is designed for collaborative kitting, i.e. some parts are collected by warehouseman while other parts are picked by the robot. To address safe human-robot collaboration, fast arm trajectory replanning considering previously unforeseen obstacles is realized. The developed system was evaluated in the European Robotics Challenge 2, where the Miiwa robot demonstrated autonomous kitting, part variant recognition, and avoidance of unforeseen obstacles.



This research received funding from the European Union’s Seventh Framework Programme grant agreement no. 608849 (EuRoC). It was performed in collaboration with our end-user partner Peugeot Citroën Automobiles S.A. (PSA). We also gratefully acknowledge the support of the EuRoC Challenge 2 host: DLR Institute of Robotics and Mechatronics in Oberpfaffenhofen, Germany.


  1. 1.
    Dömel, A., Kriegel, S., Kaßecker, M., Brucker, M., Bodenmüller, T., Suppa, M.: Toward fully autonomous mobile manipulation for industrial environments. Int. J. Adv. Robot. Syst. 14 (2017)CrossRefGoogle Scholar
  2. 2.
    Nieuwenhuisen, M., Droeschel, D., Holz, D., Stückler, J., Berner, A., Li, J., Klein, R., Behnke, S.: Mobile bin picking with an anthropomorphic service robot. In: IEEE International Conference on Robotics and Automation (ICRA) (2013)Google Scholar
  3. 3.
    Stückler, J., Schwarz, M., Behnke, S.: Mobile manipulation, tool use, and intuitive interaction for cognitive service robot Cosero. Frontiers in Robotics and AI (2016)Google Scholar
  4. 4.
    Berner, A., Li, J., Holz, D., Stückler, J., Behnke, S., Klein, R.: Combining contour and shape primitives for object detection and pose estimation of prefabricated parts. In: IEEE International Conference on Image Processing (ICIP) (2013)Google Scholar
  5. 5.
    Krueger, V., Chazoule, A., Crosby, M., Lasnier, A., Pedersen, M.R., Rovida, F., Nalpantidis, L., Petrick, R., Toscano, C., Veiga, G.: A vertical and cyber-physical integration of cognitive robots in manufacturing. Proc. IEEE 104, 1114–1127 (2016)CrossRefGoogle Scholar
  6. 6.
    Rovida, F., Crosby, M., Holz, D., Polydoros, A.S., Großmann, B., Petrick, R.P.A., Krüger, V.: SkiROS—a skill-based robot control platform on top of ROS. In: Robot Operating System (ROS): The Complete Reference, pp. 121–160 (2017)Google Scholar
  7. 7.
    Pedersen, M.R., Nalpantidis, L., Andersen, R.S., Schou, C., Bøgh , S., Krüger, V., Madsen, O.: Robot skills for manufacturing: from concept to industrial deployment. Robot. Comput. Integr. Manuf., 282–291 (2016)CrossRefGoogle Scholar
  8. 8.
    Holz, D., Topalidou-Kyniazopoulou, A., Stückler, J., Behnke, S.: Real-time object detection, localization and verification for fast robotic depalletizing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015, pp. 1459–1466 (2015)Google Scholar
  9. 9.
    Crosby, M., Petrick, R., Toscano, C., Dias, R., Rovida, F., Krüger, V.: Integrating mission, logistics, and task planning for skills-based robot control in industrial kitting applications. In: Proceedings of the 34th Workshop of the UK Planning and Scheduling Special Interest Group (PlanSIG), pp. 135–174 (2017)Google Scholar
  10. 10.
    Krug, R., Stoyanov, T., Tincani, V., Andreasson, H., Mosberger, R., Fantoni, G., Lilienthal, A.J.: The next step in robot commissioning: autonomous picking and palletizing. IEEE Robot. Autom. Lett., 546–553 (2016)CrossRefGoogle Scholar
  11. 11.
    Berenson, D., Srinivasa, S., Kuffner, J.: Task space regions: a framework for pose-constrained manipulation planning. Int. J. Robot. Res. (IJRR), 1435–1460 (2011)CrossRefGoogle Scholar
  12. 12.
    Gienger, M., Toussaint, M., Goerick, C.: Task maps in humanoid robot manipulation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2008)Google Scholar
  13. 13.
    Srinivasa, S.S., Johnson, A.M., Lee, G., Koval, M.C., Choudhury, S., King, J.E., Dellin, C.M., Harding, M., Butterworth, D.T., Velagapudi, P., Thackston, A.: A system for multi-step mobile manipulation: architecture, algorithms, and experiments. In: 2016 International Symposium on Experimental Robotics, pp. 254–265 (2017)Google Scholar
  14. 14.
    Holz, D., Behnke:, S.: Fast edge-based detection and localization of transport boxes and pallets in RGB-D images for mobile robot bin picking. In: International Symposium on Robotics (ISR) (2016)Google Scholar
  15. 15.
    Choi, C., Trevor, A.J.B., Christensen, H.I.: RGB-D edge detection and edge-based registration. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2013)Google Scholar
  16. 16.
    Holz, D., Holzer, S., Rusu, R.B., Behnke, S.: Real-time plane segmentation using RGB-D cameras. In: RoboCup 2011: Robot Soccer World Cup XV, pp. 306–317 (2012)Google Scholar
  17. 17.
    Holz, D., Ichim, A.E., Tombari, F., Rusu, R.B., Behnke, S.: Registration with the point cloud library: a modular framework for aligning in 3-D. IEEE Robot. Autom. Mag. 22, 110–124 (2015)CrossRefGoogle Scholar
  18. 18.
    Pavlichenko, D., Behnke, S.: Efficient stochastic multicriteria arm trajectory optimization. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)Google Scholar
  19. 19.
    Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., Schaal, S.: STOMP: stochastic trajectory optimization for motion planning. In: IEEE International Conference on Robotics and Automation (ICRA) (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Dmytro Pavlichenko
    • 1
    Email author
  • Germán Martín García
    • 1
  • Seongyong Koo
    • 1
  • Sven Behnke
    • 1
  1. 1.Autonomous Intelligent SystemsComputer Science Institute VI, University of BonnBonnGermany

Personalised recommendations