Autonomous Robots

, Volume 43, Issue 5, pp 1063–1078 | Cite as

Parking objects by pushing using uncalibrated visual servoing

  • Gonzalo López-NicolásEmail author
  • Erol Özgür
  • Youcef Mezouar


Pushing is one of the strategies to perform robotic manipulation when the object is too large or too heavy. Motivated by this, we address the problem of how to push an object on a plane to a target pose with two cooperating robots. The main contribution is a new uncalibrated image-based control scheme that computes the required motion of the object to reach the target pose. Then, as an application of this control scheme, we study the conditions that allow performing the task of pushing the object with two robots. The setup consists of a fixed external uncalibrated camera looking at the workspace where the object and the robots stand. The task is defined with a target image of the object in the desired pose. The proposed control scheme computes the motion commands of the pusher robots and, as a result, they translate and rotate the object by imposing non-holonomic velocity constraints. This yields smooth, continuous and efficient trajectories. The stability of the control scheme is also proven. Experiments illustrate the performance of the control scheme.


Pushing Manipulation Visual servoing 



This work was supported by French Government research program Investissements d’avenir through the RobotEx Equipment of Excellence (ANR-10-EQPX-44), the LabEx IMobS3 (ANR7107LABX716701) and I-SITE Project (CAP 20-25) and by Spanish Government/European Union through Project DPI2015-69376-R (MINECO/FEDER).

Supplementary material

10514_2018_9782_MOESM1_ESM.avi (9.7 mb)
Supplementary material 1 (avi 9944 KB)

Supplementary material 2 (avi 1647 KB)


  1. Adachi, J., & Sato, J. (2004). Uncalibrated visual servoing from projective reconstruction of control values. In 17th international conference on pattern recognition, ICPR (pp. 297–300).Google Scholar
  2. Akella, S., & Mason, M. T. (1998). Posing polygonal objects in the plane by pushing. The International Journal of Robotics Research, 17(1), 70–88.CrossRefGoogle Scholar
  3. Akella, S., & Mason, M. T. (1999). Using partial sensor information to orient parts. International Journal of Robotics Research, 18(10), 963–997.CrossRefGoogle Scholar
  4. Balorda, Z., & Bajd, T. (1994). Reducing positioning uncertainty of objects by robot pushing. IEEE Transactions on Robotics and Automation, 10(4), 535–541.CrossRefGoogle Scholar
  5. Amor, H. B., Saxena, A., Hudson, N., & Peters, J. (2014). Special issue on autonomous grasping and manipulation. Autonomous Robots, 36, 1–3.CrossRefGoogle Scholar
  6. Brost, R. C. (1986). Automatic grasp planning in the presence of uncertainty. IEEE International Conference on Robotics and Automation, 3, 1575–1581.Google Scholar
  7. Cai, C., Somani, N., & Knoll, A. (2016). Orthogonal image features for visual servoing of a 6-dof manipulator with uncalibrated stereo cameras. IEEE Transactions on Robotics, 32(2), 452–461.CrossRefGoogle Scholar
  8. Caron, G., Marchand, E., & Mouaddib, E. M. (2013). Photometric visual servoing for omnidirectional cameras. Autonomous Robots, 35, 177–193.CrossRefGoogle Scholar
  9. Chaumette, F., & Hutchinson, S. (2006). Visual servo control, part I: Basic approaches. IEEE Robotics and Automation Magazine, 13(4), 82–90.CrossRefGoogle Scholar
  10. Chen, H., Sun, D., & Yang, J. (2009). Global localization of multirobot formations using ceiling vision SLAM strategy. Mechatronics, 19(5), 617–628.CrossRefGoogle Scholar
  11. Chesi, G., & Hashimoto, K. (Eds.) (2010). Visual servoing via advanced numerical methods (Vol. 401). Lecture notes in control and information sciences. Heidelberg: Springer.Google Scholar
  12. Dogar, M., & Srinivasa, S. (2011). A framework for push-grasping in clutter. In Proceedings of robotics: Science and systems.Google Scholar
  13. Gandolfo, F., Tistarelli, M., & Sandini, G. (1991). Visual monitoring of robot actions. In Workshop on intelligence for mechanical systems. IEEE/RSJ international conference on intelligent robots and systems (Vol. 1, pp. 269–275).Google Scholar
  14. Golkar, M.A., Namin, S.T., & Aminaiee, H. (2009). Fuzzy controller for cooperative object pushing with variable line contact. In IEEE international conference on mechatronics (pp. 1–6).Google Scholar
  15. Jara, C.A., Pomares, J., F.A. Candelas Herías, & Torres, F. (2014). Optimal control for robot-hand manipulation of an object using dynamic visual servoing. In IEEE/RSJ international conference on intelligent robots and systems (pp. 89–94).Google Scholar
  16. Kopicki, M., Zurek, S., Stolkin, R., Moerwald, T., & Wyatt, J. L. (2016). Learning modular and transferable forward models of the motions of push manipulated objects. Autonomous Robots, 41, 1–22.Google Scholar
  17. Li, B., Fang, Y., & Zhang, X. (2014). Projection homography based uncalibrated visual servoing of wheeled mobile robots. In IEEE conference on decision and control (pp. 2167–2172).Google Scholar
  18. Li, Q., & Payandeh, S. (2007). Manipulation of convex objects via two-agent point-contact push. The International Journal of Robotics Research, 26(4), 377–403.CrossRefGoogle Scholar
  19. Liang, X., Wang, H., Chen, W., Guo, D., & Liu, T. (2015). Adaptive image-based trajectory tracking control of wheeled mobile robots with an uncalibrated fixed camera. IEEE Transactions on Control Systems and Technology, 23(6), 2266–2282.CrossRefGoogle Scholar
  20. López-Nicolás, G., Özgür, E., & Mezouar, Y. (2015). Image-based control of two mobile robots for object pushing. In IEEE/RSJ international conference on intelligent robots and systems (pp. 5472–5478).Google Scholar
  21. López-Nicolás, G., & Sagüés, C. (2011). Vision-based exponential stabilization of mobile robots. Autonomous Robots, 30, 293–306.CrossRefGoogle Scholar
  22. Lynch, K. M. (1992). The mechanics of fine manipulation by pushing. IEEE International Conference on Robotics and Automation, 3, 2269–2276.Google Scholar
  23. Lynch, K. M., Maekawa, H., & Tanie, K. (1992). Manipulation and active sensing by pushing using tactile feedback. IEEE/RSJ International Conference on Intelligent Robots and Systems, 1, 416–421.CrossRefGoogle Scholar
  24. Lynch, K. M., & Mason, M. T. (1996). Stable pushing: Mechanics, controllability, and planning. The International Journal of Robotics Research, 15(6), 533–556.CrossRefGoogle Scholar
  25. MacMillan, W. D. (1936). Dynamics of rigid bodies. NY: Dover.zbMATHGoogle Scholar
  26. Malis, E. (2004). Visual servoing invariant to changes in camera-intrinsic parameters. IEEE Transactions on Robotics and Automation, 20(1), 72–81.CrossRefGoogle Scholar
  27. Marchand, E., Spindler, F., & Chaumette, F. (2005). Visp for visual servoing: A generic software platform with a wide class of robot control skills. IEEE Robotics Automation Magazine, 12(4), 40–52.CrossRefGoogle Scholar
  28. Mason, M. T. (1986). Mechanics and planning of manipulator pushing operations. The International Journal of Robotics Research, 5(3), 53–71.CrossRefGoogle Scholar
  29. Mason, M. T. (2001). Mechanics of robotic manipulation. Cambridge: MIT Press.CrossRefGoogle Scholar
  30. Meriçli, T., Veloso, M., & Akın, H. V. (2015). Push-manipulation of complex passive mobile objects using experimentally acquired motion models. Autonomous Robots, 38, 317–329.CrossRefGoogle Scholar
  31. Nammoto, T., Hashimoto, K., Kagami, S., & Kosuge, K. (2013). High speed/accuracy visual servoing based on virtual visual servoing with stereo cameras. In IEEE/RSJ international conference on intelligent robots and systems (pp. 44–49).Google Scholar
  32. Okawa, Y., & Yokoyama, K. (1992). Control of a mobile robot for the push-a-box operation. IEEE International Conference on Robotics and Automation, 1, 761–766.Google Scholar
  33. Park, J. S., & Chung, M. J. (2003). Path planning with uncalibrated stereo rig for image-based visual servoing under large pose discrepancy. IEEE Transactions on Robotics and Automation, 19(2), 250–258.CrossRefGoogle Scholar
  34. Peshkin, M. A., & Sanderson, A. C. (1988). The motion of a pushed, sliding workpiece. IEEE Journal of Robotics and Automation, 4(6), 569–598.CrossRefGoogle Scholar
  35. Peshkin, M. A., & Sanderson, A. C. (1988). Planning robotic manipulation strategies for workpieces that slide. IEEE Journal of Robotics and Automation, 4(5), 524–531.CrossRefGoogle Scholar
  36. Piepmeier, J. A., McMurray, G. V., & Lipkin, H. (2004). Uncalibrated dynamic visual servoing. IEEE Transactions on Robotics and Automation, 20(1), 143–147.CrossRefGoogle Scholar
  37. Ramirez, O.A., & Jägersand, M. (2016). Practical considerations of uncalibrated visual servoing. In 13th conference on computer and robot vision, CRV (pp. 164–169).Google Scholar
  38. Rezzoug, N., & Gorce, P. (1999). Dynamic control of pushing operations. Robotica, 17(6), 613–620.CrossRefGoogle Scholar
  39. Rus, D., Donald, B., & Jennings, J. (1995). Moving furniture with teams of autonomous robots. In IEEE/RSJ international conference on intelligent robots and systems (Vol. 1, pp. 235–242).Google Scholar
  40. Salganicoff, M., Metta, G., Oddera, A., & Sandini, G. (1993). A direct approach to vision guided manipulation. In International conference on advanced robotics.Google Scholar
  41. Shademan, A., & Jägersand, M. (2012). Robust sampling-based planning for uncalibrated visual servoing. In IEEE/RSJ international conference on intelligent robots and systems (pp. 2663–2669).Google Scholar
  42. Sudsang, A., Rothganger, F., & Ponce, J. (2002). Motion planning for disc-shaped robots pushing a polygonal object in the plane. IEEE Transactions on Robotics and Automation, 18(4), 550–562.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Instituto de Investigación en Ingeniería de Aragón, Universidad de ZaragozaSaragossaSpain
  2. 2.Université Clermont Auvergne, CNRS, SIGMA Clermont, Institut PascalClermont-FerrandFrance

Personalised recommendations