Reliable non-prehensile door opening through the combination of vision, tactile and force feedback
Whereas vision and force feedback—either at the wrist or at the joint level—for robotic manipulation purposes has received considerable attention in the literature, the benefits that tactile sensors can provide when combined with vision and force have been rarely explored.
In fact, there are some situations in which vision and force feedback cannot guarantee robust manipulation. Vision is frequently subject to calibration errors, occlusions and outliers, whereas force feedback can only provide useful information on those directions that are constrained by the environment. In tasks where the visual feedback contains errors, and the contact configuration does not constrain all the Cartesian degrees of freedom, vision and force sensors are not sufficient to guarantee a successful execution.
Many of the tasks performed in our daily life that do not require a firm grasp belong to this category. Therefore, it is important to develop strategies for robustly dealing with these situations. In this article, a new framework for combining tactile information with vision and force feedback is proposed and validated with the task of opening a sliding door. Results show how the vision-tactile-force approach outperforms vision-force and force-alone, in the sense that it allows to correct the vision errors at the same time that a suitable contact configuration is guaranteed.
KeywordsSensor-based manipulation Tactile sensing Service robotics
Unable to display preview. Download preview PDF.
- Albu-Schaffer, A., Eiberger, O., Grebenstein, M., Haddadin, S., Ott, C., Wimbock, T., Wolf, S., & Hirzinger, G. (2008). Soft robotics: From torque feedback controlled light-weight robots to intrinsically compliant systems. IEEE Robotics & Automation Magazine, 15(3), 20–30. doi: 10.1109/MRA.2008.927979. CrossRefGoogle Scholar
- Allen, P. K., Miller, A., Oh, P. Y., & Leibowitz, B. (1999). Integration of vision, force and tactile sensing for grasping. International Journal of Intelligent Machines, 4(1), 129–149. Google Scholar
- Baeten, J., Bruyninckx, H., & Schutter, J. D. (2003). Integrated vision/force robotic servoing in the task frame formalism. International Journal of Robotics Research, 22(10–11), 941–954. Google Scholar
- Broxvall, M., Gritti, M., Saffiotti, A., Seo, B., & Cho, Y. (2006). PEIS ecology: Integrating robots into smart environments. In IEEE international conference on robotics and automation (pp. 212–218). Orlando, FL. Google Scholar
- Bruyninckx, H., Schutter, J. D., Lefebvre, T., Gadeyne, K., Soetens, P., Rutgeerts, J., Slaets, P., & Meeussen, W. (2003). Building blocks for slam in autonomous compliant motion. In International symposium of robotics research, Siena, Italy. Google Scholar
- Castellanos, J., Neira, J., Strauss, O., & Tardos, J. (1996). Detecting high level features for mobile robot localization. In IEEE international conference on multisensor fusion and integration (pp. 611–618). Washington, DC, USA. Google Scholar
- Comport, A. I., Marchand, E., & Chaumette, F. (2004a). Object-based visual 3d tracking of articulated objects via kinematic sets. In CVPRW ’04: Proceedings of the 2004 conference on computer vision and pattern recognition workshop (CVPRW’04) (Vol. 1, p. 2). Washington: IEEE Computer Society. CrossRefGoogle Scholar
- Comport, A. I., Marchand, E., & Chaumette, F. (2004b). Robust model-based tracking for robot vision. In IEEE/RSJ int. conf. on intelligent robots and systems, IROS’04 (pp. 692–697). Google Scholar
- Comport, A. I., Kragic, D., Marchand, E., & Chaumette, F. (2005). Robust real-time visual tracking: Comparison, theoretical analysis and performance evaluation. In Proc. IEEE int. conference on robotics and automation (pp. 2852–2857). Barcelona, Spain. Google Scholar
- Dune, C., Marchand, E., Collewet, C., & Leroux, C. (2008). Active rough shape estimation of unknown objects. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 3622–3627). Nice, France. Google Scholar
- Edsinger, A., & Weber, J. (2004). Domo: a force sensing humanoid robot for manipulation research. In Proc. 4th IEEE/RAS international conference on humanoid robots (Vol. 1, pp. 273–291). Google Scholar
- Hosoda, K., Igarashi, K., & Asada, M. (1996). Hybrid visual servoing/force control in unknown environment. In: IEEE/RSJ international conference on intelligent robots and systems (pp. 1097–1103). Osaka, Japan. Google Scholar
- Hu, Y., Eagleson, R., & Goodale, M. (1999). Human visual servoing for reaching and grasping: The role of 3-d geometric features. In Proc. IEEE int. conference on robotics and automation (pp. 3209–3216). Detroit, Michigan, USA. Google Scholar
- Khalil, W., & Dombre, E. (2002). Modeling identification and control of robots. St Alban: Hermes Penton Science. Google Scholar
- Kragic, D., & Christensen, H. (2002). Model based techniques for robotic servoing and grasping. In IEEE/RSJ international conference on intelligent robots and systems (Vol. 1, pp. 299–304). Google Scholar
- Lasky, T., & Hsia, T. (1991). On force-tracking impedance control of robot manipulators. In IEEE international conference on robotics and automation (Vol. 1, pp. 274–280). Sacramento, California. doi: 10.1109/ROBOT.1991.131587.
- Marchand, E., & Chaumette, F. (2002). Virtual visual servoing: a framework for real-time augmented reality. In EUROGRAPHICS 2002 (Vol. 21(3), pp. 289–298). Saarebrücken, Germany. Google Scholar
- Martinet, P., & Gallice, J. (1999). Position based visual servoing using a nonlinear approach. In IEEE/RSJ int. conf. on intelligent robots and systems (Vol. 1, pp. 531–536). Kyongju, Korea. Google Scholar
- Mezouar, Y., Prats, M., & Martinet, P. (2007). External hybrid vision/force control. In Int. conference on advanced robotics (ICAR’07). Jeju, Korea. Google Scholar
- Morel, G., & Bidaud, P. (1996). A reactive external force loop approach to control manipulators in the presence of environmental disturbances. In IEEE international conference on robotics and automation (Vol. 2, pp. 1229–1234). Minneapolis, Minnesota, USA. doi: 10.1109/ROBOT.1996.506875.
- Morel, G., Malis, E., & Boudet, S. (1998). Impedance based combination of visual and force control. In IEEE international conference on robotics and automation (ICRA’98) (Vol. 2, pp. 1743–1748). Leuven, Belgium. Google Scholar
- Nelson, B., Morrow, J., & Khosla, P. (1995). Improved force control through visual servoing. In Proceedings of the American control conference (Vol. 1, pp. 380–386). Google Scholar
- Petrovskaya, A., & Ng, A. (2007). Probabilistic mobile manipulation in dynamic environments with application to opening doors. In Int. joint conf. on artificial intelligence. Hyderabad, India. Google Scholar
- Petrovskaya, A., Khatib, O., Thrun, S., & Ng, A. (2006). Bayesian estimation for autonomous object manipulation based on tactile sensors. In Proc. IEEE international conference on robotics and automation ICRA 2006 (pp. 707–714). doi: 10.1109/ROBOT.2006.1641793.
- Prats, M., Martinet, P., del Pobil, A. P., & Lee, S (2007a). Vision/force control in task-oriented grasping and manipulation. In IEEE/RSJ international conference on intelligent robots and systems (pp. 1320–1325). San Diego, USA. Google Scholar
- Prats, M., del Pobil, A. P., & Sanz, P. J. (2007b). Task-oriented grasping using hand preshapes and task frames. In Proc. of IEEE international conference on robotics and automation (pp. 1794–1799). Rome, Italy. Google Scholar
- Salisbury, J. (1980). Active stiffness control of a manipulator in Cartesian coordinates. In IEEE international conference on decision and control (pp. 95–100). Albuquerque, USA. Google Scholar
- Schmid, A. J., Gorges, N., Göger, D., & Wörn, H. (2008). Opening a door with a humanoid robot using multi-sensory tactile feedback. In IEEE international conference on robotics and automation (pp. 285–291). Pasadena, CA. Google Scholar
- Se, S., Lowe, D., & Little, J. (2001). Vision-based mobile robot localization and mapping using scale-invariant features. In IEEE international conference on robotics and automation (pp. 2051–2058). Seoul, Korea. Google Scholar
- Son, J. S., Howe, R., Wang, J., & Hager, G. (1996). Preliminary results on grasping with vision and touch. In IEEE/RSJ int. conf. on intelligent robots and systems (IROS’96) (Vol. 3, pp. 1068–1075). Osaka, Japan. Google Scholar
- Stemmer, A., Schreiber, G., Arbter, K., & Albu-Schäffer, A. (2006). Robust assembly of complex shaped planar parts using vision and force. In IEEE international conference on multisensor fusion and integration for intelligent systems (pp. 493–500). Heidelberg, Germany. Google Scholar
- Wyrobek, K., Berger, E., Van der Loos, H., & Salisbury, J. (2008). Towards a personal robotics development platform: Rationale and design of an intrinsically safe personal robot. In IEEE international conference on robotics and automation (pp. 2165–2170). doi: 10.1109/ROBOT.2008.4543527.