Skip to main content
SpringerLink
Log in

Grasping and Manipulation of Unknown Objects Based on Visual and Tactile Feedback

  • Chapter
  • First Online:
Motion and Operation Planning of Robotic Systems

Part of the book series: Mechanisms and Machine Science ((Mechan. Machine Science,volume 29))

Abstract

The sense of touch allows humans and higher animals to perform coordinated and efficient interactions within their environment. Recently, tactile sensor arrays providing high force, spatial, and temporal resolution became available for robotics, which allows us to consider new control strategies to exploit this important and valuable sensory channel for grasping and manipulation tasks. Successful dexterous manipulation strongly depends on tight feedback loops integrating proprioceptive, visual, and tactile feedback. We introduce a framework for tactile servoing that can realize specific tactile interaction patterns, for example to establish and maintain contact (grasping) or to explore and manipulate objects. We demonstrate and evaluate the capabilities of the proposed control framework in a series of preliminary experiments employing a 16 \(\times \) 16 tactile sensor array attached to a Kuka LWR arm as a large fingertip.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The sensor’s sensitivity and force range can be adjusted to the task. Here, we have chosen the characteristics to provide a linear range from 0.1–1 kPa.

  2. 2.

    The steady state errors and standard deviations are computed from a time series of 20 s duration starting after convergence (response time). All values are obtained by averaging over 20 trials.

References

  1. Jenmalm P, Johansson RS (1997) Visual and somatosensory information about object shape control manipulative fingertip forces. J Neurosci 17:4486–4499

    Google Scholar 

  2. Johansson RS, Westling G (1984) Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp Brain Res 56:550–564

    Article  Google Scholar 

  3. Steffen JF, Elbrechter C, Haschke R, Ritter H (2010) Bio-inspired motion strategies for a bimanual manipulation task. In: Proceedings of international conference on humanoid robots

    Google Scholar 

  4. Dang H, Weisz J, Allen PK (2011) Blind grasping: stable robotic grasping using tactile feedback and hand kinematics. In: Proceedings of ICRA

    Google Scholar 

  5. Ho V, Nagatani T, Noda A, Hirai Sh (2012) What can be inferred from a tactile arrayed sensor in autonomous in-hand manipulation? In: Proceedings of CASE, p 461

    Google Scholar 

  6. Schürmann C, Kõiva R, Haschke R (2011) A modular high-speed tactile sensor for human manipulation research. In: World haptics conference

    Google Scholar 

  7. Li Q, Haschke R, Bolder B, Ritter H (2012) Grasp point optimization by online exploration of unknown object surface. In: Proceedings of international conference on humanoid robots

    Google Scholar 

  8. Pezzementi Z, Plaku E, Reyda C, Hager GD (2011) Tactile-object recognition from appearance information. Trans Robot 27(3):473–487

    Article  Google Scholar 

  9. Hart S, Sen S, Ou S, Grupen R (2009) The control basis API—a layered software architecture for autonomous robot learning. In: 2009 workshop on software development and integration in robotics at ICRA

    Google Scholar 

  10. Huber M (2000) A hybrid architecture for adaptive robot control. PhD thesis, University of Massachusetts

    Google Scholar 

  11. León B, Ulbrich S, Diankov R, Puche G, Przybylski M, Morales A, Asfour T, Moisio S, Bohg J, Kuffner J (2010) OpenGRASP: a toolkit for robot grasping simulation. In: Proceedings of SIMPAR. Springer, Darmstadt, pp 109–120

    Google Scholar 

  12. Gienger M, Toussaint M, Goerick C (2010) Whole-body motion planning—building blocks for intelligent systems. In: Harada K, Yoshida E, Yokoi K (eds) Motion planning for humanoid robots. Springer, London, pp 67–98

    Chapter  Google Scholar 

  13. Gienger M, Janßen H, Goerick C (2006) Exploiting task intervals for whole body robot control. In: Proceedings of IROS, pp 2484–2490

    Google Scholar 

  14. Platt R, Fagg AH, Grupen RA (2010) Null-space grasp control: theory and experiments. IEEE Trans Robot 26(2):282–295

    Article  Google Scholar 

  15. Liegeois A (1977) Automatic supervisory control of configuration and behavior of multibody mechanisms. IEEE Trans Syst, Man Cybern 7(12):861–871

    Google Scholar 

  16. Sugiura H, Gienger M, Jannsen H, Goerick C (2010) Reactive self collision avoidance with dynamic task prioritization for humanoid robots. Int J Humanoid Robot 7(01):31–54

    Article  Google Scholar 

  17. Behnisch M, Haschke R, Ritter H, Gienger M (2011) Deformable trees—exploiting local obstacle avoidance. In: Proceedings of international conference on humanoid robots

    Google Scholar 

  18. Catalano MG, Grioli G, Farnioli E, Serio A, Piazza A, Bicchi C (2014) Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Int J Robot Res 33(5):768–782

    Article  Google Scholar 

  19. Odhner LU, Ma RR, Dollar AM (2013) Open-loop precision grasping with underactuated hands inspired by a human manipulation strategy. IEEE Trans Autom Sci Eng 10(3):625–633

    Article  Google Scholar 

  20. Cutkosky M, Howe RD (1990) Human grasp choice and robotic grasp analysis. In: Venkataraman ST, Iberall T (eds) Dextrous robot hands. Springer, New York

    Google Scholar 

  21. Ückermann A, Haschke R, Ritter H (2013) Realtime 3D segmentation for human-robot interaction. In: Proceedings of IROS

    Google Scholar 

  22. Ückermann A, Haschke R, Ritter H (2012) Real-time 3D segmentation of cluttered scenes for robot grasping. In: Proceedings of international conference on humanoid robots. Video: www.youtube.com/watch?v=Z2SwggQTBC8

  23. Schöpfer M, Schmidt F, Pardowitz M, Ritter H (2010) Open source real-time control software for the Kuka light weight robot. In: Proceedings of WCICA, pp 444–449

    Google Scholar 

  24. Dahiya RS, Metta G, Valle M, Sandini G (2010) Tactile sensing: from humans to humanoids. IEEE Trans Robot 26(1):1–20

    Article  Google Scholar 

  25. Wettels N, Santos VJ, Johansson RS, Loeb GE (2008) Biomimetic tactile sensor array. Adv Robot 22(8):829–849

    Article  Google Scholar 

  26. Fishel JA, Loeb GE (2012) Sensing tactile microvibrations with the BioTac— comparison with human sensitivity. In: International conference on biomedical robotics and biomechatronics (BioRob), pp 1122–1127

    Google Scholar 

  27. Xu D, Loeb GE, Fishel JA (2013) Tactile identification of objects using Bayesian exploration. In: Proceedings of ICRA, pp 3056–3061

    Google Scholar 

  28. Schürmann C, Schöpfer M, Haschke R, Ritter H (2012) A high-speed tactile sensor for slip detection. In: Prassler E, Burgard W, Handmann U, Haschke R, Hägele M, Lawitzky G, Nebel B, Nowak W, Plöger P, Reiser U, Zöllner M (eds) Towards service robots for everyday environments, vol 76. Springer, New York, pp 403–415. Video: www.youtube.com/watch?v=mSq8e4PU90s

  29. Shadow Robot Company. Shadow Dexterous Hand (2013). http://www.shadowrobot.com/products/dexterous-hand

  30. Kõiva R, Zenker M, Schürmann C, Haschke R, Ritter H (2013). A highly sensitive 3D-shaped tactile sensor. In: International conference on advanced intelligent mechatronics (AIM)

    Google Scholar 

  31. Büscher G, Kõiva R, Schürmann C, Haschke R, Ritter H (2012) Tactile dataglove with fabric-based sensors. In: Proceedings of international conference on humanoid robots

    Google Scholar 

  32. Maycock J, Essig K, Haschke R, Schack T, Ritter H (2011) Towards an understanding of grasping using a multi-sensing approach. In: Proceedings of ICRA, pp 1–8

    Google Scholar 

  33. Roa M, Kõiva R, Castellini C (2012) Experimental evaluation of human grasps using a sensorized object. In: International conference on biomedical robotics and biomechatronics (BioRob)

    Google Scholar 

  34. Chen N, Zhang H, Rink R (1995) Edge tracking using tactile servo. In: Proceedings of IROS, vol 2. August 1995, pp 84–89

    Google Scholar 

  35. Martinez-Hernandez U, Lepora NF, Barron-Gonzalez H, Dodd TJ, Prescott TJ (2012) Towards contour following exploration based on tactile sensing with the iCub fingertip. In: Herrmann G, Studley M, Pearson M, Conn A, Melhuish C, Witkowski M, Kim J-H, Vadakkepat P (eds) Advances in autonomous robotics. Lecture notes in computer science, vol 7429. Springer, Berlin, pp 459–460

    Google Scholar 

  36. Schmitz A, Maiolino P, Maggiali M, Natale L, Cannata G, Metta G (2011) Methods and technologies for the implementation of large-scale robot tactile sensors. Trans Robot 27(3):389–400

    Article  Google Scholar 

  37. Wettels N, Loeb GE (2011) Haptic feature extraction from a biomimetic tactile sensor: force, contact location and curvature. In: Proceedings of ROBIO, pp 2471–2478

    Google Scholar 

  38. Suzuki K, Horiba I, Sugie N (2003) Linear-time connected-component labeling based on sequential local operations. Comput Vis Image Underst 89(1):1–23

    Article  MATH  Google Scholar 

  39. Duda RO, Hart PE (1972) Use of the Hough transformation to detect lines and curves in pictures. Commun ACM 15(1):11–15

    Article  MATH  Google Scholar 

  40. Li Q (2013) A control framework for tactile sensing. Video: https://www.youtube.com/watch?v=TcWipks3qJ0

  41. Schöpfer M, Ritter H, Heidemann G (2007) Acquisition and application of a tactile database. In: Proceedings of ICRA, pp 1517–1522

    Google Scholar 

  42. Meier M, Schöpfer M, Haschke R, Ritter H (2011) A probabilistic approach to tactile shape reconstruction. Trans Robot 27(3):630–635

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Interaction Technology Excellence Cluster (CITEC), Bielefeld University, Inspiration 1, 33619, Bielefeld, Germany

    Robert Haschke

Authors
  1. Robert Haschke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Haschke .

Editor information

Editors and Affiliations

  1. University of Cassino, Cassino, Frosinone, Italy

    Giuseppe Carbone

  2. Engineering School, University of Huelva, La Rábida, Huelva, Spain

    Fernando Gomez-Bravo

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Haschke, R. (2015). Grasping and Manipulation of Unknown Objects Based on Visual and Tactile Feedback. In: Carbone, G., Gomez-Bravo, F. (eds) Motion and Operation Planning of Robotic Systems. Mechanisms and Machine Science, vol 29. Springer, Cham. https://doi.org/10.1007/978-3-319-14705-5_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14705-5_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14704-8

  • Online ISBN: 978-3-319-14705-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation