Skip to main content
SpringerLink
Log in

Toward a user-guided manipulation framework for high-DOF robots with limited communication

  • Special Issue
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

This paper presents our progress toward a user-guided manipulation framework for high degree-of-freedom robots operating in environments with limited communication. The system we propose consists of three components: (1) a user-guided perception interface that assists the user in providing task-level commands to the robot, (2) planning algorithms that autonomously generate robot motion while obeying relevant constraints, and (3) a trajectory execution and monitoring system which detects errors in execution. We report quantitative experiments performed on these three components and qualitative experiments of the entire pipeline with the PR2 robot turning a valve for the DARPA robotics challenge. We also describe how the framework was ported to the Hubo2+ robot with minimal changes which demonstrates its applicability to different types of robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. A video of the framework in operation can be seen at: http://www.youtube.com/watch?v=xRcUO2mXt3s.

References

  1. Almetwally I, Mallem M (2013) Real-time tele-operation and tele-walking of humanoid robot nao using kinect depth camera. In: IEEE International Conference on Networking, Sensing and Control, pp 463–466

  2. Argall BD, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5):469–483

    Article  Google Scholar 

  3. Arkin RC, Balch T (1997) AuRA: Principles and Practice in Review. Journal of Experimental and Theoretical Artificial Intelligence 9:175–189

    Article  Google Scholar 

  4. Berenson D, Srinivasa S, Kuffner J (2011) Task Space Regions: A Framework for Pose-Constrained Manipulation Planning. International Journal of Robotics Research 30(12):1435–1460

    Article  Google Scholar 

  5. Berenson D, Abbeel P, Goldberg K (2012) A robot path planning framework that learns from experience. In: IEEE International Conference on Robotics and Automation, pp 3671–3678, doi:10.1109/ICRA.2012.6224742

  6. Besl P, McKay ND (1992) A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2):239–256

    Article  Google Scholar 

  7. Biesiadecki J, Leger C, Maimone M (2007) Tradeoffs between directed and autonomous driving on the mars exploration rovers. In: Thrun S, Brooks R, Durrant-Whyte H (eds) Robotics Research, Springer Tracts in Advanced Robotics, vol 28. Springer, Berlin Heidelberg, pp 254–267

    Google Scholar 

  8. Burridge R, Hambuchen K (2009) Using prediction to enhance remote robot supervision across time delay. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 5628–5634

  9. Butner S, Ghodoussi M (2003) Transforming a surgical robot for human telesurgery. IEEE Transactions on Robotics and Automation 19(5):818–824

    Article  Google Scholar 

  10. Carbone M, Rizzo L (2010) Dummynet revisited. ACM SIGCOMM Computer Communication Review 40(2):12

    Article  Google Scholar 

  11. Chen TL, Ciocarlie M, Cousins S, Grice P, Hawkins K, Hsiao K, Kemp CC, King CH, Lazewatsky DA, Leeper A, Nguyen H, Paepcke A, Pantofaru C, Smart WD, Takayama L (2012) Robots for humanity: User-centered design for assistive mobile manipulation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 5434–5435

  12. Chitta S, Jones EG, Ciocarlie M, Hsiao K (2012) Perception, Planning, and Execution for Mobile Manipulation in Unstructured Environments. IEEE Robotics and Automation Magazine, Special Issue on Mobile Manipulation 19(2):58–71

    Article  Google Scholar 

  13. Cho BK, Park SS, ho Oh J (2009) Controllers for running in the humanoid robot, HUBO. In: IEEE-RAS International Conference on Humanoid Robots

  14. Craighead J, Murphy R, Burke J, Goldiez B (2007) A survey of commercial open source unmanned vehicle simulators. In: IEEE International Conference on Robotics and Automation, pp 852–857

  15. Dang H, Jun Y, Oh P, Allen P (2013) Planning complex physical tasks for disaster response with a humanoid robot. In: IEEE International Conference on Technologies for Practical Robot Applications, pp 1–6

  16. Dantam N, Stilman M (2012) Robust and efficient communication for real-time multi-process robot software. In: IEEE-RAS International Conference on Humanoid Robots

  17. DARPA Tactical Technology Office (2012) DARPA Robotics Challenge Broad Agency Announcement. [Online; accessed 16-Jan-2013]

  18. Diankov R, Kuffner J (2008) OpenRAVE: A planning architecture for autonomous robotics. Tech. Rep. CMU-RI-TR-08-34, Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania

  19. Diftler M, Mehling J, Abdallah M, Radford N, Bridgwater L, Sanders AM, Askew RS, Linn D, Yamokoski J, Permenter F, Hargrave B, Piatt R, Savely R, Ambrose R (2011) Robonaut 2 - the first humanoid robot in space. In: IEEE International Conference on Robotics and Automation, pp 2178–2183

  20. Ferrell W, Sheridan T (1967) Supervisory control of remote manipulation. IEEE Spectrum 4(10):81–88

    Article  Google Scholar 

  21. Gossow D, Leeper A, Hershberger D, Ciocarlie M (2011) Interactive Markers: 3-D User Interfaces for ROS Applications. IEEE Robotics and Automation Magazine 18(4):14–15

    Article  Google Scholar 

  22. Grey M, Dantam N, Lofaro D, Bobick A, Egerstedt M, Oh P, Stilman M (2013) Multi-process control software for HUBO2 Plus robot. In: IEEE International Conference on Technologies for Practical Robot Applications, pp 1–6

  23. Hannaford B (1989) A design framework for teleoperators with kinesthetic feedback. IEEE Transactions on Robotics and Automation 5(4):426–434

    Article  Google Scholar 

  24. Harris A, Conrad J (2011) Survey of popular robotics simulators, frameworks, and toolkits. In: IEEE Southeastcon, pp 243–249

  25. Hobbelen D, de Boer T, Wisse M (2008) System overview of bipedal robots flame and tulip: Tailor-made for limit cycle walking. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2486–2491

  26. Hornung A, Phillips M, Jones E, Bennewitz M, Likhachev M, Chitta S (2012) Navigation in three-dimensional cluttered environments for mobile manipulation. In: IEEE International Conference on Robotics and Automation, pp 423–429

  27. Hsiao K, Ciocarlie M, Brook P (2011) Bayesian Grasp Planning. ICRA 2011 Workshop on Mobile Manipulation: Integrating Perception and Manipulation

  28. Khatib O (1987) A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal of Robotics and Automation 3(1):43–53

    Article  Google Scholar 

  29. Koenig N, Howard A (2004) Design and use paradigms for Gazebo, an open-source multi-robot simulator. IEEE/RSJ International Conference on Intelligent Robots and Systems 3:2149–2154

    Google Scholar 

  30. Leeper A, Hsiao K, Ciocarlie M, Takayama L, Gossow D (2012) Strategies for human-in-the-loop robotic grasping. In: ACM/IEEE International Conference on Human-Robot Interaction, pp 1–8

  31. Lofaro DM, Oh P (2012) Humanoid Throws Inaugural Pitch at Major League Baseball Game: Challenges, Approach, Implementation and Lessons Learned. In: IEEE/RSJ International Conference on Intelligent Robots and Systems

  32. Lum M, Rosen J, King H, Friedman D, Lendvay T, Wright A, Sinanan M, Hannaford B (2009) Teleoperation in surgical robotics - network latency effects on surgical performance. In: International Conference of the IEEE Engineering in Medicine and Biology Society, pp 6860–6863

  33. Luo J, Zhang Y, Hauser K, Park H, Paldhe M, Lee C, Grey M, Stilman M, Oh J, Lee J, Kim I, Oh P (2014) Robust ladder-climbing with a humanoid robot with application to the darpa robotics challenge. In: IEEE International Conference on Robotics and Automation (to appear)

  34. Medeiros AAD (1998), A survey of control architectures for autonomous mobile robots. Journal of the Brazilian Computer Society 4

  35. Miller A, Allen P (2004) Graspit! a versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine 11(4):110–122

    Article  Google Scholar 

  36. Oestges C, Montenegro-Villacieros B, Vanhoenacker-Janvier D (2009) Modeling propagation into collapsed buildings for radio-localization-based rescue search missions. In: IEEE Antennas and Propagation Society International Symposium, pp 1–4

  37. O’Flaherty R, Vieira P, Grey M, Oh P, Bobick A, Egerstedt M, Stilman M (2013) Humanoid robot teleoperation for tasks with power tools. In: IEEE International Conference on Technologies for Practical Robot Applications, pp 1–6

  38. Phillips-Grafflin C (2013) Hubo ROS Core. https://github.com/WPI-ARC/hubo_ros_core

  39. Phillips-Grafflin C (2013) Unreliable Network Communication Toolkit. https://github.com/WPI-ARC/teleop_toolkit

  40. Pirjanian P, Huntsberger T, Trebi-Ollennu A, Aghazarian H, Das H, Joshi S, Schenker P (2000) CAMPOUT: A control architecture for multi-robot planetary outposts. Proceedings of SPIE Symposium on Sensor Fusion and Decentralized Control in Robotic Systems III 4196:221–230

    Article  Google Scholar 

  41. Pordel M, Hellstrom T (2013) Robotics architecture frameworks, available tools and further requirements. UMINF

  42. Quigley M, Gerkey B, Conley K, Faust J, Foote T, Leibs J, Berger E, Wheeler R, Ng A (2009) ROS: an open-source Robot Operating System. In: ICRA workshop on open source software

  43. Rasmussen C, Yuvraj K, Vallett R, Sohn K, Oh P (2013) Towards functional labeling of utility vehicle point clouds for humanoid driving. In: IEEE International Conference on Technologies for Practical Robot Applications, pp 1–6

  44. Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). IEEE International Conference on Robotics and Automation. Shanghai, China, pp 1–4

    Google Scholar 

  45. Rybski P, Stoeter S, Gini M, Hougen D, Papanikolopoulos N (2001) Effects of limited bandwidth communications channels on the control of multiple robots. IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE 1:369–374

    Google Scholar 

  46. Sarabia M, Ros R, Demiris Y (2011) Towards an open-source social middleware for humanoid robots. In: IEEE-RAS International Conference on Humanoid Robots, pp 670–675

  47. Sart D, Mueen A, Najjar W, Keogh E, Niennattrakul V (2010) Accelerating Dynamic Time Warping Subsequence Search with GPUs and FPGAs. In: IEEE International Conference on Data Mining, pp 1001–1006

  48. Senin P (2008) Dynamic time warping algorithm review. Information and Computer Science Department University of Hawaii at Manoa Honolulu, USA pp 1–23

  49. Skrzypczyliski P (1997) Supervision and teleoperation system for an autonomous mobile robot. IEEE/RSJ International Conference on Intelligent Robots and Systems 2:1177–1181

    Google Scholar 

  50. Stilman M (2007) Task constrained motion planning in robot joint space. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, pp 3074–3081

  51. Stulp F, Fedrizzi A, Mösenlechner L, Beetz M (2012) Learning and reasoning with action-related places for robust mobile manipulation. Artificial Intelligence Research 43(1):1–42

    MATH  MathSciNet  Google Scholar 

  52. Taylor ME, Suay HB, Chernova S (2011) Integrating reinforcement learning with human demonstrations of varying ability. In: The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2

  53. Yakey J, LaValle SM, Kavraki LE (2001) Randomized Path Planning for Linkages with Closed Kinematics Chains. IEEE Transactions on Robotics and Automation 17(6):951–959

    Article  Google Scholar 

  54. Zheng Y, Wang H, Li S, Liu Y, Orin D, Sohn K, Jun Y, Oh P (2013) Humanoid robots walking on grass, sands and rocks. In: IEEE International Conference on Technologies for Practical Robot Applications, pp 1–6

  55. Zucker M, Jun Y, Killen B, Kim TG, Oh P (2013) Continuous trajectory optimization for autonomous humanoid door opening. In: IEEE International Conference on Technologies for Practical Robot Applications, pp 1–5

Download references

Acknowledgments

This work was supported in part by the Defense Advanced Research Projects Agency (DARPA) award #N65236-12-1-1005 for the DARPA Robotics Challenge. We would also like to thank Russ Tedrake of CSAIL, MIT for allowing us to use his Hubo for testing.

Author information

Authors and Affiliations

  1. Worcester Polytechnic Institute, Worcester, MA, USA

    Calder Phillips-Grafflin, Nicholas Alunni, Halit Bener Suay, Jim Mainprice, Dmitry Berenson, Sonia Chernova & Robert W. Lindeman

  2. Drexel University, Philadelphia, PA, USA

    Daniel Lofaro & Paul Oh

Authors
  1. Calder Phillips-Grafflin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicholas Alunni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Halit Bener Suay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jim Mainprice
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel Lofaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dmitry Berenson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sonia Chernova
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Robert W. Lindeman
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Paul Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Calder Phillips-Grafflin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Phillips-Grafflin, C., Alunni, N., Suay, H.B. et al. Toward a user-guided manipulation framework for high-DOF robots with limited communication. Intel Serv Robotics 7, 121–131 (2014). https://doi.org/10.1007/s11370-014-0156-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-014-0156-8

Keywords

Search

Navigation