Autonomous Robots

, Volume 23, Issue 2, pp 83–96 | Cite as

Hybrid image plane/stereo (HIPS) manipulation for robotic space applications

  • Matthew L. Robinson
  • Eric T. Baumgartner
  • Kevin M. Nickels
  • Todd E. Litwin


Manipulation systems for planetary exploration operate under severe limitations due to power and weight restrictions and extreme environmental conditions. Typically such systems employ carefully calibrated stereo cameras and carefully calibrated manipulators to achieve precision on the order of ten millimeters with respect to instrument placement activities. The environmental and functional restrictions under which these systems are used limit the operational accuracy of these approaches. This paper presents a novel approach to stereo-based manipulation designed to robustly achieve high precision levels despite the aforementioned limitations. The basic principle of the approach, known as Hybrid Image Plane/Stereo (HIPS) Manipulation, is the generation of camera models through direct visual sensing of the manipulator’s end-effector. The HIPS method estimates and subsequently uses these models to position the manipulator at a target location specified in the image-planes of a stereo camera pair using stereo correlation and triangulation. In-situ estimation and adaptation of the manipulator/camera models in this method accounts for changes in the system configuration, thus ensuring consistent precision for the life of the mission. The end result is a increase in positioning precision by a factor of approximately two for a limited version of HIPS, and an order of magnitude increase in positioning precision for the full on-line version of HIPS.


Vision-based manipulation Stereo triangulation Stereo vision Space robotics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Allen, P. K., Timcenko, A., Yoshimi, B., & Michelman, P. (1992). Trajectory filtering and prediction for automated tracking and grasping of a moving object. In Proceedings of the 1992 IEEE international conference on robotics and automation (Vol. 2, pp. 1850–1856), May 1992. Google Scholar
  2. Baumgartner, E. T., Leger, P. C., Schenker, P. S., & Huntsberger, T. L. (1998). Sensor-fused navigation and manipulation from a planetary rover. In Proceedings of the SPIE symposium on sensor fusion and decentralized control in robotics systems, Boston, November 1998. Google Scholar
  3. Baumgartner, E. T., et al. (2005). The Mars Exploration Rover instrument positioning system. In Proceedings of the IEEE aerospace conference, Big Sky, MT, March 2005. Google Scholar
  4. Bonitz, R. G. (1997). Mars surveyor ’98 lander MVACS robotic arm control system design concepts. In Proceedings of the 1997 IEEE international conference on robotics and automation (pp. 2465–2470), Alburquerque, NM. Google Scholar
  5. Chen, W. Z., Korde, U., & Skaar, S. B. (1994). Position-control experiments using vision. International Journal of Robotics Research, 13(3), 199–208. CrossRefGoogle Scholar
  6. Erickson, J., et al. (2002). Mars Exploration Rover surface operations. In Proceedings of the 53rd international astronautical congress, the world space congress, Houston, TX, October 2002. paper No. IAC-02-Q.3.1.03. Google Scholar
  7. Feddema, J. T., Lee, C. S. G., & Mitchell, O. (1992). Model-based visual feedback control for a hand–eye coordinated robotic system. IEEE Computer, 25(8), 21–31. Google Scholar
  8. Feddema, J. T., Lee, C. S. G., & Mitchell, O. (1993). Visual servoing: Real-time control of robot manipulators based on visual sensory feedback. In K. Hashimoto (Ed.), Visual servoing (Vol. 7, pp. 105–138). Singapore: World Scientific. Google Scholar
  9. Gennery, D. (2001). Least-squares camera calibration including lens distortion and automatic edition of calibration points. In A. Grun & T. Huang (Eds.) Calibration and orientation of cameras in computer vision (pp. 123–136). Berlin: Springer. Google Scholar
  10. Gennery, D. B. (2006). Generalized camera calibration including fish-eye lenses. International Journal of Computer Vision, 68(3), 239–266. CrossRefGoogle Scholar
  11. Hager, G. D., Kriegman, D. J., & Morse, A. S. (1998). The block island workshop: A summary. In D. J. Kriegman, G. D. Hager, & A. S. Morse (Eds.) The confluence of vision and control (pp. 273–281). London: Springer. Google Scholar
  12. Huntsberger, T., et al. (2002). Rover autonomy for long range navigation and science data acquisition on planetary surfaces. In Proceedings of the 1992 IEEE international conference on robotics and automation (Vol. 32, pp. 3161–3168). Google Scholar
  13. Hutchinson, S., Hager, G., & Corke, P. (1996). A tutorial on visual servo control. IEEE Transactions on Robotics and Automation, 12(5), 651–670. CrossRefGoogle Scholar
  14. Kennedy, B., et al. (2001). LEMUR: limbed excursion mechanical utility rover. Autonomous Robots, 11, 201–205. MATHCrossRefGoogle Scholar
  15. Kennedy, B., et al. (2002). Limbed excursion mechanical utility rover: LEMUR II. In Proceedings of the 53rd annual international astronautical congress, Houston, TX. Google Scholar
  16. Maki, J. N., et al. (2003). Mars Exploration Rover engineering cameras 12. Journal of Geophysical Research, 108(E12), 8071. CrossRefGoogle Scholar
  17. Matijevic, J. R. (1998). The Pathfinder mission to Mars: Autonomous navigation and the Sojourner microrover. Science, 5362, 454–455. CrossRefGoogle Scholar
  18. Matijevic, J. R., Goldstein, B. G., & Welch, R. V. (2001). The Mars Exploration Rover: An in situ science mission to Mars. In Proceedings of the 31st international conference on environmental systems, Orlando, FL, July 2001. paper No. 2001-01-2136. Google Scholar
  19. Nickels, K., et al. (2006). Vision-guided self-alignment and manipulation in a walking robot. In Proceedings of the 2006 IEEE international conference on system of systems engineering, Los Angeles, CA, USA, April 2006. Google Scholar
  20. Powell, M. W., et al. (2005). Scientific visualization for the Mars Exploration Rovers. In Proceedings of the 2005 IEEE international conference on robotics and automation. Google Scholar
  21. Ruf, A., & Horaud, R. (1999). Visual servoing of robot manipulator, Part I: Projective kinematics. International Journal of Robotics Research, 11, 1101–1118. Google Scholar
  22. Savage, D., & Cook-Anderson, G. (2004). NASA selects investigations for the Mars Science Laboratory. Jet Propulsion Laboratory, December 2004, Press Release 04-398. Google Scholar
  23. Skaar, S. B., Brockman, W. H., & Hanson, R. (1987). Camera space manipulation. International Journal of Robotics Research, 6(4), 20–32. CrossRefGoogle Scholar
  24. Smith, P. (2004). The Phoenix mission to Mars. In Proceedings of the IEEE aerospace conference (Vol. 1, p. 342), March 2004. Google Scholar
  25. Squyres, S. et al. (2003). Athena Mars rover science investigation. Journal of Geophysical Research, 108(E12), 8062. CrossRefGoogle Scholar
  26. Stroupe, A., et al. (2005). Behavior-based multi-robot collaboration for autonomous construction tasks. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Edmonton, AB, Canada, August 2005. Google Scholar
  27. Tunstel, E., et al. (2002). FIDO rover field trials as rehearsal for the NASA 2003 Mars Exploration Rovers Mission. In Proceedings of 9th international symposium on robotics & applications, 5th world automation congress (pp. 320–327), Orlando, FL, June 2002. Google Scholar
  28. Urmson, C., et al. (2001). A sensor arm for robotic Antarctic meteorite search. In Proceedings of the 3rd international conference on field and service robotics, Helsinki, Finland, July 2001. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Matthew L. Robinson
    • 1
  • Eric T. Baumgartner
    • 2
  • Kevin M. Nickels
    • 3
  • Todd E. Litwin
    • 1
  1. 1.Jet Propulsion LaboratoryPasadenaUSA
  2. 2.Ohio Northern UniversityAdaUSA
  3. 3.Trinity UniversitySan AntonioUSA

Personalised recommendations