Autonomous Robots

, Volume 36, Issue 1–2, pp 153–167 | Cite as

Object search by manipulation

  • Mehmet R. Dogar
  • Michael C. Koval
  • Abhijeet Tallavajhula
  • Siddhartha S. Srinivasa


We investigate the problem of a robot searching for an object. This requires reasoning about both perception and manipulation: some objects are moved because the target may be hidden behind them, while others are moved because they block the manipulator’s access to other objects. We contribute a formulation of the object search by manipulation problem using visibility and accessibility relations between objects. We also propose a greedy algorithm and show that it is optimal under certain conditions. We propose a second algorithm which takes advantage of the structure of the visibility and accessibility relations between objects to quickly generate plans. Our empirical evaluation strongly suggests that our algorithm is optimal under all conditions. We support this claim with a partial proof. Finally, we demonstrate an implementation of both algorithms on a real robot using a real object detection system.


Robotic manipulation Manipulation planning Object search 



Special thanks to the members of the Personal Robotics Lab at Carnegie Mellon University for insightful comments and discussions. This material is based upon work supported by NSF-IIS-0916557 and NSF-EEC-0540865.


  1. Anand, A., Koppula, H. S., Joachims, T., & Saxena, A. (2013). Contextually guided semantic labeling and search for three-dimensional point clouds. International Journal of Robotics Research, 32(1), 19–34.CrossRefGoogle Scholar
  2. Baird, D. (1995). Experimentation: An introduction to measurement theory and experiment design. Englewood Cliffs: Prentice-Hall.Google Scholar
  3. Ben-Shahar, O., & Rivlin, E. (1998). Practical pushing planning for rearrangement tasks. IEEE Transactions on Robotics and Automation, 14, 549–565.CrossRefGoogle Scholar
  4. Chen, P., & Hwang, Y. (1991). Practical path planning among movable obstacles. IEEE: In IEEE International Conference on Robotics and Automation.Google Scholar
  5. de Berg, M., Cheong, O., van Kreveld, M., & Overmars, M. (2008). Computational geometry: Algorithms and applications. Berlin: Springer.Google Scholar
  6. Diankov, R., & Kuffner, J. (2008). OpenRAVE: A planning architecture for autonomous robotics. Technical Report CMU-RI-TR-08-34, Robotics Institute.Google Scholar
  7. Dogar, M., & Srinivasa, S. (2012). A planning framework for non-prehensile manipulation under clutter and uncertainty. Autonomous Robots, 33(3), 217–236.CrossRefGoogle Scholar
  8. Espinoza, J., Sarmiento, A., Murrieta-Cid, R., & Hutchinson, S. (2011). Motion planning strategy for finding an object with a mobile manipulator in three-dimensional environments. Advanced Robotics, 25, 1627–1650.CrossRefGoogle Scholar
  9. Gupta, M., & Sukhatme, G. (2012). Interactive perception in clutter. In R. S. S. The (Ed.), 2012. Workshop on Robots in Clutter: Manipulation, Perception and Navigation in Human Environments.Google Scholar
  10. Hart, P. E., Nilsson, N. J., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2), 100–107.CrossRefGoogle Scholar
  11. Hauser, K. (2012). The minimum constraint removal problem with three robotics applications. In Workshop on the Algorithmic Foundations of Robotics (WAFR).Google Scholar
  12. Hopcroft, J., & Tarjan, R. (1973). Algorithm 447: Efficient algorithms for graph manipulation. Communications of the ACM, 16(6), 372–378.CrossRefGoogle Scholar
  13. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Researc, 4, 237–285.Google Scholar
  14. Kaelbling, L., & Lozano-Perez, T. (2012). Unifying perception, estimation and action for mobile manipulation via belief space planning. IEEE: In IEEE International Conference on Robotics and Automation.Google Scholar
  15. Kollar, T., & Roy, N. (2009). Utilizing object-object and object-scene context when planning to find things. IEEE: In IEEE International Conference on Robotics and Automation.Google Scholar
  16. Ma, J., Chung, T. H., & Burdick, J. (2011). A probabilistic framework for object search with 6-dof pose estimation. International Journal of Robotics Research, 30(10), 1209–1228.CrossRefGoogle Scholar
  17. Martinez, M., llet, A., & Srinivasa, S. (2010). MOPED: A scalable and low latency object recognition and pose estimation system. In IEEE International Conference on Robotics and Automation. IEEE.Google Scholar
  18. Moizumi, K., & Cybenko, G. (2001). The traveling agent problem. Mathematics of Control, Signals, and Systems, 14(3), 213–232.CrossRefzbMATHMathSciNetGoogle Scholar
  19. Ota, J. (2009). Rearrangement planning of multiple movable objects by a mobile robot. Advanced Robotics, 23, 1–18.CrossRefGoogle Scholar
  20. Overmars, M. H., Nieuwenhuisen, D., Nieuwenhuisen, D., Frank, A., & Overmars, H. (2006). An effective framework for path planning amidst movable obstacles. In International Workshop on the Algorithmic Foundations of Robotics.Google Scholar
  21. Shubina, K., & Tsotsos, J. (2010). Visual search for an object in a 3D environment using a mobile robot. Computer Vision and Image Understanding, 114, 535–547.CrossRefGoogle Scholar
  22. Sjo, K., Lopez, D., Paul, C., Jensfelt, P., & Kragic, D. (2009). Object search and localization for an indoor mobile robot. Journal of Computing and Information Technology, 17, 67–80.Google Scholar
  23. Srinivasa, S., Berenson, D., Cakmak, M., Dogar, M., Dragan, A., Knepper, R. A., et al. (2012). Herb 2.0: Lessons learned from developing a mobile manipulator for the home. Proceedings of the IEEE, 100(8), 1–19.CrossRefGoogle Scholar
  24. Stilman, M., Schamburek, J. U., Kuffner, J., & Asfour, T. (2007). Manipulation planning among movable obstacles. In IEEE International Conference on Robotics and Automation. IEEE.Google Scholar
  25. van den Berg, J. P., Stilman, M., Kuffner, J., Lin, M. C., & Manocha, D. (2008). Path planning among movable obstacles: A probabilistically complete approach. In International Workshop on the Algorithmic Foundations of Robotics (pp. 599–614).Google Scholar
  26. van Hoof, H., Kroemer, O., Amor, H. B., & Peters, J. (2012). Maximally informative interaction learning for scene exploration. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 5152–5158). IEEE.Google Scholar
  27. Wilfong, G. (1988). Motion planning in the presence of movable obstacles. In Proceedings of the Fourth Annual Symposium on Computational Geometry (pp. 279–288).Google Scholar
  28. Wong, L. L., Kaelbling, L. P., & Lozano-Pérez, T. (2013). Manipulation-based active search for occluded objects. In IEEE International Conference on Robotics and Automation. IEEE.Google Scholar
  29. Ye, Y., & Tsotsos, J. (1995). Where to look next in 3D object search. In IEEE International Symposium on Computer Vision. IEEE.Google Scholar
  30. Ye, Y., & Tsotsos, J. (1999). Sensor planning for 3D object search. Computer Vision and Image Understanding, 73(2), 145–168.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Mehmet R. Dogar
    • 1
  • Michael C. Koval
    • 1
  • Abhijeet Tallavajhula
    • 2
  • Siddhartha S. Srinivasa
    • 1
  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA
  2. 2.Indian Institute of Technology KharagpurKharagpurIndia

Personalised recommendations