Skip to main content
SpringerLink
Log in

Hallucinating Humans for Learning Robotic Placement of Objects

  • Chapter
Experimental Robotics

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 88))

Abstract

While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences.We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects.We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cutkosky, M.R.: Robotic Grasping and Fine Manipulation. Kluwer Academic Publishers, Norwell (1985)

    Book  Google Scholar 

  2. Salganicoff, M., Ungar, L.H., Bajcsy, R.: Active learning for vision-based robot grasping. Machine Learning 23, 251–278 (1996)

    Google Scholar 

  3. Bicchi, A., Kumar, V.: Robotic grasping and contact: A review. In: ICRA, pp. 348–353 (2000)

    Google Scholar 

  4. Hsiao, K., Nangeroni, P., Huber, M., Saxena, A., Ng, A.Y.: Reactive grasping using optical proximity sensors. In: ICRA (2009)

    Google Scholar 

  5. Rodriguez, A., Mason, M., Ferry, S.: From caging to grasping. In: Proceedings of Robotics: Science and Systems (June 2011)

    Google Scholar 

  6. Saxena, A., Driemeyer, J., Kearns, J., Ng, A.: Robotic grasping of novel objects. In: Neural Information Processing Systems (2006)

    Google Scholar 

  7. Saxena, A., Driemeyer, J., Ng, A.: Robotic grasping of novel objects using vision. The International Journal of Robotics Research 27(2), 157 (2008)

    Article  Google Scholar 

  8. Saxena, A., Wong, L., Ng, A.Y.: Learning grasp strategies with partial shape information. In: AAAI (2008)

    Google Scholar 

  9. Le, Q., Kamm, D., Kara, A., Ng, A.: Learning to grasp objects with multiple contact points. In: ICRA (2010)

    Google Scholar 

  10. Jiang, Y., Moseson, S., Saxena, A.: Efficient grasping from rgbd images: Learning using a new rectangle representation. In: ICRA (2011)

    Google Scholar 

  11. Jiang, Y., Zheng, C., Lim, M., Saxena, A.: Learning to place new objects. In: ICRA (2012)

    Google Scholar 

  12. Jiang, Y., Lim, M., Zheng, C., Saxena, A.: Learning to place new objects in a scene. The International Journal of Robotics Research, IJRR (2012)

    Google Scholar 

  13. Jiang, Y., Lim, M., Saxena, A.: Learning object arrangements in 3d scenes using human context. In: ICML (2012)

    Google Scholar 

  14. Edsinger, A., Kemp, C.: Manipulation in human environments. In: Int’l. Conf. Humanoid Robots (2006)

    Google Scholar 

  15. Schuster, M., Okerman, J., Nguyen, H., Rehg, J., Kemp, C.: Perceiving clutter and surfaces for object placement in indoor environments. In: Int’l. Conf. Humanoid Robots (2010)

    Google Scholar 

  16. Lozano-Pérez, T., Jones, J., Mazer, E., O’Donnell, P.: Task-level planning of pick-and-place robot motions. Computer 22(3), 21–29 (2002)

    Article  Google Scholar 

  17. Sugie, H., Inagaki, Y., Ono, S., Aisu, H., Unemi, T.: Placing objects with multiple mobile robots-mutual help using intention inference. In: ICRA (1995)

    Google Scholar 

  18. Jain, D., Mosenlechner, L., Beetz, M.: Equipping robot control programs with first-order probabilistic reasoning capabilities. In: ICRA (2009)

    Google Scholar 

  19. Mösenlechner, L., Beetz, M.: Parameterizing Actions to have the Appropriate Effects. In: IROS (2011)

    Google Scholar 

  20. Aker, E., Erdogan, A., Erdem, E., Patoglu, V.: Housekeeping with multiple autonomous robots: Knowledge representation and automated reasoning for a tightly integrated robot control architecture. In: IROS (2011)

    Google Scholar 

  21. Torralba, A., Murphy, K., Freeman, W.T.: Using the forest to see the trees: object recognition in context. Communications of the ACM, Research Highlights 53(3), 107–114 (2010)

    Article  Google Scholar 

  22. Saxena, A., Chung, S., Ng, A.: 3-d depth reconstruction from a single still image. International Journal of Computer Vision 76(1), 53–69 (2008)

    Article  Google Scholar 

  23. Saxena, A., Sun, M., Ng, A.: Make3d: Learning 3d scene structure from a single still image. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(5), 824–840 (2009)

    Article  Google Scholar 

  24. Heitz, G., Gould, S., Saxena, A., Koller, D.: Cascaded classification models: Combining models for holistic scene understanding. In: Neural Information Processing Systems (2008)

    Google Scholar 

  25. Li, C., Kowdle, A., Saxena, A., Chen, T.: Towards holistic scene understanding: Feedback enabled cascaded classification models. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(7), 1394–1408 (2012)

    Article  Google Scholar 

  26. Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered rooms. In: ICCV (2009)

    Google Scholar 

  27. Xiong, X., Huber, D.: Using context to create semantic 3d models of indoor environments. In: BMVC (2010)

    Google Scholar 

  28. Koppula, H., Anand, A., Joachims, T., Saxena, A.: Semantic labeling of 3d point clouds for indoor scenes. In: NIPS (2011)

    Google Scholar 

  29. Anand, A., Koppula, H., Joachims, T., Saxena, A.: Contextually guided semantic labeling and search for 3d point clouds. IJRR (2012)

    Google Scholar 

  30. Fisher, M., Hanrahan, P.: Context-based search for 3d models. ACM TOG 29(6) (2010)

    Google Scholar 

  31. Fisher, M., Savva, M., Hanrahan, P.: Characterizing structural relationships in scenes using graph kernels. In: SIGGRAPH (2011)

    Google Scholar 

  32. Nabbe, B., Kumar, S., Hebert, M.: Path planning with hallucinated worlds. In: IROS (2004)

    Google Scholar 

  33. Kuffner Jr., J., LaValle, S.: Rrt-connect: An efficient approach to single-query path planning. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA 2000, vol. 2, pp. 995–1001. IEEE (2000)

    Google Scholar 

  34. Stoll, C., Gall, J., de Aguiar, E., Thrun, S., Theobalt, C.: Video-based reconstruction of animatable human characters. ACM Transactions on Graphics (Proc. SIGGRAPH ASIA) (2010)

    Google Scholar 

  35. Shin, D., Sardellitti, I., Park, Y.-L., Khatib, O., Cutkosky, M.: Design and control of a bio-inspired human-friendly robot. The International Journal of Robotics Research 29(5), 571–584 (2010)

    Article  Google Scholar 

  36. Sung, J., Ponce, C., Selman, B., Saxena, A.: Unstructured human activity detection from rgbd images. In: International Conference on Robotics and Automation, ICRA (2012)

    Google Scholar 

  37. Ly, D., Saxena, A., Lipson, H.: Co-evolutionary predictors for kinematic pose inference from rgbd images. In: Genetic and Evolutionary Computation Conference, GECCO (2012)

    Google Scholar 

  38. Demircan, E., Besier, T.F., Khatib, O.: Muscle force transmission to operational space accelerations during elite golf swings. In: Proc. of the IEEE International Conference on Robotics and Automation, St. Paul, MN, USA, pp. 1464–1469 (May 2012)

    Google Scholar 

  39. Teh, Y.W.: Dirichlet process. Encyclopedia of Machine Learning, 280–287 (2010)

    Google Scholar 

  40. Neal, R.: Markov chain sampling methods for dirichlet process mixture models. Journal of Computational and Graphical Statistics, 249–265 (2000)

    Google Scholar 

  41. Diankov, R., Kuffner, J.: Openrave: A planning architecture for autonomous robotics, Robotics Institute, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-08-34 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Cornell University, Ithaca, NY, 14853, USA

    Yun Jiang & Ashutosh Saxena

Authors
  1. Yun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ashutosh Saxena
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yun Jiang .

Editor information

Editors and Affiliations

  1. (RAMS) Institute for Systems Research, Maryland Robotics Center, University of Maryland, Room 0160, Bldg 088, Glenn L. Martin Hall, College Park, 20742, Maryland, USA

    Jaydev P. Desai

  2. , Centre for Intelligent Machines, McGill University, Room 419, 3480 University Street, Montreal, H3A 2A7, Québec, Canada

    Gregory Dudek

  3. , Department of Computer Science, Stanford University, Stanford, 94305-9010, California, USA

    Oussama Khatib

  4. , School of Engineering & Applied Science, University of Pennsylvania, 107 Towne Building, 220 South 33rd Street, Philadelphia, 19104-6391, Pennsylvania, USA

    Vijay Kumar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Jiang, Y., Saxena, A. (2013). Hallucinating Humans for Learning Robotic Placement of Objects. In: Desai, J., Dudek, G., Khatib, O., Kumar, V. (eds) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 88. Springer, Heidelberg. https://doi.org/10.1007/978-3-319-00065-7_61

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-00065-7_61

  • Publisher Name: Springer, Heidelberg

  • Print ISBN: 978-3-319-00064-0

  • Online ISBN: 978-3-319-00065-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation