Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

View/state planning for three-dimensional object reconstruction under uncertainty

Abstract

We propose a holistic approach for three-dimensional (3D) object reconstruction with a mobile manipulator robot with an eye-in-hand sensor; considering the plan to reach the desired view/state, and the uncertainty in both observations and controls. This is one of the first methods that determines the next best view/state in the state space, following a methodology in which a set of candidate views/states is directly generated in the state space, and later only a subset of these views is kept by filtering the original set. It also determines the controls that yield a collision free trajectory to reach a state using rapidly-exploring random trees. To decrease the processing time we propose an efficient evaluation strategy based on filters, and a 3D visibility calculation with hierarchical ray tracing. The next best view/state is selected based on the expected utility, generating samples in the control space based on an error distribution according to the dynamics of the robot. This makes the method robust to positioning error, significantly reducing the collision rate and increasing the coverage, as shown in the experiments. Several experiments in simulation and with a real mobile manipulator robot with 8 degrees of freedom show that the proposed method provides an effective and fast method for a mobile manipulator to build 3D models of unknown objects. To our knowledge, this is one of the first works that demonstrates the reconstruction of complex objects with a real mobile manipulator considering uncertainty in the controls.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

References

  1. Aleotti, J., Rizzini, D. L., & Caselli, S. (2014). Perception and grasping of object parts from active robot exploration. Journal of Intelligent and Robotic Systems, 76(3–4), 401–425.

  2. Connolly, C. I. (1985). The determination of next best views. In Proceedings of the IEEE international conference on robotics and automation (Vol. 2, pp. 432–435). St. Louis, MO, USA.

  3. Espinoza, J., Sarmiento, A., Murrieta-Cid, R., & Hutchinson, S. (2011). A motion planning strategy for finding an object with a mobile manipulator in 3-d environments. Advanced Robotics, 25(13–14), 1627–1650.

  4. Foissotte, T., Stasse, O., Escande, A., Wieber, P.-B., & Kheddar, A. (2009). A two-steps next-best-view algorithm for autonomous 3d object modeling by a humanoid robot. In Proceedings of the IEEE international conference on robotics and automation (pp. 1078–1083). Kobe, Japan.

  5. Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., & Burgard, W. (2013). Octomap: An efficient probabilistic 3d mapping framework based on octrees. Autonomous Robots, 34(3), 189–206.

  6. Huang, Y., & Gupta, K. (2009). Collision-probability constrained prm for a manipulator with base pose uncertainty. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 1426–1432). St. Louis, MO, USA.

  7. Karaman, S., & Frazzoli, E. (2011). Sampling-based algorithms for optimal motion planning. International Journal of Robotics Research, 30(7), 846–894.

  8. Kavraki, L. E., Svestka, P., Latombe, J.-C., & Overmars, M. H. (1996). Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4), 566–580.

  9. Kewlani, G., Ishigami, G., & Iagnemma, K. (2009). Stochastic mobility-based path planning in uncertain environments. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 1183–1189). St. Louis, MO, USA.

  10. Khalfaoui, S., Seulin, R., Fougerolle, Y. D., & Fofi, D. (2013). An efficient method for fully automatic 3d digitization of unknown objects. Computers in Industry, 64(9), 1152–1160.

  11. Krainin, M., Curless, B., & Fox, D. (2011). Autonomous generation of complete 3d object models using next best view manipulation planning. In Proceedings of the IEEE international conference on robotics and automation (pp. 5031–5037). Shanghai, China.

  12. Kriegel, S., Rink, C., Bodenmller, T., & Suppa, M. (2013). Efficient next-best-scan planning for autonomous 3d surface reconstruction of unknown objects. Journal of Real-Time Image Processing, 10, 1–21.

  13. Kriegel, S., Rink, C., Bodenmuller, T., Narr, A., Suppa, M., & Hirzinger, G. (2012). Next-best-scan planning for autonomous 3d modeling. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 2850–2856). Vilamoura, Algarve, Portugal.

  14. Kuffner, J. J., Nishiwaki, K., Kagami, S., Inaba, M., & Inoue, H. (2003). Motion planning for humanoid robots. In Proceedings of the eleventh international symposium on robotics research (pp. 365–374). Siena, Italy.

  15. LaValle, S. M. (2006). Planning algorithms. New York: Cambridge University Press.

  16. LaValle, S. M., & Kuffner, J. J. (2001). Randomized kinodynamic planning. The International Journal of Robotics Research, 20(5), 378–400.

  17. Low, K., & Lastra, A. (2006). An adaptive hierarchical next-best-view algorithm for 3d reconstruction of indoor scenes. In Proceedigs of the 14th Pacific conference on computer graphics and applications, Taipei, Taiwan.

  18. Low, K., & Lastra, A. (2007). Predetermination of icp registration errors and its application to view planning. In Proceedings of the international conference on 3D digital imaging and modeling (pp. 73–80). Montreal, QC, Canada.

  19. Lu, C. (1989). The error propagation in robots. PhD thesis, University of Ottawa.

  20. Marchand, E., & Chaumette, F. (1999). An autonomous active vision system for complete and accurate 3d scene reconstruction. The International Journal of Robotics Research, 32(3), 171–194.

  21. Melchior, N.A., & Simmons, R. (2007). Particle rrt for path planning with uncertainty. In Proceedings of the IEEE international conference on robotics and automation (pp. 1617–1624). Roma, Italy.

  22. Oriolo, G., Vendittelli, M., Freda, L., & Troso, G. (2001). The srt method: Randomized strategies for exploration. In Proceedings of the IEEE international conference on robotics and automation (pp. 4688–4694). Seoul, Korea.

  23. Potthast, C., & Sukhatme, G. (2014). A probabilistic framework for next best view estimation in a cluttered environment. Journal of Visual Communication and Image Representation, 25(1), 148–164.

  24. Scott, W. R., Roth, G., & Rivest, J.-F. (2002). Pose error effects on range sensing.Technical report: NRC Institute for Information Technology; National Research Council Canada.

  25. Scott, W. R., Roth, G., & Rivest, J.-F. (2003). View planning for automated three-dimensional object reconstruction and inspection. ACM Computing Surveys, 35, 64–96.

  26. Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic robotics. Cambridge, MA: The MIT Press.

  27. Torabi, L. (2012). Integrated view and path planning for a fully autonomous mobile-manipulator system for 3D object modeling. PhD thesis, Simon Fraser University.

  28. Torabi, L., & Gupta, K. (2012). An autonomous 9-dof mobile-manipulator system for in situ 3d object modeling. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 4540–4541). Vilamoura, Algarve, Portugal.

  29. Torabi, L., & Gupta, K. (2012). An autonomous six-dof eye-in-hand system for in situ 3d object modeling. The International Journal of Robotics Research, 31(1), 82–100.

  30. Urmson, C., & Simmons, R. (2003). Approaches for heuristically biasing rrt growth. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 1178–1183). Las Vegas, Nevada, USA.

  31. van den Berg, J., Abbeel, P., & Goldberg, K. (2010). Lqg-mp: Optimized path planning for robots with motion uncertainty and imperfect state information. In Proceedings of the robotics: science and systems, Zaragoza, Spain.

  32. van den Berg, J., Patil, S., & Alterovitz, R. (2012). Motion planning under uncertainty using iterative local optimization in belief space. The International Journal of Robotics Research, 31(11), 1263–1278.

  33. Vasquez, J. I., Lopez-Damian, E., & Sucar, L. E. (2009). View Planning for 3D Object Reconstruction. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 4015–4020). St. Louis, MO, USA.

  34. Vasquez, J. I., Sucar, L. E., & Murrieta-Cid, R. (2013). Hierarchical ray tracing for fast volumetric next-best-view planning. In Proceedings of the tenth conference on computer and robot vision (pp. 181–187). Regina, SK, Canada.

  35. Vasquez, J. I., Sucar, L. E., & Murrieta-Cid, R. (2014) View planning for 3d object reconstruction with a mobile manipulator robot. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 4227–4233). Chicago, IL, USA.

  36. Vasquez-Gomez, J. I., Sucar, L. E., Murrieta-Cid, R., & Lopez-Damian, E. (2014). Volumetric next-best-view planning for 3d object reconstruction with positioning error. International Journal of Advanced Robotic Systems, 11(159), 1–13.

Download references

Acknowledgments

I. Vasquez-Gomez was supported by a Conacyt PhD. scholarship, he thanks Luis Valentín, Rigoberto López and David Carrillo for their help in the development of the robotic system. R. Murrieta-Cid was supported in part by Conacyt Grant No. 220796.

Author information

Correspondence to J. Irving Vasquez-Gomez.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Vasquez-Gomez, J.I., Sucar, L.E. & Murrieta-Cid, R. View/state planning for three-dimensional object reconstruction under uncertainty. Auton Robot 41, 89–109 (2017). https://doi.org/10.1007/s10514-015-9531-3

Download citation

Keywords

  • Next best view
  • Motion planning
  • Object reconstruction
  • Uncertainty