Autonomous Robots

, Volume 40, Issue 7, pp 1123–1137 | Cite as

Human–robot planning and learning for marine data collection



We propose an integrated learning and planning framework that leverages knowledge from a human user along with prior information about the environment to generate trajectories for scientific data collection in marine environments. The proposed framework combines principles from probabilistic planning with nonparametric uncertainty modeling to refine trajectories for execution by autonomous vehicles. These trajectories are informed by a utility function learned from the human operator’s implicit preferences using a modified coactive learning algorithm. The resulting techniques allow for user-specified trajectories to be modified for reduced risk of collision and increased reliability. We test our approach in two marine monitoring domains and show that the proposed framework mimics human-planned trajectories while also reducing the risk of operation. This work provides insight into the tools necessary for combining human input with vehicle navigation to provide persistent autonomy.


Marine robotics Adaptive planning Informative path planning Coactive learning 



The authors thank Robby Goetschalckx, Alan Fern, and Prasad Tadepalli from Oregon State University for their insightful comments. Further thanks go to Gaurav Sukhatme from the University of Southern California for providing access to the Ecomapper vehicle in the field experiments. This work was supported in part by the following Grants: NSF IIS-1317815.


  1. Abbeel, P., Coates, A., & Ng, A. Y. (2010). Autonomous helicopter aerobatics through apprenticeship learning. The International Journal of Robotics Research, 29(13), 1608–1639.CrossRefGoogle Scholar
  2. Abbeel, P., & Ng, A. Y. (2004). Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the International Conference on Machine Learning, pp. 1–8.Google Scholar
  3. Alvarez, M., & Lawrence, N. (2011). Computationally efficient convolved multiple output gaussian processes. The Journal of Machine Learning Research, 12, 1459–1500.MathSciNetMATHGoogle Scholar
  4. Applegate, D. L., Bixby, R. E., Chvatal, V., & Cook, W. J. (2006). The traveling salesman problem: A computational study. Princeton: Princeton University Press. doi: 10.1109/ICICTA.2009.96.MATHGoogle Scholar
  5. Bryson, M., Reid, A., Ramos, F., & Sukkarieh, S. (2010). Airborne vision-based mapping and classification of large farmland environments. Journal of Field Robotics, 27(5), 632–655.CrossRefGoogle Scholar
  6. Cao, N., Low, K. H., & Dolan, J. M. (2013). Multi-robot informative path planning for active sensing of environmental phenomena: A tale of two algorithms. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems, pp. 7–14.Google Scholar
  7. Das, J., Maughan, T., McCann, M., Godin, M., O’Reilly, T., Messie, M., Bahr, F., Gomes, K., Py, F., Bellingham, J. G., Sukhatme, G. S., & Rajan, K. (2011). Towards mixed-initiative, multi-robot field experiments: Design, deployment, and lessons learned. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3132–3139.Google Scholar
  8. Das, J., Rajany, K., Frolovy, S., Pyy, F., Ryany, J., Caron, D. A., & Sukhatme, G. S. (2010). Towards marine bloom trajectory prediction for AUV mission planning. In IEEE International Conference on Robotics and Automation, pp. 4784–4790.Google Scholar
  9. Ellison, R., & Cook, M. (2009). Cost-effective automated water quality monitoring systems providing high-resolution data in near real-time. In: World Environmental and Water Resources Congress, pp. 1–10.Google Scholar
  10. Goetschalckx, R., Fern, A., & Tadepalli, P. (2014). Coactive learning for locally optimal problem solving. In AAAI Conference on Artificial Intelligence, pp. 1824–1830.Google Scholar
  11. Hoang, T. N., Low, K. H., Jaillet, P., & Kankanhalli, M. (2014). Nonmyopic epsilon-Bayes-optimal active learning of Gaussian processes.Google Scholar
  12. Hollinger, G. A., Englot, B., Hover, F. S., Mitra, U., & Sukhatme, G. S. (2012). Active planning for underwater inspection and the benefit of adaptivity. The International Journal of Robotics Research, 32(1), 3–18.CrossRefGoogle Scholar
  13. Hollinger, G. A., Pereira, A. A., & Sukhatme, G. S. (2013). Learning uncertainty models for reliable operation of autonomous underwater vehicles. In IEEE International Conference on Robotics and Automation, pp. 5593–5599.Google Scholar
  14. Hollinger, G. A., & Sukhatme, G. S. (2014). Sampling-based robotic information gathering algorithms. The International Journal of Robotics Research, 33(9), 1271–1287.CrossRefGoogle Scholar
  15. Hollinger, G. A., & Sukhatme, G. S. (2014). Trajectory learning for human-robot scientific data collection. In IEEE International Conference on Robotics and Automation, pp. 6600–6605.Google Scholar
  16. Jain, A., Wojcik, B., Joachims, T., & Saxena, A. (2013). Learning trajectory preferences for manipulators via iterative improvement. In Advances in Neural Information Processing Systems, pp. 575–583.Google Scholar
  17. Karaman, S., & Frazzoli, E. (2011). Sampling-based algorithms for optimal motion planning. The International Journal of Robotics Research, 30, 846–894.CrossRefMATHGoogle Scholar
  18. Kim, Y. H., Shell, D. A., Ho, C., & Saripalli, S. (2013). Spatial Interpolation for robotic sampling: Uncertainty with two models of variance. International Symposium on Experimental Robotics, 88, 759–774.CrossRefGoogle Scholar
  19. Krause, A., & Guestrin, C. (2011). Submodularity and its applications in optimized information gathering. ACM Transactions on Intelligent Systems and Technology, 2(4), 32:1–32:20.CrossRefGoogle Scholar
  20. Latombe, J. C. (1991). Robot motion planning. Boston, ma: Kluwer Academic Publishers.CrossRefMATHGoogle Scholar
  21. LaValle, S. M. (2006). Planning algorithms. Cambridge: Cambridge University Press.CrossRefMATHGoogle Scholar
  22. Lermusiaux, P. F. (2006). Uncertainty estimation and prediction for interdisciplinary ocean dynamics. Journal of Computational Physics, 217(1), 176–199.MathSciNetCrossRefMATHGoogle Scholar
  23. Li, P., Chao, Y., Vu, Q., Li, Z., Farrara, J., Zhang, H., & Wang, X. (2006). OurOcean—an integrated solution to Ocean monitoring and forecasting. In: OCEANS, pp. 1–6.Google Scholar
  24. Low, K. H., Dolan, J. M., & Khosla, P. K. (2009). Information-theoretic approach to efficient adaptive path planning for mobile robotic environmental sensing. In Proceedings of the International Conference on Automated Planning and Scheduling, pp. 233–240.Google Scholar
  25. Mora, A., Ho, C., & Saripalli, S. (2013). Analysis of adaptive sampling techniques for underwater vehicles. Autonomous Robots, 35(2), 111–122.CrossRefGoogle Scholar
  26. Nooner, S. L., & Chadwick, W. W. (2009). Volcanic inflation measured in the caldera of axial seamount: Implications for magma supply and future eruptions. Geochemistry, Geophysics, Geosystems, 10(2), 1–14.CrossRefGoogle Scholar
  27. Pereira, A. A., Binney, J., Hollinger, G. A., & Sukhatme, G. S. (2013). Risk-aware path planning for autonomous underwater vehicles using predictive ocean models. Journal of Field Robotics, 30(5), 741–762.CrossRefGoogle Scholar
  28. Raman, K., Joachims, T., Shivaswamy, P., & Schnabel, T. (2013). Stable coactive learning via perturbation. Proceedings of the International Conference on Machine Learning, 28, 837–845.Google Scholar
  29. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge: The MIT Press.MATHGoogle Scholar
  30. Ratliff, N. D. (2009). Learning to search: Structured prediction techniques for imitation learning. Ph.D. Thesis, Carnegie Mellon University.Google Scholar
  31. Reif, J. H. (1979). Complexity of the mover’s problem and generalizations. In Symposium on Foundations of Computer Science, pp. 421–427.Google Scholar
  32. Sattar, J., & Dudek, G. (2011). Towards quantitative modeling of task confirmations in human-robot dialog. In IEEE International Conference on Robotics and Automation, pp. 1957–1963.Google Scholar
  33. Shchepetkin, A. F., & McWilliams, J. C. (2005). The regional oceanic modeling system (ROMS): A split-explicit, free-surface, topography-following-coordinate oceanic model. Ocean Modelling, 9(4), 347–404.CrossRefGoogle Scholar
  34. Shivaswamy, P., & Joachims, T. (2012). Online structured prediction via coactive learning. Proceedings of the International Conference on Machine Learning pp. 1–8.Google Scholar
  35. Silver, D., Bagnell, J. A., & Stentz, A. (2010). Learning from demonstration for autonomous navigation in complex unstructured terrain. The International Journal of Robotics Research, 29(12), 1565–1592.CrossRefGoogle Scholar
  36. Singh, A., Krause, A., Guestrin, C., Kaiser, W. J., & Batalin, M. A. (2007). Efficient planning of informative paths for multiple robots. International Joint Conference on Artificial Intelligence, 7, 2204–2211.Google Scholar
  37. Singh, A., Krause, A., & Kaiser, W. J. (2009). Nonmyopic adaptive informative path planning for multiple robots. International Joint Conference on Artificial Intelligence pp. 1843–1850.Google Scholar
  38. Smith, R., Das, J., Heidarsson, H., Pereira, A., Arrichiello, F., Cetnic, I., et al. (2010). USC CINAPS builds bridges: Observing and monitoring the Southern California bight. IEEE Robotics & Automation Magazine, 17(1), 20–30.CrossRefGoogle Scholar
  39. Somers, T., & Hollinger, G. A. (2014). Coactive learning with a human expert for robotic monitoring. Workshop on Robotic Monitoring at Robotics Science and Systems, 2014, 1–2.Google Scholar
  40. Somers, T., & Hollinger, G. A. (2015). Coactive learning with a human expert for robotic information gathering. In IEEE International Conference on Robotics and Automation, pp. 559–564.Google Scholar
  41. Thompson, D. R., Chien, S., Yi Chao, Li, P., Cahill, B., Levin, J., Schofield, O., Balasuriya, A., Petillo, S., Arrott, M., & Meisinger, M. (2010). Spatiotemporal path planning in strong, dynamic, uncertain currents. In IEEE International Conference on Robotics and Automation, pp. 4778–4783.Google Scholar
  42. Willmott, C. J., Ackleson, S. G., Davis, R. E., Feddema, J. J., Klink, K. M., Legates, D. R., et al. (1985). Statistics for the evaluation and comparison of models. Journal of Geophysical Research, 90(C5), 8995–9005.CrossRefGoogle Scholar
  43. Yamamoto, J. K. (2000). An alternative measure of the reliability of ordinary Kriging estimates. Mathematical Geology, 32(4), 489–509.MathSciNetCrossRefMATHGoogle Scholar
  44. Yamamoto, J. K., & Monteiro, M. (2008). Properties and applications of the interpolation variance associated with ordinary kriging estimates. In: International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, pp. 70–77.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Robotics Program, School of Mechanical, Industrial & Manufacturing Engineering, College of EngineeringOregon State UniversityCorvallisUSA

Personalised recommendations