Abstract
Intelligent physical skills are a fundamental element needed by robots to interact with the real world. Instead of learning from individual sources in single cases, continuous robot learning from crowdsourced mentors over long terms provides a practical path towards realizing ubiquitous robot physical intelligence. The mentors can be human drivers that teleoperate robots when their intelligence is not yet enough for acting autonomously. A large amount of sensorimotor data can be obtained constantly from a group of teleoperators, and processed by machine learning to continuously generate and improve the autonomous physical skills of robots. This paper presents a learning method that utilizes state space discretization to sustainably manage constantly collected data and synthesize autonomous robot skills. Two types of state space discretization have been proposed. Their advantages and limits are examined and compared. Simulation and physical tests of two object manipulation challenges are conducted to examine the proposed learning method. The capability of handling system uncertainty, sustainably managing high-dimensional state spaces, as well as synthesizing new skills or ones that have only been partly demonstrated are validated. The work is expected to provide a long-term and big-scale measure to produce advanced robot physical intelligence.
Similar content being viewed by others
References
Ackerman, E.: Agility robotics introduces cassie, a dynamic and talented robot delivery ostrich. IEEE Spectrum 2017, 28 (2017)
Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Autonomous Syst. 57(5), 469–483 (2009)
Benchmarks of ANN. https://github.com/erikbern/ann-benchmarks (2020). Accessed 16 Oct 2020
Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)
Chung, M.J.Y., Forbes, M., Cakmak, M., Rao, R.P.N.: Accelerating imitation learning through crowdsourcing. In: IEEE international conference on robotics and automation, pp. 4777–4784 (2014)
Deisenroth, M.P., Neumann, G., Peters, J., et al.: A survey on policy search for robotics. Found. Trends Robot. 2(1–2), 1–142 (2013)
Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)
Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world-wide web. Commun. ACM 54(4), 86–96 (2011)
Forbes, M., Chung, M., Cakmak, M., Rao, R.P.N.: Robot programming by demonstration with crowdsourced action fixes. In: The Second AAAI conference on human computation and crowdsourcing (2014)
Fu, C., Xiang, C., Wang, C., Cai, D.: Fast approximate nearest neighbor search with the navigating spreading-out graph. Proc. VLDB Endowment 12(5), 461–474 (2019)
Furukawa, N., Namiki, A., Taku, S., Ishikawa, M.: Dynamic regrasping using a high-speed multifingered hand and a high-speed vision system. In: IEEE international conference on robotics and automation (ICRA), pp. 181–187 (2006)
Geiger, D., Seedorf, S., Schulze, T., Nickerson, R.C., Schader, M.: Managing the crowd: towards a taxonomy of crowdsourcing processes. In: Americas conference on information systems (2011)
Gionis, A., Indyk, P., Motwani, R.: Similarity search in high dimensions via hashing. In: Proceedings of The 25th international conference on very large data bases, pp. 518–529 (1999)
Harwood, B., Drummond, T.: Fanng: Fast approximate nearest neighbour graphs. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 5713–5722 (2016)
Howe, J.: Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business. Crown Business, New York (2009)
Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. Adv. Neural Inf. Process. Syst. 2003, 1547–1554 (2003)
Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)
Joe, S., Kuo, F.Y.: Remark on algorithm 659: implementing sobol’s quasirandom sequence generator. ACM Trans. Math. Softw. 29(1), 49–57 (2003)
Johnson, M.K., Hasher, L.: Human learning and memory. Annu. Rev. Psychol. 38(1), 631–668 (1987)
Knight, W.: An AI-driven robot hand spent a hundred years teaching itself to rotate a cube. https://www.technologyreview.com/s/611724 (2018) Accessed 16 Oct 2020
Lee, I.S., Lau, H.Y.: Adaptive state space partitioning for reinforcement learning. Eng. Appl. Artif. Intell. 17(6), 577–588 (2004)
Li, W., Zhang, Y., Sun, Y., Wang, W., Li, M., Zhang, W., Lin, X.: Approximate nearest neighbor search on high dimensional data-experiments, analyses, and improvement. IEEE Trans. Knowl. Data Eng. 2019, 1 (2019)
Lipton, J.I., Fay, A.J., Rus, D.: Baxter’s homunculus: virtual reality spaces for teleoperation in manufacturing. IEEE Robot. Autom. Lett. 3(1), 179–186 (2018)
Little, G., Chilton, L.B., Goldman, M., Miller, R.C.: Turkit: Human computation algorithms on mechanical turk. In: The 23rd Annual ACM symposium on user interface software and technology, pp. 57–66 (2010)
Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 1 (2018)
Mandlekar, A., Zhu, Y., Garg, A., Booher, J., Spero, M., Tung, A., Gao, J., Emmons, J., Gupta, A., Orbay, E., et al.: Roboturk: A crowdsourcing platform for robotic skill learning through imitation. In: Proceedings of The 2nd Conference on Robot Learning, vol. 87, pp. 879–893 (2018)
Nguyen-Tuong, D., Peters, J.: Online kernel-based learning for task-space tracking robot control. IEEE Trans. Neural Netw. Learn. Syst. 23(9), 1417–1425 (2012)
Niederreiter, H.: Low-discrepancy and low-dispersion sequences. J. Number Theory 30(1), 51–70 (1988)
Quinn, A.J., Bederson, B.B.: Human computation: a survey and taxonomy of a growing field. In: The SIGCHI conference on human factors in computing systems, pp. 1403–1412 (2011)
Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning, vol. 1. MIT Press, Cambridge (2006)
Sorokin, A., Berenson, D., Srinivasa, S.S., Hebert, M.: People helping robots helping people: crowdsourcing for grasping novel objects. In: IEEE/RSJ international conference on intelligent robots and systems, pp. 2117–2122 (2010)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Uther, W.T., Veloso, M.M.: Tree based discretization for continuous state space reinforcement learning. In: The annual conference on innovative applications of artificial intelligence (IAAI), pp. 769–774 (1998)
Van Hoof, H., Hermans, T., Neumann, G., Peters, J.: Learning robot in-hand manipulation with tactile features. In: The 15th IEEE-RAS international conference on humanoid robots, pp. 121–127 (2015)
von Ahn, L.: Human Computation. PhD Thesis, Carnegie Mellon University, Pittsburgh, PA (2005)
Wang, C., Zhao, Y., Lin, C.Y., Tomizuka, M.: Fast planning of well conditioned trajectories for model learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1460–1465 (2014)
Wang, C., Lin, C.Y., Tomizuka, M.: Statistical learning algorithms to compensate slow visual feedback for industrial robots. J. Dyn. Syst. Measure. Control 137(3), 031011 (2015)
Wang, C., Zhao, Y., Chen, Y., Tomizuka, M.: Nonparametric statistical learning control of robot manipulators for trajectory or contour tracking. Robot. Comput.-Integr. Manuf. 35, 96–103 (2015)
Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., Abbeel, P.: Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1–8 (2018)
Zhao, L., Lawhorn, R., Wang, C., Lu, L., Ouyang, B.: Synthesis of robot hand skills powered by crowdsourced learning. In: IEEE International Conference on Mechatronics, pp. 211–216 (2019)
Zhao, L., Zhao, Y., Patil, S., Davies, D., Wang, C., Lu, L., Ouyang, B.: Robot composite learning and the nunchaku flipping challenge. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3160–3165 (2018)
Zhao, L., Lu, L., Wang, C.: Data-oriented state space discretization for crowdsourced robot learning of physical skills. ASME Lett. Dyn. Syst. Control 1, 2 (2020)
Funding
This work is part of the project “CAREER: Enhancing Robot Physical Intelligence via Crowdsourced Surrogate Learning” funded by the National Science Foundation (Award number 1944069).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, L., Lu, L. & Wang, C. Handling crowdsourced data using state space discretization for robot learning and synthesizing physical skills. Int J Intell Robot Appl 4, 390–402 (2020). https://doi.org/10.1007/s41315-020-00152-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41315-020-00152-1