Making Sense of Indoor Spaces Using Semantic Web Mining and Situated Robot Perception
Intelligent Autonomous Robots deployed in human environments must have understanding of the wide range of possible semantic identities associated with the spaces they inhabit – kitchens, living rooms, bathrooms, offices, garages, etc. We believe robots should learn this information through their own exploration and situated perception in order to uncover and exploit structure in their environments – structure that may not be apparent to human engineers, or that may emerge over time during a deployment. In this work, we combine semantic web-mining and situated robot perception to develop a system capable of assigning semantic categories to regions of space. This is accomplished by looking at web-mined relationships between room categories and objects identified by a Convolutional Neural Network trained on 1000 categories. Evaluated on real-world data, we show that our system exhibits several conceptual and technical advantages over similar systems, and uncovers semantic structure in the environment overlooked by ground-truth annotators.
KeywordsRobotics Artificial intelligence Semantic web-mining Deep vision Service robots Machine learning Space classification Semantic mapping Imagenet Convolutional Neural Networks
The research leading to these results has received funding from EU FP7 grant agreement No. 600623, STRANDS, and CHIST-ERA Project ALOOF.
The authors also wish to thank Daniel Hayes for his careful and attentive contributions upon which this work depends.
- 1.Faeulhammer, T., et al.: Autonomous learning of object models on a mobile robot. IEEE RAL PP(99), 1 (2016)Google Scholar
- 2.Kilgarriff, A., Fellbaum, C.: Wordnet: An electronic lexical database (2000)Google Scholar
- 4.Krizhevsky, A., et al.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates Inc. (2012)Google Scholar
- 5.Wu, Z., Palmer, M.: Verbs semantics and lexical selection. In: ACL. ser. ACL 1994, pp. 133–138. Association for Computational Linguistics, Stroudsburg (1994)Google Scholar
- 6.Basile, V., Jebbara, S., Cabrio, E., Cimiano, P.: Populating a knowledge base with object-location relations using distributional semantics. In: Blomqvist, E., Ciancarini, P., Poggi, F., Vitali, F. (eds.) EKAW 2016. LNCS (LNAI), vol. 10024, pp. 34–50. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49004-5_3 CrossRefGoogle Scholar
- 7.Camacho-Collados, J., et al.: Nasari: a novel approach to a semantically-aware representation of items. In: Mihalcea, R., Chai, J.Y., Sarkar, A. (eds.) HLT-NAACL, pp. 567–577. The Association for Computational Linguistics (2015)Google Scholar
- 8.Ambruş, R., et al.: Meta-rooms: building and maintaining long term spatial models in a dynamic world. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1854–1861. IEEE (2014)Google Scholar
- 9.Young, J., et al.: Towards lifelong object learning by integrating situated robot perception and semantic web mining. In: Proceedings of the European Conference on Artificial Intelligence (ECAI) (2016)Google Scholar
- 10.Young, J., et al.: Semantic web-mining and deep vision for lifelong object discovery. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2017)Google Scholar
- 12.Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
- 15.Pronobis, A., Jensfelt, P.: Large-scale semantic mapping and reasoning with heterogeneous modalities. In: IEEE International Conference on Robotics and Automation (2012)Google Scholar
- 18.Hanheide, M., et al.: Dora, a robot exploiting probabilistic knowledge under uncertain sensing for efficient object search. In: Proceedings of Systems Demonstration of the 21st International Conference on Automated Planning and Scheduling (ICAPS), Freiburg, Germany (2011)Google Scholar
- 19.Potapova, E., et al.: Attention-driven object detection and segmentation of cluttered table scenes using 2.5 d symmetry. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 4946–4952. IEEE (2014)Google Scholar
- 20.He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar