Situated Bayesian Reasoning Framework for Robots Operating in Diverse Everyday Environments

Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 10)


General-purpose robots operating in unstructured environments have the potential to benefit by leveraging abstract, commonsense knowledge for task execution. In this paper, we present an approach for automatically generating a compact semantic knowledge base, relevant to a robot’s particular operating environment, given only a small number of object labels obtained from object recognition or a robot’s task description. In order to cope with noise and non-deterministic data across our data sources, we formulate our representation as a statistical relational model represented as a Baysian Logic Network. We validate our approach in both abstract and real-world domains, demonstrating the robot’s ability to perform inference about object categories, locations and properties given a small amount of local information. Additionally, we present an approach for interactively validating the mined information with the help of a co-located user.



This work is supported in part by NSF IIS 1564080 and ONR N000141612835.


  1. 1.
    Boteanu, A., Chernova, S.: Solving and explaining analogy questions using semantic networks. In: AAAI Conference on Artificial Intelligence, pp. 1–8 (2015)Google Scholar
  2. 2.
    Buschka, P., Saffiotti, A.: Some notes on the use of hybrid maps for mobile robots. In: International Conference on Intelligent Autonomous Systems, pp. 547–556 (2004)Google Scholar
  3. 3.
    Chen, J., Liu, J.: Combining conceptnet andwordnet forword sense disambiguation. In: International Joint Conference on Natural Language Processing, pp. 686–694 (2011)Google Scholar
  4. 4.
    Deng, J., et al.: Imagenet: a large-scale hierarchical image database. In: Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)Google Scholar
  5. 5.
    Fung, R.M., Chang, K-C.: Weighing and integrating evidence for stochastic simulation in Bayesian networks. In: Conference on Uncertainty in Artificial Intelligence, pp. 209–220 (1990)Google Scholar
  6. 6.
    Gabrilovich, E., Markovitch, S.: Computing semantic relatedness using wikipedia-based explicit semantic analysis. In: International Joint Conference on Artificial Intelligence, pp. 1606–1611 (2007)Google Scholar
  7. 7.
    Jain, D., Kirchlechner, B., Beetz, M.: Extending markov logic to model probability distributions in relational domains. In: KI 2007: Advances in Artificial Intelligence, pp. 129–143. Springer (2007)Google Scholar
  8. 8.
    Jain, D., Waldherr, S., Beetz, M.: Bayesian logic networks. Technical report, Technische Universität München, pp. 1–21 (2009)Google Scholar
  9. 9.
    Kucera, H., Francis, W.N.: Computational Analysis of Present-Day American English. Brown University Press, Providence (1967)Google Scholar
  10. 10.
    Lenat, D.B.: CYC: a large-scale investment in knowledge infrastructure. Commun. ACM 38(11), 33–38 (1995)Google Scholar
  11. 11.
    Lin, T-Y., et al. Microsoft COCO: common objects in context. In: CoRR arXiv:1405.0312 (2014)
  12. 12.
    Matuszek, C., et al.: An introduction to the syntax and content of Cyc. In: AAAI Spring Symposium: Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering, pp. 44–49 (2006)Google Scholar
  13. 13.
    Miller, George A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  14. 14.
    Redmon, J., et al.: You only look once: unified, real-time object detection. In: Conference on Computer Vision and Pattern Recognition, pp. 779–788. IEEE (2016)Google Scholar
  15. 15.
    Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1–2), 107–136 (2006)CrossRefGoogle Scholar
  16. 16.
    Saxena, A., et al.: Robobrain: large-scale knowledge engine for robots. arXiv:1412.0691 (2014)
  17. 17.
    Seco, N., et al.: An intrinsic information content metric for semantic similarity in wordnet (2004)Google Scholar
  18. 18.
    Speer, R., Havasi, C.: Representing general relational knowledge in conceptnet 5. In: International Conference on Language Resources and Evaluation. Istanbul (2012)Google Scholar
  19. 19.
    Stoica, E., Hearst, M.A.: Nearly-automated metadata hierarchy creation. In: North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 117–120 (2004)Google Scholar
  20. 20.
    Tenorth, M., Beetz, M.: KnowRob-knowledge processing for autonomous personal robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4261–4266. IEEE (2009)Google Scholar
  21. 21.
    Tsatsaronis, G., Varlamis, I., Vazirgiannis, M.: Word sense disambiguation with semantic networks". In: Sojka, P., et al. (eds.) Text, Speech, and Dialogue, pp. 219–226. Springer, Berlin (2008)Google Scholar
  22. 22.
    Waibel, Markus, et al.: Roboearth. IEEE Robot. Autom. Mag. 18(2), 69–82 (2011)CrossRefGoogle Scholar
  23. 23.
    Xiao, J., et al.: SUN database: large-scale scene recognition from abbey to zoo. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3485–3492. IEEE (2010)Google Scholar
  24. 24.
    Zhu, Y., Fathi, A., Fei-Fei, L.: Reasoning about object affordances in a knowledge base representation. In: European Conference on Computer Vision, pp. 408–424. Springer (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute for Robotics and Intelligent MachinesGeorgia Institute of TechnologyAtlantaUSA
  2. 2.Electrical and Computer EngineeringThe University of Texas at AustinAustinUSA

Personalised recommendations