Skip to main content

Advertisement

SpringerLink
Knowledge graph construction with structure and parameter learning for indoor scene design
Download PDF
Download PDF
  • Research Article
  • Open Access
  • Published: 21 March 2018

Knowledge graph construction with structure and parameter learning for indoor scene design

  • Yuan Liang1,
  • Fei Xu1,
  • Song-Hai Zhang1,
  • Yu-Kun Lai2 &
  • …
  • Taijiang Mu1 

Computational Visual Media volume 4, pages 123–137 (2018)Cite this article

  • 1800 Accesses

  • 26 Citations

  • Metrics details

Abstract

We consider the problem of learning a representation of both spatial relations and dependencies between objects for indoor scene design. We propose a novel knowledge graph framework based on the entity-relation model for representation of facts in indoor scene design, and further develop a weaklysupervised algorithm for extracting the knowledge graph representation from a small dataset using both structure and parameter learning. The proposed framework is flexible, transferable, and readable. We present a variety of computer-aided indoor scene design applications using this representation, to show the usefulness and robustness of the proposed framework.

Download to read the full article text

Working on a manuscript?

Avoid the most common mistakes and prepare your manuscript for journal editors.

Learn more

References

  1. Autodesk REVIT. Available at https://www.autodesk. com/products/revit-family/overview.

  2. SketchUp. Available at https://www.sketchup.com/.

  3. ARCHICAD. Available at http://www.graphisoft. com/archicad/.

  4. Fisher, M.; Hanrahan, P. Context-based search for 3D models. ACM Transactions on Graphics Vol. 29, No. 6, Article No. 182, 2010.

    Google Scholar 

  5. Yeh, Y.-T.; Yang, L.; Watson, M.; Goodman, N. D.; Hanrahan, P. Synthesizing open worlds with constraints using locally annealed reversible jump MCMC. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 56, 2012.

    Google Scholar 

  6. Dalton, J.; Dietz, L.; Allan, J. Entity query feature expansion using knowledge base links. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, 365–374, 2014.

    Google Scholar 

  7. Li, F.; Dong, X. L.; Langen, A.; Li, Y. Knowledge verification for long-tail verticals. Proceedings of the VLDB Endowment Vol. 10, No. 11, 1370–1381, 2017.

    Article  Google Scholar 

  8. Zhu, X.; Anguelov, D.; Ramanan, D. Capturing long-tail distributions of object subcategories. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 915–922, 2014.

    Google Scholar 

  9. Savva, M.; Chang, A. X.; Agrawala, M. Scenesuggest: Context-driven 3D scene design. arXiv preprint arXiv:1703.00061, 2017.

    Google Scholar 

  10. Yu, L. F.; Yeung, S.-K.; Terzopoulos, D. The clutterpalette: An interactive tool for detailing indoor scenes. IEEE Transactions on Visualization and Computer Graphics Vol. 22, No. 2, 1138–1148, 2016.

    Article  Google Scholar 

  11. Fisher, M.; Ritchie, D.; Savva, M.; Funkhouser, T.; Hanrahan, P. Example-based synthesis of 3D object arrangements. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 135, 2012.

  12. Xu, K.; Chen, K.; Fu, H.; Sun, W.-L.; Hu, S.- M. Sketch2Scene: Sketch-based co-retrieval and coplacement of 3D models. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 123, 2013.

  13. Chang, A. X.; Eric, M.; Savva, M.; Manning, C. D. SceneSeer: 3D scene design with natural language. arXiv preprint arXiv:1703.00050, 2017.

    Google Scholar 

  14. Chen, K.; Lai, Y.-K.; Hu, S.-M. 3D indoor scene modeling from RGB-D data: a survey. Computational Visual Media Vol. 1, No. 4, 267–278, 2015.

    Article  Google Scholar 

  15. Mihalcea, R.; Radev, D. Graph-based Natural Language Processing and Information Retrieval. Cambridge University Press, 2011.

    Book  MATH  Google Scholar 

  16. Yao, X.; Van Durme, B. Information extraction over structured data: Question answering with freebase. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 956–966, 2014.

    Google Scholar 

  17. Socher, R.; Chen, D.; Manning, C. D.; Ng, A. Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of the Advances in Neural Information Processing Systems 26, 926–934, 2013.

    Google Scholar 

  18. Marino, K.; Salakhutdinov, R.; Gupta, A. The more you know: Using knowledge graphs for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2673–2681, 2017.

    Google Scholar 

  19. Lehmann, J.; Isele, R.; Jakob, M.; Jentzsch, A.; Kontokostas, D.; Mendes, P. N.; Hellmann, S.; Morsey, M.; van Kleef, P.; Auer, S.; Bizer, C. DBpedia—A large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web Vol. 6, No. 2, 167–195, 2015.

    Google Scholar 

  20. Chang, A. X.; Savva, M.; Manning, C. D. Learning spatial knowledge for text to 3D scene generation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2028–2038, 2014.

    Google Scholar 

  21. Dong, X.; Gabrilovich, E.; Heitz, G.; Horn, W.; Lao, N.; Murphy, K.; Strohmann, T.; Sun, S.; Zhang, W. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 601–610, 2014.

    Google Scholar 

  22. Hoffart, J.; Suchanek, F. M.; Berberich, K.; Lewis-Kelham, E.; de Melo, G.; Weikum, G. YAGO2: Exploring and querying world knowledge in time, space, context, and many languages. In: Proceedings of the 20th International Conference Companion on World Wide Web, 229–232, 2011.

    Google Scholar 

  23. Chang, A. X.; Funkhouser, T.; Guibas, L.; Hanrahan, P.; Huang, Q.; Li, Z.; Savarese, S.; Savva, M.; Song, S.; Su, H.; Xiao, J.; Yi, L.; Yu, F. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015.

    Google Scholar 

  24. Miller. G. A. WordNet: A lexical database for English. Communications of the ACM Vol. 38, No. 11, 39–41, 1995.

    Article  Google Scholar 

  25. Fisher, M.; Savva, M.; Hanrahan, P. Characterizing structural relationships in scenes using graph kernels. ACM Transactions on Graphics Vol. 30, No. 4, Article No. 34, 2011.

  26. Zeng, J.; Cheung, W. K.; Liu, J. Learning topic models by belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 5, 1121–1134, 2013.

    Article  Google Scholar 

  27. Kschischang, F. R.; Frey, B. J. Iterative decoding of compound codes by probability propagation in graphical models. IEEE Journal on Selected Areas in Communications Vol. 16, No. 2, 219–230, 1998.

    Article  Google Scholar 

  28. Blei, D. M.; Ng, A. Y.; Jordan, M. I. Latent dirichlet allocation. Journal of Machine Learning Research Vol. 3, 993–1022, 2003.

    MATH  Google Scholar 

  29. Abbeel, P.; Koller, D.; Ng, A. Y. Learning factor graphs in polynomial time and sample complexity. Journal of Machine Learning Research Vol. 7, 1743–1788, 2006.

    MathSciNet  MATH  Google Scholar 

  30. Gelfand, A. E.; Hills, S. E.; Racine-Poon, A.; Smith, A. F. M. Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of the American Statistical Association Vol. 85, No. 412, 972–985, 1990.

    Article  Google Scholar 

  31. Schwarz, G. Estimating the dimension of a model. The Annals of Statistics Vol. 6, No. 2, 461–464, 1978.

    Article  MathSciNet  MATH  Google Scholar 

  32. Soleimani, H.; Miller, D. J. Parsimonious topic models with salient word discovery. IEEE Transactions on Knowledge and Data Engineering Vol. 27, No. 3, 824–837, 2015.

    Article  Google Scholar 

  33. Lepage, G. P. A new algorithm for adaptive multidimensional integration. Journal of Computational Physics Vol. 27, No. 2, 192–203, 1978.

    Article  MathSciNet  MATH  Google Scholar 

  34. The SIMS 4. Available at https://www.ea.com/games/ the-sims?isLocalized=true.

  35. Su, H.; Maji, S.; Kalogerakis, E.; Learned-Miller, E. Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, 945–953, 2015.

    Google Scholar 

  36. Ma, R.; Li, H.; Zou, C.; Liao, Z.; Tong, X.; Zhang, H. Action-driven 3D indoor scene evolution. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 173, 2016.

    Google Scholar 

  37. Merrell, P.; Schkufza, E.; Li, Z.; Agrawala, M.; Koltun, V. Interactive furniture layout using interior design guidelines. ACM Transactions on Graphics Vol. 30, No. 4, Article No. 87, 2011.

    Google Scholar 

  38. Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; Zhu, X. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence, 2181–2187, 2015.

    Google Scholar 

  39. Fisher, M.; Savva, M.; Li, Y.; Hanrahan, P.; Niener, M. Activity-centric scene synthesis for functional 3D scene modeling. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 179, 2015.

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China (No. 2017YFB1002604), the National Natural Science Foundation of China (No. 61772298), a Research Grant of Beijing Higher Institution Engineering Research Center, and the Tsinghua–Tencent Joint Laboratory for Internet Innovation Technology.

Author information

Authors and Affiliations

  1. TNList, Department of Computer Science, Tsinghua University, Beijing, 100084, China

    Yuan Liang, Fei Xu, Song-Hai Zhang & Taijiang Mu

  2. School of Computer Science and Informatics, Cardiff University, Cardiff, CF24 3AA, UK

    Yu-Kun Lai

Authors
  1. Yuan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Song-Hai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu-Kun Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Taijiang Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taijiang Mu.

Additional information

This article is published with open access at Springerlink.com

Yuan Liang is a Ph.D. candidate in the Department of Computer Science and Technology, Tsinghua University, China. His research interests include interactive multimedia analysis and geometry processing. He received his B.S. degree from the Department of Computer Science and Technology, Tsinghua University.

Fei Xu received his B.S. degree from Guangdong University of Technology. He is an interdisciplinary master student in the Department of Information Art and Design, Tsinghua University. His interests include interactive multimedia analysis, VR and AR, and human computer interaction.

Song-Hai Zhang received his Ph.D. degree from Tsinghua University, China, in 2007. He is currently an associate professor of computer science in Tsinghua University. His research interests include image and video processing as well as geometric computing.

Yu-Kun Lai received his bachelor and Ph.D. degrees in computer science from Tsinghua University, China, in 2003 and 2008, respectively. He is currently a senior lecturer at the School of Computer Science & Informatics, Cardiff University. His research interests include computer graphics, geometry processing, image processing, and computer vision. He is on the Editorial Board of The Visual Computer.

Taijiang Mu is currently a postdoctoral researcher in the Department of Computer Science and Technology, Tsinghua University, where he received his Ph.D. and B.S. degrees in 2016 and 2011, respectively. His research area is computer graphics, mainly focusing on stereoscopic image and video processing, and stereoscopic perception.

Rights and permissions

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liang, Y., Xu, F., Zhang, SH. et al. Knowledge graph construction with structure and parameter learning for indoor scene design. Comp. Visual Media 4, 123–137 (2018). https://doi.org/10.1007/s41095-018-0110-3

Download citation

  • Received: 08 January 2018

  • Accepted: 13 January 2018

  • Published: 21 March 2018

  • Issue Date: June 2018

  • DOI: https://doi.org/10.1007/s41095-018-0110-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • knowledge graph
  • scene design
  • structure learning
  • parameter learning
Download PDF

Working on a manuscript?

Avoid the most common mistakes and prepare your manuscript for journal editors.

Learn more

Advertisement

Over 10 million scientific documents at your fingertips

Switch Edition
  • Academic Edition
  • Corporate Edition
  • Home
  • Impressum
  • Legal information
  • Privacy statement
  • California Privacy Statement
  • How we use cookies
  • Manage cookies/Do not sell my data
  • Accessibility
  • FAQ
  • Contact us
  • Affiliate program

Not affiliated

Springer Nature

© 2023 Springer Nature Switzerland AG. Part of Springer Nature.