Skip to main content
SpringerLink
Log in

Semantic 3D indoor scene enhancement using guide words

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We propose a novel framework for semantically enhancing a 3D indoor scene in agreement with a user-provided guide word. To do so, we make changes to furniture colors and place small objects in the scene. The relevance of specific furniture colors and small objects to each guide word is learned from a database of annotated images, taking into account both their frequency and specificity to that guide word. Enhancement suggestions are generated by optimizing a scoring function, which combines the relevance of both enhancement factors, i.e., furniture colors and small objects. During optimization, a submodular set function is adopted to ensure that a diverse set of enhancement suggestions is produced. Our experiments show that this framework can generate enhancement suggestions that are both compatible with the input guide word, and comparable to ones designed by humans.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Barinova, O., Lempitsky, V.S., Kohli, P.: On detection of multiple object instances using hough transforms. In: IEEE CVPR, pp. 2233–2240 (2010)

  2. Chang, A.X., Savva, M., Manning, C.D.: Interactive learning of spatial knowledge for text to 3D scene generation. In: Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces (2014)

  3. Chen, G., Li, G., Nie, Y., Xian, C., Mao, A.: Stylistic indoor colour design via bayesian network. Comput. Graph. 60, 34–45 (2016)

    Article  Google Scholar 

  4. Chen, K., Xu, K., Yu, Y., Wang, T.Y., Hu, S.M.: Magic decorator: automatic material suggestion for indoor digital scenes. ACM Trans. Graph. 34(6), 232:1–232:11 (2015)

    Google Scholar 

  5. Chen, X., Li, J., Li, Q., Gao, B., Zou, D., Zhao, Q.: Image2scene: transforming style of 3d room. In: Proceedings of ACM Multimedia, pp. 321–330 (2015)

  6. Gilks, W.R.: Markov chain Monte Carlo in practice. Chapman and Hall/CRC, London (1999)

  7. Gomez-Rodriguez, M., Leskovec, J., Krause, A.: Inferring networks of diffusion and influence. ACM Trans. Knowl. Discov. Data 5(4), 21:1–21:37 (2012)

    Article  Google Scholar 

  8. Hastings, W.K.: Monte carlo sampling methods using markov chains and their applications. Biometrika 57(1), 97–109 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  9. Koppula, H.S., Anand, A., Joachims, T., Saxena, A.: Semantic labeling of 3d point clouds for indoor scenes. In: Proceedings of NIPS, pp. 244–252 (2011)

  10. Liu, M., Guo, Y., Wang, J.: Indoor scene modeling from a single image using normal inference and edge features. Vis. Comput. (2017). doi:10.1007/s00371-016-1348-3

    Google Scholar 

  11. Majerowicz, L., Shamir, A., Sheffer, A., Hoos, H.H.: Filling your shelves: synthesizing diverse style-preserving artifact arrangements. IEEE Trans. Vis. Comput. Graph. 20(11), 1507–1518 (2014)

    Article  Google Scholar 

  12. Merrell, P., Schkufza, E., Li, Z., Agrawala, M., Koltun, V.: Interactive furniture layout using interior design guidelines. ACM Trans. Graph. 30(4), 87:1–87:10 (2011)

    Article  Google Scholar 

  13. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)

    Article  Google Scholar 

  14. Nemhauser, G.L., Wolsey, L.A., Fisher, M.L.: An analysis of approximations for maximizing submodular set functions-I. Math. Program. 14(1), 265–294 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  15. Schwing, A.G., Urtasun, R.: Efficient exact inference for 3d indoor scene understanding. In: ECCV, pp. 299–313 (2012)

  16. Shao, T., Xu, W., Zhou, K., Wang, J., Li, D., Guo, B.: An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM Trans. Graph. 31(6), 136:1–136:11 (2012)

    Article  Google Scholar 

  17. Xu, K., Stewart, J., Fiume, E.: Constraint-based automatic placement for scene composition. In: Proceedings of Graphics Interface, pp. 25–34 (2002).

  18. Yu, L., Yeung, S.K., Terzopoulos, D.: The clutterpalette: an interactive tool for detailing indoor scenes. IEEE Trans. Vis. Comput. Graph. 22(2), 1138–1148 (2016)

    Article  Google Scholar 

  19. Zhang, S., Han, Z., Zhang, H.: User guided 3d scene enrichment. In: Proceedings of Virtual-Reality Continuum and Its Applications in Industry, pp. 353–362 (2016)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (61373070), the National Key Technologies R&D Program of China (2015BAF23B03), and an EPSRC Travel Grant.

Author information

Authors and Affiliations

  1. School of Software, Tsinghua University, Beijing, China

    Suiyun Zhang & Hui Zhang

  2. Northwestern Polytechnical University, Xi’an, Shaanxi, China

    Zhizhong Han

  3. School of Computer Science and Informatics, Cardiff University, Cardiff, UK

    Ralph R. Martin

Authors
  1. Suiyun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhizhong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ralph R. Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Zhang.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2692 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Han, Z., Martin, R.R. et al. Semantic 3D indoor scene enhancement using guide words. Vis Comput 33, 925–935 (2017). https://doi.org/10.1007/s00371-017-1394-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-017-1394-5

Keywords

Search

Navigation