Abstract
3D building plays the essential role in digital city construction, city augmented reality and smart urban planning & design. Conventional building construction is accomplished by modeling software which requires significant human intervention. In this paper, a method of 3D building fabrication via Hybrid generative adversarial network (GAN) is proposed, in which a loss function with the introduction of cycle consistency loss and perceptual loss is given, a multi-properties GAN chain is built to create the building with complex architectures. Additionally, a mixed GAN network to generate the geometry and texture coordination is put forward. The discussed method can refine rough architectural models for outputting realistic buildings. Experiments show that generated 3D buildings utilizing the presented method are realistic, with geometry and textural consistency, which improves performance by 20% over traditional methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arora S, Ge R, Liang Y, Ma T, Zhang Y (2017) Generalization and equilibrium in generative adversarial nets. In: Proceedings of Thirty-fourth International Conference on Machine Learning (ICML 2017), PMLR, pp 224–232, https://doi.org/10.1145/3188745.3232194
Bao F, Schwarz M, Wonka P (2013) Procedural facade variations from a single layout. ACM Trans Graph 32(1):1–13. https://doi.org/10.1145/2421636.2421644 Article No.8
Bao J, Chen D, Wen F, Li H, Hua G (2017) Cvae-gan: fine-grained image generation through asymmetric training. In: Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV’2017), IEEE press, Venice, Italy, pp 2745–2754, https://doi.org/10.1109/ICCV.2017.299
Chaudhuri S, Kalogerakis E, Guibas L, Koltun V (2011) Probabilistic reasoning for assembly-based 3d modeling. ACM Trans Graph 30(4):1–10. https://doi.org/10.1145/2010324.1964930 Article No.35
Chen X, Li H, Fu CW, Zhang H, Cohen-Or D, Chen B (2018) 3d fabrication with universal building blocks and pyramidal shells. ACM Trans Graph 37(6):1–15. https://doi.org/10.1145/3272127.3275033 Article No.189
Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2018), IEEE press, Salt Lake City, UT, USA, pp 8789–8797, https://doi.org/10.1109/cvpr.2018.00916
Dang M, Ceylan D, Neubert B, Pauly M (2014) Safe: structure-aware facade editing. Comput Graph Forum 33(2):83–93. https://doi.org/10.1111/cgf.12313
Girdhar R, Fouhey DF, Rodriguez M, Gupta A (2016) Learning a predictable and generative vector representation for objects. In: Proceedings of European conference on computer vision (ECCV’2016), IEEE press, Amsterdam, pp 484–499, https://doi.org/10.1007/978-3-319-46466-4_29
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems 27, Curran Associates, Inc., pp 2672–2680, https://doi.org/10.5555/2969033.2969125
Hu R, Wen C, Kaick OV, Chen L, Lin D, Cohen-Or D, Huang H (2018a) Semantic object reconstruction via casual handheld scanning. ACM Trans Graph 37(6):1–12. https://doi.org/10.1145/3272127.3275024 Article No.219
Hu R, Yan Z, Zhan J, Kaick OV, Shamir A, Zhang H, Huang H (2018b) Predictive and generative neural networks for object functionality. ACM Trans Graph 37(4):1–13. https://doi.org/10.1145/3197517.3201287 Article No.151
Huang H, Kalogerakis E, Marlin B (2015) Analysis and synthesis of 3d shape families via deep-learned generative models of surfaces. Comput Graph 34(5):25–38. https://doi.org/10.1111/cgf.12694
Ilik M, Musialski P, Auzinger T, Wimmer M (2015) Layer-based procedural design of facades. Comput Graph Forum 34(2):205–216. https://doi.org/10.1111/cgf.12553 Article No.2
Kelly T, Guerrero P, Steed A, Wonka P, Mitra NJ (2018) Frankengan: guided detail synthesis for building mass-models using style-synchonized gans. Acm Trans Graph 37(6):1–14. https://doi.org/10.1145/3272127.3275065 Article No.216
Kim JC, Chung K (2020) Neural-network based adaptive context prediction model for ambient intelligence. J Ambient Intell Humaniz Comput 11:1451–1458. https://doi.org/10.1007/s12652-018-0972-3
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539
Lin J, Cohen-Or D, Zhang H, Liang C, Sharf A, Deussen O, Chen B (2011) Structure-preserving retargeting of irregular 3d architecture. ACM Trans Graph 30(6):1–10. https://doi.org/10.1145/2070781.2024217 Article No.183
Lu H, Li Y, Chen M, Kim H, Serikawa S (2018) Brain intelligence: go beyond artificial intelligence. Mobile Netw Appl 23(2):368–375. https://doi.org/10.1007/s11036-017-0932-8
Lu H, Wang D, Li Y, Li J, Li X, Kim H, Serikawa S, Humar I (2019) Conet: a cognitive ocean network. IEEE Wirel Commun 26(3):90–96. https://doi.org/10.1109/mwc.2019.1800325
Nash C, Williams CKI (2017) The shape variational autoencoder: a deep generative model of part-segmented 3d objects. Comput Graph 36(5):1–12. https://doi.org/10.1111/cgf.13240
Schwarz M, Muller P (2015) Advanced procedural modeling of architecture. ACM Trans Graph 34(4):1–12. https://doi.org/10.1145/2766956 Article No.107
Singhal S, Passricha V, Sharma P, Aggarwal RK (2019) Multi-level region-of-interest cnns for end to end speech recognition. J Ambient Intell Humaniz Comput 10:4615–4624. https://doi.org/10.1007/s12652-018-1146-z
Uemura T, Nppi JJ, Lu H, Kim H, Tachibana R, Hironaka T, Yoshida H (2019) Ensemble 3d residual network (e3d-resnet) for reduction of false-positive polyp detections in ct colonography. In: Medical Imaging 2019: Computer-Aided Diagnosis, SPIE, CA, USA, pp 276 – 282, https://doi.org/10.1117/12.2512173
Vanegas CA, Garcia-Dorado I, Aliaga DG, Benes B, Waddell P (2012) Inverse design of urban procedural models. Acm Trans Graph 31(6):1–11. https://doi.org/10.1145/2366145.2366187 Article No.168
Wang H, Schor N, Hu R, Huang H, Cohen-Or D, Huang H (2018) Global-to-local generative model for 3d shapes. ACM Trans Graph 37(6):1–10. https://doi.org/10.1145/3272127.3275025 Article No.214
Wen Z, Liu D, Liu X, Zhong L, Lv Y, Jia Y (2019) Deep learning based smart radar vision system for object recognition. J Ambient Intell Humaniz Comput 10:829–839. https://doi.org/10.1007/s12652-018-0853-9
Wu J, Zhang C, Xue T, Freeman WT, BTenenbaum J (2016) Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In: Proceedings of 30th Conference on Neural Information Processing Systems (NIPS’2016), Curran Associates, Inc., Barcelona, Spain, pp 82–90, https://doi.org/10.5555/3157096.3157106
Wu J, Wang Y, Xue T, Sun X, Freeman WT, Tenenbaum JB (2017) Marrnet: 3d shape reconstruction via 2.5d sketches. In: Proceedings of 31th Conference on Neural Information Processing Systems (NIPS’2017), Curran Associates, Inc., Long beach, CA, USA, pp 540–550, https://doi.org/10.5555/3294771.3294823
Wu W, Fan L, Liu L, Wonka P (2018) Miqp-based layout design for building interiors. Comput Graph Forum 37(2):511–521. https://doi.org/10.1111/cgf.13380
Xu X, He L, Lu H, Gao L, Ji Y (2019a) Deep adversarial metric learning for cross-modal retrieval. World Wide Web pp 657–672, https://doi.org/10.1007/s11280-018-0541-x
Xu X, Lu H, Song J, Yang Y, Shen HT, Li X (2019b) Ternary adversarial networks with self-supervision for zero-shot cross-modal retrieval. IEEE Trans Cybern pp 1–14, https://doi.org/10.1109/tcyb.2019.2928180
Zhang H, Xu K, Jiang W, Lin J, Cohen-Or D, Chen B (2013) Layered analysis of irregular facades via symmetry maximization. ACM Trans Graph 32(4):1–13. https://doi.org/10.1145/2461912.2461923 Article No.121
Zhao W, Lu H, Wang D (2018) Multisensor image fusion and enhancement in spectral total variation domain. IEEE Trans Multimed 20(4):866–879. https://doi.org/10.1109/tmm.2017.2760100
Zhu JY, Zhang R, Pathak D, Darrell T, Efros AA, Wang O, Shechtman E (2017) Toward multimodal image-to-image translation. In: Proceedings of 31th Conference on Neural Information Processing Systems (NIPS’2017), Curran Associates, Inc., Long beach, CA, USA, pp 465–476, https://doi.org/10.5555/3294771.3294816
Acknowledgements
This work is supported by the National Natural Science Foundation of China under Grant No. 61672279.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Du, Z., Shen, H., Li, X. et al. 3D building fabrication with geometry and texture coordination via hybrid GAN. J Ambient Intell Human Comput 13, 5177–5188 (2022). https://doi.org/10.1007/s12652-020-02488-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-020-02488-9