The Visual Computer

, Volume 32, Issue 2, pp 151–166 | Cite as

Procedure-based component and architecture modeling from a single image

  • Fei HouEmail author
  • Hong Qin
  • Yue Qi
Original Article


This paper advocates a new component-aware framework to reconstruct 3D architecture from a single image. Different from existing work, our motivation is to obtain a complete set of semantically correct 3D architectural components, which enables part reusability towards rapid model reproduction and facilitates model variation. The core of our system is a novel algorithm to adaptively segment repeated curved stripes (e.g., roof tiles, building floors) into individual elements, based on which 3D dimensions as well as architectural components are derived from a single image. Specially for Chinese architectures, we further devise an interactive method to identify outer columns based on user-specified inner columns. Finally, 3D components are generated and shape rules are derived, from which the buildings and their variants are constructed. Our new component-aware framework minimizes the use of data resource (i.e., one single image) and emphasizes component utility during rapid 3D architecture reproduction by advocating a component-aware approach.


Architectural modeling Image-based modeling Procedural modeling 



The authors would like to thank the anonymous reviewers for their constructive comments that help improve this paper. This work is supported in part by National Natural Science Foundation of China (Grant No. 61300068, 61190120, 61190121, 61190125, 61300067, 61272348, 61202235, 61073078), National Science Foundation of USA (Grant No. IIS-0949467, IIS-1047715, and IIS-1049448), the National High-tech R&D Program of China (Grant No. 012AA011503), Postdoctoral Science Foundation of China (Grant No. 2013M530512), Ph.D. Program Foundation of Ministry of Education of China (Grant No. 20111102110018), National Key Technology R&D Program of China (Grant No. 2014BAK18B01).

Supplementary material

Supplementary material 1 (mp4 27647 KB)


  1. 1.
    Zheng, Q., Sharf, A., Wan, G., Li, Y., Mitra, N.J., Cohen-Or, D., Chen, B.: Non-local scan consolidation for 3d urban scenes. ACM Trans. Graph. 29, 94:1–94:9 (2010)Google Scholar
  2. 2.
    Poullis, C., You, S.: Photorealistic large-scale urban city model reconstruction. IEEE Trans. Vis. Comput. Graph. 15, 654–669 (2009)CrossRefGoogle Scholar
  3. 3.
    Xiao, J., Fang, T., Zhao, P., Lhuillier, M., Quan, L.: Image-based street-side city modeling. ACM Trans. Graph. 28(5), 1–12 (2009)CrossRefGoogle Scholar
  4. 4.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Towards internet-scale multi-view stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1434–1441 (2010)Google Scholar
  5. 5.
    Musialski, P., Wimmer, M., Wonka, P.: Interactive coherence-based façade modeling, Computer Graphics Forum (Proceedings of EUROGRAPHICS 2012) 31(2), 661–670 (2012)Google Scholar
  6. 6.
    AlHalawani, S., Yang, Y.-L., Liu, H., Mitra, N.J.: Interactive facades analysis and synthesis of semi-regular facades. Comput. Graph. Forum 32(2), 215–224 (2013)CrossRefGoogle Scholar
  7. 7.
    Musialski, P., Wonka, P., Aliaga, D.G., Wimmer, M., van Gool, L., Purgathofer, W.: A survey of urban reconstruction. In: EUROGRAPHICS 2012 State of the Art Reports, Eurographics Association, pp. 146–177 (2012)Google Scholar
  8. 8.
    Wonka, P., Wimmer, M., Sillion, F., Ribarsky, W.: Instant architecture. ACM Trans. Graph. 22(3), 669–677 (2003)CrossRefGoogle Scholar
  9. 9.
    Müller, P., Wonka, P., Haegler, S., Ulmer, A., Van Gool, L.: Procedural modeling of buildings. ACM Trans. Graph. 25(3), 614–623 (2006)CrossRefGoogle Scholar
  10. 10.
    Lipp, M., Wonka, P., Wimmer, M.: Interactive visual editing of grammars for procedural architecture. ACM Trans. Graph. 27(3), 1–10 (2008)CrossRefGoogle Scholar
  11. 11.
    Talton, J.O., Lou, Y., Lesser, S., Duke, J., Měch, R., Koltun, V.: Metropolis procedural modeling. ACM Trans. Graph. 30(2), 11:1–11:14 (2011)Google Scholar
  12. 12.
    Hou, F., Qi, Y., Qin, H.: Drawing-based procedural modeling of chinese architectures. IEEE Trans. Vis. Comput. Graph. 18(1), 30–42 (2012)CrossRefGoogle Scholar
  13. 13.
    Kelly, T., Wonka, P.: Interactive architectural modeling with procedural extrusions. ACM Trans. Graph. 30, 14:1–14:15 (2011)Google Scholar
  14. 14.
    Merrell, P., Manocha, D.: Model synthesis: a general procedural modeling algorithm. IEEE Trans. Vis. Comput. Graph. 17(6), 715–728 (2011)CrossRefGoogle Scholar
  15. 15.
    Bokeloh, M., Wand, M., Seidel, H.-P.: A connection between partial symmetry and inverse procedural modeling. ACM Trans. Graph. 29, 104:1–104:10 (2010)Google Scholar
  16. 16.
    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV, pp. 1–8 (2007)Google Scholar
  17. 17.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S., Szeliski, R.: Building rome in a day. In: IEEE International Conference on Computer Vision, pp. 72–79 (2009)Google Scholar
  18. 18.
    Pollefeys, M., Nistér, D., Frahm, J.M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3d reconstruction from video. Int. J. Comput. Vis. 78, 143–167 (2008)CrossRefGoogle Scholar
  19. 19.
    Sinha, S.N., Steedly, D., Szeliski, R., Agrawala, M., Pollefeys, M.: Interactive 3d architectural modeling from unordered photo collections. ACM Trans. Graph. 27(5), 159:1–159:10 (2008)Google Scholar
  20. 20.
    Xiao, J., Fang, T., Tan, P., Zhao, P., Ofek, E., Quan, L.: Image-based façade modeling. ACM Trans. Graph. 27(5), 161:1–161:10 (2008)Google Scholar
  21. 21.
    Zhao, P., Fang, T., Xiao, J., Zhang, H., Zhao, Q., Quan, L.: Rectilinear parsing of architecture in urban environment. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 342–349 (2010)Google Scholar
  22. 22.
    Vanegas, C., Aliaga, D., Beneš, B.: Building reconstruction using manhattan-world grammars. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 358–365 (2010)Google Scholar
  23. 23.
    Ceylan, D., Mitra, N.J., Zheng, Y., Pauly, M.: Coupled structure-from-motion and 3d symmetry detection for urban facades. ACM Trans. Graph. 33(1), 2:1–2:15 (2014)Google Scholar
  24. 24.
    Hong, W., Yang, A.Y., Huang, K., Ma, Y.: On symmetry and multiple-view geometry: structure, pose, and calibration from a single image. Int. J. Comput. Vis. 60, 241–265 (2004)CrossRefGoogle Scholar
  25. 25.
    Liebowitz, D., Criminisi, A., Zisserman, A.: Creating architectural models from images. In: Annual Conference of the European Association for Computer Graphics (Eurographics), vol. 18, pp. 39–50 (1999)Google Scholar
  26. 26.
    Wilczkowiak, M., Sturm, P., Boyer, E.: Using geometric constraints through parallelepipeds for calibration and 3d modeling. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 194–207 (2005)CrossRefGoogle Scholar
  27. 27.
    Jiang, N., Tan, P., Cheong, L.-F.: Symmetric architecture modeling with a single image. ACM Trans. Graph. 28(5), 1–8 (2009)CrossRefGoogle Scholar
  28. 28.
    Wu, C., Frahm, J., Pollefeys, M.: Repetition-based dense single-view reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3113–3120 (2011)Google Scholar
  29. 29.
    Chen, X., Kang, S.B., Xu, Y.-Q., Dorsey, J., Shum, H.-Y.: Sketching reality: realistic interpretation of architectural designs. ACM Trans. Graph. 27(2), 1–15 (2008)CrossRefGoogle Scholar
  30. 30.
    Prasad, M., Fitzgibbon, A.: Single view reconstruction of curved surfaces. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1345–1354 (2006)Google Scholar
  31. 31.
    Zhang, Z., Liang, X., Ma, Y.: Unwrapping low-rank textures on generalized cylindrical surfaces. In: Proceedings of the 2011 International Conference on Computer Vision, IEEE Computer Society, Washington, DC, USA, pp. 1347–1354 (2011)Google Scholar
  32. 32.
    Liu, Y., Hel-Or, H., Kaplan, C.S., Gool, L.J.V.: Computational symmetry in computer vision and computer graphics. Found. Trends Comput. Graph. Vis. 5(1–2), 1–195 (2010)zbMATHGoogle Scholar
  33. 33.
    Mitra, N.J., Pauly, M., Wand, M., Ceylan, D.: Symmetry in 3d geometry: extraction and applications. Comput. Graph. Forum 32(6), 1–23 (2013)CrossRefGoogle Scholar
  34. 34.
    Müller, P., Zeng, G., Wonka, P., Van Gool, L.: Image-based procedural modeling of facades. ACM Trans. Graph. 26(3), 85 (2007)CrossRefGoogle Scholar
  35. 35.
    Wu, C., Frahm, J.-M., Pollefeys, M.: Detecting large repetitive structures with salient boundaries. In: Proceedings of the 11th European conference on Computer vision: Part II, ECCV’10, Springer, Berlin, Heidelberg, pp. 142–155 (2010)Google Scholar
  36. 36.
    C.-H. Shen, S.-S. Huang, H. Fu, S.-M. Hu, Adaptive partitioning of urban facades, ACM Trans. Graph. 30 (6) (2011) 184:1–184:10Google Scholar
  37. 37.
    Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision, 2nd Edn. Cambridge University Press. ISBN: 0521540518 (2004)Google Scholar
  38. 38.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22, 277–286 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.State Key Laboratory of Virtual Reality Technology and SystemsBeihang UniversityBeijingChina
  2. 2.School of Computer EngineeringNanyang Technological UniversitySingaporeSingapore
  3. 3.Department of Computer ScienceNew York University (SUNY Stony Brook)New YorkUSA

Personalised recommendations