Animage-based approach to the reconstruction of ancient architectures by extracting and arranging 3D spatial components

Article

Abstract

The objective of this research is the rapid reconstruction of ancient buildings of historical importance using a single image. The key idea of our approach is to reduce the infinite solutions that might otherwise arise when recovering a 3D geometry from 2D photographs. The main outcome of our research shows that the proposed methodology can be used to reconstruct ancient monuments for use as proxies for digital effects in applications such as tourism, games, and entertainment, which do not require very accurate modeling. In this article, we consider the reconstruction of ancient Mughal architecture including the Taj Mahal. We propose a modeling pipeline that makes an easy reconstruction possible using a single photograph taken from a single view, without the need to create complex point clouds from multiple images or the use of laser scanners. First, an initial model is automatically reconstructed using locally fitted planar primitives along with their boundary polygons and the adjacency relation among parts of the polygons. This approach is faster and more accurate than creating a model from scratch because the initial reconstruction phase provides a set of structural information together with the adjacency relation, which makes it possible to estimate the approximate depth of the entire structural monument. Next, we use manual extrapolation and editing techniques with modeling software to assemble and adjust different 3D components of the model. Thus, this research opens up the opportunity for the present generation to experience remote sites of architectural and cultural importance through virtual worlds and real-time mobile applications. Variations of a recreated 3D monument to represent an amalgam of various cultures are targeted for future work.

Key words

Digital reconstruction 3D virtual world 3D spatial components Vision and scene understanding 

CLC number

TP391.7 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. AlHalawani, S., Yang, Y.L., Liu, H., et al., 2013. Interactive facades analysis and synthesis of semi-regular facades. Comput. Graph. Forum, 32(2pt2):215–224. [doi:10.1111/cgf.12041]CrossRefGoogle Scholar
  2. Bao, F., Yan, D.M., Mitra, N.J., et al., 2013. Generating and exploring good building layouts. ACM Trans. Graph., 32(4):122.1–122.10. [doi:10.1145/2461912.2461977]CrossRefGoogle Scholar
  3. Bay, H., Tuytelaars, T., van Gool, L., 2006. SURF: speeded up robust features. Proc. 9th European Conf. on Computer Vision, p.404–417. [doi:10.1007/11744023_32]Google Scholar
  4. Bokeloh, M., Berner, A., Wand, M., et al., 2009. Symmetry detection using feature lines. Comput. Graph. Forum, 28(2):697–706. [doi:10.1111/j.1467-8659.2009.01410.x]CrossRefGoogle Scholar
  5. Ceylan, D., Mitra, N.J., Li, H., et al., 2012. Factored facade acquisition using symmetric line arrangements. Comput. Graph. Forum, 31(2pt3):671–680. [doi:10.1111/j.1467-8659.2012.03046.x]CrossRefGoogle Scholar
  6. Ceylan, D., Mitra, N.J., Zheng, Y., et al., 2014. Coupled structure-from-motion and 3D symmetry detection for urban facades. ACM Trans. Graph., 33(1):2.1–2.15. [doi:10.1145/2517348]CrossRefGoogle Scholar
  7. Chen, E., Williams, L., 1993. View interpolation for image synthesis. Proc. 20th Annual Conf. on Computer Graphics and Interative Techniques, p.279–288. [doi:10.1145/166117.166153]Google Scholar
  8. Chen, T., Zhu, Z., Shamir, A., et al., 2013. 3-Sweep: extruding editable objects from a single photo. ACM Trans. Graph., 32(6):195.1–195.10. [doi:10.1145/2508363.2508378]Google Scholar
  9. Çıçek, A., Gülesñ, M., 2004. Reconstruction of 3D models from 2D orthographic views using solid extrusion and revolution. J. Mater. Process. Technol., 152(3):291–298. [doi:10.1016/j.jmatprotec.2004.04.368]CrossRefGoogle Scholar
  10. Cignoni, P., Rocchini, C., Scopigno, R., 1998. Metro: measuring error on simplified surfaces. Comput. Graph. Forum, 17(2):167–174. [doi:10.1111/1467-8659.00236]CrossRefGoogle Scholar
  11. Criminisi, A., Reid, I., Zisserman, A., 2000. Single view metrology. Int. J. Comput. Vis., 40(2):123–148. [doi:10.1023/A:1026598000963]CrossRefMATHGoogle Scholar
  12. Davies, E.R., 2005. Machine Vision: Theory, Algorithms, Practicalities. Morgan Kauffman Press, San Francisco, USA.Google Scholar
  13. Debevec, P.E., Taylor, C.J., Malik, J., 1996. Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. Proc. 23rd Annual Conf. on Computer Graphics and Interative Techniques, p.11–20. [doi:10.1145/237170.237191]Google Scholar
  14. Dung, L.R., Huang, C.M., Wu, Y.Y., 2013. Implementation of RANSAC algorithm for feature-based image registration. J. Comput. Commun., 1:46–50. [doi:10.4236/jcc.2013.16009]CrossRefGoogle Scholar
  15. Encyclopedia, 2014. Mughal Architecture, Britannica Online. Available from http://global.britannica.com/EBchecked/topic/396119/Mughal-architecture [Accessed on Dec. 11, 2014].Google Scholar
  16. Faugeras, O., Laveau, S., Robert, L., 1995. 3-D reconstruction of urban scenes from sequences of images. Automatic Extraction of Man-Made Objects from Aerial and Space Images, p.145–168. [doi:10.1007/978-3-0348-9242-1_15]CrossRefGoogle Scholar
  17. Felzenszwalb, P.F., Huttenlochet, D.P., 2004. Efficient graphbased image segmentation. Int. J. Comput. Vis., 59(2):167–181. [doi:10.1023/B:VISI.0000022288.19776.77]CrossRefGoogle Scholar
  18. Frahm, J.M., Fite-Georgel, P., Gallup, D., et al., 2010. Building Rome on a cloudless day. Proc. 11th European Conf. on Computer Vision, p.368–381. [doi:10.1007/978-3-642-15561-1_27]Google Scholar
  19. Garcia-Gago, J., Gomez-Lahoz, J., Rodríguez-Méndez, J., et al., 2014. Historical single image-based modeling: the case of Gobierna Tower, Zamora (Spain). Remote Sens., 6(2):1085–1101. [doi:10.3390/rs6021085]CrossRefGoogle Scholar
  20. Geman, S., Geman, D., 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Patt. Anal. Mach. Intell., 6(6):721–741. [doi:10.1109/TPAMI.1984.4767596]CrossRefMATHGoogle Scholar
  21. Gormen, T.H., Leiserson, C.E., Rivest, R.L., et al., 1990. Introduction to Algorithms. MIT Press, McGraw-Hill Book Company, New York, USA.Google Scholar
  22. Guillou, E., Meneveaux, D., Maisel, E., et al., 2000. Using vanishing points for camera calibration and coarse 3D reconstruction from a single image. Vis. Comput., 16(7):396–410. [doi:10.1007/PL00013394]CrossRefMATHGoogle Scholar
  23. Hoiem, D., Efros, A.A., Hebert, M., 2005. Geometric context from a single image. Proc. 10th IEEE Int. Conf. on Computer Vision, p.654–661. [doi:10.1109/ICCV.2005.107]Google Scholar
  24. Horn, B.K.P., 1990. Height and gradient from shading. Int. J. Comput. Vis., 5(1):37–75. [doi:10.1007/BF00056771]CrossRefGoogle Scholar
  25. Horry, Y., Anjyo, K., Arai, K., 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. Proc. 24th Annual Conf. on Computer Graphics and Interactive Techniques, p.225–232. [doi:10.1145/258734.258854]Google Scholar
  26. Kang, S., 1998. Depth Painting for Image-Based Rendering Applications. Technical Report, Compaq Computer Corporation, Cambridge Research Lab.Google Scholar
  27. Laveau, S., Faugeras, O., 1994. 3D scene representation as a collection of images. Proc. 12th Int. Conf. on Pattern Recognition, p.689–691. [doi:10.1109/ICPR.1994.576404]CrossRefGoogle Scholar
  28. Liebowitz, D., Criminisi, A., Zisserman, A., 1999. Creating architectural models from images. Comput. Graph. Forum, 18(3):39–50. [doi:10.1111/1467-8659.00326]CrossRefGoogle Scholar
  29. Lowe, D.G., 2004. Distinctive image features from scaleinvariant keypoints. Int. J. Comput. Vis., 60(2):91–110. [doi:10.1023/B:VISI.0000029664.99615.94]CrossRefGoogle Scholar
  30. Ma, J., Chan, J.C., Canters, F., 2010. Fully automatic subpixel image registration of multiangle CHRIS/Proba data. IEEE Trans. GeoSci. Remote Sens., 48(7):2829–2839. [doi:10.1109/TGRS.2010.2042813]CrossRefGoogle Scholar
  31. Manferdini, A.M., 2012. A methodology for the promotion of cultural heritage sites through the use of low-cost technologies and procedures. Proc. 17th Int. Conf. on 3D Web Technology, p.180. [doi:10.1145/2338714.2338747]CrossRefGoogle Scholar
  32. McMillan, L., Bishop, G., 1995. Plenoptic modeling: an image-based rendering system. Proc. 22nd Annual Conf. on Computer Graphics and Interactive Techniques, p.39–46. [doi:10.1145/218380.218398]Google Scholar
  33. Mitra, N.J., Pauly, M., 2008. Symmetry for architectural design. Advances in Architectural Geometry, p.13–16.Google Scholar
  34. Mitra, N.J., Pauly, M., Wand, M., et al., 2013. Symmetry in 3D geometry: extraction and applications. Comput. Graph. Forum, 32(6):1–23. [doi:10.1111/cgf.12010]CrossRefGoogle Scholar
  35. Müller, P., Zeng, G., Wonka, P., et al., 2007. Image-based procedural modeling of facades. ACM Trans. Graph., 26(3):85.1–85.9. [doi:10.1145/1276377.1276484]CrossRefGoogle Scholar
  36. Nagai, T., Ikehara, M., Kurematsu, A., 2007. HMM-based surface reconstruction from single images. Syst. Comput. Jpn., 38(11):80–89. [doi:10.1002/scj.10685]CrossRefGoogle Scholar
  37. Nan, L., Sharf, A., Zhang, H., et al., 2010. SmartBoxes for interactive urban reconstruction. ACM Trans. Graph., 29(4):93.1–93.10. [doi:10.1145/1778765.1778830]CrossRefGoogle Scholar
  38. Nevatia, R., Babu, K.R., 1980. Linear feature extraction and description. Comput. Graph. Image Process., 13(3):257–269. [doi:10.1016/0146-664X(80)90049-0]CrossRefGoogle Scholar
  39. Oh, B.M., Chen, M., Dorsey, J., et al., 2001. Image-based modeling and photo editing. Proc. 28th Annual Conf. on Computer Graphics and Interactive Techniques, p.433–442. [doi:10.1145/383259.383310]Google Scholar
  40. Poulin, P., Ouimet, M., Frasson, M.C., 1998. Interactively modeling with photogrammetry. Proc. Eurographics Workshop on Rendering, p.93–104. [doi:10.1007/978-3-7091-6453-2_9]Google Scholar
  41. Pylvanainen, T., Berclaz, J., Korah, T., et al., 2012. 3D city modeling from street-level data for augmented reality applications. Proc. 2nd Int. Conf. on 3D Imaging, Modeling, Processing, Visualization and Transmission, p.238–245. [doi:10.1109/3DIMPVT.2012.19]Google Scholar
  42. Remondino, F., 2011. Heritage recording and 3D modeling with photogrammetry and 3D scanning. Remote Sens., 3(6):1104–1138. [doi:10.3390/rs3061104]CrossRefGoogle Scholar
  43. Saxena, A., Chung, S.H., Ng, A.Y., 2008a. 3-D depth reconstruction from a single still image. Int. J. Comput. Vis., 76(1):53–69. [doi:10.1007/s11263-007-0071-y]CrossRefGoogle Scholar
  44. Saxena, A., Sun, M., Ng, A.Y., 2008b. Make3D: depth perception from a single still image. Proc. 23rd AAAI Conf. on Artificial Intelligence, p.1571–1576.Google Scholar
  45. Shade, J., Gortler, S., He, L., et al., 1998. Layered depth images. Proc. 25th AAAI Annual Conf. on Computer Graphics and Interactive Techniques, p.231–242. [doi:10.1145/280814.280882]Google Scholar
  46. Shen, C.H., Fu, H., Chen, K., et al., 2012. Structure recovery by part assembly. ACM Trans. Graph., 31(6):180.1–180.11. [doi:10.1145/2366145.2366199]CrossRefGoogle Scholar
  47. Styliadis, A.D., Sechidis, L.A., 2011. Photography-based facade recovery & 3D modeling: a CAD application in cultural heritage. J. Cult. Herit., 12(3):243–252. [doi:10.1016/j.culher.2010.12.008]CrossRefGoogle Scholar
  48. Super, B.J., Bovik, A.C., 1995. Shape from texture using local spectral moments. IEEE Trans. Patt. Anal. Mach. Intell., 17(4):333–343. [doi:10.1109/34.385983]CrossRefGoogle Scholar
  49. Wang, Y., Olano, M., 2011. A framework for GPU 3D model reconstruction using structure-from-motion. Proc. 38th Annual Conf. on Computer Graphics and Interactive Techniques, p.27.1. [doi:10.1145/2037715.2037748]Google Scholar
  50. Wei, Y.M., Kang, L., Yang, B., et al., 2013. Applications of structure from motion: a survey. J. Zhejiang Univ.-Sci. C (Comput. & Electron.), 14(7):486–494. [doi:10.1631/jzus.CIDE1302]CrossRefGoogle Scholar
  51. Yang, M.D., Chao, C.F., Huang, K.S., et al., 2013. Imagebased 3D scene reconstruction and exploration in augmented reality. Autom. Constr., 33:48–60. [doi:10.1016/j.autcon.2012.09.017]CrossRefGoogle Scholar
  52. Zhang, H., Xu, K., Jiang, W., et al., 2013. Layered analysis of irregular facades via symmetry maximization. ACM. Trans. Graph., 32(4):121.1–121.10. [doi:10.1145/2461912.2461923]CrossRefGoogle Scholar
  53. Zhang, L., Dugas-Phocion, G., Samson, J.S., et al., 2002. Single-view modeling of free-form scenes. J. Visual. Comput. Animat., 13(4):225–235. [doi:10.1002/vis.291]CrossRefMATHGoogle Scholar

Copyright information

© Journal of Zhejiang University Science Editorial Office and Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Internet and Multimedia EngineeringKonkuk UniversitySeoulKorea
  2. 2.Department of Advanced Technology FusionKonkuk UniversitySeoulKorea

Personalised recommendations