Single-View 3D Reconstruction by Learning 3D Game Scenes

  • Makoto Okabe
  • Ken Anjyo
  • Rikio Onai
Part of the Mathematics for Industry book series (MFI, volume 4)


We report a method for reconstructing a three-dimensional (3D) depth map using a single two-dimensional (2D) image. Our method is designed to reconstruct manmade objects, such as buildings. We first estimate the normal map, and then integrate it to obtain the depth map. To estimate the normal map, we analyze the co-occurrence relation between the normal vectors and image features in a training dataset. We consider the corners and lines to be the image features. The training dataset is formed of 3D game scenes. In the offline learning phase, we detect corners and lines in the normal map of each game scene using a detection algorithm, and observe the normal vectors around them. Then we construct a database of the co-occurrence relations, i.e., how frequently each corner or line appears with each normal vector. In the online reconstruction phase, given an input image, we detect the corners and lines using the same detection algorithm, and estimate the normal vectors around them based on the learned co-occurrence relation. We formulate this estimation using a Markov random field. Finally, the estimated normal map is integrated by solving Poisson’s equation, and we obtain a depth map.


Single-view modeling Tour into the picture Image-based modeling Image database Markov random field Corner detection 


  1. 1.
    Shan Q, Adams R, Curless B, Furukawa Y, Seitz SM (2013) The visual turing test for scene reconstruction. In: Proceedings of 3DV13Google Scholar
  2. 2.
    Debevec PE, Taylor CJ, Malik J (1996) Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. Comput Graph, Ann Conf Ser 30:11–20Google Scholar
  3. 3.
    Horry Y, Anjyo K, Arai K (1997) Tour into the picture: using a spidery mesh interface to make animation from a single image. In: Proceedings of SIGGRAPH ’97, pp 225–232Google Scholar
  4. 4.
    Li Z, Guillaume D-P, Jean-Sebastien S, SM Seitz (2001) Single view modeling of free-form scenes. In: Proceedings of CVPR 2001, pp 990–997Google Scholar
  5. 5.
    Hoiem D, Efros AA, Hebert M (2005) Automatic photo pop-up. ACM Trans. Graph. 24(3):577–584CrossRefGoogle Scholar
  6. 6.
    Saxena A, Chung SH, Ng AY (2008) 3-D depth reconstruction from a single still image. Int. J. Comput. Vis. 76(1):53–69CrossRefGoogle Scholar
  7. 7.
    UBISOFT: Anno 1404 (2010)Google Scholar
  8. 8.
    Szeliski R, Zabih R, Scharstein D, Veksler O, Kolmogorov V, Agarwala A, Tappen M, Rother C (2008) A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6):1068–1080CrossRefGoogle Scholar

Copyright information

© Springer Japan 2014

Authors and Affiliations

  1. 1.The University of Electro-Communications/JST CRESTChofuJapan
  2. 2.OLM Digital, Inc./JST CRESTSetagaya-kuJapan
  3. 3.The University of Electro-CommunicationsChofuJapan

Personalised recommendations