Single-View 3D Reconstruction by Learning 3D Game Scenes
We report a method for reconstructing a three-dimensional (3D) depth map using a single two-dimensional (2D) image. Our method is designed to reconstruct manmade objects, such as buildings. We first estimate the normal map, and then integrate it to obtain the depth map. To estimate the normal map, we analyze the co-occurrence relation between the normal vectors and image features in a training dataset. We consider the corners and lines to be the image features. The training dataset is formed of 3D game scenes. In the offline learning phase, we detect corners and lines in the normal map of each game scene using a detection algorithm, and observe the normal vectors around them. Then we construct a database of the co-occurrence relations, i.e., how frequently each corner or line appears with each normal vector. In the online reconstruction phase, given an input image, we detect the corners and lines using the same detection algorithm, and estimate the normal vectors around them based on the learned co-occurrence relation. We formulate this estimation using a Markov random field. Finally, the estimated normal map is integrated by solving Poisson’s equation, and we obtain a depth map.
KeywordsSingle-view modeling Tour into the picture Image-based modeling Image database Markov random field Corner detection
- 1.Shan Q, Adams R, Curless B, Furukawa Y, Seitz SM (2013) The visual turing test for scene reconstruction. In: Proceedings of 3DV13Google Scholar
- 2.Debevec PE, Taylor CJ, Malik J (1996) Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. Comput Graph, Ann Conf Ser 30:11–20Google Scholar
- 3.Horry Y, Anjyo K, Arai K (1997) Tour into the picture: using a spidery mesh interface to make animation from a single image. In: Proceedings of SIGGRAPH ’97, pp 225–232Google Scholar
- 4.Li Z, Guillaume D-P, Jean-Sebastien S, SM Seitz (2001) Single view modeling of free-form scenes. In: Proceedings of CVPR 2001, pp 990–997Google Scholar
- 7.UBISOFT: Anno 1404 (2010)Google Scholar