Abstract
In this paper, a method for the automated reconstruction of architectures from two views of a monocular camera is proposed. While this research topic has been studied over the last few decades, we contend that a satisfactory approach has not yet been devised. Here, a new method to solve the same problem with several points of novelty is proposed. First, reference planes are automatically detected using color, straight lines, and edge/vanishing points. This approach is quite robust and fast even when different views and complicated conditions are presented. Second, the camera pose and 3D points are accurately estimated by a two-view geometry constraint in the convex optimization approach. It has been demonstrated that camera rotations are appropriately estimated, while translations induce a significant error in short baseline images. To overcome this problem, we rely only on reference planes to estimate image homography instead of using the conventional camera pose estimation method. Thus, the problem associated with short baseline images is adequately addressed. The 3D points and translation are then simultaneously triangulated. Furthermore, both the homography and 3D point triangulation are computed via the convex optimization approach. The error of back-projection and measured points is minimized in L ∞-norm so as to overcome the local minima problem of the canonical L 2-norm method. Consequently, extremely accurate homography and point clouds can be achieved with this scheme. In addition, a robust plane fitting method is introduced to describe a scene. The corners are considered as properties of the plane in order to limit the boundary. Thus, it is necessary to find the exact corresponding corner positions by searching along the epipolar line in the second view. Finally, the texture of faces is mapped from 2D images to a 3D plane. The simulation results demonstrate the effectiveness of the proposed method for scenic images in an outdoor environment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. C. Longuet-Higgins, “A computer algorithm for reconstructing a scene from two projections,” Nature, vol. 293, no. 10, pp. 133–135, 1981.
R. I. Hartley, “Estimation of relative camera positions for uncalibrated cameras,” Proc. of ECCV, Lecture Notes in Computer Science, vol. 588, pp. 579–587, 1992. [click]
S. J. Maybank and O. D. Faugeras, “A theory of selfcalibration of a moving camera,” International Journal of Computer Vision, vol. 8, no. 2, pp. 123–151, 1992. [click]
Q.-T. Luong and O. D. Faugeras, “The fundamental matrix: theory, algorithms, and stability analysis,” International Journal of Computer Vision, vol. 17, no. 1, pp. 43–75, 1996.
R. I. Hartley and F. Kahl, “Critical configurations for projective reconstruction from multiple views,” International Journal of Computer Vision, vol. 71, no. 1, pp. 5–47, 2007. [click]
B. Caprile and V. Torre, “Using vanishing points for camera calibration,” International Journal of Computer Vision, vol. 4, no. 2, pp. 127–140, 1990. [click]
B. Triggs, “Autocalibration from planar scenes,” Proc. of ECCV’ 92, Lecture Notes in Computer Science, vol. 1, pp. 89–105, 1998.
C. Rother and S. Carlsson, “Linear multi view reconstruction and camera recovery using a reference plane,” International Journal of Computer Vision, vol. 49, no. 2, pp. 117–141, 2002.
F. Kahl and R. Hartley, “Multiple view geometry under the L ∞-norm,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 30, no. 9, pp. 1603–1617, 2008. [click]
H.-H. Trinh, D.-N. Kim, and K.-H. Jo, “Supervised training database for building recognition by using cross ratio invariance and SVD-based method,” International Journal of Applied Intelligence, Vol. 32, no. 2, pp. 216–230, 2010.
H.-H. Trinh and K.-H. Jo, “Image-based structural analysis of building using line segments and their geometrical vanishing points,” Proc. of the SICE-ICASE International Joint Conference, pp. 566–571. 2006. [click]
O. Enqvist, F. Kahl, and C. Olsson, “Non-Sequential Structure from Motion,” Proc. of the Eleventh Workshop on Omnidirectional Vision, Camera Networks and Non-classical Camera, pp. 264–271, 2011. [click]
R. Hartley and F. Schaffalitzky, “L ∞ minimization in geometric reconstruction problems,” Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 504–509, 2004. [click]
D. Lowe, “Distinctive image features from scale-invariant interest points,” International Journal of Computer Vision, Vol. 60, no. 2, pp. 91–110, 2004. [click]
M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with application to image analysis and automated car-tography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. [click]
A. Agarwal, C. V. Jawahar, and P. J. Narayanan, “A survey of planar homography estimation techniques,” IIIT Technical Report IIIT/TR/2005/12, 2005. [click]
R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, 2004.
D. Nistér, “An efficient solution to the five-point relative pose problem,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 756–770, 2004. [click]
C. Harris and M. Stephens, “A combined corner and edge detector,” Proc. of the 4th Alvey Vision Conference, pp. 147–151, 1998.
H. Bay, T. Tuytelaars, L. V. Gool, “SURF: speeched up robust features,” Proc. of ECCV, Vol. 3951, pp. 404–417, 2006. [click]
K. Mikolajczyk and C. Schmid, “A performance evaluation of local descriptors,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 21, no. 4, 2005. [click]
S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, 2004.
A. Dalalyan and R. Keriven, “L1-penalized robust estimation for a class of inverse problems arising in multiview geometry,” Proc. of the 23rd Annual Conference on Neural Information Processing Systems, pp. 441–449, 2009.
T. Werner and A. Zisserman, “New techniques for automated architectural reconstruction from photographs,” Proc. of ECCV, vol. 2, pp. 514–555, 2002.
R. T. Collins, “A space-sweep approach to true multi-image matching,” Proc. of Computer Vision and Pattern Recognition, pp. 358–363, 1996. [click]
“The MOSEK optimization toolbox for MATLAB manual,” www.mosek.com
H. Shao and L. V. Gool, “Zubud-zurich buildings database for image based recognition,” Swiss FI of Tech., Technical Report, no. 260, 2003.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Gon-Woo Kim under the direction of Editor Euntai Kim. This work was supported by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MOE) (2013R1A1A2009984). Also, we would like to express our thanks to Ho Chi Minh City University of Technology and Education.
My-Ha Le received his B.E. and M.E. degrees from Department of Electrical and Electronic Engineering of Ho Chi Minh University of Technology, Viet Nam, in 2005 and 2008, respectively. Since 2007, he has been serving as a faculty member in Department of Electrical and Electronic Engineering, Ho Chi Minh City University of Technology and Education, Viet Nam. He received his Ph.D. degree from Electrical Engineering Department of University of Ulsan, Korea, in 2013. His research interests include 3D computer vision, pattern recognition, and vision based robotics.
Van-Dung Hoang received his bachelor of informatics from Hue University, Viet Nam in 2002, and master of computer sciences from Hanoi National University of Education, Vietnam in 2007. Since 2002, he has been serving as a lecturer in University of Quang Binh, Vietnam. He received Ph.D. degree from Electrical Engineering Department of University of Ulsan, Korea, in 2014. His research interests include pattern recognition, machine learning, computer vision, and vision based robotics.
Hoang-Hon Trinh was born in Dong Nai, Viet Nam, in 1973. He received his B.E. and M.E. degrees in Electrical-Electronic Engineering of Ho Chi Minh City University of Technology, Viet Nam, in 1997 and 2002 respectively. He received his Ph.D. degree from Electrical Engineering Department of University of Ulsan, Korea, in 2008. His research interests include computer vision, pattern recognition, understanding and reconstructing outdoor scenes, designing the outdoor mobile robot for civil and special applications.
Kang-Hyun Jo received his Ph.D. degree from Osaka University, Japan, in 1997. He joined the School of Electrical Eng., University of Ulsan right after having one year experience at ETRI as a post-doc research fellow. Dr. Jo has been active to serve for the societies for many years as directors of ICROS (Institute of Control, Robotics and Systems) and SICE (Society of Instrumentation and Control Engineers, Japan) as well as IEEE IES. He is currently contributing himself as an AE for a few journals, such as IJCAS (International Journal of Control, Automation and Systems), TCCI (Transactions on Computational Collective Intelligence) and IteN (IES Technical News, online publication of IEEE), TIE. He had involved in organizing many international conferences such as ICCAS, FCV, ICIC and IECON. He had visited for performing his research activity to Kyushu University, KIST and University of California Riverside. His research interest covers in a wide area where focuses on computer vision, robotics, and ambient intelligence.
Rights and permissions
About this article
Cite this article
Le, MH., Trinh, HH., Hoang, VD. et al. Automated architectural reconstruction using reference planes under convex optimization. Int. J. Control Autom. Syst. 14, 814–826 (2016). https://doi.org/10.1007/s12555-014-0203-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-014-0203-4