Abstract
Reconstruction of architectural structures from photographs has recently experienced intensive efforts in computer vision research. This is achieved through the solution of nonlinear least squares (NLS) problems to obtain accurate structure and motion estimates. In Photogrammetry, NLS contribute to the determination of the 3-dimensional (3D) terrain models from the images taken from photographs. The traditional NLS approach for solving the resection-intersection problem based on implicit formulation on the one hand suffers from the lack of provision by which the involved variables can be weighted. On the other hand, incorporation of explicit formulation expresses the objectives to be minimized in different forms, thus resulting in different parametric values for the estimated parameters at non-zero residuals. Sometimes, these objectives may conflict in a Pareto sense, namely, a small change in the parameters results in the increase of one objective and a decrease of the other, as is often the case in multi-objective problems. Such is often the case with error-in-all-variable (EIV) models, e.g., in the resection-intersection problem where such change in the parameters could be caused by errors in both image and reference coordinates. This study proposes the Pareto optimal approach as a possible improvement to the solution of the resection-intersection problem, where it provides simultaneous estimation of the coordinates and orientation parameters of the cameras in a two or multistation camera system on the basis of a properly weighted multi-objective function. This objective represents the weighted sum of the square of the direct explicit differences of the measured and computed ground as well as the image coordinates. The effectiveness of the proposed method is demonstrated by two camera calibration problems, where the internal and external orientation parameters are estimated on the basis of the collinearity equations, employing the data of a Manhattan-type test field as well as the data of an outdoor, real case experiment. In addition, an architectural structural reconstruction of the Merton college court in Oxford (UK) via estimation of camera matrices is also presented. Although these two problems are different, where the first case considers the error reduction of the image and spatial coordinates, while the second case considers the precision of the space coordinates, the Pareto optimality can handle both problems in a general and flexible way.
Similar content being viewed by others
References
Ameller MA, Triggs B, Quan L (2000) Camera pose revisited—new linear algorithms. European Conference on Computer Vision (ECCV)
Atkinson KB (1996) Close range photogrammetry and machine vision. Whittles Publishers
Awange JL, Kiema JBK (2013) Environmental geoinformatics—monitoring and management. Springer, Berlin
Awange JL, Grafarend EW (2005) Solving algebraic computational problems in geodesy and geoinformatics. Springer, Berlin
Awange JL, Grafarend EW, Pala´ncz B, Zaletnyik P (2010) Algebraic geodesy and geoinformatics. Springer, Berlin
Bartoli A (2002) A unified framework for quasi-linear bundle adjustment. ICPR02—In: Proceedings of the sixteenth IAPR international conference on pattern recognition, vol Ii, Quebec City, Canada, pp 560–563
Börlin N (2002) Comparison of resection: intersection algorithms and projection geometries in radiostereometry. ISPRS J Photogramm Remote Sens 56:390–400. doi:10.1016/S0924-2716(02)00068-0
Censor Y (1977) Pareto optimality in multiobjective problems. Appl Math Optim 4:41–59
Chen Q, Medioni G (1999) Efficient iterative solution to Mview projective reconstruction In: Computer vision and pattern recognition, 1999, IEEE computer society conference. doi:10.1109/CVPR.1999.784608
Coello CA (1999) A comprehensive survey of evolutionary-based multi-objective optimization techniques. Knowl Inf Syst 1(3):269–308
Dunn E, Olague G, Lutton E, Schoenauer M (2004) Pareto optimal sensing strategies for an active vision system. In: IEEE congress on evolutionary computation, 19–23 June 2004, vol 1, Portland, Oregon, USA, pp 457–463
Fekete K, Schrott P (2008) Qualification of optical capturing devices for data gathering phase of the face reconstruction process. In: Proc. of the third Hungarian conference on biomechanics, Budapest, Hungary, pp 83–88
Felus YA, Schaffrin B (2005) Performing similarity transformations using the errors-in-variable model, Baltimore, Maryland
Forsyth DA, Ponce J (2003) Computer vision—a modern approach. Pearson Education
Geisler J, Trächtler A (2009) Control of the Pareto optimality of systems with unknown disturbances. In: IEEE international conference on control and automation christchurch, 9–11 December, New Zealand, pp 695–700
Golub GH, van Loan CF (1980) An analysis of the total least-squares problem. SIAM J Numer Anal 17(6):883–893
Grafarend E, Awange JL (2012) Applications of linear and nonlinear models. Springer, New York, Berlin
Grafarend E, Shan J (1997a) Closed form solution to the P4P or the three dimensional resection problem in terms of Möbius barycentric coordinates. J Geod 71:217–231. doi:10.1007/s001900050089
Grafarend E, Shan J (1997b) Closed form solution to the twin P4P or the combined three dimensional resection-intersection problem in terms of M¨obius barycentric coordinates. J Geod 71(4):232–239. doi:10.1007/s001900050090
Grafarend E, Shan J (1997c) Estimable quantities in projective geometry I and II. Zeitschrift fuer Vermessungswessen 122(Heft 5 and 7):218–225, 323–333
Grussenmeyer P, Al Khalil O (2002) Solution of exterior orientation in photogrammetry, a review. Photogramm Rec- Int J Photogramm 17(100):615–634. doi:10.1111/j.1477-9730.2002.tb01907.x
Han JY, Guo J, Chou JY (2011) A direct determination of the orientation parameters in the collinearity equations. IEEE Geo Sci Rem Sens Letts 8:313–316. doi:10.1109/LGRS.2010.2066955
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press
Hochman HM, Rodgers JD (1969) Pareto optimal redistribution, Part 1. Am Econ Rev 59(4):542–557
Lin JG (1976) Multiple-objective problems—Pareto-optimal solutions by method of proper equality constraints. IEEE Trans Automat Contr AC-21:641–650
Marler RT, Arora JS (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscipl Optim 26:369–395
McGlone JC (1989) Analytic data-reduction schemes in nontopographic photogrammetry. American society of photogrammetry and remote sensing, chapter 4, vol 554, Falls Church, Virginia, pp 37–55
Mikhail EM, Bethel JS, McGlone CJ (2001) Introduction to modern photogrammetry. Wiley
Mirza P, Almir K (2010) Pareto-based genetic algorithm in multi-objective geospatial analysis. In:MIPRO, 2010 proceedings of the 33rd international convention, 24–28 May 2010, Opatija, Croatia, pp 680–685
Mahamud S, Herbert M, Omori Y, Ponce J (2001) Provably-convergent iterative methods for projective structure and motion. In: Computer vision and pattern recognition, 2001, CVPR 2001, Proceedings of the 2001 IEEE computer society conference page(s): I-1018 - I-1025, vol 1. doi:10.1109/CVPR.2001.990642
Neitzel F (2010) Generalization of total least-squares on example of unweighted and weighted 2D similarity transformation. J Geod 84(12):751–762. doi:10.1007/s00190-010-0408-0
Olague G, Trujillo L (2012) Interest point detection through multiobjective genetic programming. Appl Soft Comp 12(8):2566–2582. doi:10.1016/j.asoc.2012.03.058
Olague G, Trujillo L (2011) Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis Comput 29:484–498. doi:10.1016/j.imavis.2011.03.004
Olsson C, Byröd M, Kahl F (2009) Globally optimal least squares solutions for quasiconvex 1D vision problems. In: Salberg A-B, Hardeberg JY, Jenssen R (eds) SCIA 2009, LNCS 5575, pp 686–695
Paláncz B, Awange JL (2012) Application of Pareto optimality to linear models with errors-in-all-variables. J Geod 86(7):531–545. doi:10.1007/s00190-011-0536-1
Pressl B, Mader C, Wieser M (2010) User-specific web-based route planning. In: Miesenberger K et al (eds) ICCHP 2010, Part I, LNCS 6179. Springer, Berlin, Heidelberg, pp 280–287
Remondino F (2002) 3-D reconstruction of articulated objects from uncalibrated images, USA
Saadatseresht M,Mansourian A, Taleai M(2009) Evacuation planning using multi-objective evolutionary optimization approach. Eur J Oper Res 198:305–314. doi:10.1016/j.ejor.2008.07.032
Schaffrin B, Snow K (2010) Total least-squares regularization of Tykhonov type and an ancient racetrack in Corinth. Linear Algebra Appl 432:2061–2076. doi:10.1016/j.laa.2009.09.014
Sonnier DL (2010) A Pareto-optimality based routing and wavelength assignment algorithm for WDM networks. J Comp Sci Coll Arch 25(5):118–123
Triggs B, McLauchlan P, Hartley R, Fitzgibbon A (2000) Bundle adjustment—a modern synthesis. Springer, London
Werner T, Schaffalitzky F, Zisserman A (1999) Automated architecture reconstruction fromclose-range photogrammetry. In: The proceedings of the international CIPA symposium, 2001. http://www.robots.ox.ac.uk/vgg/publications/papers/werner01a.pdf. Accessed 19 Oct 2012
Warr PG (1982) Pareto optimal redistribution and private charity. J Public Econ 19(1):131–138. doi:10.1016/0047-2727(82)90056-1
Wilson PB, Macleod MD (1993) Low implementation cost IIR digital filter design using genetic algorithms. In: IEE/IEEE workshop on natural algorithms in signal processing. Chelmsford, UK, pp 4/1–4/8
Kwon Y-H (1998) DLT method. www.kwon3d.com/theory/dlt/dlt.html. Accessed 17 Nov 2011
Zitler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and the strength of Pareto approach. IEEE Trans Evol Comput 3(4):257–271. doi:10.1109/4235.797969
Zwanzig S (2006) On an application of deconvolution techniques to local linear regression with errors in variables. Department of Mathematics Uppsala University. U.U.D.M. Report 2006:12
Acknowledgements
The author are grateful to the Editor Prof. Babaie and the anonymous reviewers for their comments that helped improve the quality of the manuscript. Special thanks to Prof. Han for providing their measurements data that was used in one of the examples in this study. They are further grateful to Dr. Kevin Fleming for proof reading the manuscript, but take full responsibility for any errors. J.L. Awange acknowledges the financial support of the Alexander von Humboldt Foundation (Ludwig Leichhardt’s Memorial Fellowship) and a Curtin Research Fellowship. He is grateful for the warm welcome and the conducive working atmosphere provided by his host, Prof. Bernhard Heck and his team, at the Geodetic Institute, Karlsruhe Institute of Technology (KIT). This is a TiGER publication No. 438.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: H. A. Babaie
Appendix
Appendix
Summary of the steps of the algorithm
Read input data
The coordinates of the points on the photo-planes:
The space coordinates:
where n is the number of points on a photo-plane, and m is the number of the photo-planes.
Defining the objective functions
-
a)
for the photo-planes:
$$\begin{array}{lll} G_{\text{xy}}\left(\pi _1,\pi _2,\text{...}\pi _m\right)\\ \quad= \displaystyle\sum _{i=1}^m \sum _{j=1}^n w_{x_j}\left(x_j^{(i)}-p_x\left(\pi _i,X_j,Y_j,Z_j\right)\right)^2\\ \quad\quad{\kern6pt} +w_{y_j}\left(y_j^{(i)}-p_y\left(\pi _i,X_j,Y_j,Z_j\right)\right)^2 \end{array} $$ -
b)
for the space coordinates
-
use one-point intersection to express the space coordinates explicitely, see Eq. 14:
$$\begin{array}{lll} G_{\text{XYZ}}\left(\pi _1,\pi _2,\text{...}\pi _m\right)\\ \quad= \sum\limits_{j=1}^n W_{X_j}\left(X_j^{(i)}-p_X\left(\pi _1,\pi _2,\text{...}\pi _m,x_j^{(i)},y_j^{(i)}\right)\right)^2\\ \quad\quad+W_{Y_j}\left(Y_j^{(i)}-p_Y\left(\pi _1,\pi _2,\text{...}\pi _m,x_j^{(i)},y_j^{(i)}\right)\right)^2\\ \quad\quad+W_{Z_j}\left(Z_j^{(i)}-p_Z\left(\pi _1,\pi _2,\text{...}\pi _m,x_j^{(i)},y_j^{(i)}\right)\right)^2 \end{array} $$ -
alternatively use implicit expression of the space coordinates as constraint while minimizing the adjustments of the space coordinates, see Eq. 20.
$$\begin{array}{rll} \text{ }G_{\text{XYZ}} \left(\pi _1,\pi _2,\text{...}\pi _m\right)&=&\sum\limits_{j=1}^n W_{X_j}\Delta \text{X}_j^2+W_{Y_j}\Delta \text{Y}_j^2\\ &&+W_{Z_j}\Delta \text{Z}_j^2, \end{array} $$
-
with the constraints
Computing the dimensionless form of the conflicting objective functions
-
a)
Minimize \(\,G_{\text {xy} }\,\text {to} \,\text {get} \,\pi _1^{(\text {xy})},\pi _2^{(\text {xy})},\text {...}\pi _m^{(\text {xy})}\)
-
b)
Minimize \(G_{\text {XYZ} }\,\text {to} \,\text {get} \,\pi _1^{(\text {XYZ})},\pi _2^{(\text {XYZ})},\text {...}\pi _m^{(\text {XYZ})}\)
-
c)
The maximum values of the objective functions
$$G_{\text{xymax}}=G_{\text{xy}}\left(\pi _1^{(\text{XYZ})},\pi _2^{(\text{XYZ})},\text{...}\pi _m^{(\text{XYZ})}\right)$$$$G_{\text{XYZmax}}=G_{\text{XYZ}}\left(\pi _1^{(\text{xy})},\pi _2^{(\text{xy})},\text{...}\pi _m^{(\text{xy})}\right)$$ -
d)
Compute the dimensionless forms
$$\tilde{G}_{\text{xy}}\left(\pi _1,\pi _2,\text{...}\pi _m\right)=\frac{G_{\text{xy}}\left(\pi _1,\pi _2,\text{...}\pi _m\right)-G_{\text{xymin}}}{G_{\text{xymax}}-G_{\text{xymin}}}$$and
$$\tilde{G}_{\text{XYZ}}\left(\pi _1,\pi _2,\text{...}\pi _m\right)=\frac{G_{\text{XYZ}}(\pi _1,\pi _2,\text{...}\pi _m)-G_{\text{XYZmin}}}{G_{\text{XYZmax}}-G_{\text{XYZmin}}}$$
Computing the Pareto set
-
a)
Set discrete values λ k \(\in \)[0, 1], k \(=\) 1,2, ...N
-
b)
Minimize the mono-objective function for all λ k
$$\begin{array}{rll} \tilde{G}\left(\pi _1,\pi _2,\text{...}\pi _m,\lambda _k\right)&=& \lambda _k \tilde{G}_{\text{XYZ}}\left(\pi _1,\pi _2,\text{...}\pi _m\right)\\ &&+\left(1-\lambda _k\right)\tilde{G}_{\text{xy}}\left(\pi _1,\pi _2,\text{...}\pi _m\right) \end{array} $$to get the Pareto-set of ( λ k , Π1(k), Π2(k), ...Π m(k)) , k \(=\) 1,2, ...N
Computing the Pareto front
-
a)
Set the interpolation functions: Π 1(\(\lambda \)), Π 2(\(\lambda \))... Π m (\(\lambda \)) from the discrete values
-
b)
Substitute these functions into the objective functions,
$$\tilde{G}_{\text{xy}}(\lambda )=\tilde{G}_{\text{xy}}\left(\pi _1(\lambda ),\pi _2(\lambda ),\text{...}\pi _m(\lambda )\right)$$and
$$\tilde{G}_{\text{XYZ}}(\lambda )=\tilde{G}_{\text{XYZ}}\left(\pi _1(\lambda ),\pi _2(\lambda ),\text{...}\pi _m(\lambda )\right)$$Then we get the Pareto-front represented in param- etric form: \(\tilde {G}_{\rm xy}\)(λ) - \(\tilde {G}_{\rm XYZ}\)(λ)
Selecting a single solution
-
a)
\(\lambda \) = 0
$$\begin{array}{lll} \textrm{we get }\tilde{G}_{\min }= \tilde{G}_{\text{xymin}}\textrm{ therefore the point} \\ \quad\textrm{ of the Pareto }-\textrm{ front for }\lambda =0\, \textrm{is}\\ \quad\quad{} \left(\tilde{G}_{\text{XYZmax}},\text{ }\tilde{G}_{\text{xymin}}\right). \end{array} $$ -
b)
\(\lambda \) = 1
$$\begin{array}{lll} \textrm{we get }\tilde{G}_{\min }= \tilde{G}_{\text{XYZmin}}\textrm{ therefore the point} \\ \quad{} \textrm{ of the Pareto - front - front for }\lambda =1\, \textrm{is}\\ \quad\quad \left(\tilde{G}_{\text{XYZmin}}, \tilde{G}_{\text{xymax}}\right). \end{array} $$Consequently to minimize the error of the coordinates of the photo-planes we should select a point of the Pareto-front represented by the parameter λ ∗ \(<<\) 1, and vica versa to minimize the error of the space coordinates one should select a point of the Pareto-front with λ ∗ \(>>\) 0.This is therefore a trade-off job for the decision maker.
-
c)
compute the camera parameters \(\pi _i^*\) employing the selected \(\lambda ^*\) as \(\pi _i^*(\lambda ^*)\) for i \(=\) 1,2,\(\ldots \)m.
Selecting the Pareto-balanced solution
This solution can minimize the overall errors of the coordinates of photo-planes as well as the space coordinates. The point of the Pareto-front representing this solution is the closest point to the ideal point (0, 0), which represents zero error for \(\tilde {G}_{\rm xy}\) as well as for \(\tilde {G}_{\rm XYZ}\).
-
a)
use L 1norm
$$\underset{\lambda }{\min }\,\tilde{G}_{\text{xy}}(\lambda )+\tilde{G}_{\text{XYZ}}(\lambda )\longrightarrow \lambda ^{\text{*}}$$ -
b)
alternatively use L 2norm
$$\underset{\lambda }{\min }\sqrt{\left(\tilde{(G}_{\text{xy}}(\lambda )\right)^2+\left(\tilde{G}_{\text{XYZ}}(\lambda )\right)^2}\longrightarrow \lambda ^{\text{*}}$$
Rights and permissions
About this article
Cite this article
Paláncz, B., Awange, J.L. Pareto optimality solution of the multi-objective photogrammetric resection-intersection problem. Earth Sci Inform 6, 1–20 (2013). https://doi.org/10.1007/s12145-012-0107-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-012-0107-x