An optimality property of the integer least-squares estimator

Abstract.

A probabilistic justification is given for using the integer least-squares (LS) estimator. The class of admissible integer estimators is introduced and classical adjustment theory is extended by proving that the integer LS estimator is best in the sense of maximizing the probability of correct integer estimation. For global positioning system ambiguity resolution, this implies that the success rate of any other integer estimator of the carrier phase ambiguities will be smaller than or at the most equal to the ambiguity success rate of the integer LS estimator. The success rates of any one of these estimators may therefore be used to provide lower bounds for the LS success rate. This is particularly useful in case of the bootstrapped estimator.