Skip to main content

Monte Carlo methods

GEM - International Journal on Geomathematics volume 9, pages 117–143 (2018)Cite this article

Abstract

Monte Carlo methods deal with generating random variates from probability density functions in order to estimate unknown parameters or general functions of unknown parameters and to compute their expected values, variances and covariances. One generally works with the multivariate normal distribution due to the central limit theorem. However, if random variables with the normal distribution and random variables with a different distribution are combined, the normal distribution is not valid anymore. The Monte Carlo method is then needed to get the expected values, variances and covariances for the random variables with distributions different from the normal distribution. The error propagation by Monte Carlo methods is discussed and methods for generating random variates from the multivariate normal distribution and from the multivariate uniform distribution. The Monte Carlo integration is presented leading to the sampling–importance-resampling algorithm. Markov chain Monte Carlo methods provide by the Metropolis algorithm and the Gibbs sampler additional ways of generating random variates. A special topic is the Gibbs sampler for computing and propagating large covariance matrices. This task arises, for instance, when the geopotential is determined from satellite observations. The example of the minimal detectable outlier shows, how the Monte Carlo method is used to determine the power of a hypothesis test.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Acko, B., Godina, A.: Verification of the conventional measuring uncertainty evaluation model with Monte Carlo simulation. Int. J. Simul. Model. 4, 76–84 (2005)

    Article  Google Scholar 

  • Alkhatib, H., Kutterer, H.: Estimation of measurement uncertainty of kinematic TLS observation process by means of Monte-Carlo methods. J. Appl. Geod. 7, 125–133 (2013)

    Google Scholar 

  • Alkhatib, H., Schuh, W.D.: Integration of the Monte Carlo covariance estimation strategy into tailored solution procedures for large-scale least squares problems. J. Geod. 81, 53–66 (2007)

    Article  MATH  Google Scholar 

  • Alkhatib, H., Neumann, I., Kutterer, H.: Uncertainty modeling of random and systematic errors by means of Monte Carlo and fuzzy techniques. J. Appl. Geod. 3, 67–79 (2009)

    Google Scholar 

  • Arnold, S.: The Theory of Linear Models and Multivariate Analysis. Wiley, New York (1981)

    MATH  Google Scholar 

  • Baarda, W.: Statistical Concepts in Geodesy. Publications on Geodesy, Vol. 2, Nr. 4. Netherlands Geodetic Commission, Delft (1967)

    Google Scholar 

  • Baarda, W.: A Testing Procedure for Use in Geodetic Networks. Publications on Geodesy, Vol. 2, Nr. 5. Netherlands Geodetic Commission, Delft (1968)

    Google Scholar 

  • Baselga, S.: Nonexistence of rigorous tests for multiple outlier detection in least-squares adjustment. J. Surv. Eng. 137, 109–112 (2011)

    Article  Google Scholar 

  • Beckman, R., Cook, R.: Outlier....s. Technometrics 25, 119–149 (1983)

    MathSciNet  MATH  Google Scholar 

  • Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. B 36, 192–236 (1974)

    MathSciNet  MATH  Google Scholar 

  • Box, G., Muller, M.: A note on the generation of random normal deviates. Ann. Math. Stat. 29, 610–611 (1958)

    Article  MATH  Google Scholar 

  • Cramér, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)

    MATH  Google Scholar 

  • Dagpunar, J.: Principles of Random Variate Generation. Clarendon Press, Oxford (1988)

    MATH  Google Scholar 

  • Devroye, L.: Non-uniform Random Variate Generation. Springer, Berlin (1986)

    Book  MATH  Google Scholar 

  • Dietrich, C.: Uncertainty, Calibration and Probability, 2nd edn. Taylor & Francis, Boca Raton (1991)

    Google Scholar 

  • Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10, 197–208 (2000)

    Article  Google Scholar 

  • Falk, M.: A simple approach to the generation of uniformly distributed random variables with prescribed correlations. Commun. Stat. Simul. 28, 785–791 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  • Gaida, W., Koch, K.R.: Solving the cumulative distribution function of the noncentral \(F\)-distribution for the noncentrality parameter. Sci. Bull. Stanisl. Staszic Univ. Min. Metall. Geod. B 90(1024), 35–44 (1985)

    Google Scholar 

  • Gelfand, A., Smith, A.: Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 398–409 (1990)

    MathSciNet  Article  MATH  Google Scholar 

  • Gelman, A., Carlin, J., Stern, H., Rubin, D.: Bayesian Data Analysis, 2nd edn. Chapman and Hall, Boca Raton (2004)

    MATH  Google Scholar 

  • Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–6, 721–741 (1984)

    Article  MATH  Google Scholar 

  • Geman, S., McClure, D.: Statistical methods for tomographic image reconstruction. Bull. Int. Stat. Inst. 52–21(1), 5–21 (1987)

    MathSciNet  Google Scholar 

  • Geman, D., Geman, S., Graffigne, C.: Locating texture and object boundaries. In: Devijver, P., Kittler, J. (eds.) Pattern Recognition Theory and Applications, pp. 165–177. Springer, Berlin (1987)

    Chapter  Google Scholar 

  • Gentle, J.: Random Number Generation and Monte Carlo Methods, 2nd edn. Springer, Berlin (2003)

    MATH  Google Scholar 

  • Gilks, W.: Full conditional distributions. In: Gilks, W., Richardson, S., Spiegelhalter, D. (eds.) Markov Chain Monte Carlo in Practice, pp. 75–88. Chapman and Hall, London (1996)

    Google Scholar 

  • Golub, G., van Loan, C.: Matrix Computations. The Johns Hopkins University Press, Baltimore (1984)

    MATH  Google Scholar 

  • Gordon, N., Salmond, D.: Bayesian state estimation for tracking and guidance using the bootstrap filter. J. Guidance Control Dyn. 18, 1434–1443 (1995)

    Article  Google Scholar 

  • Gundlich, B., Kusche, J.: Monte Carlo integration for quasi-linear models. In: Xu, P., Liu, J., Dermanis, A. (eds.) VI Hotine-Marussi Symposium on Theoretical and Computational Geodesy, pp. 337–344. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Gundlich, B., Koch, K.R., Kusche, J.: Gibbs sampler for computing and propagating large covariance matrices. J. Geod. 77, 514–528 (2003)

    Article  MATH  Google Scholar 

  • Guo, J.F., Ou, J.K., Yuan, Y.B.: Reliability analysis for a robust M-estimator. J. Surv. Eng. 137, 9–13 (2011)

    Article  Google Scholar 

  • Hennes, M.: Konkurrierende Genauigkeitsmaße—Potential und Schwächen aus der Sicht des Anwenders. Allg. Vermess. Nachr. 114, 136–146 (2007)

    Google Scholar 

  • Huber, P.: Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964)

    MathSciNet  Article  MATH  Google Scholar 

  • ISO: Guide to the Expression of Uncertainty in Measurement. International Organization for Standardization, Geneve (1995)

  • JCGM: Evaluation of measurement data—supplement 2 to the “Guide to the expression of uncertainty in measurement”—extension to any number of output quantities. JCGM 102:2011. Joint Committee for Guides in Metrology (2011). www.bipm.org/en/publications/guides/

  • Kacker, R., Jones, A.: On use of Bayesian statistics to make the guide to the expression of uncertainty in measurement consistent. Metrologia 40, 235–248 (2003)

    Article  Google Scholar 

  • Kargoll, B.: On the theory and application of model misspecification tests in geodesy. Universität Bonn, Institut für Geodäsie und Geoinformation, Schriftenreihe 8, Bonn (2008)

  • Knight, N., Wang, J., Rizos, C.: Generalised measures of reliability for multiple outliers. J. Geod. 84, 625–635 (2010)

    Article  Google Scholar 

  • Koch, K.R.: Ausreißertests und Zuverlässigkeitsmaße. Vermess. Raumordn. 45, 400–411 (1983)

    Google Scholar 

  • Koch, K.R.: Parameter Estimation and Hypothesis Testing in Linear Models, 2nd edn. Springer, Berlin (1999)

    Book  MATH  Google Scholar 

  • Koch, K.R.: Monte-Carlo-Simulation für Regularisierungsparameter. ZfV-Z Geod. Geoinf. Landmanag. 127, 305–309 (2002)

    Google Scholar 

  • Koch, K.R.: Determining the maximum degree of harmonic coefficients in geopotential models by Monte Carlo methods. Stud. Geophys. Geod. 49, 259–275 (2005)

    Article  Google Scholar 

  • Koch, K.R.: Gibbs sampler by sampling–importance-resampling. J. Geod. 81, 581–591 (2007a)

    Article  MATH  Google Scholar 

  • Koch, K.R.: Introduction to Bayesian Statistics, 2nd edn. Springer, Berlin (2007b)

    MATH  Google Scholar 

  • Koch, K.R.: Determining uncertainties of correlated measurements by Monte Carlo simulations applied to laserscanning. J. Appl. Geod. 2, 139–147 (2008a)

    Google Scholar 

  • Koch, K.R.: Evaluation of uncertainties in measurements by Monte Carlo simulations with an application for laserscanning. J. Appl. Geod. 2, 67–77 (2008b)

    Google Scholar 

  • Koch, K.R.: Minimal detectable outliers as measures of reliability. J. Geod. 89, 483–490 (2015)

    Article  Google Scholar 

  • Koch, K.R.: Bayesian statistics and Monte Carlo methods. J. Geod. Sci. 8 (in preparation) (2018)

  • Koch, K.R., Brockmann, J.: Systematic effects in laser scanning and visualization by confidence regions. J. Appl. Geod. 10(4), 247–257 (2016)

    Google Scholar 

  • Koch, K.R., Kargoll, B.: Outlier detection by the EM algorithm for laser scanning in rectangular and polar coordinate systems. J. Appl. Geod. 9, 162–173 (2015)

    Google Scholar 

  • Koch, K.R., Schmidt, M.: Deterministische und stochastische Signale. Dümmler, Bonn (1994). ftp://skylab.itg.uni-bonn.de/koch/00_textbooks/Determ_u_stoch_Signale.pdf

  • Koch, K.R., Kusche, J., Boxhammer, C., Gundlich, B.: Parallel Gibbs sampling for computing and propagating large covariance matrices. ZfV-Z Geod. Geoinf. Landmanag. 129, 32–42 (2004)

    Google Scholar 

  • Kok, J.: Statistical analysis of deformation problems using Baarda’s testing procedures. In: “Forty Years of Thought”. Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday 2, 470–488 (1982). Delft

  • Kok, J.: On data snooping and multiple outlier testing. NOAA Technical Report NOS NGS 30. US Department of Commerce, National Geodetic Survey, Rockville (1984)

  • Lehmann, R.: Improved critical values for extreme normalized and studentized residuals in Gauss–Markov models. J. Geod. 86, 1137–1146 (2012)

    Article  Google Scholar 

  • Lehmann, R.: On the formulation of the alternative hypothesis for geodetic outlier detection. J. Geod. 87, 373–386 (2013)

    Article  Google Scholar 

  • Leonard, T., Hsu, J.: Bayesian Methods. Cambridge University Press, Cambridge (1999)

    MATH  Google Scholar 

  • Liu, J.: Monte Carlo Strategies in Scientific Computing. Springer, Berlin (2001)

    MATH  Google Scholar 

  • Marsaglia, G., Bray, T.: A convenient method for generating normal variables. SIAM Rev. 6, 260–264 (1964)

    MathSciNet  Article  MATH  Google Scholar 

  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    Article  Google Scholar 

  • Nowel, K.: Application of Monte Carlo method to statistical testing in deformation analysis based on robust M-estimation. Surv. Rev. 48(348), 212–223 (2016)

    Article  Google Scholar 

  • O’Hagan, A.: Bayesian Inference, Kendall’s Advanced Theory of Statistics, vol. 2B. Wiley, New York (1994)

    MATH  Google Scholar 

  • Pope, A.: The statistics of residuals and the detection of outliers. NOAA Technical Report NOS65 NGS1, US Department of Commerce, National Geodetic Survey, Rockville (1976)

  • Proszynski, W.: Another approach to reliability measures for systems with correlated observations. J. Geod. 84, 547–556 (2010)

    Article  Google Scholar 

  • Roberts, G., Smith, A.: Simple conditions for the convergence of the Gibbs sampler and Metropolis–Hastings algorithms. Stoch. Process. Appl. 49, 207–216 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  • Rubin, D.: Using the SIR algorithm to simulate posterior distributions. In: Bernardo, J., DeGroot, M., Lindley, D., Smith, A. (eds.) Bayesian Statistics 3, pp. 395–402. Oxford University Press, Oxford (1988)

    Google Scholar 

  • Rubinstein, R.: Simulation and the Monte Carlo Method. Wiley, New York (1981)

    Book  MATH  Google Scholar 

  • Schader, M., Schmid, F.: Distribution function and percentage points for the central and noncentral F-distribution. Stat. Pap. 27, 67–74 (1986)

    MATH  Google Scholar 

  • Siebert, B., Sommer, K.D.: Weiterentwicklung des GUM und Monte-Carlo-Techniken. Tech. Messen 71, 67–80 (2004)

    Article  Google Scholar 

  • Smith, A., Gelfand, A.: Bayesian statistics without tears: a sampling-resampling perspective. Am. Stat. 46, 84–88 (1992)

    MathSciNet  Google Scholar 

  • Smith, A., Roberts, G.: Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. J. R. Stat. Soc. B 55, 3–23 (1993)

    MathSciNet  MATH  Google Scholar 

  • Staff of the Geodetic Computing Center, S.: The Delft approach for the design and computation of geodetic networks. In: “Forty Years of Thought”, Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday 1, 202–274 (1982). Delft

  • Teunissen, P.: Adjusting and testing with the models of the affine and similarity transformation. Manuscr. Geod. 11, 214–225 (1986)

    Google Scholar 

  • Teunissen, P.: Testing theory: an introduction. MGP, Delft University of Technology, Department of Mathematical Geodesy and Positioning, Delft (2000)

  • Teunissen, P., de Bakker, P.: Single-receiver single-channel multi-frequency GNSS integrity: outliers, slips, and ionospheric disturbances. J. Geod. 87, 161–177 (2013)

    Article  Google Scholar 

  • van Dorp, J., Kotz, S.: Generalized trapezoidal distributions. Metrika 58, 85–97 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  • Wilks, S.: Mathematical Statistics. Wiley, New York (1962)

    MATH  Google Scholar 

  • Xu, P.: Random simulation and GPS decorrelation. J. Geod. 75, 408–423 (2001)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

The author is indebted to Willi Freeden for his invitation of this paper for GEM and to Jan Martin Brockmann for his valuable comments.

Author information

Authors and Affiliations

  1. Theoretical Geodesy Group, Institute for Geodesy and Geoinformation, University of Bonn, Nussallee 17, 53115, Bonn, Germany

    Karl-Rudolf Koch

Authors
  1. Karl-Rudolf Koch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karl-Rudolf Koch.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Koch, KR. Monte Carlo methods. Int J Geomath 9, 117–143 (2018). https://doi.org/10.1007/s13137-017-0101-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13137-017-0101-z

Keywords

  • Bayesian statistics
  • SIR algorithm
  • Metropolis algorithm
  • Gibbs sampler
  • Markov chain Monte Carlo methods

Mathematics Subject Classification

  • 62 Statistics