Monte Carlo Methods

• 177 Accesses

Part of the Springer Reference Naturwissenschaften book series (SRN)

Abstract

Monte Carlo methods deal with generating random variates from probability density functions in order to estimate unknown parameters or general functions of unknown parameters and to compute their expected values, variances and covariances. One generally works with the multivariate normal distribution due to the central limit theorem. However, if random variables with the normal distribution and random variables with a different distribution are combined, the normal distribution is not valid anymore. The Monte Carlo method is then needed to get the expected values, variances and covariances for the random variables with distributions different from the normal distribution.

The error propagation by the Monte Carlo method is discussed and methods for generating random variates from the multivariate normal distribution and from the multivariate uniform distribution. The Monte Carlo integration is presented leading to the sampling-importance-resampling (SIR) algorithm. Markov Chain Monte Carlo methods provide by the Metropolis algorithm and the Gibbs sampler additional ways of generating random variates. A special topic is the Gibbs sampler for computing and propagating large covariance matrices. This task arises when the geopotential is determined from satellite observations. The example of the minimal detectable outlier shows, how the Monte Carlo method is used to determine the power of a hypothesis test.

Keywords

• Bayesian statistics
• SIR algorithm
• Metropolis algorithm
• Gibbs sampler
• Markov Chain Monte Carlo method

This chapter is part of the series Handbuch der Geodäsie, volume “Mathematical Geodesy/ Mathematische Geodäsie”, edited by Willi Freeden, Kaiserslautern.

This contribution is based on the article: Koch [49].

This is a preview of subscription content, access via your institution.

References

1. Acko, B., Godina, A.: Verification of the conventional measuring uncertainty evaluation model with Monte Carlo simulation. Int. J. Simul. Model. 4, 76–84 (2005)

2. Alkhatib, H., Kutterer, H.: Estimation of measurement uncertainty of kinematic TLS observation process by means of Monte-Carlo methods. J. Appl. Geodesy 7, 125–133 (2013)

3. Alkhatib, H., Neumann, I., Kutterer, H.: Uncertainty modeling of random and systematic errors by means of Monte Carlo and fuzzy techniques. J. Appl. Geodesy 3, 67–79 (2009)

4. Alkhatib, H., Schuh, W.D.: Integration of the Monte Carlo covariance estimation strategy into tailored solution procedures for large-scale least squares problems. J. Geodesy 81, 53–66 (2007)

5. Arnold, S.: The Theory of Linear Models and Multivariate Analysis. Wiley, New York (1981)

6. Baarda, W.: Statistical Concepts in Geodesy. Publications on Geodesy, vol. 2, Nr. 4. Netherlands Geodetic Commission, Delft (1967)

7. Baarda, W.: A Testing Procedure for Use in Geodetic Networks. Publications on Geodesy, vol. 2, Nr. 5. Netherlands Geodetic Commission, Delft (1968)

8. Baselga, S.: Nonexistence of rigorous tests for multiple outlier detection in least-squares adjustment. J. Surv. Eng. 137, 109–112 (2011)

9. Beckman, R., Cook, R.: Outlier…. S. Technometrics 25, 119–149 (1983)

10. Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. B 36, 192–236 (1974)

11. Box, G., Muller, M.: A note on the generation of random normal deviates. Ann. Math. Stat. 29, 610–611 (1958)

12. Cramér, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)

13. Dagpunar, J.: Principles of Random Variate Generation. Clarendon Press, Oxford (1988)

14. Devroye, L.: Non-Uniform Random Variate Generation. Springer, Berlin (1986)

15. Dietrich, C.: Uncertainty, Calibration and Probability, 2nd edn. Taylor & Francis, Boca Raton (1991)

16. van Dorp, J., Kotz, S.: Generalized trapezoidal distributions. Metrika 58, 85–97 (2003)

17. Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10, 197–208 (2000)

18. Falk, M.: A simple approach to the generation of uniformly distributed random variables with prescribed correlations. Commun. Stat. Simul. 28, 785–791 (1999)

19. Gaida, W., Koch, K.R.: Solving the cumulative distribution function of the noncentral F-distribution for the noncentrality parameter. Sci. Bull. Stanislaw Staszic Univ. Min. Metall. Geodesy b.90(1024), 35–44 (1985)

20. Gelfand, A., Smith, A.: Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 398–409 (1990)

21. Gelman, A., Carlin, J., Stern, H., Rubin, D.: Bayesian Data Analysis, 2nd edn. Chapman and Hall, Boca Raton (2004)

22. Geman, D., Geman, S., Graffigne, C.: Locating texture and object boundaries. In: Devijver, P., Kittler, J. (eds.) Pattern Recognition Theory and Applications, pp. 165–177. Springer, Berlin (1987)

23. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 6, 721–741 (1984)

24. Geman, S., McClure, D.: Statistical methods for tomographic image reconstruction. Bull. Int. Stat. Inst. 52, 5–21 (1987)

25. Gentle, J.: Random Number Generation and Monte Carlo Methods, 2nd edn. Springer, Berlin (2003)

26. Gilks, W.: Full conditional distributions. In: Gilks, W., Richardson, S., Spiegelhalter, D. (eds.) Markov Chain Monte Carlo in Practice, pp. 75–88. Chapman and Hall, London (1996)

27. Golub, G., van Loan, C.: Matrix Computations. The Johns Hopkins University Press, Baltimore (1984)

28. Gordon, N., Salmond, D.: Bayesian state estimation for tracking and guidance using the bootstrap filter. J. Guid. Control. Dyn. 18, 1434–1443 (1995)

29. Gundlich, B., Koch, K.R., Kusche, J.: Gibbs sampler for computing and propagating large covariance matrices. J. Geod. 77, 514–528 (2003)

30. Gundlich, B., Kusche, J.: Monte Carlo integration for quasi–linear models. In: Xu, P., Liu, J., Dermanis, A. (eds.) VI Hotine-Marussi Symposium on Theoretical and Computational Geodesy, pp. 337–344. Springer, Berlin/Heidelberg (2008)

31. Guo, J.F., Ou, J.K., Yuan, Y.B.: Reliability analysis for a robust M-estimator. J. Surv. Eng. 137, 9–13 (2011)

32. Hennes, M.: Konkurrierende Genauigkeitsmaße – Potential und Schwächen aus der Sicht des Anwenders. Allgemeine Vermessungs-Nachrichten 114, 136–146 (2007)

33. Huber, P.: Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964)

34. ISO: Guide to the Expression of Uncertainty in Measurement. International Organization for Standardization, Geneve (1995)

35. JCGM: Evaluation of measurement data–supplement 2 to the “Guide to the Expression of Uncertainty in Measurement”–Extension to any number of output quantities. JCGM 102:2011. Joint Committee for Guides in Metrology. (2011). www.bipm.org/en/publications/guides

36. Kacker, R., Jones, A.: On use of Bayesian statistics to make the guide to the expression of uncertainty in measurement consistent. Metrologia 40, 235–248 (2003)

37. Kargoll, B.: On the Theory and Application of Model Misspecification Tests in Geodesy. Universität Bonn, Institut für Geodäsie und Geoinformation, Schriftenreihe 8, Bonn (2008)

38. Knight, N., Wang, J., Rizos, C.: Generalised measures of reliability for multiple outliers. J. Geod. 84, 625–635 (2010)

39. Koch, K.R.: Ausreißertests und Zuverlässigkeitsmaße. Vermessungswesen und Raumordnung 45, 400–411 (1983)

40. Koch, K.R.: Parameter Estimation and Hypothesis Testing in Linear Models, 2nd edn. Springer, Berlin (1999)

41. Koch, K.R.: Monte-Carlo-Simulation für Regularisierungsparameter. ZfV–Z Geodäsie, Geoinformation und Landmanagement 127, 305–309 (2002)

42. Koch, K.R.: Determining the maximum degree of harmonic coefficients in geopotential models by Monte Carlo methods. Studia Geophysica et Geodaetica 49, 259–275 (2005)

43. Koch, K.R.: Gibbs sampler by sampling-importance-resampling. J. Geod. 81, 581–591 (2007)

44. Koch, K.R.: Introduction to Bayesian Statistics, 2nd edn. Springer, Berlin (2007)

45. Koch, K.R.: Determining uncertainties of correlated measurements by Monte Carlo simulations applied to laserscanning. J. Appl. Geod. 2, 139–147 (2008)

46. Koch, K.R.: Evaluation of uncertainties in measurements by Monte Carlo simulations with an application for laserscanning. J. Appl. Geod. 2, 67–77 (2008)

47. Koch, K.R.: Minimal detectable outliers as measures of reliability. J. Geod. 89, 483–490 (2015)

48. Koch, K.R.: Bayesian statistics and Monte Carlo methods. J. Geod. Sci. 8, 18–29 (2018)

49. Koch, K.R.: Monte Carlo methods. GEM–Int. J. Geomath. 9(1), 117–143 (2018)

50. Koch, K.R., Brockmann, J.: Systematic effects in laser scanning and visualization by confidence regions. J. Appl. Geod. 10(4), 247–257 (2016)

51. Koch, K.R., Kargoll, B.: Outlier detection by the EM algorithm for laser scanning in rectangular and polar coordinate systems. J. Appl. Geod. 9, 162–173 (2015)

52. Koch, K.R., Kusche, J., Boxhammer, C., Gundlich, B.: Parallel Gibbs sampling for computing and propagating large covariance matrices. ZfV–Z Geodäsie, Geoinformation und Landmanagement 129, 32–42 (2004)

53. Koch, K.R., Schmidt, M.: Deterministische und stochastische Signale. Dümmler, Bonn (1994). http://skylab.itg.uni-bonn.de/koch/00_textbooks/Determ_u_stock_Signale.pdf

54. Kok, J.: Statistical analysis of deformation problems using Baarda’s testing procedures in: “Forty Years of Thought”. Anniversary Volume Occasion of Prof. Baarda’s 65th Birthday 2, 470–488 (1982). Delft

55. Kok, J.: On data snooping and multiple outlier testing. In: NOAA Technical Report NOS NGS 30. US Department of Commerce, National Geodetic Survey, Rockville (1984)

56. Lehmann, R.: Improved critical values for extreme normalized and studentized residuals in Gauss-Markov models. J. Geod. 86, 1137–1146 (2012)

57. Lehmann, R.: On the formulation of the alternative hypothesis for geodetic outlier detection. J. Geod. 87, 373–386 (2013)

58. Leonard, T., Hsu, J.: Bayesian Methods. Cambridge University Press, Cambridge (1999)

59. Liu, J.: Monte Carlo Strategies in Scientific Computing. Springer, Berlin (2001)

60. Marsaglia, G., Bray, T.: A convenient method for generating normal variables. SIAM Rev. 6, 260–264 (1964)

61. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

62. Nowel, K.: Application of Monte Carlo method to statistical testing in deformation analysis based on robust M-estimation. Surv. Rev. 48(348), 212–223 (2016)

63. O’Hagan, A.: Bayesian Inference, Kendall’s Advanced Theory of Statistics, vol. 2B. Wiley, New York (1994)

64. Pope, A.: The statistics of residuals and the detection of outliers. In: NOAA Technical Report NOS65 NGS1. US Department of Commerce, National Geodetic Survey, Rockville (1976)

65. Proszynski, W.: Another approach to reliability measures for systems with correlated observations. J. Geod. 84, 547–556 (2010)

66. Roberts, G., Smith, A.: Simple conditions for the convergence of the Gibbs sampler and Metropolis-Hastings algorithms. Stoch. Process. Appl. 49, 207–216 (1994)

67. Rubin, D.: Using the SIR algorithm to simulate posterior distributions. In: Bernardo, J., DeGroot, M., Lindley, D., Smith, A. (eds.) Bayesian Statistics, vol. 3, pp. 395–402. Oxford University Press, Oxford (1988)

68. Rubinstein, R.: Simulation and the Monte Carlo Method. Wiley, New York (1981)

69. Schader, M., Schmid, F.: Distribution function and percentage points for the central and noncentral F-distribution. Stat. Pap. 27, 67–74 (1986)

70. Siebert, B., Sommer, K.D.: Weiterentwicklung des GUM und Monte-Carlo-Techniken. tm–Technisches Messen 71, 67–80 (2004)

71. Smith, A., Gelfand, A.: Bayesian statistics without tears: a sampling-resampling perspective. Am. Stat. 46, 84–88 (1992)

72. Smith, A., Roberts, G.: Bayesian computation via the Gibbs sampler and related Markov Chain Monte Carlo methods. J. R. Stat. Soc. B 55, 3–23 (1993)

73. Staff of the Geodetic Computing Center, S.: The Delft approach for the design and computation of geodetic networks. In: “Forty Years of Thought”. Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday vol. 1, pp. 202–274. Delft (1982)

74. Teunissen, P.: Adjusting and testing with the models of the affine and similarity transformation. Manuscr. Geodaet. 11, 214–225 (1986)

75. Teunissen, P.: Testing theory; An introduction. MGP, Department of Mathematical Geodesy and Positioning, Delft University of Technology, Delft (2000)

76. Teunissen, P., de Bakker, P.: Single-receiver single-channel multi-frequency GNSS integrity: outliers, slips, and ionospheric disturbances. J. Geod. 87, 161–177 (2013)

77. Wilks, S.: Mathematical Statistics. Wiley, New York (1962)

78. Xu, P.: Random simulation and GPS decorrelation. J. Geod. 75, 408–423 (2001)

Acknowledgments

The author is indebted to Willi Freeden for his invitation to this contribution for HbMG and to Jan Martin Brockmann for his valuable comments.

Author information

Authors

Corresponding author

Correspondence to Karl-Rudolf Koch .

Rights and permissions

Reprints and Permissions

© 2019 Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature

Cite this entry

Koch, KR. (2019). Monte Carlo Methods. In: Freeden, W., Rummel, R. (eds) Handbuch der Geodäsie. Springer Reference Naturwissenschaften . Springer Spektrum, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-46900-2_100-2

• DOI: https://doi.org/10.1007/978-3-662-46900-2_100-2

• Published:

• Publisher Name: Springer Spektrum, Berlin, Heidelberg

• Print ISBN: 978-3-662-46900-2

• Online ISBN: 978-3-662-46900-2

• eBook Packages: Springer Referenz Naturwissenschaften

Chapter History

1. Latest

Monte Carlo Methods
Published:
23 February 2020

DOI: https://doi.org/10.1007/978-3-662-46900-2_100-2

2. Original

Monte Carlo Methods
Published:
31 May 2018

DOI: https://doi.org/10.1007/978-3-662-46900-2_100-1