Skip to main content
SpringerLink
Log in

A statistical model of roundoff error for varying length floating-point arithmetic

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

A model is presented which explains the behavior of the roundoff error in a result quantity when computing precision is varied. A set of hypotheses concerning this a posteriori model is tested in a matrix inversion algorithm. The characteristics of the algorithms where the error model is valid are discussed. As an application of the model, the usual estimation procedure for roundoff error consisting of comparing the results computed in two different precisions is analyzed statistically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. W. Daniel,Correcting approximations to multiple roots of polynomials, Num. Math. 9, 99–102, 1966.

    Google Scholar 

  2. P. Henrici,Discrete variable methods in ordinary differential equations, Wiley, New York, 1962.

    Google Scholar 

  3. P. Henrici,Error propagation for difference methods, Wiley, New York, 1964.

    Google Scholar 

  4. P. Henrici,Elements of numerical analysis, Wiley, New York, 1964.

    Google Scholar 

  5. T. E. Hull and J. R. Swenson,Tests of probabilistic models for propagation of roundoff errors, Comm. ACM, vol. 9, 108–113, 1966.

    Article  Google Scholar 

  6. D. E. Knuth,The art of computer programming, vol. 2, chapter 4, Addison Wesley, New York, 1969.

    Google Scholar 

  7. M. Tienari and V. Suokonautio,A set of procedures making real arithmetic of unlimited accuracy possible within Algol 60, BIT 6, 332–338, 1966.

    Google Scholar 

  8. M. Tienari,Varying length floating point arithmetic: a necessary tool for numerical analyst, Technical report no. CS 62, Computer Science Department, Stanford University, Stanford, California, 1967.

    Google Scholar 

  9. J. Wilkinson,Rounding errors in algebraic processes, Prentice Hall, Englewood Cliffs, N. J., 1963.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, University of Helsinki, Töölönkatu 11, Helsinki 10, Finland

    Martti Tienari

Authors
  1. Martti Tienari
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tienari, M. A statistical model of roundoff error for varying length floating-point arithmetic. BIT 10, 355–365 (1970). https://doi.org/10.1007/BF01934204

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01934204

Keywords

Search

Navigation