Abstract
Rounding errors may change the flow of control in numerical computing processes by leading to changes in some branching decisions of the process. In this paper some general topological and measure theoretical results associated with this effect of rounding errors are derived. The approach is based on a model of numerical computation related to program schemes. Each computing process specified by the model computes a partial functionR n →R m using rational operations and simple tests on real numbers. The topological structure of input point sets inR n on which the computation follows the same execution path is studied. We also investigate input points, called sensitive, on which rounding errors may change the execution path followed. Conditions concerning computing processes are given which guarantee that the Lebesgue measure of sensitive points approaches zero (i.e. the probability of a branching error gets arbitrarily small) as the precision of the arithmetic increases. Most numerical processes used in practice are easily seen to satisfy these conditions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. V. Aho, J. E. Hopcroft and J. D. Ullman,The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, Mass., 1974.
F. L. Bauer,Computational graphs and rounding error, SIAM J. Numer. Anal. 11 (1974), 87–96.
H. Federer,Geometric Measure Theory, Springer, Berlin - Heidelberg- New York, 1969.
S. A. Greibach,Theory of Program Structures: Schemes, Semantics, Verification, Lect. Notes in Computer Science Vol. 36, Springer, Berlin - Heidelberg - New York, 1975.
D. E. Knuth,The Art of Computer Programming, Vol. 2: Seminumerical Algorithms, Addison-Wesley, Reading, Mass., 1969.
D. E. Knuth, Private communication to M. Tienari, 1972.
M. LaPorte and J. Vignes, Methode numérique de detection de la singularité d'une matrice, Numer. Math. 23 (1974), 73–81.
S. Linnainmaa,Taylor expansion of the accumulated rounding error, BIT 16 (1976), 146–160.
W. Miller,Computer search for numerical instability, J. Assoc. Comput. Mach. 22 (1975), 512–521.
W. Miller,Software for roundoff analysis, ACM Transactions on Mathematical Software 1 (1975), 108–128.
K. Nickel, Über die Notwendigkeit einer Fehlerschranken-Arithmetik für Rechnenautomaten, Numer. Math. 9 (1966), 69–79.
W. Rudin,Principles of Mathematical Analysis (2. ed.), McGraw-Hill, N.Y., 1964.
P. H. Sterbenz,Floating-Point Computation, Prentice-Hall, N.Y., 1974.
M. Tienari,On some topological properties of numerical algorithms, BIT 12 (1972), 409–433.
E. Ukkonen.On the effect of rounding errors on the flow of control in numerical processes, Report A-1977-7, Department of Computer Science, University of Helsinki, 1977.
A. van Wijngaarden,Numerical analysis as an independent science, BIT 6 (1966), 66–81.
J. H. Wilkinson,Rounding Errors in Algebraic Processes, Prentice-Hall, Englewood Cliffs, N.J., 1963.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ukkonen, E. An analysis of the effect of rounding errors on the flow of control in numerical processes. BIT 19, 116–133 (1979). https://doi.org/10.1007/BF01931229
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01931229