Skip to main content
Log in

Unrealistic models for realistic computations: how idealisations help represent mathematical structures and found scientific computing

  • Unrealistic Models
  • Published:
Synthese Aims and scope Submit manuscript

Abstract

We examine two very different approaches to formalising real computation, commonly referred to as “Computable Analysis” and “the BSS approach”. The main models of computation underlying these approaches—bit computation (or Type-2 Effectivity) and BSS, respectively—have also been put forward as appropriate foundations for scientific computing. The two frameworks offer useful computability and complexity results about problems whose underlying domain is an uncountable space (such as \({\mathbb {R}}\) or \({\mathbb {C}}\)). Since typically the problems dealt with in physical sciences, applied mathematics, economics, and engineering are also defined in uncountable domains, it is fitting that we choose between these two approaches a foundational framework for scientific computing. However, the models are incompatible as to their results. What is more, the BSS model is highly idealised and unrealistic; yet, it is the de facto implicit model in various areas of computational mathematics, with virtually no problems for the everyday practice. This paper serves three purposes. First, we attempt to delineate what the goal of developing foundations for scientific computing exactly is. We distinguish between two very different interpretations of that goal, and on the separate basis of each one, we put forward answers about the appropriateness of each framework. Second, we provide an account of the fruitfulness and wide use of BSS, despite its unrealistic assumptions. Third, according to one of our proposed interpretations of the scope of foundations, the target domain of both models is a certain mathematical structure (namely, floating-point arithmetic). In a clear sense, then, we are using idealised models to study a purely mathematical structure (actually a class of such structures). The third purpose is to point out and explain this intriguing (perhaps unique) phenomenon and attempt to make connections with the typical case of idealised models of empirical domains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Quoted and translated in Avigad and Brattka (2014).

  2. See Avigad and Brattka (2014), for a wonderfully written history of the development of real computability.

  3. See, Papayannopoulos (2018, ch. 4) for a detailed account of that view.

  4. “We say that a function is computable if its value is computable for any computable value of the variable. In other words, if \(\alpha \) is a computable number, one has to know how to compute the value of \(f(\alpha )\) with precision \(\frac{1}{n}\) for any n.” (Borel (1912), quote from Avigad and Brattka 2014; emphasis added).

    And, also:

    “We cannot define general computable functions of a real variable, since there is no general method of describing a real number, but we can define a computable function of a computable variable.” (Turing 1936, p. 254)

  5. Often referred to as a ‘Type-2 Turing Machine’, and the theory as ‘Type-2 Effectivity’ (abbrv. ‘TTE’).

  6. For example, on an alphabet containing at least {0,1,#}, if x is \(\frac{1}{3}\), \({\bar{x}}\) could be a finite string of the form:

    $$\begin{aligned} \#b\left( \frac{1}{2}\right) \#b\left( \frac{2}{4}\right) \#b\left( \frac{3}{8}\right) \#b\left( \frac{6}{16}\right) \#\ldots \end{aligned}$$

    where b(k) means the binary expansion of k.

  7. The above model is sometimes modified as follows. We assume that M has access, via its input tape, to a rational sequence that rapidly converges to x. Furthermore, we require from the machine to produce not only one rational approximation to f(x) with some precision n, but approximations for every \(n\in {\mathbb {N}}\); that is, to produce a rational sequence that rapidly converges to f(x). We then get an extensionally equivalent model, which dispenses with the extra input n. This modified model is in effect the one considered in Weihrauch (2000). Note that in this modification, it is necessary that the output tape is one-way, so that although a computation may need an infinite number of steps to be completed, we are nevertheless assured that after a finite number of steps we have always available a correct first approximation of the output (hence the equivalence with the oracle formulation).

  8. “A function cannot be computable, if it is not continuous at all computable values of the variable.” (Quoted in Avigad and Brattka 2014)

  9. Two implicit assumptions here are that the same naming system is used for both inputs, and that the output tape is one-way.

  10. This too was informally pointed out by Borel (1912): “The first problem in the theory of computable numbers is the problem of equality of two such numbers. If two computable numbers are unequal, this can obviously be noticed by computing both with sufficient precision, but in general it will not be known a priori.” (Quoted in Avigad and Brattka 2014)

  11. If instead of the entire \({\mathbb {R}}\), only the computable reals \({\mathbb {R}}_C\) are considered, then the undecidability of equality comes as a consequence of Rice’s theorem.

  12. Recall, for example, how the BSS-decidability of a set was defined in Sect. 2.2, based on computing its discontinuous characteristic function.

  13. Almost all commonly encountered continuous functions in analysis are CA-computable on an appropriate domain, including, for example, polynomials, trigonometric functions, \(\frac{1}{x}\), \(e^{x}\), \(\log x\), \(\root n \of {x}\), |x|, etc., and compositions thereof.

  14. For a formal proof of the non BSS-computability of exponentials, see, e.g., Brattka (2000, p. 8). See also Weihrauch (2000, p. 265) for the sketch of a similar proof. See Calvert et al. (2011), for various results of uncomputability in BSS, including transcendental and algebraic functions.

  15. For example, by using a machine implementing Newton’s method (Fig. 1), to approximate the zero of the function \(f(x)=x-a^2\), and obtain \(\sqrt{x}\approx a\). Nevertheless, such approximate algorithms may turn out problematic in case of compositions of functions, with a known desired output precision (e.g., \(f(x)=\sqrt{5+\sqrt{x}}\), with precision \(\varepsilon >0\)), since we need to be able to find the required precision of the input based on \(\varepsilon \) (Sect. 2.1.1). This is more straightforward in bit-computation (see Braverman and Cook 2006, p. 322 for an example) than in BSS.

  16. Recall that Borel had arrived at the same conclusions mainly by informal reasoning (see, fn.8,10).

  17. Although the motivation here is based on the properties of modern-day FPA implementations, interestingly, Turing himself had also regarded as useful a similar cost measure, based on the number of operations. The following quotation is from (1948)—the same paper where he also introduces the notion of condition number (long before FPA was implemented in actual computers).

    It is convenient to have a measure of the amount of work involved in a computing process, even though a very crude one [..] We might, for instance, count the number of additions, subtractions, multiplications, divisions, ...” (Turing 1948, p. 288).

  18. We explain what that means in Sect. 5.

  19. By ‘effective computation’ we mean computation performed by an idealised agent (machine or otherwise), following a systematic method and having unlimited space and time. Such a computation should be in principle simulable by pen-and-paper.

  20. We do not offer further arguments for that claim here, besides ascertaining that it is a generalised version of classical Turing computability, which admittedly regiments ‘effectivity’. We refer, however, the unconvinced reader to the discussion in Pégny (2016) for more conceptual arguments to that effect.

  21. See, for example, Weihrauch (2000, ch. 9), for a standard criticism.

  22. Other similar models are the Real-RAM and Real Number model. In fact, all these three models are practically equivalent, and the difference is mostly terminological. The former term is typically found in the computational geometry literature, whereas the latter typically in information-based complexity.

  23. Wozniakowski, coming from the area of information-based complexity, uses the term ‘real number model’.

  24. One could retort here that ordinary complexity theory is grounded in idealisations as well, apparent in the asymptotic formalism describing costs and complexity classes. I think, however, that this would be a mistaken take on this practice, and that asymptotic results should rather be understood as approximations. There is a conceptual difference between approximations and idealisations, but a discussion of it would be beyond the scope of this paper. For illuminating clarifications, one can see Norton (2012).

  25. Similarly to the classical case of discrete computability theory.

  26. We ignore conceptual complications arising from the possibility of hypercomputers, or other non-conventional forms of computing here.

  27. Their reliance on Turing machine formalisms is presumably harmless, for a tacit assumption is the validity of the Church-Turing thesis.

  28. As a historical aside, it seems that the seminal articles initiating the field of scientific computing, by means of introducing thorough discussions of computer errors, were von Neumann and Goldstine (1947), and Turing (1948).

  29. Such a framework would also need to provide an appropriate formal definition of ‘algorithm’, for in order to prove lower bounds of computational costs (and thereby characterise complexity classes), we need to generalise over all possible algorithms.

  30. Thanks to an anonymous referee for pressing on this point.

  31. One can also see the concern with implementational issues reflected in how the research in the fields of numerical analysis, computational geometry, and information-based complexity have changed through the last decades and become much more implementation-focused. Textbook presentations make this turn apparent too.

  32. For example, there is no discussion of representation or round-off errors in Weihrauch (2000).

  33. We are always referring to this particular formulation whenever we use the term ‘bit-computability’ in this paper.

  34. The distinction is in reality not as clear-cut as presented here. For example, computer algebra involves representations of numbers as well. Perhaps, the most broadly known computer algebra systems today are Mathematica and Maple, whereas Matlab is perhaps the most well-known environment for numerical calculations. Of course, many more such systems exist, and recently many of them are enhanced with packages that allow them to cross boundaries and perform both kinds of computations (Matlab being a recent case in point.)

  35. Symbolic algorithms, on the other hand, are not implemented in floating-point arithmetic.

  36. Hereafter, we consider BSS computations over \(R={\mathbb {R}}\).

  37. This may seem even more surprising, given some of this arithmetic’s bizarre properties, mentioned in the appendix.

  38. Woźniakowski’s discussion is about the use of the Real Number model in information-based complexity, but his results bear on the context of our discussion as well. We heavily draw upon his discussion, and Cucker’s (1999), in what follows.

  39. We remind the reader that the cost of a numerical algorithm, in general, is measured by the sum cost of all particular operation costs, expressed as a function of some input size n, in asymptotic form. If we take the worst-case cost for any particular algorithm that solves a particular problem, and then the minimum algorithmic cost (of known and unknown algorithms), then we can identify this cost with the complexity of the problem. We note in passing here the unfortunate fact about the lack of standard terminology in the literature, which can cause confusion. Many practitioners use the term ‘complexity’ both for algorithms and problems. Others, distinguish between saying that an algorithm has a time complexity, whereas a problem belongs in a complexity class. Whatever the terminology one may adopt, one should always bear in mind that there is conceptual difference between the two cases. Here, we choose to talk about the ‘cost’ of an algorithm, and the ‘complexity’ of a problem.

  40. See, e.g., Cucker (1999, sec.3). Extensions of the BSS model, allowing for machines with round-off errors, have also been proposed; see, e.g., Cucker and Smale (1999). Here, we only deal with exact BSS machines.

  41. It is important to keep in mind the distinction that ‘stability’ is a property of an algorithm, whereas ‘conditioning’ is a property of the problem. If a problem is ill-conditioned, a stable algorithm is not enough to save the situation; the computation error will be large. But if a problem is well-conditioned, then all stable algorithms will give results with small errors.

  42. The operator norm of a (matrix) linear operator expresses the largest value by which the operator A stretches the vector x. In this case, it is the square root of the largest eigenvalue of the symmetric matrix \(A^TA\). The definition of the condition number of the problem \(Ax=b\) is due to Turing (1948).

  43. The derivation is quick and straightforward; see, e.g., Cucker (1999, p. 114).

  44. For a thorough introduction to backward analysis, one can see, e.g., Corless and Fillion (2013).

  45. Typically, within the interval [\(10^{-8},10^{-2}\)] for most applications (Woźniakowski 1999).

  46. See, Woźniakowski (1999).

  47. To see this, consider eq.(2), and recall that the backward relative error in this case comprises the relative error due to FP rounding (bounded by u; eq.(5) in the appendix) and the relative error due to the stable algorithm itself (bounded by C).

  48. To put the contrast between the two interpretations in a different, maybe clearer, way: consider the possibility that usable quantum computers are finally developed and put to commercial use. Then, of their many initial applications, scientific computing is expected to be a prominent one (e.g., simulating quantum systems). A foundational framework for scientific computing in the first sense, then, would subsume this subfield too. Since, quantum computers do not go beyond Turing computability limits, (un)computability and complexity results proven in the context of Type-2 Effectivity will be relevant to quantum scientific computations as well. On the other hand, a foundational framework in the second sense would be rather irrelevant for quantum scientific computations, since the latter may not employ anything similar to FPA, or exhibit round-offs arising from digital representation (it remains to see what they will actually be using, and what their round-offs will look like).

  49. As a toy example, consider the high school algorithm for finding the roots of the quadratic \(ax^2+bx+c=0\):

    $$\begin{aligned} x=\dfrac{-b\pm \sqrt{b^2-4ac}}{2a} \end{aligned}$$
    (4)

    In developing an algorithm like this, there is no initial assumption to the effect that ab, and c are rational, or floating-point numbers, or Cauchy sequences of rationals, etc. The algorithm is thought of as if it’s operating over the exact reals, as elements of the field \({\mathbb {R}}\). Representational concerns arise at a later stage. (After all, when Newton, Gauss, et al. developed their methods, there were no digital computers around.) For a detailed discussion of this practice, and its implications for the conceptualisation of ‘algorithm’ in logic and mathematics, see, Papayannopoulos (2018, ch. 2–3).

  50. See, e.g., Maddy (1997, 1992) and Pincock (2012, 2014).

  51. There is a precise sense in which FPA is a mathematical structure; i.e., has an underlying set \({\mathbb {F}}\) endowed with specific operations and functions. See the appendix for a very brief exposition.

  52. What could such an equation look like? A typical example is the two-dimensional Laplace equation: \(\frac{\partial ^2u}{\partial x^2}+\frac{\partial ^2u}{\partial y^2}=0\). Given determinate boundary conditions along the the y-axis, u(0, y) and \(\frac{\partial u}{\partial x}(0,x)\), a unique solution u(xy) can be obtained. More specifically, if we let: \(u(0,y)=0\) and \(\frac{\partial u}{\partial x}(0,y)=g_n(y)=\frac{\sin (ny)}{n}\), the unique solution for the \(n^\text {th}\) problem (\(n=1,2,3,\ldots \)), is:

    $$\begin{aligned} u(x,y)=\frac{1}{2n^2}(e^{nx}-e^{-nx})\sin (ny) \end{aligned}$$

    The initial data u(0, y) and \(\frac{\partial u}{\partial x}(0,x)=g_n(y)\) can be kept in this case arbitrarily close to zero while n grows very large. Assume a determinate x (\(\ne 0\)); say, \(x=1\). The solution becomes:

    $$\begin{aligned} u(1,y)=\frac{1}{2n^2}(e^{n}-e^{-n})\sin (ny) \end{aligned}$$

    In this sort of situation, then, the solution can become extremely large—by means of increasing n as much as we want, thereby exploding the term \(\frac{e^n}{n}\)—while at the same time keeping \(|g_n(y)|=|\frac{\sin (ny)}{n}|<\varepsilon \) for any \(\varepsilon >0\) and any y (and even assuming \(u(0,y)=0\)). This means that no matter how accurate a measurement of the data could be, if we don’t know the exact value of \(\frac{\partial u}{\partial x}(0,y)\), it is impossible to approximate the solution u(1, y) to any degree of accuracy. This is an example of an ill-posed problem, and it’s far worse than a chaotic system. In the words of J. Hadamard:

    [..T]he mere replacing of the value zero for [\(g_n(y)\)] by the (however small) value [\(\frac{\sin (ny)}{n}\)] changes the solution not by very small but by very great quantities. Everything takes place, physically speaking, as if the knowledge of [the initial] data would not determine the unknown function. This shows how very differently things behave in this case and in those which correspond to physical questions. If a physical phenomenon were to be dependent on such an analytical problem as [the one here] it would appear to us as being governed by pure chance. (Hadamard 1923, p. 38)

    By the word ‘knowledge’ in this quote, Hadamard refers to experimental knowledge, within a certain approximation. In his analysis of the problem, he defines different orders of continuity, and pinpoints the source of this problem to the fact that the solution u(xy) of the problem is not continuous in the data u(0, y) and \(g_n(y)\) of any order at all.

    I took this example from Myrvold (1995). It is originally due to Hadamard (1923), and was later discussed in Courant and Hilbert (1962) too, in the context of characterising well-posed problems in mathematical physics. As Myrvold notices, the construction by Pour-El and Richards (1981) of the well-known Turing-uncomputable solution of the wave equation with computable initial data, exploits the same fact that underlies Hadamard’s example; namely, that the solution is not continuously dependent on the initial data.

  53. The other two are that a solution exists and is uniquely determined. Courant and Hilbert present Hadamard’s example as well, as a case of an ill-posed problem.

  54. We think that the context here does not allow for misinterpretation between the sense of ‘stability’ in C&H’s quote (which refers to a problem) and the sense of ‘numerical stability’ used earlier, which is a property of algorithms.

  55. Courant and Hilbert, for example, mention non-linear systems and quantum theory as cases in which studied problems reflect real phenomena, even though they are not properly-posed in the above sense.

  56. ”[T]here are those [hypotheses] that are quite natural and without which we could hardly do. It is difficult not to suppose that the influence of very distant bodies is quite negligible, that small movements obey a linear law, that the effect is a continuous function of the cause. [..] All these hypotheses form, so to speak, the common foundation of all theories in mathematical physics.” (Poincaré 1902, p. 524; emphasis added).

  57. Again, the aim of this discussion is not to delimit what meaningful representations in empirical science are. As Courant and Hilbert (1962, p. 230) say, there exist not well-posed problems whose study however can be of physical significance (a classic example might be the inverse heat equation). Still, however, for the purposes of this paper, we think it adequate to restrict attention to the success of idealised models that are concerned with well-posed problems.

  58. Or if, like Poincaré (1902), one is comfortable with cause-effect terminology, the “effect” depends continuously on the “cause” (see, fn.56).

  59. For example, the floating-point numbers can be seen as the “analogue” of experimental values. The latter are always measured with some error, which corresponds to the representation error of floating-point numbers, since in both cases a real value gets approximated and represented by a rational one. Round-off errors can be considered to corresponded to propagation of errors in experimental physics; that is, uncertainties in derived quantities caused by initial uncertainties in the experimentally measured quantities.

  60. In analogy with the Hadamard example, in fn.52, which too is mathematically correct.

  61. Assuming the validity of UCTT and ignoring the possibility of hypercomputation.

  62. We also hoped to achieve a further, indirect goal: that of drawing attention to a debate that has long been going on within the mathematics and computer science communities, yet remains, by and large, unnoticed by the philosophical community.

  63. To the author’s knowledge, the earliest explicit description of this model was in Preparata and Shamos (1985), under the name ‘Real-RAM’ (though, according to the Wikipedia entry “Real-RAM”, the model was originally formulated in 1978 in Shamos’s PhD thesis).

  64. This appendix provided only a very sketchy description of FPA, omitting important concepts related to errors, such as undeflow, overflow, rounding operations, and others. We refer the interested reader to various online resources as well as to the very illuminating discussion in Corless and Fillion (2013), upon which this appendix has also extensively drawn.

References

  • Aberth, O. (1980). Computable analysis. New York: McGraw-Hill.

    Google Scholar 

  • Avigad, J., & Brattka, V. (2014). Computability and analysis: the legacy of Alan Turing. In R. Downey (Ed.), Turing’s legacy: Developments from Turing’s ideas in logic (pp. 1–47). Cambridge: Cambridge University Press.

    Google Scholar 

  • Blum, L. (2004). Computing over the reals: Where Turing meets Newton. Notices of the AMS, 51(9), 1024–1034.

    Google Scholar 

  • Blum, L., Cucker, F., Shub, M., & Smale, S. (1997). Complexity and real computation. Berlin: Springer Science.

    Google Scholar 

  • Borel, É. (1912). Le calcul des intégrales définies. Journal de Mathématiques pures et appliquées, 8, 159–210.

    Google Scholar 

  • Brattka, V. (2000). The emperor’s new recursiveness: The epigraph of the exponential function in two models of computability. Words, languages and combinatorics, 3, 63–72.

    Google Scholar 

  • Brattka, V., Hertling, P., & Weihrauch, K. (2008). A tutorial on computable analysis. In S. B. Cooper, B. Löwe, & A. Sorbi (Eds.), New computational paradigms: Changing conceptions of what is computable (pp. 425–491). New York: Springer.

    Google Scholar 

  • Braverman, M. (2005). On the complexity of real functions. In Proceedings of the 46th annual IEEE symposium on foundations of computer science (FOCS’05), (pp. 155–164).

  • Braverman, M., & Cook, S. (2006). Computing over the reals: Foundations for scientific computing. Notices of the AMS, 53(3), 318–329.

    Google Scholar 

  • Calvert, W., Kramer, K., & Miller, R. (2011). Noncomputable functions in the Blum–Shub–Smale model. Logical Methods in Computer Science, 7(2), 1–20.

    Google Scholar 

  • Church, A. (1936). An unsolvable problem of elementary number theory. American Journal of Mathematics, 58(2), 345–363.

    Google Scholar 

  • Corless, R. M., & Fillion, N. (2013). A graduate introduction to numerical methods: From the viewpoint of backward error analysis. New York: Springer.

    Google Scholar 

  • Courant, R., & Hilbert, D. (1962). Methods of mathematical physics. Volume II: Partial differential equations. Hoboken: Wiley.

    Google Scholar 

  • Cucker, F. (1999). Real computations with fake numbers. In J. Wiedermann, P. van Emde Boas, & M. Nielsen (Eds.), Automata, languages and programming (pp. 55–73). Berlin: Springer.

    Google Scholar 

  • Cucker, F., & Smale, S. (1999). Complexity estimates depending on condition and round-off error. Journal of the ACM, 46(1), 113–184.

    Google Scholar 

  • Gherardi, G. (2008). Computability and incomputability of differential equations. In R. Lupacchini & G. Corsi (Eds.), Deduction, computation, experiment (pp. 223–242). Milano: Springer.

    Google Scholar 

  • Grzegorczyk, A. (1955). Computable functionals. Fundamenta Mathematicae, 42(19553), 168–202.

    Google Scholar 

  • Hadamard, J. (1923). Lectures on Cauchy’s problem in linear partial differential equations. New Haven: Yale University Press.

    Google Scholar 

  • Higham, N. J., Dennis, M. R., Glendinning, P., Martin, P. A., Santosa, F., & Tanner, J. (2015). The Princeton companion to applied mathematics. Princeton: Princeton University Press.

    Google Scholar 

  • Knuth, D. E. (1997). The art of computer programming: Seminumerical algorithms (3rd ed., Vol. 2). Boston: Addison-Wesley Longman Publishing Co., Inc.

    Google Scholar 

  • Ko, K.-I. (1991). Complexity theory of real functions. Cambridge: Birkhauser Boston Inc.

    Google Scholar 

  • Lacombe, D. (1955). Extension de la notion de fonction recursive aux fonctions dune ou plusieurs variables reelles. Comptes Rendus Hebdomadaires Des Seances De L’ Academie Des Sciences, 240(26), 2478–2480.

    Google Scholar 

  • Maddy, P. (1992). Indispensability and practice. The Journal of Philosophy, 89(6), 275–289.

    Google Scholar 

  • Maddy, P. (1997). Naturalism in mathematics. Oxford: Oxford University Press.

    Google Scholar 

  • Mazur, S. (1963). Computable analysis. Warsaw: Instytut Matematyczny Polskiej Akademi Nauk.

    Google Scholar 

  • Myrvold, W. C. (1994). Constructivism, computability, and physical theories. Ph. D. thesis, Boston University.

  • Myrvold, W. C. (1995). Computability in quantum mechanics. In W. D. Pauli-Schimanovich, E. Köhler, & F. Stadler (Eds.), Vienna circle institute yearbook (pp. 33–46). Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Norton, J. D. (2012). Approximation and idealization: Why the difference matters. Philosophy of Science, 79(2), 207–232.

    Google Scholar 

  • Novak, E. (1995). The real number model in numerical analysis. Journal of Complexity, 11(1), 57–73.

    Google Scholar 

  • Papayannopoulos, P. (2018). Computing, modelling, and scientific practice: Foundational analyses and limitations. Ph. D. thesis, University Of Western Ontario.

  • Pégny, M. (2016). How to make a meaningful comparison of models: The Church–Turing thesis over the reals. Minds and Machines, 26(4), 359–388.

    Google Scholar 

  • Pincock, C. (2012). Mathematics and scientific representation. Oxford: Oxford University Press.

    Google Scholar 

  • Pincock, C. (2014). How to avoid inconsistent idealizations. Synthese, 191(13), 2957–2972.

    Google Scholar 

  • Poincaré, H. (1902). Relations between experimental physics and mathematical physics. The Monist, 12(4), 516–543.

    Google Scholar 

  • Pour-El, M. B., & Richards, I. (1981). The wave equation with computable initial data such that its unique solution is not computable. Advances in Mathematics, 39(3), 215–239.

    Google Scholar 

  • Pour-El, M. B., & Richards, I. (1989). Computability in analysis and physics. Berlin: Springer.

    Google Scholar 

  • Preparata, F. P., & Shamos, M. I. (1985). Computational geometry: An introduction. New York: Springer.

    Google Scholar 

  • Smale, S. (1990). Some remarks on the foundations of numerical analysis. SIAM Review, 32(2), 211–220.

    Google Scholar 

  • Smale, S. (1997). Complexity theory and numerical analysis. Acta Numerica, 6, 523–551.

    Google Scholar 

  • Tisza, L. (1963). The conceptual structure of physics. Reviews of Modern Physics, 35, 151–184.

    Google Scholar 

  • Turing, A. M. (1936). On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, 42(1), 230–265.

    Google Scholar 

  • Turing, A. M. (1948). Rounding-off errors in matrix processes. The Quarterly Journal of Mechanics and Applied Mathematics, 1(1), 287–308.

    Google Scholar 

  • von Neumann, J., & Goldstine, H. H. (1947). Numerical inverting of matrices of high order. Bulletin of the American Mathematical Society, 53(11), 1021–1099.

    Google Scholar 

  • Weihrauch, K. (2000). Computable analysis: An introduction. Berlin: Springer Science.

    Google Scholar 

  • Woźniakowski, H. (1999). Why does information-based complexity use the real number model? Theoretical Computer Science, 219(1), 451–465.

    Google Scholar 

Download references

Acknowledgements

I am indebted to Wayne Myrvold for discussions on some of the topics dealt with here, and to Alexander Regueiro for thoroughly reading this paper and giving advice on its final form. I have also been greatly benefited from the comments and suggestions of two anonymous referees of Synthese. Part of this research was carried out while I held a post-doctoral fellowship at the Sidney M. Edelstein Centre for History and Philosophy of Science Technology and Medicine, at the Hebrew University of Jerusalem.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Philippos Papayannopoulos.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Floating-point arithmetic

Appendix: Floating-point arithmetic

Physical computing requires ways of representing the entities operated upon. When you are computing on a napkin the generous tip you should leave with your restaurant bill, you typically represent the added numbers in decimal notation. If you are trying to solve (perhaps on the same napkin) a physics problem with larger numbers (e.g., Avogadro’s number), you might employ scientific notation, in powers of 10, as it is convenient for computing only with the significant figures. Accordingly, digital, silicon-based computing requires representation of numbers in some convenient system. Many possible systems exist (see, e.g., chapter 4 in Knuth 1997), but in modern computers, the near ubiquitous representation system is Floating-Point Arithmetic (FPA).

By ‘representation’, in this context, we mean a mapping \(\delta :\subseteq \ R \rightarrow \varSigma ^*\) from a set of numbers (e.g., the reals, \(R={\mathbb {R}}\)) to a set of finite strings \(\varSigma ^*\) over a finite alphabet \(\varSigma \). A possible string from \(\varSigma ^*\) can be of the form: \(d_{n}d_{n-1}\ldots d_1d_0\) (\(d_i\in \varSigma \)). Additionally, it can also allow for decimal points, and be of the form:

$$\begin{aligned} d_{n}d_{n-1}\ldots d_1d_0.d_{-1}d_{-2}\ldots d_{-p+1}d_{-p} \end{aligned}$$

In floating-point arithmetic, numbers are represented in a way that is structurally reminiscent of scientific notation; that is, in the form:

$$\begin{aligned} \text {significan}{\textit{d}}\times \text {base}^\text {exponent} \end{aligned}$$

For example, assuming base 10, the number 152853.5047, can be represented in the above form as \(1.528535047\times 10^5\). Also, if the base is fixed and known in advance, then there is no need to include a representation of it as part of the actual FP string used by the system. The representation of the number, then, can comprise just two parts: the significand and the exponent (for example: 1.528535047|5).

More precisely, then, an n-bit FP word consists of two parts: a k-bit word, the significand (aka mantissa, or coefficient), representing a signed rational number, and an l-bit word (\(k+l=n\)), the exponent (aka characteristic, or scale), representing a signed integer.

In concrete applications, there are two main ways of representing a positive or negative number, either in the significand or in the exponent. The first is to use a designated bit to denote the sign and the remaining available bits for the magnitude. The second is to use all the available bits for the magnitude, and add a bias.

The most widely used system of FPA today is the IEEE-754 standard. This standard requires that the significand is normalised as a number of the form (Corless and Fillion 2013)

$$\begin{aligned} \pm 1.d_{-1}d_{-2}\ldots d_{-p+1}d_{-p} \end{aligned}$$

That is, there is one bit used for the sign and the rest for the magnitude. However, since the first digit is always 1, it can be implied and hidden, and hence all the bits (except the one for the sign) can be used for the fractional part of the number. For the exponent, on the other hand, the standard requires a biased representation. Furthermore, IEEE-754 defines five formats, in terms of precision, the most common of which are single precision (32-bit) and double precision (64-bit).

To make things clearer, we present in Fig. 5 a 32-bit example. In this precision format, the exponent can be either an 8-bit signed integer from \(-128\) to 127, or an 8-bit unsigned integer from 0 to 255 (i.e., biased); IEEE-754 requires the latter.

An important concept, with respect to the particular system, is the unit round-off, u. It is meant to express an upper bound on the relative error due to the representation and round-off errors of the particular FP system. A related term is the machine epsilon, \(\epsilon \). Unfortunately, there are no standard definitions of these ubiquitous terms, and approaches vary, even between the numerical analysis and the numerical software literature. In the former tradition, the two terms are often used interchangeably to refer to the maximum relative error due to the particular FP representation. In the latter, \(\epsilon \) is defined as the difference between 1 and the next larger floating-point number (equivalently, the spacing of floating-point numbers when the exponent is zero; see, e.g., Corless and Fillion 2013, p. 805). In this case, the unit round-off is defined as \(\epsilon /2\), where rounding to the nearest is assumed. In any case, u is meant as the maximum relative error due to FP representation (and round-offs).

Fig. 5
figure 5

Single-precision (32-bit) floating-point representation (base 2) of the number 0.15625, in accordance with the IEEE-754 standard. The first bit signifies the sign of the significand, the next 8 bits are assigned to the representation of the exponent, and the last 23 bits are assigned to the fractional part of the significand

It is clear that only countably many reals can be represented exactly by means of floating-point systems. The finite set \({\mathbb {F}}\) of floating-point numbers is the set of all numbers that have an exact floating-point representation. More precisely, for some set \(\varSigma _{fl}^*\) of floating-point words, and some floating-point representation mapping \(\delta _{fl}:\subseteq {\mathbb {R}}\rightarrow \varSigma _{fl}^*\),

$$\begin{aligned} {\mathbb {F}}=\{x\in {\mathbb {R}} | \delta _{fl}(x)=s,\text { }s\in \varSigma _{fl}^*\} \end{aligned}$$

Let \({\text {fl}}(x)\in {\mathbb {F}}\) be the floating-point number to which some real number \(x\in {\mathbb {R}}\) is rounded. Then, the following holds for the unit round-off, u (\(0<u<1\)):

$$\begin{aligned} \frac{|{\text {fl}}(x)-x|}{|x|}\leqslant u \end{aligned}$$
(5)

The exact operation \({\text {fl}}:\subset {\mathbb {R}}\rightarrow {\mathbb {F}}\) is determined by the particular FPA system employed.

In the body of this paper, we have repeatedly asserted that the considered models of computation are meant to model FPA as a mathematical structure. We obtain such a structure by endowing the set \({\mathbb {F}}\) with (counterparts of) the basic arithmetic operations (\(\pm ,\times ,\div \)), along with some additional ones related to rounding, such as \({\text {fl}}(x)\). The resulted structure exhibits peculiar mathematical properties. For instance, addition is generally not associative or distributive, the set \({\mathbb {F}}\) is not closed under addition, and it is neither a field, nor a ring or a group. These (and other) bizarre properties mainly occur due to the accumulation of representation and round-off errors. However, the exact mathematical properties depend on the specific representation format used. Therefore, significant effort has been put into developing satisfactory formats, in which the implementations of the above operations are precisely specified, and the accumulation of round-offs and representation errors are held at a minimum. The IEEE-754 standard, with double (64-bit) precision, guarantees many desirable properties, whence its popularity. It is exactly these messy details and cumbersome practicalities that make the majority of numerical analysts, and other practitioners, turn to easier models, such as BSS and Real-RAM.Footnote 64

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Papayannopoulos, P. Unrealistic models for realistic computations: how idealisations help represent mathematical structures and found scientific computing. Synthese 199, 249–283 (2021). https://doi.org/10.1007/s11229-020-02654-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11229-020-02654-8

Keywords

Navigation