Motivation for This Work

Abstract

The fundamental laws of Nature that govern the physics of atoms, elementary particles, light quanta, and everything else that is small and light, have been found to fit in one grandiose framework called Quantum Mechanics. The internal logic of this framework appears to be counter intuitive and mysterious, while extremely coherent and effective. In this general introduction the author explains why he feels the need for expressing his views on how to understand these facts, even if many known scientists have already given extensive reviews of numerous ideas and approaches.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This book is about a theory, and about an interpretation. The theory, as it stands, is highly speculative. It is born out of dissatisfaction with the existing explanations of a well-established fact. The fact is that our universe appears to be controlled by the laws of quantum mechanics. Quantum mechanics looks weird, but nevertheless it provides a very solid basis for doing calculations of all sorts that explain the peculiarities of the atomic and sub-atomic world. The theory developed in this book starts from assumptions that, at first sight, seem to be natural and straightforward, and we think they can be very well defended.

Regardless whether the theory is completely right, partly right, or dead wrong, one may be inspired by the way it looks at quantum mechanics. We are assuming the existence of a definite ‘reality’ underlying quantum mechanical descriptions. The assumption that this reality exists leads to a rather down-to-earth interpretation of what quantum mechanical calculations are telling us. The interpretation works beautifully and seems to remove several of the difficulties encountered in other descriptions of how one might interpret the measurements and their findings. We propose this interpretation that, in our eyes, is superior to other existing dogmas.

However, numerous extensive investigations have provided very strong evidence that the assumptions that went into our theory cannot be completely right. The earliest arguments came from von Neumann [86], but these were later hotly debated [6, 15, 49]. The most convincing arguments came from John S. Bell’s theorem, phrased in terms of inequalities that are supposed to hold for any local classical interpretation of quantum mechanics, but are strongly violated by quantum mechanics. Later, many other variations were found of Bell’s basic idea, some even more powerful. We will discuss these repeatedly, and at length, in this work. Basically, they all seemed to point in the same direction: from these theorems, it was concluded by most researchers that the laws of Nature cannot possibly be described by a local, deterministic automaton. So why this book?

There are various reasons why the author decided to hold on to his assumptions anyway. The first reason is that they fit very well with the quantum equations of various very simple models. It looks as if Nature is telling us: “wait, this approach is not so bad at all!”. The second reason is that one could regard our approach simply as a first attempt at a description of Nature that is more realistic than other existing approaches. We can always later decide to add some twists that introduce indeterminism, in a way more in line with the afore mentioned theorems; these twists could be very different from what is expected by many experts, but anyway, in that case, we could all emerge out of this fight victorious. Perhaps there is a subtle form of non-locality in the cellular automata, perhaps there is some quantum twist in the boundary conditions, or you name it. Why should Bell’s inequalities forbid me to investigate this alley? I happen to find it an interesting one.

But there is a third reason. This is the strong suspicion that all those “hidden variable models” that were compared with thought experiments as well as real experiments, are terribly naive.¹ Real deterministic theories have not yet been excluded. If a theory is deterministic all the way, it implies that not only all observed phenomena, but also the observers themselves are controlled by deterministic laws. They certainly have no ‘free will’, their actions all have roots in the past, even the distant past. Allowing an observer to have free will, that is, to reset his observation apparatus at will without even infinitesimal disturbances of the surrounding universe, including modifications in the distant past, is fundamentally impossible.² The notion that, also the actions by experimenters and observers are controlled by deterministic laws, is called superdeterminism. When discussing these issues with colleagues the author got the distinct impression that it is here that the ‘no-go’ theorems they usually come up with, can be put in doubt.³

We hasten to add that this is not the first time that this remark was made [50, 51]. Bell noticed that superdeterminism could provide a loophole around his theorem, but as most researchers also today, he was quick to dismiss it as “absurd”. As we hope to be able to demonstrate, however, superdeterminism may not quite be as absurd as it seems.⁴

In any case, realizing these facts sheds an interesting new light on our questions, and the author was strongly motivated just to carry on.

Having said all this, I do admit that what we have is still only a theory. It can and will be criticized and attacked, as it already was. I know that some readers will not be convinced. If, in the mind of some others, I succeed to generate some sympathy, even enthusiasm for these ideas, then my goal has been reached. In a somewhat worse scenario, my ideas will be just used as an anvil, against which other investigators will sharpen their own, superior views.

In the mean time, we are developing mathematical notions that seem to be coherent and beautiful. Not very surprisingly, we do encounter some problems in the formalism as well, which we try to phrase as accurately as possible. They do indicate that the problem of generating quantum phenomena out of classical equations is actually quite complex. The difficulty we bounce into is that, although all classical models allow for a reformulation in terms of some ‘quantum’ system, the resulting quantum system will often not have a Hamiltonian that is local and properly bounded from below. It may well be that models that do produce acceptable Hamiltonians will demand inclusion of non-perturbative gravitational effects, which are indeed difficult and ill-understood at present.

It is unlikely, in the mind of the author, that these complicated schemes can be wiped off the table in a few lines, as is asserted by some.⁵ Instead, they warrant intensive investigation. As stated, if we can make the theories more solid, they would provide extremely elegant foundations that underpin the Cellular Automaton Interpretation of quantum mechanics. It will be shown in this book that we can arrive at Hamiltonians that are almost both local and bounded from below. These models are like quantized field theories, which also suffer from mathematical imperfections, as is well-known. We claim that these imperfections, in quantum field theory on the one hand, and our way of handling quantum mechanics on the other, may actually be related to one another.

Furthermore, one may question why we would have to require locality of the quantum model at all, as long as the underlying classical model is manifestly local by construction. What we exactly mean by all this will be explained, mostly in Part II where we allow ourselves to perform detailed calculations.

1 Why an Interpretation Is Needed

The discovery of quantum mechanics may well have been the most important scientific revolution of the 20th century. Not only the world of atoms and subatomic particles appears to be completely controlled by the rules of quantum mechanics, but also the worlds of solid state physics, chemistry, thermodynamics, and all radiation phenomena can only be understood by observing the laws of the quanta. The successes of quantum mechanics are phenomenal, and furthermore, the theory appears to be reigned by marvellous and impeccable internal mathematical logic.

Not very surprisingly, this great scientific achievement also caught the attention of scientists from other fields, and from philosophers, as well as the public in general. It is therefore perhaps somewhat curious that, even after nearly a full century, physicists still do not quite agree on what the theory tells us—and what it does not tell us—about reality.

The reason why quantum mechanics works so well is that, in practically all areas of its applications, exactly what reality means turns out to be immaterial. All that this theory⁶ says, and that needs to be said, is about the reality of the outcomes of an experiment. Quantum mechanics tells us exactly what one should expect, how these outcomes may be distributed statistically, and how these can be used to deduce details of its internal parameters. Elementary particles are one of the prime targets here. A theory⁶ has been arrived at, the so-called Standard Model, that requires the specification of some 25 internal constants of Nature, parameters that cannot be predicted using present knowledge. Most of these parameters could be determined from the experimental results, with varied accuracies. Quantum mechanics works flawlessly every time.

So, quantum mechanics, with all its peculiarities, is rightfully regarded as one of the most profound discoveries in the field of physics, revolutionizing our understanding of many features of the atomic and sub-atomic world.

But physics is not finished. In spite of some over-enthusiastic proclamations just before the turn of the century, the Theory of Everything has not yet been discovered, and there are other open questions reminding us that physicists have not yet done their job completely. Therefore, encouraged by the great achievements we witnessed in the past, scientists continue along the path that has been so successful. New experiments are being designed, and new theories are developed, each with ever increasing ingenuity and imagination. Of course, what we have learned to do is to incorporate every piece of knowledge gained in the past, in our new theories, and even in our wilder ideas.

But then, there is a question of strategy. Which roads should we follow if we wish to put the last pieces of our jig-saw puzzle in place? Or even more to the point: what do we expect those last jig-saw pieces to look like? And in particular: should we expect the ultimate future theory to be quantum mechanical?

It is at this point that opinions among researchers vary, which is how it should be in science, so we do not complain about this. On the contrary, we are inspired to search with utter concentration precisely at those spots where no-one else has taken the trouble to look before. The subject of this book is the ‘reality’ behind quantum mechanics. Our suspicion is that it may be very different from what can be read in most text books. We actually advocate the notion that it might be simpler than anything that can be read in the text books. If this is really so, this might greatly facilitate our quest for better theoretical understanding.

Many of the ideas expressed and worked out in this treatise are very basic. Clearly, we are not the first to advocate such ideas. The reason why one rarely hears about the obvious and simple observations that we will make, is that they have been made many times, in the recent and the more ancient past [86], and were subsequently categorically dismissed.

The primary reason why they have been dismissed is that they were unsuccessful; classical, deterministic models that produce the same results as quantum mechanics were devised, adapted and modified, but whatever was attempted ended up looking much uglier than the original theory, which was plain quantum mechanics with no further questions asked. The quantum mechanical theory describing relativistic, subatomic particles is called quantum field theory (see Part II, Chap. 20), and it obeys fundamental conditions such as causality, locality and unitarity. Demanding all of these desirable properties was the core of the successes of quantum field theory, and that eventually gave us the Standard Model of the sub-atomic particles. If we try to reproduce the results of quantum field theory in terms of some deterministic underlying theory, it seems that one has to abandon at least one of these demands, which would remove much of the beauty of the generally accepted theory; it is much simpler not to do so, and therefore, as for the requirement of the existence of a classical underlying theory, one usually simply drops that.

Not only does it seem to be unnecessary to assume the existence of a classical world underlying quantum mechanics, it seems to be impossible also. Not very surprisingly, researchers turn their heads in disdain, but just before doing so, there was one more thing to do: if, invariably, deterministic models that were intended to reproduce typically quantum mechanical effects, appear to get stranded in contradictions, maybe one can prove that such models are impossible. This may look like the more noble alley: close the door for good.

A way to do this was to address the famous Gedanken experiment designed by Einstein, Podolsky and Rosen [33, 53]. This experiment suggested that quantum particles are associated with more than just a wave function; to make quantum mechanics describe ‘reality’, some sort of ‘hidden variables’ seemed to be needed. What could be done was to prove that such hidden variables are self-contradictory. We call this a ‘no-go theorem’. The most notorious, and most basic, example was Bell’s theorem [6], as we already mentioned. Bell studied the correlations between measurements of entangled particles, and found that, if the initial state for these particles is chosen to be sufficiently generic, the correlations found at the end of the experiment, as predicted by quantum mechanics, can never be reproduced by information carriers that transport classical information. He expressed this in terms of the so-called Bell inequalities, later extended as CHSH inequality [20]. They are obeyed by any classical system but strongly violated by quantum mechanics. It appeared to be inevitable to conclude that we have to give up producing classical, local, realistic theories. They don’t exist.

So why the present treatise? Almost every day, we receive mail from amateur physicists telling us why established science is all wrong, and what they think a “theory of everything” should look like. Now it may seem that I am treading in their foot steps. Am I suggesting that nearly one hundred years of investigations of quantum mechanics have been wasted? Not at all. I insist that the last century of research has led to magnificent results, and that the only thing missing so-far was a more radical description of what has been found. Not the equations were wrong, not the technology, but only the wording of what is often referred to as the Copenhagen Interpretation, should be replaced. Up to now, the theory of quantum mechanics consisted of a set of very rigorous rules as to how amplitudes of wave functions refer to the probabilities for various different outcomes of an experiment. It was stated emphatically that they are not referring to ‘what is really happening’. One should not ask what is really happening, one should be content with the predictions concerning the experimental results. The idea that no such ‘reality’ should exist at all sounds mysterious. It is my intention to remove every single bit of mysticism from quantum theory, and we intend to deduce facts about reality anyway.

Quantum mechanics is one of the most brilliant results of one century of science, and it is not my intention to replace it by some mutilated version, no matter how slight the mutilation would be. Most of the text books on quantum mechanics will not need the slightest revision anywhere, except perhaps when they state that questions about reality are forbidden. All practical calculations on the numerous stupefying quantum phenomena can be kept as they are. It is indeed in quite a few competing theories about the interpretation of quantum mechanics where authors are led to introduce non-linearities in the Schrödinger equation or violations of the Born rule that will be impermissible in this work.

As for ‘entangled particles’, since it is known how to produce such states in practice, their odd-looking behaviour must be completely taken care of in our approach.

The ‘collapse of the wave function’ is a typical topic of discussion, where several researchers believe a modification of Schrödinger’s equation is required. Not so in this work, as we shall explain. We also find surprisingly natural answers to questions concerning ‘Schrödinger’s cat’, and the ‘arrow of time’.

And as of ‘no-go theorems’, this author has seen several of them, standing in the way of further progress. One always has to take the assumptions into consideration, just as the small print in a contract.

2 Outline of the Ideas Exposed in Part I

Our starting point will be extremely simple and straightforward, in fact so much so that some readers may simply conclude that I am losing my mind. However, with questions of the sort I will be asking, it is inevitable to start at the very basic beginning. We start with just any classical system that vaguely looks like our universe, with the intention to refine it whenever we find this to be appropriate. Will we need non-local interactions? Will we need information loss? Must we include some version of a gravitational force? Or will the whole project run astray? We won’t know unless we try.

The price we do pay seems to be a modest one, but it needs to be mentioned: we have to select a very special set of mutually orthogonal states in Hilbert space that are endowed with the status of being ‘real’. This set consists of the states the universe can ‘really’ be in. At all times, the universe chooses one of these states to be in, with probability 1, while all others carry probability 0. We call these states ontological states, and they form a special basis for Hilbert space, the ontological basis. One could say that this is just wording, so this price we pay is affordable, but we will assume this very special basis to have special properties. What this does imply is that the quantum theories we end up with all form a very special subset of all quantum theories. This then, could lead to new physics, which is why we believe our approach will warrant attention: eventually, our aim is not just a reinterpretation of quantum mechanics, but the discovery of new tools for model building.

One might expect that our approach, having such a precarious relationship with both standard quantum mechanics and other insights concerning the interpretation of quantum mechanics, should quickly strand in contradictions. This is perhaps the more remarkable observation one then makes: it works quite well! Several models can be constructed that reproduce quantum mechanics without the slightest modification, as will be shown in much more detail in Part II. All our simple models are quite straightforward. The numerous responses I received, saying that the models I produce “somehow aren’t real quantum mechanics” are simply mistaken. They are really quantum mechanical. However, I will be the first to remark that one can nonetheless criticize our results: the models are either too simple, which means they do not describe interesting, interacting particles, or they seem to exhibit more subtle defects. In particular, reproducing realistic quantum models for locally interacting quantum particles along the lines proposed, has as yet shown to be beyond what we can do. As an excuse I can only plead that this would require not only the reproduction of a complete, renormalizable quantum field theoretical model, but in addition it may well demand the incorporation of a perfectly quantized version of the gravitational force, so indeed it should not surprise anyone that this is hard.

Numerous earlier attempts have been made to find holes in the arguments initiated by Bell, and corroborated by others. Most of these falsification arguments have been rightfully dismissed. But now it is our turn. Knowing what the locality structure is expected to be in our models, and why we nevertheless think they reproduce quantum mechanics, we can now attempt to locate the cause of this apparent disagreement. Is the fault in our models or in the arguments of Bell c.s.? What could be the cause of this discrepancy? If we take one of our classical models, what goes wrong in a Bell experiment with entangled particles? Were assumptions made that do not hold? Do particles in our models perhaps refuse to get entangled? This way, we hope to contribute to an ongoing discussion.

The aim of the present study is to work out some fundamental physical principles. Some of them are nearly as general as the fundamental, canonical theory of classical mechanics. The way we deviate from standard methods is that, more frequently than usual, we introduce discrete kinetic variables. We demonstrate that such models not only appear to have much in common with quantum mechanics. In many cases, they are quantum mechanical, but also classical at the same time. Some of our models occupy a domain in between classical and quantum mechanics, a domain often thought to be empty.

Will this lead to a revolutionary alternative view on what quantum mechanics is? The difficulties with the sign of the energy and the locality of the effective Hamiltonians in our theories have not yet been settled. In the real world there is a lower bound for the total energy, so that there is a vacuum state. The subtleties associated with that are postponed to Part II, since they require detailed calculations. In summary: we suspect that there will be several ways to overcome this difficulty, or better still, that it can be used to explain some of the apparent contradictions in quantum mechanics.

The complete and unquestionable answers to many questions are not given in this treatise, but we are homing in to some important observations. As has happened in other examples of “no-go theorems”, Bell and his followers did make assumptions, and in their case also, the assumptions appeared to be utterly reasonable. Nevertheless we now suspect that some of the premises made by Bell may have to be relaxed. Our theory is not yet complete, and a reader strongly opposed to what we are trying to do here, may well be able to find a stick that seems suitable to destroy it. Others, I hope, will be inspired to continue along this path.

We invite the reader to draw his or her own conclusions. We do intend to achieve that questions concerning the deeper meanings of quantum mechanics are illuminated from a new perspective. This we do by setting up models and by doing calculations in these models. Now this has been done before, but most models I have seen appear to be too contrived, either requiring the existence of infinitely many universes all interfering with one another, or modifying the equations of quantum mechanics, while the original equations seem to be beautifully coherent and functional.

Our models suggest that Einstein may have been right, when he objected the conclusions drawn by Bohr and Heisenberg. It may well be that, at its most basic level, there is no randomness in Nature, no fundamentally statistical aspect to the laws of evolution. Everything, up to the most minute detail, is controlled by invariable laws. Every significant event in our universe takes place for a reason, it was caused by the action of physical law, not just by chance. This is the general picture conveyed by this book. We know that it looks as if Bell’s inequalities have refuted this possibility, in particular because we are not prepared to abandon notions of locality, so yes, they raise interesting and important questions that we shall address at various levels.

It may seem that I am employing rather long arguments to make my point.⁷ The most essential elements of our reasoning will show to be short and simple, but just because I want chapters of this book to be self-sustained, well readable and understandable, there will be some repetitions of arguments here and there, for which I apologize. I also apologize for the fact that some parts of the calculations are at a very basic level; the hope is that this will also make this work accessible for a larger class of scientists and students.

The most elegant way to handle quantum mechanics in all its generality is Dirac’s bra-ket formalism (Sect. 1.6). We stress that Hilbert space is a central tool for physics, not only for quantum mechanics. It can be applied in much more general systems than the standard quantum models such as the hydrogen atom, and it will be used also in completely deterministic models (we can even use it in Newton’s description of the planetary system, see Sect. 5.7.1).

In any description of a model, one first chooses a basis in Hilbert space. Then, what is needed is a Hamiltonian, in order to describe dynamics. A very special feature of Hilbert space is that one can use any basis one likes. The transformation from one basis to another is a unitary transformation, and we shall frequently make use of such transformations. Everything written about this in Sects. 1.6, 3.1 and 11.3 is completely standard.

In Part I of the book, we describe the philosophy of the Cellular Automaton Interpretation (CAI) without too many technical calculations. After the Introduction, we first demonstrate the most basic prototype of a model, the Cogwheel Model, in Chap. 2.

In Chaps. 3 and 4, we begin to deal with the real subject of this research: the question of the interpretation of quantum mechanics. The standard approach, referred to as the Copenhagen Interpretation, is dealt with very briefly, emphasizing those points where we have something to say, in particular the Bell and the CHSH inequalities.

Subsequently, we formulate as clearly as possible what we mean with deterministic quantum mechanics. The Cellular Automaton Interpretation of quantum mechanics (Chaps. 4 and 5) must sound as a blasphemy to some quantum physicists, but this is because we do not go along with some of the assumptions usually made. Most notably, it is the assumption that space-like correlations in the beables of this world cannot possibly generate the ‘conspiracy’ that seems to be required to violate Bell’s inequality. We derive the existence of such correlations.

We end Chap. 3 with one of the more important fundamental ideas of the CAI: our hidden variables do contain ‘hidden information’ about the future, notably the settings that will be chosen by Alice an Bob, but it is fundamentally non-local information, impossible to harvest even in principle (Sect. 3.7.1). This should not be seen as a violation of causality.

Even if it is still unclear whether or not the results of these correlations have a conspiratory nature, one can base a useful and functional interpretation doctrine from the assumption that the only conspiracy the equations perform is to fool some of today’s physicists, while they act in complete harmony with credible sets of physical laws. The measurement process and the collapse of the wave function are two riddles that are completely resolved by this assumption, as will be indicated.

We hope to inspire more physicists to investigate these possibilities, to consider seriously the possibility that quantum mechanics as we know it is not a fundamental, mysterious, impenetrable feature of our physical world, but rather an instrument to statistically describe a world where the physical laws, at their most basic roots, are not quantum mechanical at all. Sure, we do not know how to formulate the most basic laws at present, but we are collecting indications that a classical world underlying quantum mechanics does exist.

Our models show how to put quantum mechanics on hold when we are constructing models such as string theory and “quantum” gravity, and this may lead to much improved understanding of our world at the Planck scale. Many chapters are reasonably self sustained; one may choose to go directly to the parts where the basic features of the Cellular Automaton Interpretation (CAI) are exposed, Chaps. 3–10, or look at the explicit calculations done in Part II.

In Chap. 5.2, we display the rules of the game. Readers might want to jump to this chapter directly, but might then be mystified by some of our assertions if one has not yet been exposed to the general working philosophy developed in the previous chapters. Well, you don’t have to take everything for granted; there are still problems unsolved, and further alleys to be investigated. They are in Chap. 9, where it can be seen how the various issues show up in calculations.

Part II of this book is not intended to impress the reader or to scare him or her away. The explicit calculations carried out there are displayed in order to develop and demonstrate our calculation tools; only few of these results are used in the more general discussions in the first part. Just skip them if you don’t like them.

3 A 19th Century Philosophy

Let us go back to the 19th century. Imagine that mathematics were at a very advanced level, but nothing of the 20th century physics was known. Suppose someone had phrased a detailed hypothesis about his world being a cellular automaton.⁸ The cellular automaton will be precisely defined in Sect. 5.1 and in Part II; for now, it suffices to characterize it by the requirement that the states Nature can be in are given by sequences of integers. The evolution law is a classical algorithm that tells unambiguously how these integers evolve in time. Quantum mechanics does not enter; it is unheard of. The evolution law is sufficiently non-trivial to make our cellular automaton behave as a universal computer [37, 61]. This means that, at its tiniest time and distance scale, initial states could be chosen such that any mathematical equation can be solved with it. This means that it will be impossible to derive exactly how the automaton will behave at large time intervals; it will be far too complex.

Mathematicians will realize that one should not even try to deduce exactly what the large-time and large-distance properties of this theory will be, but they may decide to try something else. Can one, perhaps, make some statistical statements about the large scale behaviour?

In first approximation, just white noise may be seen to emerge, but upon closer inspection, the system may develop non-trivial correlations in its series of integers; some of the correlation functions may be calculable, just the way these may be calculated in a Van der Waals gas. We cannot rigorously compute the trajectories of individual molecules in this gas, but we can derive free energy and pressure of the gas as a function of density and temperature, we can derive its viscosity and other bulk properties. Clearly, this is what our 19th century mathematicians should do with their cellular automaton model of their world. In this book we will indicate how physicists and mathematicians of the 20th and 21st centuries can do even more: they have a tool called quantum mechanics to derive and understand even more sophisticated details, but even they will have to admit that exact calculations are impossible. The only effective, large scale laws that they can ever expect to derive are statistical ones. The average outcomes of experiments can be predicted, but not the outcomes of individual experiments; for doing that, the evolution equations are far too difficult to handle.

In short, our imaginary 19th century world will seem to be controlled by effective laws with a large stochastic element in them. This means that, in addition to an effective deterministic law, random number generators may seem to be at work that are fundamentally unpredictable. On the face of it, these effective laws together may look quite a bit like the quantum mechanical laws we have today for the sub-atomic particles.

The above metaphor is of course not perfect. The Van der Waals gas does obey general equations of state, one can understand how sound waves behave in such a gas, but it is not quantum mechanical. One could suspect that this is because the microscopic laws assumed to be at the basis of a Van der Waals gas are very different from a cellular automaton, but it is not known whether this might be sufficient to explain why the Van der Waals gas is clearly not quantum mechanical.

What we do wish to deduce from this reasoning is that one feature of our world is not mysterious: the fact that we have effective laws that require a stochastic element in the form of an apparently perfect random number generator, is something we should not be surprised about. Our 19th century physicists would be happy with what their mathematicians give them, and they would have been totally prepared for the findings of 20th century physicists, which implied that indeed the effective laws controlling hydrogen atoms contain a stochastic element, for instance to determine at what moment exactly an excited atom decides to emit a photon.

This may be the deeper philosophical reason why we have quantum mechanics: not all features of the cellular automaton at the basis of our world allow to be extrapolated to large scales. Clearly, the exposition of this chapter is entirely non-technical and it may be a bad representation of all the subtleties of the theory we call quantum mechanics today. Yet we think it already captures some of the elements of the story we want to tell. If they had access to the mathematics known today, we may be led to the conclusion that our 19th century physicists could have been able to derive an effective quantum theory for their automaton model of the world. Would the 19th century physicists be able to do experiments with entangled photons? This question we postpone to Sect. 3.6 and onwards.

Philosophizing about the different turns the course of history could have chosen, imagine the following. In the 19th century, the theory of atoms already existed. They could have been regarded as physicists’ first successful step to discretize the world: atoms are the quanta of matter. Yet energy, momenta, and angular momenta were still assumed to be continuous. Would it not have been natural to suspect these to be discrete as well? In our world, this insight came with the discovery of quantum mechanics. But even today, space and time themselves are still strictly continuous entities. When will we discover that everything in the physical world will eventually be discrete? This would be the discrete, deterministic world underlying our present theories, as was advertised, among others, by Fredkin. In this scenario, quantum mechanics as we know it today, is the imperfect logic resulting from an incomplete discretization.⁹

4 Brief History of the Cellular Automaton

A cellular automaton is a mathematical model of a physical system that reduces the physical variables to discrete integers, defined on a one- or higher dimensional grid. The locations on the grid are referred to as ‘cells’, whose positions are indicated by a series of integers, the coordinates of the grid. At the beat of a clock, the variables in the cells on this grid are all renewed, so that they are time dependent. The rule according to which they are renewed reflects the laws of physics. For each cell, typically, the renewed values only depend on the values the cell had previously, and the contents of the neighbouring cells. This property we call ‘locality’.

The earliest mention of such a concept was by John von Neumann [87] and Stanislaw Ulam [84]. Both were interested in the question how, in a physical world with simple laws of evolution, structures could arise that reproduce themselves: the emergence of life. This was in the 1940s. However, it really became a popular subject of study in the 1970s when John Conway [39, 40] proposed an interesting example of an automaton on a two-dimensional grid, called Conway’s game of life. The evolution rules, standing for the ‘laws of physics’, for this system were very simple. The grid was a rectangular one, so that each cell had 4 closest neighbours plus 4 next-to-closest ones, diagonally separated. The data in each cell could take just two values: 0 and 1. Conveniently, these two states of a cell would be called ‘dead’ and ‘alive’. At each beat of the clock every cell was renewed as follows:

• Any live cell with fewer than 2 of its 8 neighbours alive, will die, “as if caused by loneliness”;

• Any live cell with 2 or 3 live neighbours lives on to the next generation;

• Any live cell with more than 3 live neighbours will die, “as if by over-population”;

• Any dead cell with exactly 3 live neighbours becomes alive, “as if by reproduction”.

The initial state could be assumed to be anything. At every beat of the clock, every cell was renewed according to the above rules. The evolution of the entire system could be followed indefinitely. In principle, the grid was assumed to be of infinite size, but one could also consider any type of boundary conditions.

The game became popular when Martin Gardner described it in the October 1970 issue of Scientific American [39, 40]. In that time, physicists could watch the evolution of such automata on computers, and they noticed that the “game of life” could serve as a primitive model of an evolving universe with living creatures in it. It was found that some structures, if surrounded by empty cells, would be stable, or periodic. Other structures, called “gliders” or “space ships”, would move along horizontal, vertical, or diagonal paths.

Thus it was found, that relatively simple, primary laws of physics could lead to complexity, and some asserted that a universe with ‘consciousness’ and ‘free will’ could emerge. Cellular automaton rules were divided into classes to distinguish distinct, global properties: some systems of automata would quickly evolve into stable or featureless final states, some would quickly lead to apparently completely chaotic final structures. The most interesting cellular automata would evolve into recognizable structures with increasing complexity. These class 4 automata were suspected to be applicable for performing complex calculations.

Most of the more interesting examples, such as the ‘game of life’, are not reversible in time, since many different initial patterns can lead to the same final state. This makes them less interesting for physics at first glance, since, at the atomic level, most of the physical laws are time-reversible. Most of the models studied in our book, are also time reversible. Later in our study, however, we shall observe the importance of time non-reversibility in cellular automata for physics, so that models such as the ‘game of life’ enter into the picture again. Most of the members of the interesting class 4 are not time-reversible, and this is an other reason to suspect that time non-reversibility might add an interesting form of stability to our systems, which may add to their significance for physics. More about time non reversibility in Chap. 7.

Cellular automata are often used as models for physical systems such as liquids or other complex mixtures of particles. However, there was also an interest to use cellular automata as theories of physics. Could it be that physics, at its most primordial level, is based on discrete laws? In 1967, this idea was pioneered by Konrad Zuse [97] in his book Rechnender Raum (Calculating Space), where it was suggested that the entire universe is the output of a deterministic law of computation in an automaton. Indeed, this idea did not sound so crazy, considering the fact that fundamental particles appear to behave as single bits of information running around. In particular fermions look like bits, when they are written in coordinate representation.

The concept was phrased as “It from Bit” by John Archibald Wheeler [89, 90], which is the idea that particles of matter (“it”) may well be identified with the information transmitted by them, which in turn is needed for their description (“bit”).

An extensive study of the role of cellular automata as models for addressing scientific questions was made by Stephen Wolfram in his book A New Kind of Science [92]. He attached a special philosophy to his approach. Since cellular automata have complexity and computational universality in common with many models of physical systems, Wolfram suggests that experiments with cellular automata themselves can reveal many special features of such physical systems. The reader might have the impression that our book is a follow-up on Wolfram’s pioneering work, but we do not have such ambitions as yet. The classes of models considered by Wolfram may well be too restrictive for our purposes, and furthermore, our basic question very specifically pertains to the origin of quantum mechanical phenomena.

Both Zuse and Wolfram already speculated that quantum mechanical behaviour should be explained in terms of cellular automata, but did not really attempt to get to the bottom of this—how exactly do we explain quantum mechanics in terms of a cellular automaton? Do we need a very special automaton or does every automaton sooner or later produce quantum mechanical behaviour?

Computational scientists have studied many features of cellular automata that will not be used in this work; this is because these issues involve quite special initial states, while quantum mechanics will force us to consider primarily generic states.

5 Modern Thoughts About Quantum Mechanics

The discovery of the laws of quantum mechanics has severely affected the way investigators now think about ‘reality’. Even authorities such as Richard Feynman were baffled: “I think I can safely say that nobody today understands quantum mechanics” [36]. One fact was established with very little doubt: the theory is completely coherent and it agrees amazingly well with experiment.

It would be nice if an evolution law for a cellular automaton could be found that generates the particles of the Standard Model and the characteristics of their interactions, but most investigators today find it quite unlikely that we will soon be able to identify such a system starting with what we know. What we do have, could be formulated as a dictionary of information: our particles represent information, which is passed on and is being processed. Today, we experience these processes as if it is quantum mechanical information: superpositions of eigen states of operators called observables. If one system of information carriers could be exactly transformed into another system of information carriers, with other rules of processing this information, then we would never be able to decide which of these systems is more ‘fundamental’. Consequently, we might end up with classes of cellular automaton systems, such that we cannot decide which element in one particular class represents our world. David Deutsch [28] formulates this situation in his constructor theory. The key point is distinguishability of different physical systems.

The proposal presented in this book is that at least one element in such classes should turn out to be a classical automaton, but this step is usually not made. More frequently, it is found that a ‘many world’ interpretation seems to be inevitable [88].

Also, the idea that non-linear modifications of the Schrödinger equation, no matter how small, would be needed to explain the collapse of the wave function, still seems to persist. The density matrix calculated from the Schrödinger equation contains off-diagonal terms, and no matter how fast these might oscillate, or how unstable the phases of these terms are, something seems to be needed to erase them altogether. We will show that this is not the case in our theory.

A poll held by A. Zeilinger et al. [74] concerning positions taken by participants of a conference on the foundations of quantum mechanics, was quite revealing. Although perhaps the questions themselves were somewhat biased, it appeared that a majority is divided over the exact wording to be chosen, but agrees that quantum information is fundamentally different from classical information. None of the participants believed in an underlying deterministic theory. Most of them thought that Einstein, in his criticism of Bohr’s formulation of quantum mechanics, was simply mistaken. In this book we hope to convince the reader that a deterministic underpinning theory may not be impossible at all, and, although Niels Bohr was right in a pragmatic sense, there are amendments to be made to the Copenhagen doctrine. A bird’s eye version of the views elaborated in this book, was presented in Ref. [109]. Other, preliminary excursions by the present author are described in [101, 119, 120] and [125].

Practically all investigators [22, 23] adhere to the concept called freedom of choice, which means that an observer, at any time, must enjoy the freedom to choose which observable property of a system is to be measured. Zeilinger [94] claims that this freedom can be guaranteed in experiments. In our book however, we observe that there may be a severe restriction to this freedom of choice, due to very strong spacelike correlations. By carefully defining exactly what freedom of choice means, we shall replace ‘free will’ by something mathematically more precise. We then observe that although all observers at a given time indeed have the freedom to choose their settings, correlation functions then dictate, non-locally, what the ontological states of the observed objects such as elementary photons may be. In short, the choices made by an observer will have to comply with the correlation functions imposed by physical laws. The laws are local, but the correlation functions are not. We shall see how these correlation functions may affect our conclusions concerning the mysteries of quantum mechanics.

6 Notation

In most parts of this book, quantum mechanics will be used as a tool kit, not a theory. Our theory may be anything; one of our tools will be Hilbert space and the mathematical manipulations that can be done in that space. Although we do assume the reader to be familiar with these concepts, we briefly recapitulate what a Hilbert space is.

Hilbert space ℋ is a complex¹⁰ vector space, whose number of dimensions is usually infinite, but sometimes we allow that to be a finite number. Its elements are called states, denoted as $|\psi\rangle ,|\varphi \rangle$, or any other “ket”.

We have linearity: whenever $|\psi_{1}\rangle$ and $\psi _{2}\rangle$ are states in our Hilbert space, then

$$\begin{aligned} |\varphi \rangle =\lambda |\psi_{1}\rangle +\mu |\psi _{2} \rangle , \end{aligned}$$

(1.1)

where $\lambda$ and $\mu$ are complex numbers, is also a state in this Hilbert space. For every ket-state $|\psi\rangle$ we have a ‘conjugated bra-state’, $\langle \psi|$, spanning a conjugated vector space, $\langle \psi|,\langle \varphi |$. This means that, if Eq. (1.1) holds, then

$$\begin{aligned} \langle \varphi |=\lambda ^{*}\langle \psi_{1}|+\mu ^{*}\langle \psi_{2}|. \end{aligned}$$

(1.2)

Furthermore, we have an inner product, or inproduct: if we have a bra, $\langle \chi|$, and a ket, $|\psi\rangle$, then a complex number is defined, the inner product denoted by $\langle \chi|\psi\rangle$, obeying

$$\begin{aligned} \langle \chi|\bigl(\lambda |\psi_{1}\rangle +\mu |\psi _{2} \rangle \bigr)=\lambda \langle \chi|\psi_{1}\rangle +\mu \langle \chi | \psi_{2}\rangle ;\qquad\langle \chi|\psi\rangle =\langle \psi|\chi \rangle ^{*}. \end{aligned}$$

(1.3)

The inner product of a ket state $|\psi\rangle$ with its own bra is real and positive:

$$\begin{aligned} &\|\psi\|^{2}\equiv\langle \psi|\psi\rangle = \hbox{real and}\ge0, \end{aligned}$$

(1.4)

$$\begin{aligned} &\hbox{while}\quad\langle \psi|\psi\rangle =0\quad \leftrightarrow\quad |\psi\rangle =0. \end{aligned}$$

(1.5)

Therefore, the inner product can be used to define a norm. A state $|\psi\rangle$ is called a physical state, or normalized state, if

$$\begin{aligned} \|\psi\|^{2}=\langle \psi|\psi\rangle =1. \end{aligned}$$

(1.6)

Later, we shall use the word template to denote such state (the word ‘physical state’ would be confusing and is better to be avoided). The full power of Dirac’s notation is exploited further in Part II.

Variables will sometimes be just numbers, and sometimes operators in Hilbert space. If the distinction should be made, or if clarity may demand it, operators will be denoted as such. We decided to do this simply by adding a super- or subscript “op” to the symbol in question.¹¹

The Pauli matrices, $\vec{\sigma}=(\sigma _{x},\sigma _{y}, \sigma _{z})$ are defined to be the $2\times2$ matrices

$$\begin{aligned} \sigma ^{\mathrm{op}}_{x}= \begin{pmatrix} 0&1\cr1&0\end{pmatrix},\qquad \sigma ^{\mathrm{op}}_{y}=\begin{pmatrix}0&-i\cr i&0\end{pmatrix}\qquad\sigma ^{\mathrm{op}}_{z}=\begin{pmatrix}1&0\cr0&-1\end{pmatrix} . \end{aligned}$$

(1.7)