Why was it possible to understand the quantum world?

The question in the title is answered by analyzing a specific case, Max Planck’s discovery of the radiation law. Trial and error as method of research and a rather moderate enlargement of the “knowledge space” of classical physics had been sufficient to unravel the secrets of quantum phenomena.


Introduction
The question formulated in the title occurred to me every time I taught a course in quantum mechanics. In those active years, however, other problems had priority and I carried the question with me into retirement and present my conclusions only now 1 . Let me explain why I find the question important. There is a fundamental difference between the world of our daily life and its atomic substructure, which I call the quantum world. Our senses only allow observing processes of our environment and our reasoning is adapted to understand this part of the world. This is not so for the objects of the quantum world, the atoms and molecules. These particles and their interactions are not directly accessible to our senses and mankind did not even know of their existence for most of the evolution. Therefore it is not obvious to me that our thinking should have the necessary tools to unravel the secrets of the quantum world. Yet, physicists succeeded. Why was it so? This is not a question to be answered within physics, but it is philosophical by nature and an answer should be found in that branch of philosophy which is called epistemology or the theory of knowledge. There, general conditions are formulated of how reliable knowledge can be generated in scientific research. My approach is an empirical one. I draw philosophical conclusions from an analysis of a concrete example, namely from Max Planck's discovery of the law behind black body radiation. Since I am not a trained philosopher, my conclusions may lack professional rigor and I apologize.
The paper consists of three parts. It begins with a summary of some basic notions from the theory of knowledge, followed by an analysis of Planck's research on the black body radiation and by the introduction of the notion of a "knowledge space". It ends with an answer to the question raised.

3312
The European Physical Journal Special Topics 2 Correspondence theory of truth Max Planck, who lived from 1858 to 1947, had been the first physicist who found a theoretical explanation for a quantum phenomenon. Before I describe how he proceeded, I will quote the first words from his scientific autobiography written at the age of 87. There he speaks about a deep philosophical observation which had convinced him to devote his life to research in physics: "My original decision to devote myself to science was a direct result of the discovery (which has never ceased to fill me with enthusiasm since my early youth) of the far from obvious fact that the laws of human reasoning coincide with the laws governing the sequences of the impressions we receive from the world around us; and that, therefore, pure reasoning can enable man to gain an insight into their mechanisms. In this connection, it is of paramount importance that the outside world is something independent from man, something absolute and the quest for the laws which apply to this absolute appeared to me as the most sublime scientific life-task." [1] Central to Planck's concept of scientific research is the statement that "the laws of human reasoning coincide with the laws governing the sequences of the impressions we receive from the world about us." This concept corresponds to the definition of truth given first by Aristotle and is known under the name of correspondence theory of truth. The best known formulation is the one by Thomas Aquinas: " Veritas est adaequatio rei et intellectus" (Truth is the equation of thing and intellect.) Planck is more precise. He does not speak about the correspondence between the laws of human reasoning and the laws governing the world about us, but rather about the correspondence between the laws of human reasoning and the laws governing the sequences of the impressions we receive from the world about us. I will keep this important difference in mind, but for reasons of brevity, I will speak about the correspondence between the laws of reasoning and the laws of nature.
The correspondence between the laws of reasoning and the laws of nature is the central goal of any scientific research. Understanding a phenomenon means achieving this correspondence. What is the origin of the correspondence, i.e. how does it come about? On my way to an answer, I will begin by sketching the philosophical framework within which I will argue.

Evolutionary epistemology
According to evolutionary epistemology the correspondence between the laws of reasoning and the laws of nature is the result of evolution. For details I refer to an article in the Stanford Encyclopedia of Philosophy [2]. In the following I will only summarize the ideas that are relevant to the question raised in the title.
Trial and error are the agents of progress during evolution. In biology this mechanism is familiar from Charles Darwin's theory on the evolution of living species. The properties of living organisms are encoded in their genetic material. Over time, the genetic material changes and with it the properties of the organisms. Changes are induced by mutations and sexual reproduction (for higher organisms). These changes are random and will be called the "trials" of evolution. A trial is a success, if it improves the chances for the organism to survive in a hostile environment. In this case its genetic material is transmitted to the next generation and a "better" organism will replace the parent one. A trial will be called an error of or a failure of the evolution, if the new organism is less well adapted to the environment than its parent(s). In this case the organism may die before reproduction and its genetic material will vanish.
A necessary condition for the survival in a hostile environment is that the organism correctly interprets the signals coming from the environment. Also, the senses for information gathering and the nerve cells ("brains") for information processing have evolved during the millions of years of the biological evolution in the direction of a better understanding of the environment. Over time the correspondence between information processing of an organism and the processes in nature has gradually improved. This progress is again one of trial and error and it is -according to evolutionary epistemology -the biological origin of the correspondence in humans between the laws of reasoning and the laws of nature. However, according to Gerhard Vollmer, the empirical proof of this claim is not yet well established [3].
In addition to the biological evolution of information processing, common to all organisms, humans have developed a second path of information processing: the cultural evolution. The knowledge that one generation has accumulated by learning and experience is transmitted to the next generation by teaching and is encoded by writing.
Also, progress in the cultural evolution proceeds by trial and error. When humans observe a new phenomenon, they formulate a hypothesis about its origin and nature and act accordingly. Hypotheses are the trials of the cultural evolution. If a hypothesis does not fit, it leads to a failure or even death and the trial has been an error. Successful hypotheses, however, will be remembered orally or in written form. Because of the formal analogy between biological and cultural advances, the term "evolution" is also applied for progress in the cultural realm, hence the wording "evolutionary epistemology". While the trials in the biological evolution are mostly random in nature, the hypotheses, the trials in the cultural evolution, are predominantly goaloriented. For instance experienced scientists try to define the goal of their research and often sense intuitively rather quickly when they are on the wrong track. Goalorientation shortens the way to an eventual success (or final failure).
One of the first scientists, who formulated the basic ideas of the evolutionary epistemology -without using the word -was Ludwig Boltzmann, a theoretical physicist. I quote: "According to my conviction, the laws of reasoning have their origin in that the relations between the inner ideas, which we form of the objects, slowly adjusted to the relations between objects. All rules of the relations which contradict experience are dismissed, while the always correct ones are kept with all energies and were handed down to the next generation." [4] These ideas were later worked out by Konrad Lorenz and in particular by Gerhard Vollmer [5]. Donald Campell coined the term "evolutionary epistemology" in an article about Karl Popper's epistemology [6]. Central to Popper's theory of scientific progress is the requirement of falsifiability: Each hypothesis which is constructed to explain a new phenomenon should have the property that it can be falsified, i.e. that its predictions can be tested empirically. According to Popper, a hypothesis, which cannot be falsified, is useless.
Planck's discovery of the law underlying black body radiation is a good example of scientific progress by the method of trial and error. Fortunately, we know it rather well from Planck's own writings [1,7].

Planck and the black body radiation
At the end of the 19th century, Planck was a professor for theoretical physics at the University of Berlin and worked about entropy and other aspects of heat. He was in close contact with his colleague Heinrich Rubens, a professor in experimental physics, who studied the black body radiation. He and Ferdinand Kurlbaum measured the intensity of this radiation as a function of frequency. For large and small frequencies, the shape of the experimental spectrum had already been understood within the framework of classical physics (the laws of Wien and Raleigh/Jeans), but a description of the middle part was missing. Planck was eager to find one and proceeded in three steps.
Step 1: Planck started from the experimental curve and the successful descriptions at small and large frequencies. Although he had no idea about the underlying physics, he tried out various parameterizations of the curve until he found an interpolating function which perfectly fitted the full spectrum. This was his first success. But he was not satisfied, because he had not understood the underlying physics. He writes: "But even if one assumes their absolutely precise validity, the radiation formula would only have a formal meaning in the sense of a happily guessed [my italics] law and would therefore have only a formal meaning. Therefore I immediately started new work with the aim to find a proper physical interpretation and this question led me to consider the relationship between entropy and probability, that is, to Boltzmann's train of thought." [1] Step 2: In its middle part the experimental spectrum is rather smooth and does not reveal anything about the underlying physics. Therefore the mode of discovery by induction was closed to Planck. He faced terra incognita. Imagination and courage were required. His task was twofold: He had to invent a physical model for the black body radiation and he had to find a theoretical framework to derive the formula. As for the latter, Planck chose the theory of statistical mechanics, which Boltzmann had recently developed to derive some laws of thermodynamics under the assumption that matter was built up from atoms. Probability calculus was the mathematical formalism of this new theory.
As for Planck's model for black body radiation, he imagined a set of oscillators to represent the black body which emits the radiation. In order to describe the system of body and radiation within statistical mechanics he had to calculate the number of possibilities by which the total energy of the radiation can be distributed to the oscillators. After many futile trials, which he does not mention, Planck finally came up with the idea of splitting the total energy of the radiation into energy packets of equal size and distributing them among the oscillators. This was not an easy choice for him, as Planck confessed later: "In a nutshell, I can call the whole act an act of despair. For by nature I am peaceful and reluctant to intellectual adventures. But I had been struggling with the problem of the balance between radiation and matter for six years now (since 1894) without any success; I knew that this problem was of fundamental importance for physics, I knew the formula for the energy distribution in the normal spectrum; a theoretical interpretation had to be found at all costs, however high. Classical physics was not enough, I knew that." [8] In mathematics when calculating with continuous quantities it is not uncommon to split these into finite elements and then carry out the desired operation. In the end, one lets the size of the elements approach zero. The interesting thing about Planck's result was that after having introduced the energy packets Planck was able to fit the experimental curve only if the size of the energy packets takes on a very specific value: = h · ν. Here ν denotes the frequency and h is a constant, which will later be called Planck's constant. Obviously, the introduction of energy packets was not simply a mathematical trick, but the energy packets had a physical reality. The oscillators do not absorb or emit energy in a continuous way (as it is familiar from classical physics), but only in packets, "quanta", of a well-defined size . This result was completely unexpected to Planck and gave him a big headache, because it did not fit into his concept of energy. It was outside of Planck's "knowledge space", a notion which we will explain later.
His success in deriving a formula which fits the experimental curve was the beginning of quantum physics -from hindsight only. There was still a long way to go. Planck presented his result on December 14, 1900 at the meeting of the Physical Society in Berlin. The audience was not particularly enthusiastic. Also, Planck was not satisfied with this solution. Therefore, a third step followed.
Step 3: Although he was able to describe the experimental result by means of the quantum hypothesis, Planck suspected that also other, less revolutionary hypotheses would explain experiment as well. He began to search for them and wrote about this phase: "My unsuccessful attempts to somehow incorporate the quantum of action into classic theory stretched over a number of years and cost me a lot of work. Some peers saw a kind of tragedy in it. I disagree, because for me the profit I gained from such a thorough education was all the more valuable. Now I knew exactly that the quantum of action plays a much more important role in physics than I was initially inclined to assume and thereby I gained a full understanding of the need to introduce completely new methods of reasoning and calculation in the treatment of atomistic problems." [1] When Planck was unable to find any other derivation for the radiation formula, he was satisfied, but not the community. In physics, a hypothesis is only accepted as a suitable explanation of one experiment, if it also predicts the outcome of another experiment and if this prediction is confirmed. This is Popper's condition of falsifiability. Since Planck's work failed to meet this requirement, his colleagues remained skeptical.

Photons, the quanta of light
If we imagine the evolution of science as a relay race, then, in 1905, Albert Einstein took over Planck's relay beam and succeeded in two respects: He proposed a physical interpretation to Planck's quanta, and he predicted the outcome of another experiment.
According to Einstein, Planck's mysterious energy quantum is not an auxiliary mathematical quantity, but it is a new real particle, later called photon, and the blackbody radiation is an assembly of particles and not of electromagnetic waves. At that time this was a hard-to-digest claim for the following reason: The dispute over the question whether light consisted of particles or waves had "finally" been settled by Thomas Young's interference experiment in 1801. A century later, the still unknown physicist Einstein claimed: Light consists of tiny particles. Hardly any other physicist, including Planck, could accept the idea. Could the claim be verified or falsified?
Fortunately, Einstein was able to propose a new experiment, based on the photoelectric effect, in which Planck's formula = h · ν could be tested. It was not until 1914, fourteen years after Planck's discovery and nine years after Einstein's prediction, that Robert Andrews Millikan was able to confirm Planck's hypothesis in all details.
Einstein's photon hypothesis and its experimental confirmation were so crucial to the development of quantum theory that two Nobel Prizes were awarded: to Einstein in 1921 and to Millikan in 1923. Planck had received the prize already in 1918.

Knowledge space
The way Planck discovered the radiation formula is a nearly perfect illustration of the method of trial and error. It proved successful even in a situation, where there was no prior insight into the underlying physics. But trial and error by themselves do not guarantee success. If the researcher never hits the right hypothesis among his trials, he will never be successful.
After having set up the goal of his research, a scientist invents a number of hypotheses which he wants to investigate. Where does he take them from? I propose the concept of a "knowledge space", a kind of an "intellectual supermarket". Faced with a problem a researcher visits his personal knowledge space to "shop" for suitable hypotheses. Every person has an intellectual supermarket of his own. It contains all the knowledge that the person has gathered during his life including his professional training. But one can also attribute a knowledge space to an abstract body like the science of physics. Its space contains the results of previous research in physics and mathematics. Even philosophical ideas can be found there. For example, the notion that matter is made up of atoms had been in the shelves of physics' intellectual supermarket for almost two millennia, before it became a successful hypothesis. Sometimes concepts are also removed from the shelves. This was so for the particle nature of light after Young's interference experiment.
If the researcher cannot find a suitable hypothesis on the shelves of his personal knowledge space, he has to enlarge his space. There are various possibilities for this. In most cases he will already be successful, when he talks to colleagues or dives into the literature. In rare cases, like Planck's, only a really new hypothesis can lead to success. Imagination and courage are required to find the right hypothesis to enlarge the knowledge space.
Planck's decisive moment came, when he split the total energy of the black body radiation into many little packets of equal size. What seemed to be just a simple mathematical operation was a sizable conceptual step and enlarged his knowledge space. Although he did not understand the physical significance of his step, he sensed its importance. Einstein then clarified the physics: Radiation is composed of particles. This hypothesis contradicted proven results of classical physics and therefore was outside its knowledge space.
The particle-wave duality of light is one of the revolutionary concepts by which quantum physics differs from classical physics. It was discovered by the joined effort of three geniuses, Planck, Einstein and Millikan, in a time span of just fourteen years. This is rather short when one compares it with other breakthroughs in the history of physics. Is there a reason to it?
Without wishing to belittle the achievements of the three geniuses, I claim that their revolutionary step was "relatively small". The concepts of particles and waves were already well accepted, but only as mutually exclusive concepts. The revolutionary new idea consisted in that the wave and particle properties are no longer mutually exclusive, but complementary.
I conclude: Understanding the (non-relativistic) quantum world did not require completely new physics with unknown structures and objects. The concepts of particles and waves, already familiar from classical physics, could be kept, but their properties had to be modified. In my opinion, this "relatively small" difference between the physics of our daily world and the physics of its atomic substructure is the reason why physicists succeeded to unravel the secrets of the quantum world.

Conclusion
Generalizing the results obtained in the paper, my answer to the question raised in the title is summarized in the following two statements: First: The method of trial and error proved a very powerful tool even for understanding the quantum world, where the method of induction failed because of little or no prior physical insight.
Second: A complete understanding within classical physics was not possible. Yet, the enlargement of the corresponding knowledge space was only moderate in size. In other words: The quantum world differs from the world of daily life not in a fundamental way, but only in certain details, though important ones. This was the reason why the quantum world could be understood.
Open Access funding enabled and organized by Projekt DEAL. I gratefully acknowledge the invitation to the 40th Max-Born Symposium, which gave me the opportunity to take part in the celebration of David Blaschke's 60th birthday. David and I first met 1988 in a conference, where we both reported on work on the formation of J/y in heavy ion collisions. After the fall of the iron curtain, we began to work together and over time we became friends. David is not only a good physicist, but also has a wide spectrum of interest outside physics. Therefore a more philosophically oriented article, like the present one, may be appropriate for this occasion. I also thank Professor Gerhard Vollmer for a series of stimulating discussions and correspondences.
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Publisher's Note The EPJ Publishers remain neutral with regard to jurisdictional claims in published maps and institutional affiliations.