A random walk can be defined as a stochastic process (e.g., Brownian motion) describing a path (e.g., the movement of a particle) that consists of a succession of random steps on a certain state space. It is well known that Karl Pearson first coined the term in his 1905 correspondence in Nature with Lord Rayleigh [10] and that the idea can be traced back to as early as the 1560s, when the Italian mathematician Girolamo Cardano proposed an elementary theory of gambling in his Liber de Ludo Aleae. What is less well known, or at least less commented upon, is that the Cambridge logician John Venn presented the first graph of a random walk and anticipated the notion of a fractal in the third edition of The Logic of Chance (1888). A. W. F. Edwards and H. A. Davis have drawn attention to this fact in their 2001 Annotated Readings in the History of Statistics [2, pp. 115–128], though without providing the historical context. Other authors have discussed this context without also elaborating the random walk, e.g., [14]. This synthesizing note takes a closer look at Venn’s graph, discussing its origins and nature in relation to Venn’s own views on probability and randomness.

The Logic of Chance

John Venn, FRS (1834–1923) is best known as the inventor of Venn diagrams of logic and set theory, which first appeared in their full glory in his Symbolic Logic (1881). In 1866, Venn published his most influential book, The Logic of Chance, giving the first ever systematic account of the idea that probability is relative frequency. Venn spent a lot of time attacking the view of probability as a degree of belief, and especially the use of ignorance priors, found in the work of Pierre-Simon Laplace and the British Laplacian Augustus de Morgan. Throughout the book’s three editions (1866, 1876, and 1888) Venn also wrestled heavily with the problem of the proper definition of frequentism. (Below, we follow [3, chapter 3], [5], and [13]). For all the problematic aspects of his theory, Venn may well be credited for having acknowledged and made explicit many of them some fifty years before others did in the wake of the work of Richard von Mises.

Probability

According to Venn, probability is the relative frequency of an event in a suitably long series of events. For example, probability of heads is the relative frequency of heads in a long series of coin tosses. And probability of a female birth is the relative frequency of females in a long series of births. From the very outset, it is clear that there are no single-event probabilities, just frequencies; that is, there is no probability, for instance, of a coin coming up heads when it is flipped once, only a frequency of heads in a long series of trials.

But what kind of series is required and how long should it be? Take the simple example of dice. If a die is tossed 10 times and comes up 3, 3, 7, 2, 5, 4, 7, 3, 6, 4, is the probability of getting a three 3/10? If that doesn’t seem right, how about 20 times, 50 times, or 500 times? Venn quickly concludes that only an infinite number of tosses would do, so that probability is the limit of the relative frequency as the number of trials goes to infinity. Since the world, of course, contains only a finite number of such events, Venn’s particular kind of frequentism is today sometimes called hypothetical frequentism; cf. [6] and [13].

There is another reason for this name. It relates to the question of what sort of events enter into the sequence. In brief, Venn wants a sequence of trials in which all the relevant factors remain the same, so that it is possible to say that these are all trials of the same kind. Take the simple case of the die. What is required is that the same die be thrown in the same way again and again. Venn noted, however, that in the real world, this die will, in time, become a different die due to it becoming worn, with the edges and corners gradually wearing off. So Venn again finds himself forced to accept a hypothetical relative frequency view:

The probability of this die coming up a 6 is the limiting relative frequency that it would have had if it were thrown an infinite number of times with no change in die, the tossing, or any of the relevant circumstances [3, p. 71].

Interestingly, Venn’s The Logic of Chance was written to be in accordance with John Stuart Mill’s empirical view of logic, and it received praise from the economist John Maynard Keynes for containing “no mystery,” “no new indefinables,” “no appeals to intuition” [9, p. 94]. But the view standing at the book’s core was that probability must be defined as a counterfactual. This oft-forgotten fact was first noticed by F. D. Bradley in his 1883 The Principles of Logic, where the author dismisses Venn’s frequentism as a metaphysical theory.

As Venn’s contemporaries already observed, The Logic of Chance has more peculiar features. One is that it essentially gives up the idea that probability can be a guide to life: probability as frequency does not inform degrees of belief and rational decision. Another is that it says that there are no (“objective”) probabilities in the world, but only frequencies, not minding that this excludes large portions of what traditionally belonged to the theory. This brings us to Venn’s take on randomness (for which we largely follow [8] and [15]).

Randomness

During the seventeenth and eighteenth centuries, classical probabilists often struggled to point out that studying probability theory did not lead to a gambling addiction. And in 1866, Venn was anxious to show that the theory did not undermine a belief in God. He saw two ways that it could—mistakenly, he believed—be taken as a support for atheism. When the cause of an event is attributed to chance, then chance could be understood to be a distinct agency (what Venn refers to as “Chance”) or, if chance is taken to mean “without a cause,” then events would not be under God’s control. According to Venn, probability theory is concerned only with stable relative frequencies and has no implications about the causes of individual events. Also, since the theory is postulated on the uncertainty about this or that individual event being due to ignorance, it does not imply indeterminism either. Hence, the existence of chance, whatever it may be, is irrelevant to the logic of chance.

The notion of irregular individual events producing long-run regularities is fundamental to Venn’s frequentism. It is surprising, therefore, to find out that randomness appears in the first edition of The Logic of Chance only once and in passing. Venn mentions it in a brief passage where he criticizes those who hope to conclude from a long-run regularity that the individual events did not happen at random. By 1888, Venn was convinced that randomness was worthy of serious study, if only because the irregularity of individual events seemed to warrant at least some attention in a 500-page work on probability theory.

In the third edition of The Logic of Chance, Venn included a new chapter entitled “The Conception of Randomness and Its Scientific Treatment” [12, chapter 5]. From the start, it is obvious that Venn firmly believes that all events in the world are fully causally determined, whether we know it or not. He held this to be true also of random events, since these too became uniform in the long run. Venn draws a basic distinction between the popular view of “random,” which looks to the process of events done “at random,” and the scientific view, which looks only to the results. To illustrate the former case, Venn gives an example that leads to nonrandom distributions of results: that of a person placed in front of a bookcase and asked to select a book at random, in which case the books most easily reached are those most likely to be chosen.

As an illustration of the second case, Venn describes several examples, some real-life, some physical, and others mathematical. For all of them, the key idea is that a random distribution is a distribution that becomes uniform in the limit. That is to say, random sequences are sequences whose elements are equally likely. Put a piece of paper out on the ground during a rain shower. Where the individual drops will fall is unpredictable, but it is certain that the paper will gradually be uniformly spotted over. Venn believes that all situations involving randomness concern a series of unpredictable, but equally probable, results. So if “raining down” on a piece of paper is an event with equally probable outcomes, then this model can be applied to many other situations, and the calculation of probability becomes a problem of geometry. Venn gives the example of “randomly” placing pawns on a chessboard. When the pawns are viewed as rained down on the chess board, then given the diameter of the base of the pawns and the size of the squares on the board, the probability of a pawn lying across more than one square becomes a geometric calculation. A slightly more complicated case is that of a shot fired “at random” from a gun whose maximum range (i.e., at 45º elevation) is 3000 yards. What is the chance the actual range shall exceed 2000 yards? Here, two different domains of equally probable events are obtained, depending on whether it is assumed that the entire hemisphere of possible positions of the gun are equally probable or that the direction in which the shooter is facing is first chosen (“randomly”) and then the gun is raised (“randomly”) to a particular angular height.

In all cases, the key to solving probability problems is the determination of the domain of equiprobable events. Venn’s frequentism says that this is done by looking to the result of an indefinitely large number of trials. Given this view, Venn cannot allow for randomness in different kinds of problem situations, that is, in which it is not a matter of (hypothetical) fact that certain events in the long run happen equally often. A case in point is a slight variation of the “three-point problem,” discussed by various contributors to the Educational Times as well as by Augustus de Morgan in his 1865 paper “On infinity”: “What is the chance that three straight lines, taken or drawn at random, shall be of such lengths as will admit of their forming a triangle?” [12, p. 105]. For Venn, there are two, and only two, ways to approach this problem. First, it is possible to start with the assumption of three lines not greater than a certain length n and then determine toward what limit the chance tends as n increases indefinitely. Second, it is possible to start with some particular magnitude to represent one of the lines (say the longest of them) and consider that all possible shapes of a triangle will be represented by varying the lengths of the other two. In both cases, Venn expects a definite result without having to make a “random” selection from the infinity of possible lengths.

Taken together, how does one determine whether a given distribution is random? When the ultimate distribution is known, this is merely a matter of fact, as in the case of raindrops. When the ultimate distribution is unknown, it is a matter of counterfactually supposing the ultimate distribution known or of hypothetically extrapolating the limiting relative frequency.

Venn’s focus on results allows him to uphold his view of chance as ignorance in the face of the conviction that each event is fully causally determined. Venn, for instance, dismisses the view of randomness held by M. W. Crofton (1826–1915) and Wesley S. B. Woolhouse (1809–1893)—both regular contributors to the Educational Times—for whom “random” means “according to no law” [12, p. 101]. Venn argues as follows. The notion of randomness makes sense only in light of an ultimate (or at least ultimately) uniform distribution, whether observed or hypothesized. Every uniform distribution must be the result of some law. Hence, randomness cannot mean lawlessness.

The (Wrong) Digits of Pi

The idea of determining whether a distribution is random by observing it for a while—watching the pattern of rainfall on a piece of paper or of daisies growing on a small lawn—may feel a bit archaic in today’s world of randomness at the subatomic level. Venn did give a slightly more sophisticated argument for what now would be called “pseudorandomness.” And it was in the context of this argument that he showed to have had a grasp of random walks, one version of which would soon become famous as the simple diffusion model of physics.

Venn’s example to show that randomness refers to the resulting distribution, rather than the process that produces it, is that of the digits of pi. These had been calculated out to 707 places, using a formula developed by John Machin, and published in 1873 by William Shanks (1812–1882). Shanks’s story is a rather fascinating one [6]. A human computer (which then denoted not a machine but a person skilled in arithmetic), he was an amateur and a marginal figure in the mid-Victorian mathematical community. But he published some 15 papers in the Proceedings of the Royal Society, submitted on his behalf by prominent fellows of the society such as George Stokes, George Biddell Airy, William Whewell, and De Morgan. Starting in 1850 and returning to the task at intervals over more than twenty years, his biggest stunt was his pi calculation with pencil and paper, which appeared in three stages: January 1853 (530 places), spring 1853 (from 530 to 607 places) and 1873 (up to 707 places). Shanks made a series of mistakes beginning around decimal place 530 that meant that the rest of the work was essentially for naught.

Venn himself found it quite striking that the digits of pi would show anything resembling the kind of statistical randomness he was interested in. After all, no physical causation is involved to which to ascribe the tendency of irregular individual events (i.e., digits) to gradually become regular (i.e., to occur equally often). He wanted to show his reader that the digits of pi nonetheless provided an example of randomness. The best way to illustrate this randomness was by expressing it graphically, Venn held. He actually made a claim to originality in this regard, writing, “I do not know that any one has suggested the drawing of a line whose shape as well as position shall be of a purely random character” (this claim has been seconded in the secondary literature ever since) [12, p. 114].

Venn graphically illustrates the randomness of the digits of pi by using them to generate a path described by a point. His starting point is to work with a compass with eight points (numbered 0–7) to direct the movement at each step on a square lattice: 0 is north, 1 is northeast, and so on. Next, the digits 0 to 7 from the first 707 digits of pi are used by Venn as clues to the direction of the path traced. What results is Figure 1, which may well be the first published diagram of a discrete random walk.

From the path’s apparent patternlessness, Venn concluded that the graph of the path generated by the chosen digits would enable the reader to comprehend the idea of a “chance arrangement.” Venn emphasized that he had drawn similar paths using digits from logarithm tables, finding the result to be much the same [12, p. 118].

Venn also briefly discusses the limiting (or “ideal”) form that such a random walk would take if the number of possible directions was infinite and the step length made indefinitely small. The result is described in terms that today are captured by the word fractal, which around that time were just beginning to be studied mathematically; cf. [2, p. 144]. Thus, in the “ideal case of randomness of direction,” Venn writes, the “spikiness” of the random line will diminish “without any limit”; more generally, there is, in fact, “no ‘limit’ here, intelligible to the understanding or picturable by the imagination … towards which we find ourselves continually approaching” [12, p. 116].

Figure 1
figure 1

Venn’s “random walk.” (taken from J. Venn. The Logic of Chance, p. 118. Macmillan, 1888).

Full size image

Afterword

Venn’s random walk went largely unnoticed. Others, however, have followed the lead of his particular way of thinking about whether the digits in the decimal expansion of numbers like pi are randomly arranged. Whether they are is unknown, but since the 1950s, for example, many have entertained the belief that the decimal expansion of certain irrational numbers might produce an endless source of random digits; see [1, Chapter 9] for a number of such cases.

There is a remarkable gap separating Venn’s frequentism from the twentieth-century frequentism of the German applied mathematician Richard von Mises. Although he responded to David Hilbert’s famous 1900 call for probability theory’s axiomatization rather than to Venn (with whose work he was initially totally unfamiliar), von Mises may be said to have updated Venn’s frequentism. Like Venn, von Mises defined probability as relative frequency in a certain kind of infinite sequence (called a “collective”) that was to exhibit intuitive ideas of local disorder (“individual irregularity”) and global order (the “limiting relative frequency”) already emphasized by Venn since 1866. (Here, we follow [3, Chapter 3], and [4, Chapter 5].) Unlike Venn, von Mises did not simply assume the existence of limiting relative frequencies. Instead, it needed to be postulated as an attribute of a collective. And more importantly, von Mises captured local disorder by a specific requirement of randomness. Von Mises’s idea was that a sequence is random when the relative frequencies are somehow invariant under selection of subsequences. For instance, the sequence HTHTHTHTH… of heads (H) and tails (T) is not random, because in the subsequence of odd members, H has a relative frequency of 1, and in the subsequence of even members, it has a relative frequency of 0. Already in the 1920s it was pointed out by other probabilists, such as the Russian mathematician Aleksandr Yakovlevich Khinchin, that this crude criterion was too powerful: if there exists an infinite sequence of heads and tails in which the relative frequencies are not 0 or 1, then that sequence will contain an infinite subsequence of all heads and another of all tails. Hence, the way in which subsequences were selected had to be regimented. Von Mises considered so-called place-selection functions that map initial segments of the sequence to 0 (IN) or 1 (OUT). Here, the underlying idea was that of nonpredictability, which von Mises came to express by speaking of the “impossibility of a gambling system”; see, e.g., [10, p. 25].

The question of which functions should be used as place-selection functions led to serious objections to von Mises’s entire theory. At the same time, between the 1930s and 1960s, it also gave rise to fascinating work on random sequences by Abraham Wald, Andrey Kolmogorov, Per Martin-Löf, and several others; see [14, chapter 7]. This work, in turn, was crucial to the understanding of stochastic processes such as the random walks first found, in rudimentary form, in Venn’s The Logic of Chance.