A Puzzle about Further Facts

Case A Fonda has just attended an inspiring department colloquium. On her way out, she absentmindedly takes a wrong turn and wanders into a computer lab. She approaches one of the computers. On it, a simulation of a number of humanoid agents in a virtual universe is being run. The perspective of one of these agents—call it Alpha—is continuously displayed on the monitor. Fonda is enthralled and continues to watch from Alpha’s perspective. Because Alpha takes a large variety of interesting actions in the simulated universe, it does not take Fonda long to learn the laws of physics governing it. Moreover, Fonda—whose undergraduate degree was in computer science—can easily imagine how one would write the code for simulating the environment according to these laws. Then, she has the following thought. There must be some additional code, beyond the code that executes the simulated universe’s laws of physics (and the code that gives its initial conditions). Namely, there must also be some code that governs the displaying of Alpha’s perspective on the screen. This additional code could in principle be changed without changing any of the code governing the physics and the initial conditions. For example, what is currently displayed as red on the monitor could be displayed as blue instead, by changing only the code governing the display. As another example, this code could be changed to display the perspective of a different agent instead. (It is possible that the other monitors in the lab display the perspectives of the other agents—she has not checked—but even so, one could change which agent’s perspective gets displayed on which screen.) Alternatively, the code could be changed to display nothing at all (with the simulation nevertheless running unchanged). Fonda thus concludes there are some “further facts” to this universe, in that there is additional code beyond that governing the laws of physics and the initial conditions. This additional code is, in a sense, contingent relative to the other code.¹

The “further facts” that Fonda concludes are present in Case A are related to those considered in the literature on metaphysics and philosophy of mind. The first example of a further fact in the case is related to qualia and the possibility of inverted spectra.² It especially relates to strong versions of the inverted spectrum scenario where qualia do not supervene on the physical, i.e., where two microphysically identical twins nevertheless have inverted spectra. Stated otherwise, a variant of the question for which the case above is arguably especially relevant is: did the laws of our universe (and its initial conditions³) necessitate that when I see red things they phenomenally appear the way they do, or would they have allowed for them to appear the way blue things do now? Whether the phenomenal nature of color perception is contingent (holding fixed the physical laws of our universe and its initial conditions) has been the subject of much philosophical debate. Fonda’s question is clearly related, though it would seem that in her case she is entirely right to conclude that the way things in the simulated universe appear to her is contingent, even holding fixed the physical laws and initial conditions of the simulated universe. The analogy between Fonda’s questions in Case A and the standard ones from the metaphysics and philosophy of mind literature breaks down at some points, and we will explore this in what follows. Before we do so, let us consider the other example, Fonda’s question of why it is Alpha, and not some other agent, whose perspective is displayed. This is related to questions in metaphysics about personal identity and the self. Most closely, it is related to the question of whether “I could have been someone else,” and the closely related question of whether it is contingent that this perspective is the present one.⁴

Another type of further fact that could be included in the discussion is that of further facts about time. Fonda might ask herself why this point in the simulation’s timeline is being displayed to her right now, as opposed to another point in simulated time. Also, she might ask herself why the simulation runs at the rate that it does, as opposed to (say) twice as fast.⁵ Here, it may not be clear that things could have been different. For example, perhaps the simulation simply started running when the program was first executed and it is running at the fastest possible rate on the hardware provided. On the other hand, the entire simulation may have been precomputed from beginning to end, so that a type of block universe is already stored in computer memory and Fonda is just watching a replay of some part of it. In this case, there must indeed be some further code governing which temporal part is replayed and at which rate the replay runs. While I believe that the cases presented here may indeed provide some helpful insights for the metaphysics of time, things are clearer for the other types of further facts. Hence, I will avoid discussion of time in what follows.

We are now ready to introduce the next case.

Case B This case proceeds similarly to Case A, though with an important difference. Instead of plain monitors, the computer lab now has sophisticated virtual reality (VR) headsets. Fonda puts on one of these that shows her the perspective of Alpha. The VR system is so remarkably good that Fonda spends a long time using it and becomes completely engrossed—so engrossed that she completely forgets the world outside the simulation, her own identity in it, and, we may suppose, even basic facts such as what the color of grass is, or even the existence of such a thing as grass at all (assuming there is no such thing in the simulated universe). All that is left to her is the simulation, displayed from Alpha’s perspective—so that presumably she feels rather identified with Alpha. Again, she quickly learns the laws of physics in this universe. Then she has the following thought. There must be further facts to this universe, namely the ones concerning my own perspective in it. The (simulated) sky could have appeared to me in the color in which the (simulated) ground appears to me now, without the fundamental laws of the universe changing. Moreover, there must be further facts regarding my identity—why is it this perspective that appears (to me⁶) and not that of some other agent?

Finally, Case C returns to day-to-day life.

Case C In this version, Fonda does not walk into any computer lab; she just walks outside and experiences the world as we normally do. She knows the laws of physics well from her undergraduate studies and nothing in the world seems mysterious to her (unresolved questions in physics aside). Then, she has a thought just like the one in Case B, but now about our own familiar universe. Why does the sky appear to me the way it does? Why is it this perspective that appears (to me)? There must be further facts beyond the laws of physics and any initial conditions.

This, to me, seems the least appealing option of the three, so I will not spend much space on it. Fonda’s reasoning seems entirely sound to me: if the simulation is displaying on the monitor, there must in fact be some code that governs this display. It would certainly be possible to write code for the simulation without any instructions to display anything on the screen, but then the simulation would just run silently⁸ on the machine without any output. This is in fact a mistake programmers make on occasion: they write the code (for, say, computing \(2^n\) as a function of n) correctly but forget to write instructions to display the result to (say) the screen. Such a mistake is typically easily corrected by adding a line to the code.⁹

I will now skip to Option III before returning to Option II.

While this option seems more appealing to me than Option I, it still seems difficult to argue for it. Key to this difficulty, of course, is that by assumption, Fonda has forgotten everything about the outside world in Case B. If she retains some memory of the outside world, the case will reduce to one that is not substantively different from Case A.

Is Case B substantively different from Case C? Of course: in Case B the universe under consideration is a simulation in a larger universe. But are the two cases substantively different in terms of Fonda’s epistemic situation? This is what seems difficult to argue. Whether we have reason to believe that we are not a brain in a vat or (in) a computer simulation is a topic that has been explored at length in the literature. Bostrom (2003) has argued that there is a large probability that we are in fact in a computer simulation, under some assumptions including that posthuman civilizations are likely to be reached and likely to run a large number of such simulations. In contrast, Markosian (2014) has argued that all the evidence speaks in favor of the external world being real; evidence in favor of being a brain in a vat would be exemplified by a major glitch in the simulated environment. For our purposes, it is not necessary to resolve this debate. What matters is not whether Fonda is justified in believing that the world around her is real (in the sense of not being a simulation) in either Case B or C, but rather whether there is a substantive difference between these two cases in terms of her epistemic situation.

Let us simply assume that there are no glitches in Case B. Furthermore, at least in principle, the simulation in Case B could provide Fonda with a very rich experience. As Case B has been described so far, Fonda is not able to take actions in it; she is just observing. This indeed constitutes a difference between what Fonda observes in Cases B and C. It is not immediately clear to me whether and how this particular difference is relevant to the soundness of her argument for further facts. In any case, we can easily modify the example to give her some control over Alpha’s actions (with perhaps other human beings who similarly wandered into labs controlling the other agents in the simulation).

Perhaps more interestingly, she may wonder about the place of her own thoughts in the simulated universe. In our own world, we have reasons to believe that our thoughts are generated by our brains. If a similar account does not seem reasonable in the simulated universe—for example, because there does not seem to be any physical structure in it capable of generating these thoughts—she may conclude that such a structure must exist somewhere outside of her observable universe, and from there it is a short step to conclude the existence of further facts. Then again, we could modify the example so that there appear to be brains inside the simulated agents; we could even go so far as to imagine that, unbeknownst to Fonda, her brain is being scanned while she is standing in the lab, and what goes on in it is then reflected in Alpha’s simulated brain.¹¹ (A similar idea is described by Chalmers (2005).)

Overall, it seems difficult to draw a sharp distinction between Fonda’s epistemic situation in Cases B and C that cannot be addressed with a simple modification of the cases.¹² Of course, this is precisely the point of Case B, to make Fonda’s epistemic situation in it essentially identical to that in Case C; and if we can in fact succeed at this, then Option III fails. This leaves us with Option II.

This appears to me the most attractive of the three options. A first attempt at an argument proceeds as follows. In Case A, Fonda recognizes (say) that the color of the sky in the simulation is the same as the color of grass in the outside world (i.e., green), whereas it could just as well have been displayed as the color of the sky in the outside world (i.e., blue). That is, the correspondence between colors in the simulation (as displayed on the screen) and colors in the outside world clearly could have been different, without the code that governs the laws of physics and the initial conditions in the simulation being any different. Thus there is clearly a “further fact” present, consisting in the additional code that governs how a perspective is displayed on the monitor. In Case B, however, Fonda cannot recognize the existence of any such correspondence, because she no longer remembers the outside world. Hence—so the argument goes—she cannot conclude further facts exist.

Now, this argument does not seem entirely satisfactory to me. Even in Case B, it seems entirely possible for Fonda to imagine a scenario where the color of the sky (in the simulation, though she does not know it is a simulation) would be the color that the grass is now (in the simulation), and vice versa. This is the familiar inverted spectrum scenario, except in this case, by virtue of all this taking place in a virtual reality system, it is clearly true that the spectrum could be inverted; all this would require is some changes to the code governing the display. Nevertheless, it is still possible that Fonda’s belief that the spectrum could have been inverted is not justified. Having forgotten about the outside world, she certainly does not know about the simple mechanism—changing a few lines of code in the outside world—by which the spectrum could indeed be inverted. But few would argue that awareness of a specific mechanism by which the spectrum could be inverted is necessary to justify belief in the possibility of an inverted spectrum (though it is clearly sufficient).

Nevertheless, it seems that the physicalist, arguing that Fonda’s belief is not justified, has arguments available in Case B that are unavailable in Case A. The physicalist can argue that Fonda cannot be sure that experiential properties corresponding to her seeing the simulated sky are not, at bottom, physical properties. (Again, here, “physical” refers to the physics of the environment, which we happen to know is simulated but she does not.) Even in our own case (Case C), fleshing out such an argument and addressing immediate counterarguments requires substantial work; see, for example, Hawthorne (2002). But I do not see that any additional obstacles to such an argument are introduced when moving from Case C to Case B. In contrast, in Case A such an argument becomes untenable. In that case, Fonda clearly knows that the experiential properties corresponding to her seeing the simulated sky are not, at bottom, properties of the simulated physics; she knows that the code governing the display, the monitor itself, her eyes and brain, etc., are also involved.

So, perhaps this all reduces to standard arguments about inverted spectra. Perhaps Fonda cannot reasonably reject the possibility that the way colors appear to her necessarily emerges from the laws of her universe. If so, it at least suggests that the debate on inverted spectra has been on the right track. But it also provides a clearer lens on these arguments.¹³ This is because unlike in the standard inverted spectrum scenario, in this case it is clearly true that the spectrum could have been inverted. This, I believe, reduces the intuitive appeal of the argument that which quale appears must supervene upon properties of the physical world (and that therefore a strong type of inverted spectrum is not possible). It makes it clear that if this argument is to succeed, it should be fundamentally epistemological in nature: it should argue just that we cannot know that there is no such supervenience. At least, this is so if our epistemic situation is sufficiently like that of Fonda in Case B, and it appears that it is, as discussed in the previous section.

It is useful to note that even slightly nudging Case B towards Case A—for example, allowing Fonda to remember only that she is in a simulation, but effectively nothing else about the outside world, including even whether her color experiences there were anything like the ones she is experiencing now—would again allow her to soundly conclude that an inverted spectrum in her simulated world is a genuine possibility. This is why it is important to be strict about Fonda not remembering anything in Case B.

Next, let us consider further facts about personal identity and the self. Again, in Case A, Fonda can soundly conclude that there are further facts about how the perspectives of agents in the simulation are assigned to monitors in the outside world. Even if every agent’s perspective is displayed on some monitor, clearly the correspondence—on which particular monitor each agent’s perspective is displayed—could have been different, without the code that governs the laws of physics and the initial conditions of the simulation being any different. Thus there is clearly a further fact present, consisting in the additional code that governs on which (if any) monitors each perspective is displayed. Again, however, in Case B, Fonda cannot recognize the existence of any such correspondence, because she no longer remembers the outside world. So one might argue that in Case B she is not justified in believing in the existence of further facts about personal identity and the self.

How satisfying is this argument? Could she nevertheless, in Case B, imagine the perspective of an agent other than Alpha appearing to her? Certainly it is true that a different agent’s perspective could be made to appear to her; all this would require is a change to the code governing which perspective is displayed by the VR system. But would she be justified in believing that a different perspective could have appeared to her? Again, it seems that the moment we allow her to remember even just the mere fact that she is in a simulation, even if she remembers nothing else about her identity in the outside world, she can indeed conclude that a different agent’s perspective could have been made to appear to her. But we explicitly rule out such a memory in Case B. Hence, it is not clear that the notion of a different perspective appearing to her makes sense from Fonda’s perspective. For all she knows, she is Alpha, and how could any perspective other than Alpha’s appear to Alpha?

It is interesting to note that this argument is not entirely analogous to the corresponding argument regarding color appearance given earlier. It may be plausible to Fonda that the way colors appear necessarily emerges from the laws of the universe in which she finds herself. But it seems implausible that somehow Alpha’s perspective, to the exclusion of any other, necessarily emerges as the “present” one from these laws, given that Alpha is just one agent among many similar ones as far as these laws go.¹⁴ Instead, the argument here relies on the possibility of her complete identification with Alpha.

Again, perhaps this all reduces to standard arguments about personal identity and the self. If so, then again, this on the one hand suggests that the debate has been on the right track, while on the other hand also casting a clearer lens on it. This is because unlike in standard scenarios in the literature on personal identity and the self, here it is clearly true that Fonda “could have been someone else”—i.e., she could have had a different agent’s perspective in the simulation displayed to her on the VR system. This highlights, again, that the problem fundamentally has an important epistemological component.

How can we avoid concluding that further facts exist in Case C, which corresponds to our own world? It seems that the approach most likely to succeed is to argue that, while in Case A the conclusion of further facts is justified, it is not in Case B. Moreover, the most natural way to do so is to counter the argument in favor of further facts in Case B in a way that is similar to how arguments in favor of further facts in our own world (Case C) are often countered. But it appears that these counterarguments lose at least some of their immediate intuitive appeal when moving from Case C to Case B. This is because, by construction, there are in fact further facts in Case B, making it difficult to point out where exactly the argument that further facts exist goes wrong. Moreover, this argument is essentially the same as in Case A, where presumably we do believe the argument is correct. Therefore, the counterargument needs to rely entirely on Fonda’s epistemic limitations in Case B. Again, I believe that this is the most natural way to avoid the conclusion that further facts exist. But I also believe that the counterargument is in need of further fleshing out. In any case, if one agrees that Fonda’s situation in Case B is epistemically sufficiently like ours (Case C), so that any arguments available to us against further facts should be available to Fonda in Case B as well, we obtain a nontrivial conclusion. This is that we cannot know with certainty that qualia supervene on the physical facts, because after all, (say) an inverted spectrum is genuinely possible in Case B.¹⁵ At most, one can argue that we have no good reason to believe that they fail to supervene on the physical facts, and hence are not justified in concluding that there are further facts.

One may take other routes. I do not see how one could reasonably hold that Fonda’s argument in Case A is flawed. On the other hand, perhaps one could successfully argue that there is a relevant difference in Fonda’s epistemic situation between Cases B and C. Of course, Case B is intended to be epistemically as similar to Case C as possible, and for any remaining potentially relevant epistemic difference between the two cases, it seems we can modify Case B appropriately to make the difference go away. One might perhaps argue that Case B describes a scenario that is inherently impossible—or at least that it would become so after sufficiently many of these modifications. But I see no convincing reason to think so.¹⁶

Debates about further facts are ancient and clearly I have not settled them. I do believe that the three cases presented here, besides putting a modern spin on these questions, help to disentangle some of the different aspects relevant to these debates. They also allow us to treat different types of further facts in a more uniform manner.