The interface theory of perception: the future of the science of the mind?
Perception is typically conceptualized as a neurocognitive system that evolved to reveal the truth about objects and events in the world. Hoffman et al.'s Interface Theory of Perception questions this assumption. If true, the implications for the science of the mind are profound.
KeywordsPerception Evolution Embodied cognition Computational theory of mind
If you believe in evolution, you must then believe that the world we perceive is not the world we actually inhabit. Not even close. The objects we see or hear or touch do not exist independently of the mind that constructs them. Our perception is nothing more than a useful fiction. This is the core of Hoffman, Singh, and Prakash’s Interface Theory of Perception and the claim that the present Special Issue examines in depth.
The idea is far from new. Indeed, philosophers have entertained it for centuries. But Hoffman and colleagues give the theory some empirical teeth and force us to take it seriously, along with its mind-blowing implications.
The empirical teeth come in the form of evolutionary game simulations and genetic algorithms to model natural selection. Scenarios were set up in which the probability of the evolution of different perceptual strategies could be assessed. The critical perceptual strategies in the virtual test tube consisted of (1) those that represented truth about the nature of the (simulated) world, or (2) those that favored fitness payoffs over truth. For example, imagine a world in which physically red and green objects have the same positive fitness payoff (they are equally nutritious) while blue and orange objects have the same negative fitness payoff (they are equally toxic). One can then ask, how often does a perceptual strategy evolve such that the organism perceives all four color categories veridically, i.e., they see the truth, versus another strategy in which the organism only sees two categories corresponding to fitness payoffs (red/green and blue/orange) rather than truth? The result of many such simulations under a range of scenarios is strikingly consistent: fitness wins every time and truth goes extinct. In fact, truth never gets on the evolutionary game board.
This should be no surprise. How could evolution favor veridical perception if the truth doesn’t help make babies? It can’t unless it does so accidentally (i.e., it’s a spandrel), which is highly unlikely given the complexity of our perceptual systems, or unless truth and fitness are monotonically related. Hoffman and colleagues offer arguments for why truth and fitness are generally unlikely to be monotonically related, for example, the need for homeostasis in biological systems. I can offer, in addition, a simple example. It is quite difficult to invoke “truth” to explain species-specific sexual attraction (humans are attracted to humans, baboons to baboons), whereas fitness considerations readily explain why beauty is in the eye of the (species) beholder.
Hoffman et al. call their model the Interface Theory of Perception (ITP) and illustrate it intuitively using a graphical user interface (GUI) analogy. The desktop display of your computer shows a set of icons representing files, folders, operations (such as trash/delete), and apps. You don’t take that interface literally. You understand that your latest manuscript isn’t literally a little rectangle sitting in a clutter of other little rectangles in the upper left corner of your display. Rather that little rectangle is just a convenient icon that represents your manuscript. Similarly you understand that if you drag that little rectangle to the trashcan in frustration, the rectangle isn’t literally in the trashcan. Rather, the desktop is a convenient means to interface with the underlying reality of your computer. It’s useful because it hides the truth and presents instead a set of user-friendly shortcuts for writing papers, sending messages, and manipulating photos. Notice too how notions of causality play out in the interface. The cursor, the little rectangle, and the trashcan icons themselves have no causal power. It’s not the movement of the rectangle icon to the trashcan icon that causes the file to disappear; it’s the underlying electric currents and switches that actually have causal power. We understand this because sometimes something goes wrong with our GUI and a successful move of an icon from one place to another results in no actual effect (e.g., the drive doesn’t eject). Perception, Hoffmann et al. argue, is precisely the same: what we experience is nothing more than a set of species-specific icons, user-friendly shortcuts for staying alive and reproducing. Similarly, there is no causal power inherent to the objects we perceive. It appears that a tennis racquet can cause a ball to move, but that is nothing more than a juxtaposition of icons in our interface. The real causal powers are hidden from us, according to ITP.
If right, the implications of ITP are profound. For example, it immediately rules out currently popular ideas concerning Gibsonian-style “direct perception,” a foundational idea behind the embodied cognition movement. Embodied cognition takes many forms but generally tends toward a view in which high-level cognitive states are grounded in lower-level perceptual experiences, which themselves are grounded in the body as it interacts with the physical environment. Some theorists even eschew traditional notions of computation as the basis of mental function, talking instead of direct perception of, and/or “resonance” with, the environment. According to ITP, there is nothing remotely direct about perception (it’s just a set of icons), the environment can’t serve to ground cognition because it is a product of perception, and computation is fundamental. Hoffman et al. do discuss a perception-action loop, which is a term familiar to embodied cognition theorists, but read carefully: it is a computational loop tuned to fitness, not to truth or simple resonance with an objective reality. If ITP is correct, embodied cognition will be remembered as a quaint regression in the history of psychology.
Cognitive or computational neuroscientists don’t fare much better in the jet wash of the ITP. Our beloved neurons and the brain they inhabit, you see, are just icons too, our mind’s construction of objects as viewed through microscopes and magnetic resonance imaging (MRI) machines. And similar to tennis racquets, they have no causal powers in and of themselves. For the bench neuroscientist this revelation is of little consequence in the same sense that an engineer building a bridge doesn’t have to worry about quantum uncertainty, wave-particle duality, or entanglement. Both the neuroscientist and the engineer are working within the interface, trying to build or understand states and interactions of the desktop icons. But if we want to go beyond this “applied science” and understand the true nature of the mind and the reality beyond it, we can’t look to neurons or brains, according to the ITP. We have to look deeper.
There is something about quantum theory that is different in character from any physical theory posed before. To put a finger on it, the issue is this: The basic statement of the theory—the one we have all learned from our textbooks—seems to rely on terms our intuitions balk at as having any place in a fundamental description of reality. The notions of “observer” and “measurement” are taken as primitive, the very starting point of the theory. This is an unsettling situation! Shouldn’t physics be talking about what is before it starts talking about what will be seen and who will see it? (p. 1) (Fuchs, 2010)
What exactly qualifies some physical systems to play the role of ‘measurer’? Was the wavefunction of the world waiting to jump for thousands of millions of years until a single-celled living creature appeared? Or did it have to wait a little longer, for some better qualified system . . . with a PhD? (p. 34) (Bell, 1990).
The ITP underlines these nagging problems in physics. The physical world is not something we as a species (or any species) can directly perceive. Our measurements, therefore, are species-specific and we should worry if they form the starting point for physical theories. At the same time, the ITP represents some level of convergence with quantum physics in that both ITP and quantum theory come to the same conclusion: the world we experience isn’t true reality. In this sense, quantum mechanics provides confirmation for ITP.
The point here is that the stakes are high. If ITP is on the right track, most traditional psychological and neuroscientific theories have starting assumptions that are dead wrong. It is not an exaggeration to say that, if true, ITP would represent a paradigm shift in theories of the mind, one in which evolutionary theory plays a central role and the role of the mind is to compute perception-action correspondences that subserve fitness rather than reconstruct reality.
Because there is so much on the line, the theory deserves serious scrutiny. This Special Issue on the ITP represents a first round of cross-examination. In the first article Hoffman et al. present their case for the ITP. This is followed by several hard-hitting commentaries on various components of the theory. No doubt several of the issues that occurred to you in reading the present synopsis appear in the commentaries along with several that didn’t occur to you. In the final article Hoffman et al. respond to their critics and sympathizers.
This is not light reading (nor is it impenetrable to a broad audience!), but it is worth studying, understanding, sharing, and discussing. It could be the future of the science of the mind.