Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Sensory Experiences as Cryptic Symbols of a Multimodal User Interface


According to current theories of perception, our visual experiences match or approximate the true properties of an objective physical world. Ecological optics claims this match is direct, unmediated by psychological or computational processes. Computational theories claim that this match is indirect, the result of sophisticated algorithms that instantiate, e.g., the inferential methods of Bayesian estimation. The assumption of all of these theories that there is an objective, i.e., mind independent, physical world has proved incapable, so far, of yielding a scientific theory for the mind-body problem, a scientific theory of the relationship between conscious experiences and the brain. Therefore I explore, instead, the pos-sibility that sensory experiences constitute a multimodal user interface (MuI) between the perceiver and an objective world, an interface useful precisely because it does not match, approximate, or resemble that world. I also explore conscious realism, the ontological claim that the objective world itself is comprised entirely of conscious agents. Together, MUI theory and conscious realism provide a straightforward solution to the mind-body problem, and entail epiphysicalism: physical objects, such as quarks and brains and stars, are constructed by conscious agents, but such physical objects have no causal powers.


  1. Bennett, B.M., Hoffman, D.D., & Kakarala, R.. (1993). Modeling performance in observer theory. Journal of Mathematical Psychology, 37, 220–240.

  2. Bennett, B.M., Hoffman, D.D., & Murthy, P. (1993). Lebesgue logic for probabilistic reasoning, and some applications to perception. Journal of Mathematical Psychology, 37, 63–103.

  3. Bennett, B.M., Hoffman, D.D., & Prakash, C. (1989). Observer mechanics: A formal theory ofperception. San Diego: Academic Press. Available free:

  4. Bennett, B.M., Hoffman, D.D., & Prakash, C. (1991). Unity of perception. Cognition, 38, 295–334.

  5. Bennett, B.M., Hoffman, D.D., Prakash, C., Richman, S. 1996. Observer theory, Bayes theory, and psychophysics. In D. Knill and W. Richards (Eds) Perception as Bayesian inference. Cambridge University Press, pp. 163–212.

  6. Block, N., & Fodor, J. (1972). What psychological states are not. Philosophical Review, 81, 159–181.

  7. Bradley, D.R., & Petry, H.M. (1977). Organizational determi-nants of subjective contour: The subjective Necker cube. American Journal of Psychology, 90, 253–262.

  8. Chalmers, D.J. (1995). The conscious mind: In search of a fundamental theory. Oxford University Press.

  9. Churchland, P.M. (1995). The engine of reason, the seat of the soul: A philosophical journey into the brain. Cambridge, MA: MIT Press.

  10. Churchland, P.S. (2002). Brain-wise: Studies in neurophilosophy. Cambridge, MA: MIT Press.

  11. Crick, F., & Koch, C. (1998). Consciousness and neuroscience. Cerebral Cortex, 8, 97–107.

  12. Descartes, R. (1641/1984). The philosophical writings of Descartes. Translated by J. Cottingham, R. Stoothoff, and D. Murdoch. Cambridge University Press.

  13. Dennett, D. (1983). Consciousness explained. Boston: Little, Brown and Company.

  14. Edelman, G. (1992). Bright air, brilliant fire. New York: Basic Books.

  15. Edelman, G. (2004). Wider than the sky: The phenomenal gift of consciousness. Yale University Press.

  16. Edelman, G., & Tononi, G. (2000). A universe of conscious-ness: How matter becomes imagination. New York: Basic Books.

  17. Foster, J. (1991). The immaterial self: A defence of the Carte-sian dualist conception of the mind. London: Routledge.

  18. Gibson, J.J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Publishers.

  19. Griffin, D.R. (1998). Unsnarling the world knot: Conscious-ness, freedom, and the mind-body problem. Berkeley, CA: University of California Press.

  20. Helmholtz, H.L.F. (1910/1962). Handbook of physiological optics, Volume III. New York: Dover.

  21. Hoffman, D.D. (1998). Visual intelligence: How we create what we see. New York: W.W. Norton.

  22. Jackson, F. (1982). Epiphenomenal qualia. Philosophical Quarterly, 32, 127–136.

  23. Kant, I. (1781/2003). Critique of pure reason. New York: Dover.

  24. Knill, D., & Richards, W. (Eds) (1996). Perception as Baye-sian inference. Cambridge University Press.

  25. Koch, C. (2004). The quest for consciousness: A neurobiological approach. Englewood, CO: Roberts and Company Publishers.

  26. Marr, D. (1982). Vision. San Francisco: Freeman Press.

  27. Maynard-Smith, J. (1982). Evolution and the Theory of Games. Cambridge: Cambridge University Press.

  28. Penrose, R. (1994). Shadows of the mind. Oxford University Press.

  29. Poggio, T., Torre, V., & Koch, C. (1985). Computational vision and regularization theory. Nature, 317, 314–319.

  30. Sabra, A.I. 1978. Sensation and inference in Alhazen’s theory of visual perception. In Machamer, P.K. & Turnbull, R.G. (Eds) Studies in perception: Interrelations in the history of philosophy and science. Columbus: Ohio State University Press.

  31. Schneiderman, B. (1998). Designing the user interface: Strat-egies for effective human-computer interaction. Reading, MA: Addison-Wesley.

  32. Skyrms, B. (2000). Game Theory, Rationality and Evolution of the Social Contract. Journal of Consciousness Studies, 7, 269–284.

  33. Tye, M. (1996). Ten Problems of Consciousness. Cambridge, MA: MIT Press.

  34. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press.

  35. Wheeler, J.A., & Zurek, W.H. (1983). Quantum theory and measurement. Princeton University Press.

  36. Whitehead, A.N. (1929/1979). Process and reality: An essay in cosmology. New York: Free Press.

Download references

Author information

Correspondence to Donald D. Hoffman.

Rights and permissions

This article is published under an open access license. Please check the 'Copyright Information' section for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hoffman, D.D. Sensory Experiences as Cryptic Symbols of a Multimodal User Interface. Act Nerv Super 52, 95–104 (2010).

Download citation

Key words

  • Consciousness
  • Computation
  • Sensory Experience
  • Mind-Body Problem