Courting the Visual Image: The Ability of Digital Graphics and Interfaces to Alter the Memory and Behaviour of the Viewer

  • Damian SchofieldEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10901)


An intrinsic connection exists between humans and the memories they create; they define who we are, where we came from and our accomplishments and failures. However, decades of research has shown how fragile human memory can be. Almost all human computer interfaces involve vision and most rely on vision as the primary means of passing information to the user [1]. It is worth considering that perhaps this specific form of media interaction requires special care and attention due to its inherently persuasive nature, and the undue reliance that the viewer may place on information presented through a (potentially photorealistic) visualisation medium. Their influence on human memory and behaviour cannot be underestimated.

This paper will introduce research undertaken by the author over the past 25 years that has experimented with, and examined a range of visual based presentation technology into courtrooms all over the world. Courtrooms are environments where the decisions made (based on human memory and comprehension) can significantly affect the lives of others. This paper describes research undertaken to assess the effect of visual technology on users (in particular their memory and decision making abilities) and describes some of the issues raised by the experimental results. The work presented in this paper connects psychological research with human cognitive and perceptual processes and limitations, to allow the evaluation and optimisation of visual interfaces. The paper concludes with a discussion of the potential benefits and problems of designing interactive visual technology when considering the impact on human cognition.


Visual images Computer graphics Psychology Evidence HCI 


  1. 1.
    Powers, D.: Vision in HCI: embodiment, multimodality and information capacity. In: Proceedings of the Workshop on the Use of Vision in HCI (VisHCI 2006), Canberra, Australia, vol. 56 (2006)Google Scholar
  2. 2.
    Schofield, D., Mason, S.: Using graphical technology to present evidence. In: Mason, S. (ed.) Electronic Evidence, 3rd edn, pp. 217–253. LexisNexis Butterworths, London (2012)Google Scholar
  3. 3.
    Burton, A., Schofield, D., Goodwin, L.M.: Gates of global perception: forensic graphics for evidence presentation. In: Proceedings of ACM Symposium on Virtual Reality Software and Technology, Singapore, pp. 103–111 (2005)Google Scholar
  4. 4.
    Joseph, G.P.: Modern Visual Evidence. Law Journal Seminars Press, New York (2009)Google Scholar
  5. 5.
    Feigenson, N., Spiesel, C.O.: Law on Display: The Digital Transformation of Legal Persuasion and Judgment. NYU Press, New York (2009)CrossRefGoogle Scholar
  6. 6.
    Mervis, J.: Court views engineers as scientists. Science 284(5411), 21 (1999)CrossRefGoogle Scholar
  7. 7.
    Tufte, E.R.: The visual display of quantitative information. Am. J. Phys. 53(11), 1117–1118 (1983)CrossRefGoogle Scholar
  8. 8.
    Grimes, W.D.: Classifying the Elements in a Scientific Animation, Accident Reconstruction: Technology and Animation, USA Society of Automotive Engineers, SAE Paper no. 940919, pp. 39–404 (1994)Google Scholar
  9. 9.
    Burgoon, J.K.: Interactivity in human–computer interaction: a study of credibility, understanding and influence. Comput. Hum. Behav. 16(6), 553–574 (2000)CrossRefGoogle Scholar
  10. 10.
    Shapiro, M.A., McDonald, D.G.: I’m not a doctor, but I play one in virtual reality: implications of judgments about reality. J. Commun. 42(4), 94–114 (1992)CrossRefGoogle Scholar
  11. 11.
    Gilbert, D.T.: How mental systems behave. Am. Psychol. 46(2), 107–119 (1991)CrossRefGoogle Scholar
  12. 12.
    Hopkins, R.: Picture, Image and Experience. Cambridge University Press, Cambridge (1998)Google Scholar
  13. 13.
    Schofield, D.: Animating and interacting with graphical evidence: bringing courtrooms to life with virtual reconstructions. In: Proceedings of IEEE Conference on Computer Graphics, Imaging and Visualisation, Bangkok, Thailand, pp. 321–328 (2007)Google Scholar
  14. 14.
    Spiesel, C.O., Sherwin, R.K., Feigenson, N.: Law in the age of images: the challenges of visual literacy. In: Wagner, A., Summerfield, T., Vanegas, F.S.B. (eds.) Contemporary Issues of the Semiotics of Law, Oñati International Series in Law and Society, vol. 13 (2005)Google Scholar
  15. 15.
    Kuehn, P.F.: Maximising your persuasiveness: effective computer generated exhibits. Journal of the DuPage Country Bar Association. Accessed 21 Jan 2018
  16. 16.
    Schroder, K.J.: Computer Animation: The Litigator’s Legal Ally, Computers and Law. University of Buffalo, Buffalo (1997)Google Scholar
  17. 17.
    Devine, D.J., Clayton, L.D., Dunford, B.B., Seying, R., Pryce, J.: Jury decision making, 45 years of empirical research on deliberating groups. Psychol. Pub. Policy Law 7(3), 622–727 (2001)CrossRefGoogle Scholar
  18. 18.
    Durkin, K.P., Dunn, C.H.: Building your case for the jury, Litig. J. 36(3) (2010)Google Scholar
  19. 19.
    Rubin, C.B.: A paperless trial. Litig. Magaz. 19(3), 66–88 (1992)Google Scholar
  20. 20.
    Loftus, E.R., Loftus, G.R.: On the permanence of stored information in the human brain. Am. Psychol. 35(5), 409–420 (1980)CrossRefGoogle Scholar
  21. 21.
    Leader, L., Schofield, D.: Madness in the method? Potential pitfalls in handling expert evidence. J. Pers. Inj. Law 6(1), 68–86 (2006)Google Scholar
  22. 22.
    Schofield, D.: The future of evidence: new applications of digital technologies, forensic science: classroom to courtroom In: Proceedings of 18th International Symposium of the Forensic Sciences, Fremantle, Western Australia (2006)Google Scholar
  23. 23.
    Richter, E.M., Humke, A.M.: Demonstrative evidence: evidence and technology in the courtroom. In: Weiner, R.L., Bornstein, B.H. (eds.) Handbook of Trial Consulting, pp. 187–201. Springer, Heidelberg (2011). Scholar
  24. 24.
    Schofield, D.: Playing with evidence: using video games in the courtroom. J. Entertainment Comput. 2(1), 47–58 (2011). (Special Issue: Video Games as Research Instruments)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Jones, I.S., Muir, D.W., Groo, S.W.: Computer animation – admissibility in the courtroom, accident reconstruction: technology and animation. Soc.Autom. Eng. 1, 143–151 (1991)Google Scholar
  26. 26.
    Lederer, F.I., Solomon, S.H.: Courtroom technology – an introduction to the onrushing future. In: Proceedings of Fifth National Court Technology Conference (CTC5). National Centre for State Courts, Detroit (1997)Google Scholar
  27. 27.
    Lederer, F.I.: Courtroom technology: for trial lawyers, the future is now. Crim. Justice 19(1), 14–21 (2004)Google Scholar
  28. 28.
    Seltzer, R.F.: Evidence and Exhibits at Trial, 387 PLI/Lit 371 (1990)Google Scholar
  29. 29.
    Cobo, M.E.: A Strategic Approach to Demonstrative Exhibits and Effective Jury Presentations, 3 PLI/Lit 359 (1990)Google Scholar
  30. 30.
    Krieger, R.: Sophisticated computer graphics come of age-and evidence will never be the same. J. Am. Bar Assoc. 78, 93–95 (1992)Google Scholar
  31. 31.
    Thomas, R.D.: Computer Re-enactment. Accessed 21 Jan 2018
  32. 32.
    Sherwin, R.K.: When Law Goes Pop: The Vanishing Line between Law and Popular Culture, 2nd edn. University Of Chicago Press, Chicago (2002)Google Scholar
  33. 33.
    Kassin, S.M., Dunn, M.A.: Computer-animated displays and the jury: facilitative and prejudicial effects. Law Hum. Behav. 21(3), 269–281 (1997)CrossRefGoogle Scholar
  34. 34.
    Selbak, J.: Digital litigation: the prejudicial effects of computer-generated animation in the courtroom. Berkley Technol. Law J. 19(9:2), 337 (1994)Google Scholar
  35. 35.
    Dunn, M.A., Feigenson, N., Salovey, P.: The jury persuaded (and not): computer animation in the courtroom. Law Policy 28(2), 228–248 (2006)CrossRefGoogle Scholar
  36. 36.
    Tait, D.: Rethinking the role of the image in justice: visual evidence and science in the trial process. Law Probab. Risk 6(1–4), 311–318 (2007)CrossRefGoogle Scholar
  37. 37.
    Tait, D., Goodman-Delahunty, J., Schofield, D., Jones, D.: Evidence on the holodeck: jury responses to computer simulations. In: Proceedings of the 4th Law and Technology Conference. Australian Institute of Judicial Administration, Sydney (2008)Google Scholar
  38. 38.
    Schofield, D.: Animating evidence: computer game technology in the courtroom. J. Inf. Law Technol. (JILT) 1, 1–20 (2009)Google Scholar
  39. 39.
    Loftus, G.R., Loftus, E.F.: The influence of one memory retrieval on a subsequent memory retrieval. Mem. Cognit. 2(3), 467–471 (1974)CrossRefGoogle Scholar
  40. 40.
    Young, W., Davis, M., McNeill, I.M., Malhotra, B., Russell, S., Unsworth, K., Clegg, C.W.: Changing behaviour: successful environmental programmes in the workplace. Bus. Strategy Environ. 24(8), 689–703 (2015)CrossRefGoogle Scholar
  41. 41.
    Davis, R., Campbell, R., Hildon, Z., Hobbs, L., Michie, S.: Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol. Rev. 9(3), 323–344 (2015)CrossRefGoogle Scholar
  42. 42.
    Harding, E.J., Paul, E.S., Mendl, M.: Animal behaviour: cognitive bias and affective state. Nature 427(6972), 312 (2004)CrossRefGoogle Scholar
  43. 43.
    Haselton, M.G., Nettle, D., Murray, D.R.: The Evolution of Cognitive Bias. The Handbook of Evolutionary Psychology. Wiley, New York (2005)Google Scholar
  44. 44.
    Loftus, E.F.: Eyewitness Testimony. Harvard University Press, Cambridge (1996)CrossRefGoogle Scholar
  45. 45.
    Loftus, E.F.: Our changeable memories: legal and practical implications. Nat. Rev. Neurosci. 4, 231–234 (2003)CrossRefGoogle Scholar
  46. 46.
    Biederman, I., Ju, G.: Surface vs. edge-based determinants of visual recognition. Cognit. Psychol. 20, 38–64 (1998)CrossRefGoogle Scholar
  47. 47.
    Cooper, C.: Computer Animation on Trial, The San Diego Union Tribune (27th January 1993), E1 (1993)Google Scholar
  48. 48.
    Darley, J.M., Gross, P.H.: A hypothesis-confirming bias in labeling effects. J. Pers. Soc. Psychol. 44(1), 20–33 (1983)CrossRefGoogle Scholar
  49. 49.
    Nunez, N., McCoy, M.L., Clark, H.L., Shaw, L.A.: The testimony of elderly victim/witnesses and their impact on juror decisions: the importance of examining multiple stereotypes. Law Hum. Behav. 23(4), 413–423 (1999)CrossRefGoogle Scholar
  50. 50.
    Schuller, R.A., Terry, D., McKimmie, B.: The impact of expert testimony on jurors’ decisions: gender of the expert and testimony complexity. J. Appl. Soc. Psychol. 35(6), 1266–1280 (2005)CrossRefGoogle Scholar
  51. 51.
    Wuensch, K.L., Campbell, M.W., Kesler, F.C., Moore, C.H.: Racial bias in decisions made by mock jurors evaluating a case of sexual harassment. J. Soc. Psychol. 142(5), 587–600 (2002)CrossRefGoogle Scholar
  52. 52.
    Grice, H.P.: Logic and conversation. In: Cole, P., Morgan, J.L. (eds.) Speech Acts Syntax and Semantics, vol. 3, pp. 41–58. Academic Press, London (1975)Google Scholar
  53. 53.
    Goodwin, L., Schofield, D.: Visualising uncertainty: combining evidence with statistics. In: Proceedings of Conference on Expert Evidence: Causation, Proof and Presentation, Florence, Italy (2002)Google Scholar
  54. 54.
    Klein, G.: Recognition-primed decisions. In: Rouse, W.B. (ed.) Advances in Man-Machine Systems Research, vol. 5, pp. 47–92. JAI Press, Greenwich (1989)Google Scholar
  55. 55.
    Lurie, N.H., Mason, C.: Visual representation: implications for decision making. J. Market. 71, 160–177 (2007)CrossRefGoogle Scholar
  56. 56.
    Chapman, P.: Remembering what we’ve seen: predicting recollective experience from eye movements when viewing everyday scenes. In: Underwood, G.D.M. (ed.) Cognitive Processes in Eye Guidance, pp. 237–258. Oxford University Press, Oxford (2005)CrossRefGoogle Scholar
  57. 57.
    Drummond, M.A.: Courtroom Persuasion: Eight Keys to Success. Loyola University Press, Institute for Continuing Legal Education, Chicago (1999)Google Scholar
  58. 58.
    Yale, D.: Computers on the witness stand: expert testimony that relies on data generated by computers in the age of Daubert. Accessed 21 Jan 2018
  59. 59.
    Brewer, M.B.: A dual process model of impression formation. In: Wyer, R.S., Srull, T.K. (eds.) Advances in Social Cognition, vol. 1, pp. 1–36. Psychology Press (1998)Google Scholar
  60. 60.
    Morkes, J., Kernal, H.K., Nass, C.: Task-oriented computer-mediated communication and human-computer interaction. In: Proceedings of CHI 98 Conference Summary on Human Factors in Computing Systems, pp. 215–216. ACM (1998)Google Scholar
  61. 61.
    O’Flaherty, D.: Computer-generated displays in the courtroom: for better or worse? Web J. Curr. Legal Issues 2(4), 1–14 (1996)Google Scholar
  62. 62.
    Habermas, J.: The Structural Transformation of the Public Sphere. MIT Press, Boston (1996)Google Scholar
  63. 63.
    Baudrillard, J.: Simulacra and Simulation. University of Michigan, Michigan (1994)Google Scholar
  64. 64.
    Nichols, S., Haldane, C., Wilson, J.R.: Measurement of presence and side effects in virtual environments. Int. J. Hum. Comput. Stud. 52(3), 471–491 (2000)CrossRefGoogle Scholar
  65. 65.
    Hussin, N., Schofield, D., Shalaby, T.M.: Visualising information: evidence analysis for computer-generated animation (CGA). In: Proceedings of the 8th International Conference on Information Visualisation (IV), London, UK, pp. 903–908 (2004)Google Scholar
  66. 66.
    Rossner, M., Yamada, K.M.: What’s in a picture? The temptation of image manipulation. J. Cell Biol. 166(1), 11–15 (2004)CrossRefGoogle Scholar
  67. 67.
    Porter, G.: Visual culture in forensic science. Aust. J. Forensic Sci. 39(2), 81–91 (2007)CrossRefGoogle Scholar
  68. 68.
    Worring, M., Cucchiara, R.: Multimedia in forensics. In: Proceedings of the 17th ACM International Conference on Multimedia, New York, USA, pp. 1153–1154 (2009)Google Scholar
  69. 69.
    Noond, J., Schofield, D., March, J., Evison, M.: Visualising the scene: computer graphics and evidence presentation. Sci. Justice 42(2), 89–95 (2002)CrossRefGoogle Scholar
  70. 70.
    Bryce, J., Rutter, J.: Spectacle of the deathmatch: character and narrative in first-person shooters. In: King, G., Krzywinska, T. (eds.) ScreenPlay: Cinema/Videogames/Interfaces, pp. 66–80. Wallflower Press, Boston (2002)Google Scholar
  71. 71.
    Jones, E.E., Nesbett, R.E.: The actor and the observer: divergent perceptions of the causes of behavior. In: Jones, E.E. (ed.) Attribution: Perceiving the Causes of Behavior, pp. 79–94. General Learning Press, Morristown (1971)Google Scholar
  72. 72.
    MacDorman, K.F.: Subjective ratings of robot video clips for human likeness, familiarity, and eeriness: an exploration of the uncanny valley. In: Proceedings of the ICCS/CogSci-2006 Long Symposium: Toward Social Mechanisms of Android Science, pp. 26–29 (2006)Google Scholar
  73. 73.
    Machado, L.S., Morales, R.M., Souza, D.L., Souza, L., Cunha, I.L.: A framework for development of virtual reality-based training simulators. In: Westewood, J.D. (ed.) Medicine Meets Virtual Reality. IOS Press, New York (2009)Google Scholar
  74. 74.
    Tromp, J., Schofield, D.: Practical experiences of building virtual reality systems. In: Proceedings of Designing and Evaluating Virtual Reality Systems Symposium. University of Nottingham, UK (2004)Google Scholar
  75. 75.
    Fielder, B.S.: Are your eyes deceiving you? The evidential crisis regarding the admissibility of computer-generated evidence. New York Law Sch. Law Rev. 48(1–2), 295–321 (2003)Google Scholar
  76. 76.
    Galves, F.: Where the not so wild things are: computers in the courtroom, the federal rules of evidence, and the need for institutional reform and more judicial acceptance. Harvard J. Law Technol. 13(2), 161–302 (2002)Google Scholar
  77. 77.
    Gold, S.: Forensic Animation – Its Origins, Creation, Limitations and Future, Expert Law. Accessed 21 Jan 2018
  78. 78.
    Marcotte, P.: Animated evidence: delta 191 crash re-created through computer simulations at trial. J. Am. Bar Assoc. 75, 52–57 (1989)Google Scholar
  79. 79.
    Porschmann, C., Pellegrini, R.S.: 3-D audio in mobile communication devices: effects of self-created and external sounds on presence in auditory virtual environments. J. Virtual Real. Broadcast. 7, 1–9 (2010)Google Scholar
  80. 80.
    Buck, U., Naether, S., Räss, B., Jackowski, C., Thali, M.J.: Accident or homicide–virtual crime scene reconstruction using 3D methods. Forensic Sci. Int. 225(1), 75–84 (2013)CrossRefGoogle Scholar
  81. 81.
    March, J., Schofield, D., Evison, M., Woodford, N.: Three-dimensional computer visualisation of forensic pathology data. Am. J. Forensic Med. Pathol. 25(1), 60–70 (2004)CrossRefGoogle Scholar
  82. 82.
    Maksymowicz, K., Tunikowski, W., Kościuk, J.: Crime event 3D reconstruction based on incomplete or fragmentary evidence material–case report. Forensic Sci. Int. 242, e6–e11 (2014)CrossRefGoogle Scholar
  83. 83.
    Craven, M., Taylor, I., Drozd, A., Purbrick, J., Greenhalgh, C., Benford, S.: Exploiting interactivity, influence, space and time to explore non-linear drama in virtual worlds. In: Proceedings of the SIGCHI, Human Factors in Computing Systems, pp. 30–37 (2001)Google Scholar
  84. 84.
    Fordham, J.G.: Muddying the waters with red herrings: a progress report on Western Australian jury research. In: Brooks-Gordon, B., Freeman, M. (eds.) Law and Psychology Current Legal Issues, vol. 9, pp. 338–360. Oxford University Press, Oxford (2006)CrossRefGoogle Scholar
  85. 85.
    Holmgren, J.A., Fordham, J.G.: The CSI effect and the Canadian and Australian jury. J. Forensic Sci. 56(S1), S63–S71 (2011)CrossRefGoogle Scholar
  86. 86.
    Schofield, D.: Displaying the bomb on the train: the challenge of preparing visual evidence. In: Tait, D., Goodman-Delahunty, J. (eds.) Juries, Science and Popular Culture in the Age of Terror, pp. 123–143. Palgrave Macmillan, Basingstoke (2016)Google Scholar
  87. 87.
    Burns, D.C.: When used in the criminal legal process forensic science shows a bias in favour of the prosecution. Sci. Justice 41(4), 271–277 (2001)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State University of New YorkOswegoUSA

Personalised recommendations