Advertisement

A Cognitive Approach to Audio Description

  • Jana Holsanova
Chapter
Part of the Palgrave Studies in Translating and Interpreting book series (PTTI)

Abstract

Holsanova presents theoretical and methodological approaches relevant for the research on audio description (AD) and exemplifies the application of these approaches by studies on live AD of films conducted at Lund University. A cognitive, reception-oriented perspective on AD and the framework of embodied cognition are in focus. It is claimed that previous research on scene perception, scene description and mental imagery can be preferably adopted in the study of AD. It is further argued that an interdisciplinary framework, integration of theoretical approaches and triangulation of methods is necessary in order to investigate such a complex phenomenon as AD.

Keywords

Mental Image Verbal Description Visual Scene Mental Imagery Blind Person 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Alfredson, D. (Director). (2013). Skumtimmen [Motion picture]. Sweden: Fundament Film.Google Scholar
  2. Bergen, B. K., Lindsay, S., Matlock, T., & Narayanan, S. (2007). Spatial and linguistic aspects of visual imagery in sentence comprehension. Cognitive Science, 31, 733–764.CrossRefGoogle Scholar
  3. Blomberg, J., Holsanova, J., & Gärdenfors, P. (in progress) Event structure in audio description of movies.Google Scholar
  4. Cabeza-Cáceres, C. (2013). Audiodescripció i recepció. Efecte de la velocitat de narració, l’entonació i l’explicitació en la comprensió fílmica. Published PhD. Accessed July 4, 2014, from http://www.tdx.cat/handle/10803/113556
  5. Cattaneo, Z., & Vecchi, T. (2011). Blind vision: The neuroscience of visual impairment. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
  6. Cornoldi, C., De Beni, R., Roncari, S., & Romano, S. (1989). The effects of imagery instructions on totally congenitally blind recall. European Journal of Cognitive Psychology, 1, 321–331.CrossRefGoogle Scholar
  7. Eimer, M. (2004). Multisensory integration: How visual experience shapes spatial perception. Current Biology, 14, 115–117.CrossRefGoogle Scholar
  8. Forceville, C. (2014). Relevance Theory as a model for analysing multimodal communication. In D. Machin (Ed.), Visual communication (Handbooks of Communication Science [HoCS]). Berlin: Walter De Gruyter, 51–70.Google Scholar
  9. Fresno Cañada, N. (2014). La (re)construcción de los personajes fílmicos en la audiodescripción. Efectos de la cantidad de información y de su segmentación en el recuerdo de los receptores. Published PhD. Accessed June 4, 2015, from http://www.tdx.cat/handle/10803/285420
  10. Gallese, V., & Goldman, A. (1998). Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Sciences, 2(12), 493–501.CrossRefGoogle Scholar
  11. Gärdenfors, P. (2000). Conceptual spaces. Cambridge, MA: MIT Press.Google Scholar
  12. Gärdenfors, P. (2014). The geometry of meaning: Semantics based on conceptual spaces. Cambridge, MA: MIT Press.Google Scholar
  13. Glenberg, A. M., Gutierrez, T., Levin, J. R., Japuntich, S., & Kaschak, M. P. (2004). Activity and imagined activity can enhance young children’s reading comprehension. Journal of Educational Psychology, 96, 424–436.CrossRefGoogle Scholar
  14. Goldin-Meadow, S., Nusbaum, H., Kelly, S., & Wagner, S. (2001). Explaining math: Gesturing lightens the load. Psychological Science, 12, 516–522.CrossRefGoogle Scholar
  15. Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representation of action words in human motor and premotor cortex. Neuron, 41, 301–307.CrossRefGoogle Scholar
  16. Hirvonen, M. (2013). Sampling similarity in image and languages—Figure and ground in the analysis of filmic audio description. SKY Journal of Linguistics, 26, 87–115.Google Scholar
  17. Holsanova, J. (1999). På tal om bilder. Om fokusering av uppmärksamhet i och strukturering av talad beskrivande diskurs (Speaking of pictures. On focusing attention and structuring spoken descriptive discourse). Lund University Cognitive Studies, 78.Google Scholar
  18. Holsanova, J. (2001). Picture viewing and picture description. Two Windows on the Mind. Doctoral dissertation. Lund University Cognitive Studies, 83.Google Scholar
  19. Holsanova, J. (2008). Discourse, vision, and cognition. Amsterdam: John Benjamins.CrossRefGoogle Scholar
  20. Holsanova, J. (2011). How we focus attention in picture viewing, picture description, and during mental imagery. In K. Sachs-Hombach & R. Totzke (Eds.), Bilder Sehen, Denken. Cologne: Herbert von Halem Verlag.Google Scholar
  21. Holsanova, J. (Ed.). (2012). Methodologies for multimodal research, Special issue of Visual Communication, 11(3).Google Scholar
  22. Holsanova, J. (2014a). Reception of multimodality: Applying eye tracking methodology in multimodal research. In C. Jewitt (Ed.), Routledge handbook of multimodal analysis (2nd ed.). London: Routledge. 287–298.Google Scholar
  23. Holsanova, J. (2014b). In the eye of the beholder: Visual communication from a recipient perspective. In D. Machin (Ed.), Visual communication (Handbooks of Communication Science [HoCS]). Berlin: Walter De Gruyter. 331–355.Google Scholar
  24. Holsanova, J., & Forceville, C. (in progress). Evaluating audio description of animated film.Google Scholar
  25. Holsanova, J., Hildén, A., Samlson, M., Kesen Tundell, V. (2015). Audio description and audio subtitles - a study of user preferences. With guidelines for audiovisual media. Stockholm: Tundell Salmson Lär.Google Scholar
  26. Holsanova, J., Andrén, M., & Wadensjö, C. (forthcoming). Syntolkning: forskning och praktik (Audio description: Research and practices). Lund University Cognitive Studies.Google Scholar
  27. Holsanova, J., Hedberg, B., & Nilsson, N. (1999). Visual and verbal focus patterns when describing pictures. In W. Becker, H. Deubel, & T. Mergner (Eds.), Current Oculomotor research: Physiological and psychological aspects. New York: Plenum.Google Scholar
  28. Igareda, P. (2012). The audio description of emotions and gestures in Spanish spoken film. In A. Serban, A. Matamala, & J. M. Lavaur (Eds.), Audiovisual translation in close-up: Practical and theoretical approaches. Bern: Peter Lang.Google Scholar
  29. Johansson, R. (forthcoming). Mentala bilder hos seende och blinda. In J. Holsanova, M. Andrén, & C. Wadensjö (Eds.), Syntolkning: forskning och praktik (Audio description: Research and practices). Lund University Cognitive Studies.Google Scholar
  30. Johansson, R., Holsanova, J., Dewhurst, R., & Holmqvist, K. (2012). Eye movements during scene recollection have a functional role, but they are not reinstatements of those produced during encoding. Journal of Experimental Psychology. Human Perception and Performance, 38, 1289–1314.CrossRefGoogle Scholar
  31. Johansson, R., Holsanova, J., & Holmqvist, K. (2006). Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness. Cognitive Science, 30, 1053–1079.CrossRefGoogle Scholar
  32. Johansson, R., Holsanova, J., & Holmqvist, K. (2013). Using eye movements and spoken discourse as windows to inner space. In C. Paradis, J. Hudson, & U. Magnusson (Eds.), Conceptual spaces and the construal of spatial meaning: Empirical evidence from human communication. Oxford: Oxford University Press.Google Scholar
  33. Kluckhorn, K. (2005). Informationssstrukturierung als Kompensationsstrategie – Audiodeskription und Syntax. In Ulla Fix (Eds.), Hörfilm. Bildkompensation durch Sprache (Berlin: Erich Schmidt Verlag). 49–66.Google Scholar
  34. Kozhevnikov, M., Kosslyn, S., & Shephard, J. (2005). Spatial versus object visualizers: A new characterization of visual cognitive style. Memory & Cognition, 33, 710–726.CrossRefGoogle Scholar
  35. Kruger, J.-. L. (2010). Audio description, audio narration—A new era in AVT. Perspectives: Studies in Translatology, 18(1), 141–142.Google Scholar
  36. Kruger, J.-L. (2012). Making meaning in AVT: Eye tracking and viewer construction of narrative. In I. Mazur, & J.-L. Kruger (Eds.), Perspectives: Studies in translatology, Special Issue, 20(1).Google Scholar
  37. Laeng, B., Bloem, I. M., D’Ascenzo, S., & Tommasi, L. (2014). Scrutinizing visual images: The role of gaze in mental imagery and memory. Cognition, 131, 263–283.CrossRefGoogle Scholar
  38. Matamala, A., & Orero, P. (2008). Designing a course on Audio Description: Main competences of the future professional. Linguistica Antverpiensa, 6, 329–344.Google Scholar
  39. Moulton, S. T., & Kosslyn, S. M. (2009). Imagining predictions: Mental imagery as mental emulation. Philosophical Transactions of the Royal Society, B: Biological Sciences, 364, 1273–1280.CrossRefGoogle Scholar
  40. Noordzij, M. L., Zuidhoek, S., & Postma, A. (2007). The influence of visual experience on visual and spatial imagery. Perception, 36, 101–112.CrossRefGoogle Scholar
  41. Nordqvist, S. (1990). Kackel i trädgårdslandet (Opal).Google Scholar
  42. Orero, P. (2005). Teaching audiovisual accessibility. Translating Today, 4, 12–15.Google Scholar
  43. Orero, P. (2012). Audio Description behaviour: Universals, regularities and guidelines. International Journal of Humanities and Social Science (IJHSS), 2(17), 195–202.Google Scholar
  44. Orero, P., & Vilaró, A. (2012). Eye tracking analysis of minor details in films for Audio Description. MonTI, 4, 295–319.CrossRefGoogle Scholar
  45. Pietrini, P., Furey, M. L., Ricciardi, E., Gobbini, M. I., Wu, W. H. C., Cohen, L., et al. (2004). Beyond sensory images: Object-based representation in the human ventral pathway. Proceedings of the National Academy of Sciences of the United States of America, 101(15), 5658–5663.CrossRefGoogle Scholar
  46. Postma, A., Zuidhoek, S., Noordzij, M.L., & Kappers, A.M.L. (2008). Haptic orientation perception benefits from visual experience: evidence from early blind, late blind and sighted people. Perception & Psychophysics, 70, 1197–1206.Google Scholar
  47. Richardson, D. C., Altmann, G. T. M., Spivey, M. J., & Hoover, M. A. (2009). Much ado about eye movements to nothing: A response to Ferreira et al.: Taking a new look at looking at nothing. Trends in Cognitive Science, 13(6), 235–236.CrossRefGoogle Scholar
  48. Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131–141.CrossRefGoogle Scholar
  49. Röder, B., & Rösler, F. (2004). Compensatory plasticity as a consequence of sensory loss. In G. Calvert, C. Spence, & B. E. Spence (Eds.), The handbook of multisensory processes. Cambridge, MA: MIT Press.Google Scholar
  50. Rosch, E., Thompson, E., & Varela, F. J. (1991). The embodied mind: Cognitive science and human experience (Paperback 1992nd ed.). Cambridge, MA: MIT Press.Google Scholar
  51. Snyder, J. (2005). Audio description. The visual made verbal across arts disciplines—Across the globe. Translating Today, 4, 15–17.Google Scholar
  52. Sperber, D. and Wilson, D. (1995). Relevance: Communication and Cognition. 2nd edition. Oxford: Blackwell.Google Scholar
  53. Strukelj, A. (forthcoming). Praktiska erfarenheter av syntolkning – en intervjustudie. In J. Holsanova, M. Andrén, & C. Wadensjö (Eds.), Syntolkning – forskning och praktik. (Audio description – research and practices). Lund University Cognitive Studies.Google Scholar
  54. Suddendorf, T., & Corballis, M. C. (2007). The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences, 30, 299–351.Google Scholar
  55. Vandaele, J. (2012). What meets the eye. Cognitive narratology for audio description. In I. Mazur, & J.-L. Kruger (Eds.), Perspectives: Studies in translatology, Special Issue, 20(1).Google Scholar
  56. Vercauteren, G., & Orero, P. (2013). Describing facial expressions: Much more than meets the eye. Quaderns de Traducció, 20, 187–199.Google Scholar
  57. Vilaró, A., Duchowski, A. T., Orero, P., Grindinger, T., Tetreault, S., & di Giovanni, E. (2012). How sound is the Pear Tree? Testing the effect of varying audio stimuli on visual attention distribution. Perspectives: Studies in Translatology, 20(1), 55–65.CrossRefGoogle Scholar
  58. Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625–636.CrossRefGoogle Scholar
  59. Zwaan, R. A., Magliano, J. P., & Graesser, A. C. (1995). Dimensions of situation-model construction in narrative comprehension. Journal of Experimental Psychology. Learning, Memory, and Cognition, 21, 386–397.CrossRefGoogle Scholar
  60. Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology, 135(1), 1–11.CrossRefGoogle Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s) 2016

Authors and Affiliations

  • Jana Holsanova
    • 1
  1. 1.Lund UniversityHelsingborgSweden

Personalised recommendations