Children's Literature in Education

, Volume 41, Issue 2, pp 85–104

Reading Multimodal Texts: Perceptual, Structural and Ideological Perspectives

Original Paper

Abstract

This article presents a tripartite framework for analyzing multimodal texts. The three analytical perspectives presented include: (1) perceptual, (2) structural, and (3) ideological analytical processes. Using Anthony Browne’s picturebook Piggybook as an example, assertions are made regarding what each analytical perspective brings to the interpretation of multimodal texts and how these perspectives expand readers’ interpretive repertoires. Drawing on diverse fields of inquiry, including semiotics, art theory, visual grammar, communication studies, media literacy, visual literacy and literary theory, the article suggests an expansion of the strategies and analytical perspectives readers being to multimodal texts and visual images. Each perspective is presented as necessary but insufficient in and of itself to provide the necessary foundation for comprehending texts. It is through an expansion of the interpretive strategies and perspectives that readers bring to a multimodal text, focusing on visual, textual, and design elements that readers will become more proficient in their interpretive processes.

Keywords

Multimodal texts Picturebooks Interpretation Comprehension Visual grammar 

References

  1. Aiello, Giorgia. (2006). Theoretical Advances in Critical Visual Analysis: Perception, Ideology, Mythologies, and Social Semiotics. Journal of Visual Literacy, 26(2), 89–102.Google Scholar
  2. Albers, Peggy. (2007). Visual Discourse Analysis: An Introduction to the Analysis of School-Generated Visual Texts. In D. W. Rowe, R. T. Jiminez, D. L. Compton, D. K. Dickinson, Y. Kim, K. M. Leander & V. J. Risko (Eds.), 56th Yearbook of the National Reading Conference (pp. 81–95). Oak Creek, WI: NRC.Google Scholar
  3. Albers, Peggy. (2008). Theorizing Visual Representation in Children’s Literature. Journal of Literacy Research, 40(2), 163–200.CrossRefGoogle Scholar
  4. Anstey, Michele, & Bull, Geoff. (2006). Teaching and Learning Multiliteracies: Changing Times, Changing Literacies Newark, DE: International Reading Association.Google Scholar
  5. Arnheim, Rudolf. (1986). Art and Visual Perception: A Psychology of the Creative Eye. Berkeley, CA: University of California Press.Google Scholar
  6. Barthes, Roland. (1977). The Rhetoric of the Image. In R. Barthes (Ed.), Image, Music, Text (pp. 32–51). New York: Hill and Wang.Google Scholar
  7. Beach, Richard, Encisco, Patricia, Harste, Jerome, Jenkins, Christine, Raina, Seemi Aziz, Rogers, Rebecca, Short, Kathy G., Sung, Yoo Kyung, Wilson, Melissa, & Yenika-Agbaw, Vivian. (2009). Exploring the “Critical” in Critical Content Analysis of Children’s Literature. In K. M. Leander, D. W. Rowe, D. K. Dickinson, M. K. Hundley, R. T. Jiminez & V. J. Risko (Eds.), 58th Yearbook of the National Reading Conference (pp. 129–143). Oak Creek, WI: NRC.Google Scholar
  8. Beardsley, Monroe C. (1981). Aesthetics: Problems in the Philosophy of Criticism. Indianapolis, IN: Hackett Publishing Company.Google Scholar
  9. Berger, John. (1972). Ways of Seeing. London:Penguin.Google Scholar
  10. Browne, Anthony. (1986). Piggybook. New York: Alfred A. Knopf.Google Scholar
  11. Chandler, David. (2007). Semiotics: The Basics, 2nd ed. London: Routledge.Google Scholar
  12. Chauvin, B. A. (2003). Visual or Media Literacy? Journal of Visual Literacy, 23(2), 119–128.Google Scholar
  13. Cohn, Neil. (2007). A Visual Lexicon. The Public Journal of Semiotics, 1(1), 35–56.Google Scholar
  14. Crawford, Patricia A. (2000). Inside the Picture, Outside the Frame: Semiotics and the Reading of Wordless Picture Books. Journal of Research in Childhood Education, 15(1), 66–80.Google Scholar
  15. Culler, Jonathan. (1997). Literary Theory: A Very Short Introduction. Oxford: Oxford University Press.Google Scholar
  16. Dondis, Donis A. (1973). A Primer of Visual Literacy. Cambridge, MA: MIT Press.Google Scholar
  17. Duncum, Paul. (2004). Visual Culture isn’t Just Visual: Multiliteracy, Multimodality, and Meaning. Studies in Art Education, 45(3), 252–264.Google Scholar
  18. Elkins, John. (2008). Visual Literacy. New York: Routledge.Google Scholar
  19. Fairclough, Norman. (2003). Analysing Discourse: Textual Analysis for Social Research. London: Routledge.Google Scholar
  20. Fish, Stanley. (1980). Is There a Text in This Class? The Authority of Interpretive Communities. Cambridge, MA: Harvard University Press.Google Scholar
  21. Fleckenstein, Kristie S. (2002). Inviting Imagery into Our Classrooms. In K. S. Fleckenstein, L. T. Calendrillo & D. A. Worley (Eds.), Language and Images in the Reading-Writing Classroom (pp. 3–25). Mahwah, NJ: Erlbaum.Google Scholar
  22. Freebody, Peter. (1992). A Socio-Cultural Approach: Resourcing Four Roles as a Literacy Learner. In A. Watson and A. Beadenhop (Eds.), Prevention of Reading Failure (pp. 26–41). Sydney: Ashton-Scholastic.Google Scholar
  23. Freebody, Peter, & Luke, Allan. (1990). Literacies Programs: Debates and Demands in Cultural Context. Prospect: Australian Journal of TESOL, 5(7), 7–16.Google Scholar
  24. Genette, Gerard. (1997). Paratexts: Thresholds of Interpretation. Translated by J. E. Lewin. Cambridge: Cambridge University Press.Google Scholar
  25. Golden, Joanne M., & Gerber, Annyce. (1990). A Semiotic Perspective of Text: The Picture Story Book Event. Journal of Reading Behavior, 22(3), 203–219.Google Scholar
  26. Gombrich, E. H. (1961). Art and Illusion: A Study in the Psychology of Pictorial Representation, 2nd ed. Princeton, NJ: Princeton University Press.Google Scholar
  27. Goodman, Nelson. (1976). Languages of Art. Indianapolis, IN: Hackett Publishing Company.Google Scholar
  28. Graham, Judith. (1990). Pictures on the Page. Exeter: Short Run Press.Google Scholar
  29. Halliday, Michael A. K. (1975). Learning How to Mean: Explorations in the Development of Language. London: Elsevier.Google Scholar
  30. Halliday, Michael A. K. (1978). Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.Google Scholar
  31. Harvey, Stephanie, & Goudvis, Anne. (2000). Strategies that Work: Teaching Comprehension to Enhance Understanding. Portland, ME: Stenhouse.Google Scholar
  32. Hassenmuller, Christine. (1987). Panofsky, Iconography, and Semiotics. Journal of Aesthetics and Art Criticism, 35, 289–301.Google Scholar
  33. Hassett, Dawnene D., & Schieble, Melissa B. (2007). Finding Space and Time for the Visual in K-12 Literacy Instruction. English Journal, 97(1), 62–68.CrossRefGoogle Scholar
  34. Idema, Rick. (2001). Analysing Film and Television: A Social Semiotic Account of Hospital: An Unhealthy Business. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of Visual Analysis (pp. 183–204). London: Sage.Google Scholar
  35. Jewitt, Carey, & Kress, Gunther (Eds.). (2003). Multimodal Literacy. New York: Peter Lang.Google Scholar
  36. Jewitt, Carey, & Oyama, Rumiko. (2001). Visual Meaning: A Social Semiotic Approach. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of Visual Analysis (pp. 134–156). London: Sage.Google Scholar
  37. Kress, Gunther. (2003). Literacy in the New Media Age. London: Routledge.CrossRefGoogle Scholar
  38. Kress, Gunther. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.Google Scholar
  39. Kress, Gunther, & van Leeuwen, Theo. (1996). Reading Images: The Grammar of Visual Design. London: Routledge Falmer.Google Scholar
  40. Leeds-Hurwitz, Wendy. (1993). Semiotics and Communication: Signs, Codes, Cultures. Hillsdale, NJ: Laurence Erlbaum Associates.Google Scholar
  41. Lewis, David. (2001). Reading Contemporary Picturebooks: Picturing Text. London: Routledge Falmer.Google Scholar
  42. Lister, Martin, & Wells, Liz. (2001). Seeing Beyond Belief: Cultural Studies as an Approach to Analyzing the Visual. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of Visual Analysis (pp. 61–91). London: Sage.Google Scholar
  43. Martineo, Radan, & Salway, Andrew. (2005). A System for Image-Text Relations in New (and Old) Media. Visual Communication, 4(3), 337–371.CrossRefGoogle Scholar
  44. Messaris, Paul. (1994). Visual Literacy: Image, Mind, & Reality. Boulder, CO: Westview Press.Google Scholar
  45. Messaris, Paul. (2003). Visual Communication: Theory and Research. Journal of Communication, 53(3), 551–556.CrossRefGoogle Scholar
  46. Mirzeoff, Nicholas. (1998). What is Visual Culture? In N. Mirzeoff (Ed.), The Visual Culture Reader. London: Routledge.Google Scholar
  47. Moebuis, William. (1986). Introduction to Picturebook Codes. Word & Image, 2(2), 141–158.Google Scholar
  48. Moss, Barbara. (2003). Exploring the Literature of Fact: Children’s Nonfiction Trade Books in the Elementary Classroom. New York: Guilford Press.Google Scholar
  49. Moss, Pamela A., Phillips, Denis C., Erickson, Frederick, Floden, Robert E., Lather, Patti A., & Barbara Schneider, L. (2009). Learning from Our Differences: A Dialogue Across Perspectives on Quality in Educational Research. Educational Researcher, 38(7), 501–517.CrossRefGoogle Scholar
  50. Nikolajeva, Maria, & Scott, Carole. (2000). The Dynamics of Picturebook Communication. Children’s Literature in Education, 31(4), 225–239.CrossRefGoogle Scholar
  51. Nodelman, Perry. (1984). How Picture Books Work. In H. Darling & P. Neumeyer (Eds.), Image and Maker: An Annual Dedication to the Consideration of Book Illustration (pp. 1–13). La Jolla, CA: Green Tiger Press.Google Scholar
  52. Owocki, Gretchen. (2003). Comprehension: Strategic Instruction for K-3 Students. Portsmouth, NH: Heinemann.Google Scholar
  53. Panofsky, Erwin. (1955). Meaning in the Visual Arts. Chicago: University of Chicago Press.Google Scholar
  54. Rose, Gillian. (2001). Visual Methodologies. London: Sage.Google Scholar
  55. Samara, Timothy. (2007). Design Elements: A Graphic Style Manual. Beverly, MA: Rockport.Google Scholar
  56. Scholes, Robert E. (1985). Textual Power: Literary Theory and the Teaching of English. New Haven: Yale University Press.Google Scholar
  57. Scott, Linda M. (1994). Images in Advertising: The Need for a Theory of Visual Rhetoric. Journal of Consumer Research, 21, 252–273.CrossRefGoogle Scholar
  58. Semali, Ladislaus. (2003). Ways with Visual Languages: Making the Case for Critical Media Literacy. The Clearing House, 39(3), 271–277.CrossRefGoogle Scholar
  59. Serafini, Frank. (2009). Understanding Visual Images in Picturebooks. In J. Evans (Ed.), Talking Beyond the Page: Reading and Responding to Contemporary Picturebooks (pp. 10–25). London: Routledge.Google Scholar
  60. Serafini, Frank, & Ladd, Sophie M. (2008). The Challenge of Moving Beyond the Literal in Literature Discussions. Journal of Language and Literacy Education, 4(2), 6–20.Google Scholar
  61. Barry, Seward, & Marie, Anne. (1997). Visual Intelligence: Perception, Image, and Manipulation in Visual Communication Albany, NY: State University of New York Press.Google Scholar
  62. Siegel, Marjorie. (2006). Rereading the Signs: Multimodal Transformations in the Field of Literacy Education. Language Arts, 84(1), 65–77.Google Scholar
  63. Siegel, Marjorie, & Panofsky, Carolyn P. (2009). Designs for Multimodality in Literacy Studies: Explorations in Analysis. In K. M. Leander, D. W. Rowe, D. K. Dickinson, M. K. Hundley, R. T. Jiminez & V. J. Risko (Eds.), 58th Yearbook of the National Reading Conference (pp. 99–111). Oak Creek, WI: NRC.Google Scholar
  64. Sipe, Lawrence R. (1998). How Picture Books Work: A Semiotically Framed Theory of Text-Picture Relationships. Children’s Literature in Education, 29(2), 97–108.CrossRefGoogle Scholar
  65. Smith-Shank, Deborah Lee. (2004). Semiotics and Visual Culture: Sights, Signs, and Significance. In Deborah L. Smith-Shank (Ed.), Reston, VA: National Art Education Association.Google Scholar
  66. Snow, Catherine and Sweet, Anne. (2003). Rethinking Reading Comprehension. New York: Guilford.Google Scholar
  67. Stafford, Barbara Marie. (2008). The Remaining 10 Percent: The Role of Sensory Knowledge in the Age of the Self-Organizing Brain. In J. Elkins (Ed.), Visual Literacy (pp. 31–58). New York: Routledge.Google Scholar
  68. Stolnitz, Jerome. (1960). Aesthetics and the Philosophy of Art Criticism. Boston: Houghtonn Mifflin Company.Google Scholar
  69. Sturken, Marita, & Cartwright, LIsa. (2001). Practices of Looking: An Introduction to Visual Culture. Oxford: Oxford University Press.Google Scholar
  70. Unsworth, Lee. (2002). Changing Dimensions of School Literacies. Australian Journal of Language and Literacy, 25(1), 62–77.Google Scholar
  71. Unsworth, Len, & Wheeler, Janet. (2002). Re-Valuing the Role of Images in Reviewing Picture Books. Reading, 36(2), 68–74.CrossRefGoogle Scholar
  72. van Leeuwen, Theo, & Carey, Jewitt. (2001). Handbook of Visual Analysis. London: Sage.Google Scholar
  73. van Straten, Roelof. (1994). An Introduction to Iconography. London: Taylor & Francis.Google Scholar
  74. Williamson, Judith. (1978). Decoding Advertisements: Ideology and Meaning in Advertising. London: Marion Boyars.Google Scholar
  75. Wolcott, Anne. (1996). Is What You See What You Get? A Postmodern Approach to Understanding Works of Art. Studies in Art Education, 37(2), 69–79.CrossRefGoogle Scholar
  76. Zammit, Katina. (2007). Popular Culture in the Classroom: Interpreting and Creating Multimodal Texts. In A. McCabe, M. O’Donnell & R. Whittaker (Eds.), Advances in Language and Education (pp. 60–76). New York: Continuum.Google Scholar
  77. Zeki, Semir. (1999). Art and the Brain. Journal of Consciousness Studies, 6(6–7), 76–96.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.College of Teacher Education and LeadershipArizona State UniversityScottsdaleUSA

Personalised recommendations