Skip to main content
SpringerLink
Log in

Reading Multimodal Texts: Perceptual, Structural and Ideological Perspectives

  • Original Paper
  • Published:
Children's Literature in Education Aims and scope Submit manuscript

Abstract

This article presents a tripartite framework for analyzing multimodal texts. The three analytical perspectives presented include: (1) perceptual, (2) structural, and (3) ideological analytical processes. Using Anthony Browne’s picturebook Piggybook as an example, assertions are made regarding what each analytical perspective brings to the interpretation of multimodal texts and how these perspectives expand readers’ interpretive repertoires. Drawing on diverse fields of inquiry, including semiotics, art theory, visual grammar, communication studies, media literacy, visual literacy and literary theory, the article suggests an expansion of the strategies and analytical perspectives readers being to multimodal texts and visual images. Each perspective is presented as necessary but insufficient in and of itself to provide the necessary foundation for comprehending texts. It is through an expansion of the interpretive strategies and perspectives that readers bring to a multimodal text, focusing on visual, textual, and design elements that readers will become more proficient in their interpretive processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aiello, Giorgia. (2006). Theoretical Advances in Critical Visual Analysis: Perception, Ideology, Mythologies, and Social Semiotics. Journal of Visual Literacy, 26(2), 89–102.

    Google Scholar 

  • Albers, Peggy. (2007). Visual Discourse Analysis: An Introduction to the Analysis of School-Generated Visual Texts. In D. W. Rowe, R. T. Jiminez, D. L. Compton, D. K. Dickinson, Y. Kim, K. M. Leander & V. J. Risko (Eds.), 56th Yearbook of the National Reading Conference (pp. 81–95). Oak Creek, WI: NRC.

    Google Scholar 

  • Albers, Peggy. (2008). Theorizing Visual Representation in Children’s Literature. Journal of Literacy Research, 40(2), 163–200.

    Article  Google Scholar 

  • Anstey, Michele, & Bull, Geoff. (2006). Teaching and Learning Multiliteracies: Changing Times, Changing Literacies Newark, DE: International Reading Association.

    Google Scholar 

  • Arnheim, Rudolf. (1986). Art and Visual Perception: A Psychology of the Creative Eye. Berkeley, CA: University of California Press.

    Google Scholar 

  • Barthes, Roland. (1977). The Rhetoric of the Image. In R. Barthes (Ed.), Image, Music, Text (pp. 32–51). New York: Hill and Wang.

    Google Scholar 

  • Beach, Richard, Encisco, Patricia, Harste, Jerome, Jenkins, Christine, Raina, Seemi Aziz, Rogers, Rebecca, Short, Kathy G., Sung, Yoo Kyung, Wilson, Melissa, & Yenika-Agbaw, Vivian. (2009). Exploring the “Critical” in Critical Content Analysis of Children’s Literature. In K. M. Leander, D. W. Rowe, D. K. Dickinson, M. K. Hundley, R. T. Jiminez & V. J. Risko (Eds.), 58th Yearbook of the National Reading Conference (pp. 129–143). Oak Creek, WI: NRC.

    Google Scholar 

  • Beardsley, Monroe C. (1981). Aesthetics: Problems in the Philosophy of Criticism. Indianapolis, IN: Hackett Publishing Company.

    Google Scholar 

  • Berger, John. (1972). Ways of Seeing. London:Penguin.

    Google Scholar 

  • Browne, Anthony. (1986). Piggybook. New York: Alfred A. Knopf.

    Google Scholar 

  • Chandler, David. (2007). Semiotics: The Basics, 2nd ed. London: Routledge.

    Google Scholar 

  • Chauvin, B. A. (2003). Visual or Media Literacy? Journal of Visual Literacy, 23(2), 119–128.

    Google Scholar 

  • Cohn, Neil. (2007). A Visual Lexicon. The Public Journal of Semiotics, 1(1), 35–56.

    Google Scholar 

  • Crawford, Patricia A. (2000). Inside the Picture, Outside the Frame: Semiotics and the Reading of Wordless Picture Books. Journal of Research in Childhood Education, 15(1), 66–80.

    Google Scholar 

  • Culler, Jonathan. (1997). Literary Theory: A Very Short Introduction. Oxford: Oxford University Press.

    Google Scholar 

  • Dondis, Donis A. (1973). A Primer of Visual Literacy. Cambridge, MA: MIT Press.

    Google Scholar 

  • Duncum, Paul. (2004). Visual Culture isn’t Just Visual: Multiliteracy, Multimodality, and Meaning. Studies in Art Education, 45(3), 252–264.

    Google Scholar 

  • Elkins, John. (2008). Visual Literacy. New York: Routledge.

    Google Scholar 

  • Fairclough, Norman. (2003). Analysing Discourse: Textual Analysis for Social Research. London: Routledge.

    Google Scholar 

  • Fish, Stanley. (1980). Is There a Text in This Class? The Authority of Interpretive Communities. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Fleckenstein, Kristie S. (2002). Inviting Imagery into Our Classrooms. In K. S. Fleckenstein, L. T. Calendrillo & D. A. Worley (Eds.), Language and Images in the Reading-Writing Classroom (pp. 3–25). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Freebody, Peter. (1992). A Socio-Cultural Approach: Resourcing Four Roles as a Literacy Learner. In A. Watson and A. Beadenhop (Eds.), Prevention of Reading Failure (pp. 26–41). Sydney: Ashton-Scholastic.

    Google Scholar 

  • Freebody, Peter, & Luke, Allan. (1990). Literacies Programs: Debates and Demands in Cultural Context. Prospect: Australian Journal of TESOL, 5(7), 7–16.

    Google Scholar 

  • Genette, Gerard. (1997). Paratexts: Thresholds of Interpretation. Translated by J. E. Lewin. Cambridge: Cambridge University Press.

  • Golden, Joanne M., & Gerber, Annyce. (1990). A Semiotic Perspective of Text: The Picture Story Book Event. Journal of Reading Behavior, 22(3), 203–219.

    Google Scholar 

  • Gombrich, E. H. (1961). Art and Illusion: A Study in the Psychology of Pictorial Representation, 2nd ed. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Goodman, Nelson. (1976). Languages of Art. Indianapolis, IN: Hackett Publishing Company.

    Google Scholar 

  • Graham, Judith. (1990). Pictures on the Page. Exeter: Short Run Press.

    Google Scholar 

  • Halliday, Michael A. K. (1975). Learning How to Mean: Explorations in the Development of Language. London: Elsevier.

    Google Scholar 

  • Halliday, Michael A. K. (1978). Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.

    Google Scholar 

  • Harvey, Stephanie, & Goudvis, Anne. (2000). Strategies that Work: Teaching Comprehension to Enhance Understanding. Portland, ME: Stenhouse.

    Google Scholar 

  • Hassenmuller, Christine. (1987). Panofsky, Iconography, and Semiotics. Journal of Aesthetics and Art Criticism, 35, 289–301.

    Google Scholar 

  • Hassett, Dawnene D., & Schieble, Melissa B. (2007). Finding Space and Time for the Visual in K-12 Literacy Instruction. English Journal, 97(1), 62–68.

    Article  Google Scholar 

  • Idema, Rick. (2001). Analysing Film and Television: A Social Semiotic Account of Hospital: An Unhealthy Business. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of Visual Analysis (pp. 183–204). London: Sage.

    Google Scholar 

  • Jewitt, Carey, & Kress, Gunther (Eds.). (2003). Multimodal Literacy. New York: Peter Lang.

    Google Scholar 

  • Jewitt, Carey, & Oyama, Rumiko. (2001). Visual Meaning: A Social Semiotic Approach. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of Visual Analysis (pp. 134–156). London: Sage.

    Google Scholar 

  • Kress, Gunther. (2003). Literacy in the New Media Age. London: Routledge.

    Book  Google Scholar 

  • Kress, Gunther. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.

    Google Scholar 

  • Kress, Gunther, & van Leeuwen, Theo. (1996). Reading Images: The Grammar of Visual Design. London: Routledge Falmer.

    Google Scholar 

  • Leeds-Hurwitz, Wendy. (1993). Semiotics and Communication: Signs, Codes, Cultures. Hillsdale, NJ: Laurence Erlbaum Associates.

    Google Scholar 

  • Lewis, David. (2001). Reading Contemporary Picturebooks: Picturing Text. London: Routledge Falmer.

    Google Scholar 

  • Lister, Martin, & Wells, Liz. (2001). Seeing Beyond Belief: Cultural Studies as an Approach to Analyzing the Visual. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of Visual Analysis (pp. 61–91). London: Sage.

    Google Scholar 

  • Martineo, Radan, & Salway, Andrew. (2005). A System for Image-Text Relations in New (and Old) Media. Visual Communication, 4(3), 337–371.

    Article  Google Scholar 

  • Messaris, Paul. (1994). Visual Literacy: Image, Mind, & Reality. Boulder, CO: Westview Press.

    Google Scholar 

  • Messaris, Paul. (2003). Visual Communication: Theory and Research. Journal of Communication, 53(3), 551–556.

    Article  Google Scholar 

  • Mirzeoff, Nicholas. (1998). What is Visual Culture? In N. Mirzeoff (Ed.), The Visual Culture Reader. London: Routledge.

    Google Scholar 

  • Moebuis, William. (1986). Introduction to Picturebook Codes. Word & Image, 2(2), 141–158.

    Google Scholar 

  • Moss, Barbara. (2003). Exploring the Literature of Fact: Children’s Nonfiction Trade Books in the Elementary Classroom. New York: Guilford Press.

    Google Scholar 

  • Moss, Pamela A., Phillips, Denis C., Erickson, Frederick, Floden, Robert E., Lather, Patti A., & Barbara Schneider, L. (2009). Learning from Our Differences: A Dialogue Across Perspectives on Quality in Educational Research. Educational Researcher, 38(7), 501–517.

    Article  Google Scholar 

  • Nikolajeva, Maria, & Scott, Carole. (2000). The Dynamics of Picturebook Communication. Children’s Literature in Education, 31(4), 225–239.

    Article  Google Scholar 

  • Nodelman, Perry. (1984). How Picture Books Work. In H. Darling & P. Neumeyer (Eds.), Image and Maker: An Annual Dedication to the Consideration of Book Illustration (pp. 1–13). La Jolla, CA: Green Tiger Press.

    Google Scholar 

  • Owocki, Gretchen. (2003). Comprehension: Strategic Instruction for K-3 Students. Portsmouth, NH: Heinemann.

    Google Scholar 

  • Panofsky, Erwin. (1955). Meaning in the Visual Arts. Chicago: University of Chicago Press.

    Google Scholar 

  • Rose, Gillian. (2001). Visual Methodologies. London: Sage.

    Google Scholar 

  • Samara, Timothy. (2007). Design Elements: A Graphic Style Manual. Beverly, MA: Rockport.

    Google Scholar 

  • Scholes, Robert E. (1985). Textual Power: Literary Theory and the Teaching of English. New Haven: Yale University Press.

    Google Scholar 

  • Scott, Linda M. (1994). Images in Advertising: The Need for a Theory of Visual Rhetoric. Journal of Consumer Research, 21, 252–273.

    Article  Google Scholar 

  • Semali, Ladislaus. (2003). Ways with Visual Languages: Making the Case for Critical Media Literacy. The Clearing House, 39(3), 271–277.

    Article  Google Scholar 

  • Serafini, Frank. (2009). Understanding Visual Images in Picturebooks. In J. Evans (Ed.), Talking Beyond the Page: Reading and Responding to Contemporary Picturebooks (pp. 10–25). London: Routledge.

    Google Scholar 

  • Serafini, Frank, & Ladd, Sophie M. (2008). The Challenge of Moving Beyond the Literal in Literature Discussions. Journal of Language and Literacy Education, 4(2), 6–20.

    Google Scholar 

  • Barry, Seward, & Marie, Anne. (1997). Visual Intelligence: Perception, Image, and Manipulation in Visual Communication Albany, NY: State University of New York Press.

    Google Scholar 

  • Siegel, Marjorie. (2006). Rereading the Signs: Multimodal Transformations in the Field of Literacy Education. Language Arts, 84(1), 65–77.

    Google Scholar 

  • Siegel, Marjorie, & Panofsky, Carolyn P. (2009). Designs for Multimodality in Literacy Studies: Explorations in Analysis. In K. M. Leander, D. W. Rowe, D. K. Dickinson, M. K. Hundley, R. T. Jiminez & V. J. Risko (Eds.), 58th Yearbook of the National Reading Conference (pp. 99–111). Oak Creek, WI: NRC.

    Google Scholar 

  • Sipe, Lawrence R. (1998). How Picture Books Work: A Semiotically Framed Theory of Text-Picture Relationships. Children’s Literature in Education, 29(2), 97–108.

    Article  Google Scholar 

  • Smith-Shank, Deborah Lee. (2004). Semiotics and Visual Culture: Sights, Signs, and Significance. In Deborah L. Smith-Shank (Ed.), Reston, VA: National Art Education Association.

  • Snow, Catherine and Sweet, Anne. (2003). Rethinking Reading Comprehension. New York: Guilford.

    Google Scholar 

  • Stafford, Barbara Marie. (2008). The Remaining 10 Percent: The Role of Sensory Knowledge in the Age of the Self-Organizing Brain. In J. Elkins (Ed.), Visual Literacy (pp. 31–58). New York: Routledge.

    Google Scholar 

  • Stolnitz, Jerome. (1960). Aesthetics and the Philosophy of Art Criticism. Boston: Houghtonn Mifflin Company.

    Google Scholar 

  • Sturken, Marita, & Cartwright, LIsa. (2001). Practices of Looking: An Introduction to Visual Culture. Oxford: Oxford University Press.

    Google Scholar 

  • Unsworth, Lee. (2002). Changing Dimensions of School Literacies. Australian Journal of Language and Literacy, 25(1), 62–77.

    Google Scholar 

  • Unsworth, Len, & Wheeler, Janet. (2002). Re-Valuing the Role of Images in Reviewing Picture Books. Reading, 36(2), 68–74.

    Article  Google Scholar 

  • van Leeuwen, Theo, & Carey, Jewitt. (2001). Handbook of Visual Analysis. London: Sage.

    Google Scholar 

  • van Straten, Roelof. (1994). An Introduction to Iconography. London: Taylor & Francis.

    Google Scholar 

  • Williamson, Judith. (1978). Decoding Advertisements: Ideology and Meaning in Advertising. London: Marion Boyars.

    Google Scholar 

  • Wolcott, Anne. (1996). Is What You See What You Get? A Postmodern Approach to Understanding Works of Art. Studies in Art Education, 37(2), 69–79.

    Article  Google Scholar 

  • Zammit, Katina. (2007). Popular Culture in the Classroom: Interpreting and Creating Multimodal Texts. In A. McCabe, M. O’Donnell & R. Whittaker (Eds.), Advances in Language and Education (pp. 60–76). New York: Continuum.

    Google Scholar 

  • Zeki, Semir. (1999). Art and the Brain. Journal of Consciousness Studies, 6(6–7), 76–96.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Teacher Education and Leadership, Arizona State University, 8084 E. High Point Dr., Scottsdale, AZ, 85257, USA

    Frank Serafini

Authors
  1. Frank Serafini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frank Serafini.

Additional information

Dr. Frank Serafini is an author, illustrator, photographer, educator, musician, and an Associate Professor of Literacy Education and Children’s Literature at Arizona State University.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Serafini, F. Reading Multimodal Texts: Perceptual, Structural and Ideological Perspectives. Child Lit Educ 41, 85–104 (2010). https://doi.org/10.1007/s10583-010-9100-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10583-010-9100-5

Keywords

Search

Navigation