Reading and Writing

, Volume 33, Issue 1, pp 143–170 | Cite as

Beyond first grade: examining word, sentence, and discourse text factors associated with oral reading rate in informational text in second grade

  • Laura S. TortorelliEmail author


Text complexity in elementary classrooms is typically measured by traditional readability tools, which rely on surface-level measures of word and sentence complexity. Theoretical and empirical work on text complexity, however, indicates that additional measures of semantics, syntax, and discourse structure may be equally important for understanding how young children process texts. This study evaluated the components of two available text analysis tools representing different generations of readability: the Lexile Framework and the Coh-Metrix Text Easability Assessor. Using multilevel modeling, informational passage readings were nested in 5133 second-grade children in 459 schools to identify text variables that predicted changes in oral reading rate. Coh-Metrix dimensions of word concreteness, referential cohesion, and deep cohesion were uniquely associated with oral reading rate after controlling for traditional readability components. A full model including both the Lexile components and three Coh-Metrix dimensions provided the best fit to the data. Results suggest that measures of semantic complexity and text cohesion are needed to improve prediction of text difficulty for young readers and better inform text selection in the early grades. Implications for policy and practice under the Common Core State Standards, which emphasize reading complex texts across the gradespan, are discussed.


Text complexity Oral reading rate Reading fluency Multilevel modeling 



  1. Amendum, S. J., Conradi, K., & Hiebert, E. (2017). Does text complexity matter in the elementary grades? A research synthesis of text difficulty and elementary students’ reading fluency and comprehension. Educational Psychology Review,30, 121–151.Google Scholar
  2. Anderson, R. C., Hiebert, E. H., Scott, J. A., & Wilkinson, I. A. G. (1985). Becoming a nation of readers. Champaign, IL: University of Illinois. Center for the Study of Reading.Google Scholar
  3. Ardoin, S. P., Williams, J. C., Christ, T. J., Klubnik, C., & Wellborn, C. (2010). Examining readability estimates’ predictions of students’ oral reading rate: Spache, Lexile, and Forcast. School Psychology Review,39, 277–285.Google Scholar
  4. Arya, D. J., Hiebert, E. H., & Pearson, P. D. (2011). The effects of syntactic and lexical complexity on the comprehension of elementary science texts. International Electronic Journal of Elementary Education,4, 107–125.Google Scholar
  5. Baretta, L., Tomitch, L. M. B., MacNair, N., Lim, V. K., & Waldie, K. E. (2009). Inference making while reading narrative and expository texts: An ERP study. Psychology & Neuroscience,2, 137.Google Scholar
  6. Barth, A. E., Tolar, T. D., Fletcher, J. M., & Francis, D. (2014). The effects of student and text characteristics on the oral reading fluency of middle-grade students. Journal of Educational Psychology,106, 162–180.Google Scholar
  7. Begeny, J. C., & Greene, D. J. (2014). Can readability formulas be used to successfully gauge difficulty of reading materials? Psychology in the Schools,51, 198–215.Google Scholar
  8. Bell, B.A., Ferron, J. M., & Kromrey, J. D. (2008). Cluster size in multilevel models: The impact of sparse data structures on point and interval estimates in two-level models. In JSM Proceedings, Section on Survey Research Methods (pp. 1122–1129). Alexandria, VA: American Statistical Association.Google Scholar
  9. Bell, B. A., Morgan, G. B., Kromrey, J. D., & Ferron, J. M. (2010). The impact of small cluster size on multilevel models: A Monte Carlo examination of two-level models with binary and continuous predictors. In JSM Proceedings, Survey Research Methods Section (pp. 4057–4067). Alexandria, VA: American Statistical Association.Google Scholar
  10. Benjamin, R. G. (2012). Reconstructing readability: Recent developments and recommendations in the analysis of text difficulty. Educational Psychology Review,24, 63–88.Google Scholar
  11. Bernhardt, E. B. (2011). Understanding advanced second language reading. New York: Routledge.Google Scholar
  12. Best, R., Floyd, R. G., & McNamara, D. S. (2004, April). Understanding the fourth-grade slump: Comprehension difficulties as a function of reader aptitudes and text genre. In 85th Annual Meeting of the American Educational Research Association.Google Scholar
  13. Bormuth, J. R. (1966). Readability: A new approach. Reading Research Quarterly,1, 79–132.Google Scholar
  14. Browne, M. W. (2000). Cross-validation methods. Journal of Mathematical Psychology, 44, 108–132.Google Scholar
  15. Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word dilemmas. Behavior Research Methods,46, 904–911.Google Scholar
  16. Cain, K., & Nash, H. M. (2011). The influence of connectives on young readers’ processing and comprehension of text. Journal of Educational Psychology,103, 429–441.Google Scholar
  17. Chall, J. (1983). Stages of reading development. New York: McGraw-Hill.Google Scholar
  18. Collins-Thompson, K., & Callan, J. (2005). Predicting reading difficulty with statistical language models. Journal of the Association for Information Science and Technology,56, 1448–1462.Google Scholar
  19. Compton, D. L., Appleton, A. C., & Hosp, M. K. (2004). Exploring the relationship between text leveling systems and reading accuracy and fluency in second-grade students who are average and poor decoders. Learning Disabilities Research & Practice,19, 176–184.Google Scholar
  20. Crossley, S. A., Dufty, D. F., McCarthy, P. M., & McNamara, D. S. (2007). Toward a new readability: A mixed model approach. In Proceedings of the 29th annual conference of the Cognitive Science Society (pp. 197–202).Google Scholar
  21. Crossley, S. A., Allen, D. B., & McNamara, D. S. (2011). Text readability and intuitive simplification: A comparison of readability formulas. Reading in a foreign language, 23, 84–101.Google Scholar
  22. Crossley, S. A., Greenfield, J., & McNamara, D. S. (2008). Assessing text readability using cognitively based indices. TESOL Quarterly,42, 475–493.Google Scholar
  23. Crossley, S. A., Skalicky, S., Dascalu, M., McNamara, D. S., & Kyle, K. (2017). Predicting text comprehension, processing, and familiarity in adult readers: New approaches to readability formulas. Discourse Processes,54, 340–359.Google Scholar
  24. Cunningham, J. W., Hiebert, E. H., & Mesmer, H. A. (2018). Investigating the validity of two widely used quantitative text tools. Reading and Writing,31, 813–833.Google Scholar
  25. Cunningham, J. W., & Mesmer, H. A. (2014). Quantitative measurement of text difficulty: What’s the use? The Elementary School Journal,115, 255–269.Google Scholar
  26. Cunningham, J. W., Spadorcia, S. A., Erickson, K. A., Koppenhaver, D. A., Sturm, J. M., & Yoder, D. E. (2005). Investigating the instructional supportiveness of leveled texts. Reading Research Quarterly,40, 410–427.Google Scholar
  27. Davison, A., & Kantor, R. N. (1982). On the failure of readability formulas to define readable texts: A case study from adaptations. Reading Research Quarterly,17, 187–209.Google Scholar
  28. De Clercq, O., Hoste, V., Desmet, B., Van Oosten, P., De Cock, M., & Macken, L. (2014). Using the crowd for readability prediction. Natural Language Engineering,20, 293–325.Google Scholar
  29. DuBay, W. (2004). The principles of readability. Costa Mesa, CA: Impact Information.Google Scholar
  30. Duke, N. K. (2000). 3.6 minutes per day: The scarcity of informational texts in first grade. Reading Research Quarterly,35, 202–224.Google Scholar
  31. Ehri, L. C. (2005). Learning to read words: Theory, findings, and issues. Scientific Studies of Reading,9, 167–188.Google Scholar
  32. Ehri, L. C. (2014). Orthographic mapping in the acquisition of sight word reading, spelling memory, and vocabulary learning. Scientific Studies of Reading,18, 5–21.Google Scholar
  33. Feng, L., Jansche, M., Huenerfauth, M., & Elhadad, N. (2010, August). A comparison of features for automatic readability assessment. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters (pp. 276–284). Association for Computational Linguistics.Google Scholar
  34. Fitzgerald, J., Elmore, J., Hiebert, E. H., Koons, H. H., Bowen, K., Sanford-Moore, E. E., et al. (2016). Examining text complexity in the early grades. Phi Delta Kappan,97, 60–65.Google Scholar
  35. Fitzgerald, J., Elmore, J., Koons, H., Hiebert, E. H., Bowen, K., Sanford-Moore, E. E., et al. (2015). Important text characteristics for early-grades text complexity. Journal of Educational Psychology,107, 4–29.Google Scholar
  36. Fountas, I. C., & Pinnell, G. S. (2002). Guided reading: Good first teaching for all children. Portsmouth, NH: Heinemann.Google Scholar
  37. Fuchs, L. S., Fuchs, D., Hosp, M. K., & Jenkins, J. R. (2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading,5, 239–256.Google Scholar
  38. Gardner, D. (2004). Vocabulary input through extensive reading: A comparison of words found in children’s narrative and expository reading materials. Applied Linguistics,25, 1–37.Google Scholar
  39. Gernsbacher, M. A., Varner, K. R., & Faust, M. E. (1990). Investigating differences in general comprehension skill. Journal of Experimental Psychology. Learning, Memory, and Cognition,16, 430.Google Scholar
  40. Givón, T. (1995). Functionalism and grammar. Philadelphia: John Benjamins.Google Scholar
  41. Graesser, A. C., Hauft-Smith, K., Cohen, A. D., & Pyles, L. D. (1980). Advanced outlines, familiarity, and text genre on retention of prose. The Journal of Experimental Education,48, 281–290.Google Scholar
  42. Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher,40, 223–234.Google Scholar
  43. Graesser, A. C., McNamara, D. S., & Louwerse, M. M. (2003). What do readers need to learn in order to process coherence relations in narrative and expository text? In A. P. Sweet & C. E. Snow (Eds.), Rethinking reading comprehension (pp. 82–98). New York: Guilford.Google Scholar
  44. Haberlandt, K. F., & Graesser, A. C. (1985). Component processes in text comprehension and some of their interactions. Journal of Experimental Psychology,114, 357–374.Google Scholar
  45. Halliday, M. A. K., & Hasan, R. (2013). Cohesion in English. New York, NY: Routledge. (Original work published 1976).Google Scholar
  46. Heilman, M., Collins-Thompson, K., Callan, J., & Eskenazi, M. (2006). Classroom success of an intelligent tutoring system for lexical practice and reading comprehension. In 9th International Conference on Spoken Language Processing. Pittsburgh, PA: ISCA.Google Scholar
  47. Hiebert, E. H. (Ed.). (2009). Reading more, reading better. New York: Guilford Press.Google Scholar
  48. Hiebert, E. H. (2017). The text of literacy instruction: Obstacles to or opportunities for educational equity? Literacy Research: Theory, Method, and Practice,66, 117–134.Google Scholar
  49. Hiebert, E. H., & Mesmer, H. A. E. (2013). Upping the ante of text complexity in the Common Core State Standards examining its potential impact on young readers. Educational Researcher,42, 44–51.Google Scholar
  50. Hiebert, E. H., & Pearson, P. D. (2010). An examination of current text difficulty indices with early reading texts. Reading Research Report# 10-01.Google Scholar
  51. Huang, F. L. (2014). Using a bifactor model to assess the factor structure of the Phonological Awareness Literacy Screening for grades 1 through 3. Journal of Psychoeducational Assessment,32, 638–650.Google Scholar
  52. Invernizzi, M., Meier, J. D., & Juel, C. (2003). Phonological Awareness Literacy Screening 1-3 (PALS 1–3). Charlottesville, VA: University of Virginia Press.Google Scholar
  53. Invernizzi, M. A., Meier, J., & Juel, C. (2010). PALS 13 Technical Reference. Virginia State Department of Education.Google Scholar
  54. Just, M. A., & Carpenter, P. A. (1987). The psychology of reading and language comprehension. Boston: Allyn & Bacon.Google Scholar
  55. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review,99(1), 122.Google Scholar
  56. Kim, Y. S., Petscher, Y., Schatschneider, C., & Foorman, B. (2010). Does growth rate in oral reading fluency matter in predicting reading comprehension achievement? Journal of Educational Psychology,102, 652–667.Google Scholar
  57. Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge, MA: Cambridge University Press.Google Scholar
  58. Kolker, B., & Terwilliger, P. N. (1981). Sight vocabulary learning of first and second graders. Literacy Research and Instruction,20, 251–258.Google Scholar
  59. Koons, H., Elmore, J., Sanford-Moore, E., & Stenner, A. J. (2017). The relationship between Lexile text measures and early grades Fountas & Pinnell reading levels (MetaMetrics Research Brief). Durham, NC: MetaMetrics.Google Scholar
  60. Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. Thousand Oaks, CA: Sage.Google Scholar
  61. Kuhn, M. R., Schwanenflugel, P. J., & Meisinger, E. B. (2010). Aligning theory and assessment of reading fluency: Automaticity, prosody, and definitions of fluency. Reading Research Quarterly,45, 230–251.Google Scholar
  62. LaBerge, D., & Samuels, S. J. (1974). Toward a theory of automatic information processing in reading. Cognitive Psychology,6, 293–323.Google Scholar
  63. Laing, E., & Hulme, C. (1999). Phonological and semantic processes influence beginning readers’ ability to learn to read words. Journal of Experimental Child Psychology,73, 183–207.Google Scholar
  64. Larson, S. C. (1931). The shrinkage of the coefficient of multiple correlation. Journal of Educational Psychology,22, 45–55.Google Scholar
  65. Lee, D. Y. W. (2001). Genres, registers, text types, domains and styles: Clarifying the concepts and nevigating a path through the BNC jungle. Language Learning and Technology,5, 37–72.Google Scholar
  66. Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology,1, 86–92.Google Scholar
  67. McNamara, D. S., Graesser, A. C., McCarthy, A. C., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. New York: Cambridge University Press.Google Scholar
  68. McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction,14, 1–43.Google Scholar
  69. Mesmer, H. A. E. (2008). Tools for matching readers to texts: Research-based practices. New York: Guilford Press.Google Scholar
  70. Mesmer, H. A. E., Cunningham, J. W., & Hiebert, E. H. (2012). Toward a theoretical model of text complexity for the early grades: Learning from the past, anticipating the future. Reading Research Quarterly,47, 235–258.Google Scholar
  71. MetaMetrics. (2000). The Lexile framework for reading. Durham, NC: Author. Retrieved from: Accessed 1 Nov 2016.
  72. Miles, K. P., & Ehri, L. C. (2017). Learning to read words on flashcards: Effects of sentence contexts and word class in native and nonnative English-speaking kindergartners. Early Childhood Research Quarterly,41, 103–113.Google Scholar
  73. Mok, M. (1995). Sample-size requirements for 2-level designs in educational research. Multilevel Modelling Newsletter,7, 11–15.Google Scholar
  74. Morris, D., Bloodgood, J., Perney, J., Frye, E. M., Kucan, L., Trathen, W., et al. (2011). Validating craft knowledge: An empirical examination of elementary-grade students’ performance on an informal reading assessment. Elementary School Journal,112, 205–233.Google Scholar
  75. Murphy, K. R. (1983). Fooling yourself with cross-validation: Single sample designs. Personnel Psychology,36, 111–118.Google Scholar
  76. National Governors Association Center for Best Practices [NGA], & Council of Chief State School Officers [CCSSO]. (2010). Common core state standards for English language arts & literacy in history/social studies, science, and technical subjects. Washington, DC: National Governors Association for Best Practices, Council of Chief State School Officers.Google Scholar
  77. Nelson, J., Perfetti, C., Liben, D., & Liben, M. (2012). Measures of text difficulty: Testing their predictive value for grade levels and student performance. Washington, DC: Council of Chief State School Officers.Google Scholar
  78. Newbold, N., & Gillam, L. (2010, June). The linguistics of readability: The next step for word processing. In Proceedings of the NAACL HLT 2010 workshop on computational linguistics and writing: Writing processes and authoring aids (pp. 65–72). Association for Computational Linguistics.Google Scholar
  79. Paivio, A., Yuille, J. C., & Madigan, S. A. (1968). Concreteness, imagery, and meaningfulness values for 925 nouns. Journal of Experimental Psychology,76, 1–25.Google Scholar
  80. Pearson, P. D. (1974). The effects of grammatical complexity on children’s comprehension, recall, and conception of certain semantic relations. Reading Research Quarterly,10, 155–192.Google Scholar
  81. Perfetti, C. A. (2007). Reading ability: Lexical quality to comprehension. Scientific Studies of Reading,11, 357–383.Google Scholar
  82. Peugh, J. L. (2010). A practical guide to multilevel modeling. Journal of School Psychology,48, 85–112.Google Scholar
  83. Pitler, E., & Nenkova, A. (2008). Revisiting readability: A unified framework for predicting text quality. In Proceedings of the conference on empirical methods in natural language processing (pp. 186–195). Association for Computational Linguistics.Google Scholar
  84. Rapp, D. N., van den Broek, P., McMaster, K. L., Kendeou, P., & Espin, C. A. (2007). Higher-order comprehension processes in struggling readers: A perspective for research and intervention. Scientific Studies of Reading,11, 289–312.Google Scholar
  85. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Newberry Park, CA: Sage.Google Scholar
  86. Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., Congdon, R. T., & du Toit, M. (2011). HLM-7. Lincolnwood, IL: Scientific Software International Inc.Google Scholar
  87. Raynor, A., & Pollatsek, K. (1994). The psychology of reading. Cliffs, NJ: Prentice Hall.Google Scholar
  88. Schwanenflugel, P. J., & Akin, C. E. (1994). Developmental trends in lexical decisions for abstract and concrete words. Reading Research Quarterly,29, 251–264.Google Scholar
  89. Schwarm, S. E., & Ostendorf, M. (2005). Reading level assessment using support vector machines and statistical language models. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (pp. 523–530).Google Scholar
  90. Selya, A. S., Rose, J. S., Dierker, L. C., Hedeker, D., & Mermelstein, R. J. (2012). A practical guide to calculating Cohen’s f2, a measure of local effect size, from PROC MIXED. Frontiers in Psychology,3, 111.Google Scholar
  91. Share, D. L. (1995). Phonological recoding and self-teaching: Sine qua non of reading acquisition. Cognition,55, 151–218.Google Scholar
  92. Sheehan, K. M., Kostin, I., Napolitano, D., & Flor, M. (2014). The TextEvaluator tool: Helping teachers and test developers select texts for use in instruction and assessment. The Elementary School Journal,115, 184–209.Google Scholar
  93. Smith, D., Stenner, A. J., Horabin, I., Smith, M. (1989). The Lexile scale in theory and practice: Final report. Washington, DC: MetaMetrics (ERIC Document Reproduction Service No. ED307577.Google Scholar
  94. Smith, M., & Turner, J. (2017). Neglected no more: Addressing text complexity in the early grades (MetaMetrics Research Brief). Durham, NC: MetaMetrics.Google Scholar
  95. Snijders, T. A. B. (2005). Power and sample size in multilevel linear models. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (pp. 1570–1573). Chicester: Wiley.Google Scholar
  96. Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.Google Scholar
  97. Stahl, S. A., & Heubach, K. M. (2005). Fluency-oriented reading instruction. Journal of Literacy Research,37, 25–60.Google Scholar
  98. Stenner, A. J., Burdick, H., Sanford, E. E., & Burdick, D. S. (2007). The Lexile framework for reading technical report. Durham: MetaMetrics, Inc.Google Scholar
  99. Swanson, C. E., & Fox, H. G. (1953). Validity of readability formulas. Journal of Applied Psychology,37(2), 114.Google Scholar
  100. Toyama, Y., Hiebert, E. H., & Pearson, P. D. (2017). An analysis of the text complexity of leveled passages in four popular classroom reading assessments. Educational Assessment,22, 139–170.Google Scholar
  101. Valencia, S. W., Smith, A. T., Reece, A. M., Li, M., Wixson, K. K., & Newman, H. (2010). Oral reading fluency assessment: Issues of construct, criterion, and consequential validity. Reading Research Quarterly,45, 270–291.Google Scholar
  102. Van Den Broek, P., Virtue, S., Everson, M. G., Tzeng, Y., & Sung, Y. C. (2002). Comprehension and memory of science texts: Inferential processes and the construction of a mental representation. In J. Otero, J. A. León, & A. C. Graesser (Eds.), The psychology of science text comprehension (pp. 131–154). Mahwah, NJ: Lawrence Eribaum Associates Publishers.Google Scholar
  103. Wang, C., Porfeli, E., & Algozzine, B. (2008). Development of oral reading fluency in young children at risk for failure. Journal of Education for Students Placed at Risk,13, 402–425.Google Scholar
  104. Whittaker, T. A., & Furlow, C. F. (2009). The comparison of model selection criteria when selecting among competing hierarchical linear models. Journal of Modern Applied Statistical Methods,8, 173–193.Google Scholar
  105. Whitten, I. A., & Frank, E. (2005). Data mining. San Francisco: Elsevier.Google Scholar
  106. Widman, T. (2011). Factors that influence cross-validation of hierarchical linear models (Doctoral dissertation, Georgia State University). Retrieved from Accessed 28 Dec 2018.
  107. Yang, C. L., Perfetti, C. A., & Schmalhofer, F. (2005). Less skilled comprehenders’ ERPs show sluggish word-to-text integration processes. Written Language & Literacy,8, 233–257.Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Michigan State UniversityEast LansingUSA

Personalised recommendations