Advertisement

Educational Psychology Review

, Volume 24, Issue 1, pp 63–88 | Cite as

Reconstructing Readability: Recent Developments and Recommendations in the Analysis of Text Difficulty

  • Rebekah George Benjamin
Review

Abstract

Largely due to technological advances, methods for analyzing readability have increased significantly in recent years. While past researchers designed hundreds of formulas to estimate the difficulty of texts for readers, controversy has surrounded their use for decades, with criticism stemming largely from their application in creating new texts as well as their utilization of surface-level indicators as proxies for complex cognitive processes that take place when reading a text. This review focuses on examining developments in the field of readability during the past two decades with the goal of informing both current and future research and providing recommendations for present use. The fields of education, linguistics, cognitive science, psychology, discourse processing, and computer science have all made recent strides in developing new methods for predicting the difficulty of texts for various populations. However, there is a need for further development of these methods if they are to become widely available.

Keywords

Readability Text difficulty Reading Text analysis 

References

  1. Anderson, J. R. (1983). A spreading activation theory of memory. Journal of Verbal Learning and Verbal Behavior, 22, 261–295.CrossRefGoogle Scholar
  2. Anderson, R., & Pichert, J. (1978). Recall of previously unrecallable information following a shift in perspective. Journal of Verbal Learning & Verbal Behavior, 17(1), 1–12. doi: 10.1016/S0022-5371(78)90485-1.CrossRefGoogle Scholar
  3. Bailin, A., & Grafstein, A. (2001). The linguistic assumptions underlying readability formulae: A critique. Language & Communication, 21(3), 285–301.CrossRefGoogle Scholar
  4. Blackburn, B. (2000). Best practices for using Lexiles. Popular Measurement, 3(1), 22–24.Google Scholar
  5. Bormuth, J. (1966). Readability: A new approach. Reading Research Quarterly, 1, 79–132.CrossRefGoogle Scholar
  6. Bormuth, J.R. (1969). Development of readability analyses. Final Report, Project No. 7-0052, Contract No. 1, OEC-3-7-070052-0326. Washington, DC: U.S. Office of Education.Google Scholar
  7. Bormuth, J. R. (1971). Development of standards of readability: Toward a rational criterion of passage performance. Final report, U.S. Office of Education, Project No. 9-0237. Chicago: University of Chicago.Google Scholar
  8. Britton, B., & Gülgöz, S. (1991). Using Kintsch’s computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology, 83(3), 329–345. doi: 10.1037/0022-0663.83.3.329.CrossRefGoogle Scholar
  9. Britton, B., Gülgöz, S., Glynn, S. (1993). Impact of good and poor writing on learners: Research and theory. Learning from textbooks: Theory and practice (pp. 1–46). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  10. Carroll, J. B., Davies, P., & Richman, B. (Eds.). (1971). Word frequency book. New York: Houghton Mifflin.Google Scholar
  11. Caylor, J.S., Sticht, T.G., Fox, L.C., Ford, J.P. 1973. Methodologies for determining reading requirements of military occupational specialties: Technical report No. 73-5. Alexandria, VA: Human Resources Research Organization.Google Scholar
  12. Chall, J. S., & Dale, E. (1995). Readability revisited: The new Dale–Chall readability formula. Cambridge: Brookline Books.Google Scholar
  13. Chall, J. S., Bissex, G. L., Conrad, S. S., & Harris-Sharples, S. (1996). Qualitative assessment of text difficulty: A practical guide for teachers and writers. Cambridge: Brookline Books.Google Scholar
  14. Coleman, E. (1962). Improving comprehensibility by shortening sentences. Journal of Applied Psychology, 46(2), 131–134. doi: 10.1037/h0039740.CrossRefGoogle Scholar
  15. Collins-Thompson, K., & Callan, J. (2004). A language modeling approach to predicting reading difficulty. In S. Dumais, D. Marcu, & S. Roukos (Eds.), HLT-NAACL 2004: Main proceedings (pp. 193–200). Morristown: Association for Computational Linguistics.Google Scholar
  16. Collins-Thompson, K., & Callan, J. (2005). Predicting reading difficulty with statistical language models. Journal of the American Society for Information Science & Technology, 56(13), 1448–1462. doi: 10.1002/asi.20243.CrossRefGoogle Scholar
  17. Crossley, S. A., Dufty, D. F., McCarthy, P. M., & McNamara, D. S. (2007). Toward a new readability: A mixed model approach. In D. S. McNamara & G. Trafton (Eds.), Proceedings of the 29th Annual Conference of the Cognitive Science Society (pp. 197–202). Austin: Cognitive Science Society.Google Scholar
  18. Dale, E., & Chall, J. S. (1948). A formula for predicting readability. Educational Research Bulletin, 27(1), 11–20–28.Google Scholar
  19. Dam, G., & Kaufmann, S. (2008). Computer assessment of interview data using latent semantic analysis. Behavior Research Methods, 40(1), 8–20. doi: 10.3758/BRM.40.1.8.CrossRefGoogle Scholar
  20. Davison, A., & Kantor, R. (1982). On the failure of readability formulas to define readable texts: A case study from adaptations. Reading Research Quarterly, 17(2), 187–209.CrossRefGoogle Scholar
  21. DuBay, W.H. 2004. The principles of readability. Retrieved 30 August 2010 from http://www.impact-information.com/impactinfo/readability02.pdf.
  22. Fellbaum, C. (Ed.). (1998). WordNet: An electronic lexical database. Cambridge: MIT.Google Scholar
  23. Feng, L. (2009). Automatic readability assessment for people with intellectual disabilities. ACM SIGACCESS Accessibility and Computing, 93, 84–91. doi: 10.1145/1531930.1531940.CrossRefGoogle Scholar
  24. Foltz, P., Kintsch, W., & Landauer, T. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25(2–3), 285–307. doi: 10.1080/01638539809545029.CrossRefGoogle Scholar
  25. Fountas, I. C., & Pinnell, G. S. (1999). Matching books to readers: Using leveled books in guided reading, K-3. Portsmouth: Heinemann.Google Scholar
  26. Fountas, I. C., & Pinnell, G. S. (2001). Guiding readers and writers: Grades 3–6. Portsmouth: Heinemann.Google Scholar
  27. Fry, E. (1990). A readability formula for short passages. Journal of Reading, 33(8), 594–97.Google Scholar
  28. Graesser, A. C., Gernsbacher, M. A., & Goldman, S. R. (1997). Cognition. In T. A. van Dijk & T. A. van Dijk (Eds.), Discourse as structure and process: Discourse studies: A multidisciplinary introduction, vol. 1 (pp. 292–319). Thousand Oaks: Sage.Google Scholar
  29. Graesser, A., McNamara, D., Louwerse, M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments & Computers, 36(2), 193–202.CrossRefGoogle Scholar
  30. Green, A., Ünaldi, A., & Weir, C. (2010). Empiricism versus connoisseurship: Establishing the appropriacy of texts in tests of academic reading. Language Testing, 27(2), 191–211. doi: 10.1177/0265532209349471.CrossRefGoogle Scholar
  31. Heilman, M., Collins-Thompson, K., Callan, J., Eskenazi, M. (2007). Combining lexical and grammatical features to improve readability measures for first and second language texts. In Proceedings of the NAACL Human Language Technology Conference (pp. 460–467). Morristown, NJ: Association for Computational Linguistics.Google Scholar
  32. Heilman, M., Collins-Thompson, K., Eskenazi, M. (2008). An analysis of statistical models and features for reading difficulty prediction. In EANL '08 Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, June 19–19 (pp.71–79). Morristown, NJ: Association for Computational Linguistics.Google Scholar
  33. Hiebert, E. (1999). Text matters in learning to read. Reading Teacher, 52(6), 552–66.Google Scholar
  34. Hiebert, E. H., Pearson, P. D. (2010). An examination of current text difficulty indices with early reading texts (Reading Research Report No. 10-01). Santa Cruz, CA: TextProject, Inc.Google Scholar
  35. Just, M., & Carpenter, P. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354. doi: 10.1037/0033-295X.87.4.329.CrossRefGoogle Scholar
  36. Kalyuga, S., Chandler, P., Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126–136.2000-03003-01110.1037/0022-0663.92.1.126. doi: 10.1037/0022-0663.92.1.126.Google Scholar
  37. Kalyuga, S., Chandler, P., & Sweller, J. (2001). Learner experience and efficiency of instructional guidance. Educational Psychology, 21, 5–23. doi: 10.1080/0144341012468110.1080/014434101246812001-16707-001.10.1080/01443410124681.CrossRefGoogle Scholar
  38. Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93, 579–588. doi: 10.1037/0022-0663.93.3.57910.1037/0022-0663.93.3.5792001-18059-013.10.1037/0022-0663.93.3.579.CrossRefGoogle Scholar
  39. Kalyuga, S., Ayres, P., Chandler, P., Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38, 23–31. doi: 10.1207/S15326985EP3801.  10.1207/S15326985EP3801.Google Scholar
  40. Kim, H., Goryachev, S., Rosemblat, G., Browne, A., Keselman, A., & Zeng-Treitler, Q. (2007). Beyond surface characteristics: A new health text-specific readability measurement. American Medical Informatics (AMIA) Annual Symposium (pp. 418–422). Washington, DC.Google Scholar
  41. Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction–integration model. Psychological Review, 95(2), 163–182. doi: 10.1037/0033-295X.95.2.163.CrossRefGoogle Scholar
  42. Kintsch, W., & Keenan, J. (1973). Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5(3), 257–274. doi: 10.1016/0010-0285(73)90036-4.CrossRefGoogle Scholar
  43. Kintsch, W., & van Dijk, T. (1978). Toward a model of text comprehension and production. Psychological Review, 85(5), 363–394. doi: 10.1037/0033-295X.85.5.363.CrossRefGoogle Scholar
  44. Kirsch, I. S., & Mosenthal, T. B. (1990). Exploring document literacy: Variables underlying the performance of young adults. Reading Research Quarterly, 25, 5–30.CrossRefGoogle Scholar
  45. Klare, G. R. (1963). The measurement of readability. Ames: Iowa State University Press.Google Scholar
  46. Klare, G. (1974). Assessing readability. Reading Research Quarterly, 10, 62–102.CrossRefGoogle Scholar
  47. Landauer, T., Foltz, P., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2–3), 259–284. doi: 10.1080/01638539809545028.CrossRefGoogle Scholar
  48. Leahy, W., Chandler, P., Sweller, J. (2003). When auditory presentations should and should not be a component of multimedia instruction. Applied Cognitive Psychology, 17, 401–418.2003-00690-00410.1002/acp.877. doi: 10.1002/acp.877.
  49. Lin, S., Su, C., Lai, Y., Yang, L., & Hsieh, S. (2009). Assessing text readability using hierarchical lexical relations retrieved from WordNet. Computational Linguistics and Chinese Language Processing, 14(1), 45–84.Google Scholar
  50. Liu, X., Croft, W.B., Oh, P., Hart, D. (2004). Automatic recognition of reading levels from user queries. SIGIR '04 Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval (pp. 548–549). New York, NY: ACM.Google Scholar
  51. McClelland, J., & Rumelhart, D. (1981). An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychological Review, 88(5), 375–407. doi: 10.1037/0033-295X.88.5.375.CrossRefGoogle Scholar
  52. McNamara, D., & Kintsch, W. (1996). Learning from texts: Effects of prior knowledge and text coherence. Discourse Processes, 22(3), 247–288. doi: 10.1080/01638539609544975.CrossRefGoogle Scholar
  53. McNamara, D., Kintsch, E., Songer, N., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1–43. doi: 10.1207/s1532690xci1401_1.CrossRefGoogle Scholar
  54. McNamara, D., Crossley, S., & McCarthy, P. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57–86. doi: 10.1177/0741088309351547.CrossRefGoogle Scholar
  55. McNamara, D., Louwerse, M., McCarthy, P., & Graesser, A. (2010). Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47(4), 292–330. doi: 10.1080/01638530902959943.CrossRefGoogle Scholar
  56. Meyer, B., Marsiske, M., & Willis, S. (1993). Text processing variables predict the readability of everyday documents read by older adults. Reading Research Quarterly, 28(3), 234–249. doi: 10.2307/747996.CrossRefGoogle Scholar
  57. Miller, T. (2003). Essay assessment with latent semantic analysis. Journal of Educational Computing Research, 29(4), 495–512. doi: 10.2190/W5AR-DYPW-40KX-FL99.CrossRefGoogle Scholar
  58. Miller, G. R., & Coleman, E. B. (1967). A set of thirty-six prose passages calibrated for complexity. Journal of Verbal Learning and Verbal Behavior, 6(6), 851–854.CrossRefGoogle Scholar
  59. Miller, J., & Kintsch, W. (1980). Readability and recall of short prose passages: A theoretical analysis. Journal of Experimental Psychology: Human Learning and Memory, 6(4), 335–354. doi: 10.1037/0278-7393.6.4.335.CrossRefGoogle Scholar
  60. Millis, K., Magliano, J., Wiemer-Hastings, K., Todaro, S., McNamara, D. (2007). Assessing and improving comprehension with latent semantic analysis. Handbook of latent semantic analysis (pp. 207–225). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  61. Milone, M. (2009). The development of ATOS: The renaissance readability formula. Wisconsin Rapids: Renaissance Learning.Google Scholar
  62. Miltsakaki, E., Troutt, A. (2007). Read-X: Automatic evaluation of reading difficulty of web text. Proceedings of E-Learn 2007, sponsored by the Association for the Advancement of Computing in Education. Quebec, Canada.Google Scholar
  63. Miltsakaki, E., & Troutt, A. (2008). Real-time web text classification and analysis of reading difficulty. In J. Tetreault, J. Burstein, & R. De Felice (Eds.), EANL '08 Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications (pp. 89–97). Morristown: Association for Computational Linguistics.CrossRefGoogle Scholar
  64. Peterson, B. (1991). Selecting books for beginning readers: Children’s literature suitable for young readers. In D. E. DeFord, C. A. Lyons, & G. S. Pinnell (Eds.), Bridges to literacy: Learning from reading recovery (pp. 119–147). Portsmouth: Heinemann.Google Scholar
  65. Peterson, S., & Ostendorf, M. (2009). A machine learning approach to reading level assessment. Computer Speech and Language, 23(1), 89–106.CrossRefGoogle Scholar
  66. Rog, L., & Burton, W. (2002). Matching texts and readers: Leveling early reading materials for assessment and instruction. Reading Teacher, 55(4), 348–56.Google Scholar
  67. Rosch, E., Mervis, C. B., Gray, W., Johnson, D., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology, 8(3), 382–439.CrossRefGoogle Scholar
  68. Rumelhart, D., & McClelland, J. (1982). An interactive activation model of context effects in letter perception: II. The contextual enhancement effect and some tests and extensions of the model. Psychological Review, 89(1), 60–94. doi: 10.1037/0033-295X.89.1.60.CrossRefGoogle Scholar
  69. School Renaissance Inst., Inc. (2000). The ATOS[TM] readability formula for books and how it compares to other formulas. Madison, WI: School Renaissance Inst., Inc. (ERIC Document Reproduction Service No. ED449468).Google Scholar
  70. Schriver, K. A. (2000). Readability formulas in the new millennium: What’s the use? ACM Journal of Computer Documentation, 24(3), 105–106.CrossRefGoogle Scholar
  71. Schwarm, S.E., Ostendorf, M. 2005. Reading level assessment using support vector machines and statistical language models. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pp. 523–530, Ann Arbor, MI.Google Scholar
  72. Si, L., Callan, J. 2001. A statistical model for scientific readability. CIKM’01: Proceedings of the Tenth International Conference on Information and Knowledge Management, pp. 574–576.Google Scholar
  73. Smith, R. (2000a). How the Lexile framework operates. Popular Measurement, 3(1), 18–19.Google Scholar
  74. Smith, R. (2000b). The Lexile community: From science to practice. Popular Measurement, 3(1), 20–21.Google Scholar
  75. Smith, D., Stenner, A.J., Horabin, I., Smith, M. (1989). The Lexile scale in theory and practice: Final report. Washington, DC: MetaMetrics (ERIC Document Reproduction Service No. ED307577).Google Scholar
  76. Stenner, A.J. (1996). Measuring reading comprehension with the Lexile framework. Paper presented at the 4th North American Conference on Adolescent/Adult Literacy, Washington, DC.Google Scholar
  77. Stenner, A.J. (1999). Instructional uses of the Lexile framework. Durham, NC: MetaMetrics, Inc. (ERIC Document Reproduction Service No. ED435976).Google Scholar
  78. Stenner, A., Burdick, D. (1997). The objective measurement of reading comprehension: In response to technical questions raised by the California Department of Education Technical Study Group. Durham, NC: MetaMetrics, Inc. (ERIC Document Reproduction Service No. ED435978).Google Scholar
  79. Stenner, A.J., Burdick, H., Sanford, E.E., Burdick, D.S. (2007). The Lexile framework for reading technical report. MetaMetrics, Inc.Google Scholar
  80. vor der Brück, T., Hartrumpf, S., & Helbig, H. (2008). A readability checker with supervised learning using deep indicators. Informatica, 32(4), 429–435.Google Scholar
  81. Wolfe, M., Schreiner, M., Rehder, B., Laham, D., Foltz, P., Kintsch, W., et al. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25(2–3), 309–336. doi: 10.1080/01638539809545030.CrossRefGoogle Scholar
  82. Wright, B., Stenner, A. (1998). Readability and reading ability. Paper presented to the Australian Council on Education Research (ACER) (ERIC Document Reproduction Service No. ED435979).Google Scholar
  83. Wright, B., & Stenner, A. (2000). Lexile perspectives. Popular Measurement, 3(1), 16–17.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Educational PsychologyUniversity of GeorgiaAthensUSA

Personalised recommendations