Abstract
This study investigated the effect of cloze item practice on reading comprehension, where cloze items were either created by humans, by machine using natural language processing techniques, or randomly. Participants from Amazon Mechanical Turk (\(N=302\)) took a pre-test, read a text, and took part in one of five conditions, Do-Nothing, Re-Read, Human Cloze, Machine Cloze, or Random Cloze, followed by a 24-hour retention interval and post-test. Participants used the MoFaCTS system [27], which in cloze conditions presented items adaptively based on individual success with each item. Analysis revealed that only Machine Cloze was significantly higher than the Do-Nothing condition on post-test, \(d=.58\), \(CI_{95} [.21,.94]\). Additionally, Machine Cloze was significantly higher than Human and Random Cloze conditions on post-test, \(d=.49\), \(CI_{95} [.12,.86]\) and \(d=.71\), \(CI_{95} [.34,1.09]\) respectively. These results suggest that Machine Cloze items generated using natural language processing techniques are effective for enhancing reading comprehension when delivered by an adaptive practice scheduling system.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmed, Y., Francis, D.J., York, M., Fletcher, J.M., Barnes, M., Kulesz, P.: Validation of the direct and inferential mediation (DIME) model of reading comprehension in grades 7 through 12. Contemp. Educ. Psychol. 4445, 68–82 (2016)
Aleven, V., Mclaren, B.M., Sewall, J., Koedinger, K.R.: A new paradigm for intelligent tutoring systems: example-tracing tutors. Int. J. Artif. Intell. Educ. 19(2), 105–154 (2009)
Bransford, J.D., Johnson, M.K.: Contextual prerequisites for understanding: some investigations of comprehension and recall. J. Verbal Learn. Verbal Behav. 11(6), 717–726 (1972)
Britton, B.K., Gülgöz, S.: Using Kintsch’s computational model to improve instructional text: effects of repairing inference calls on recall and cognitive structures. J. Educ. Psychol. 83(3), 329–345 (1991)
Chall, J.S., Jacobs, V.A.: Writing and reading in the elementary grades: developmental trends among low SES children. Lang. Arts 60(5), 617–626 (1983). http://www.jstor.org/stable/41961511
Chi, M.T.H., Wylie, R.: The ICAP framework: linking cognitive engagement to active learning outcomes. Educ. Psychol. 49(4), 219–243 (2014). http://dx.doi.org/10.1080/00461520.2014.965823
Fang, Z.: The language demands of science reading in middle school. Int. J. Sci. Educ. 28(5), 491–520 (2006). http://dx.doi.org/10.1080/09500690500339092
Harrell, F.: Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. Graduate Texts in Mathematics. Springer, New York (2001)
Hirsch, E.D.: Reading comprehension requires knowledge-of words and the world. Am. Educator 27(1), 10–13, 16–22, 28–29, 48 (2003)
Johansson, R., Nugues, P.: Dependency-based syntactic-semantic analysis with PropBank and NomBank. In: Proceedings of the Twelfth Conference on Computational Natural Language Learning, CoNLL 2008, pp. 183–187. Association for Computational Linguistics, Morristown (2008)
Kelly, S., Ye, F.: Accounting for the relationship between initial status and growth in regression models. J. Exp. Educ. 85(3), 353–375 (2017). http://dx.doi.org/10.1080/00220973.2016.1160357
Kintsch, W.: Comprehension: A Paradigm for Cognition. Cambridge University Press, Cambridge (1998)
Laufer, B.: Lexical thresholds for reading comprehension: what they are and how they can be used for teaching purposes. TESOL Q. 47(4), 867–872 (2013). http://www.jstor.org/stable/43267941
Leelawong, K., Biswas, G.: Designing learning by teaching agents: the Betty’s Brain system. Int. J. Artif. Intell. Educ. 18(3), 181–208 (2008)
Lipson, M.Y.: Learning new information from text: the role of prior knowledge and reading ability. J. Read. Behav. 14(3), 243–261 (1982)
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60. Association for Computational Linguistics, Baltimore, June 2014. http://www.aclweb.org/anthology/pp. 14-5010
McKeown, M.G., Beck, I.L., Omanson, R.C., Pople, M.T.: Some effects of the nature and frequency of vocabulary instruction on the knowledge and use of words. Read. Res. Q. 20(5), 522–535 (1985). http://www.jstor.org/stable/747940
McNamara, D., Levinstein, I., Boonthum, C.: iSTART: interactive strategy training for active reading and thinking. Behav. Res. Methods 36, 222–233 (2004). doi:10.3758/BF03195567
McNamara, D.S., Magliano, J.: Toward a comprehensive model of comprehension. In: The Psychology of Learning and Motivation, Psychology of Learning and Motivation, vol. 51, pp. 297–384. Academic Press (2009). https://www.sciencedirect.com/science/article/pii/S0079742109510092
Mol, S.E., Bus, A.G.: To read or not to read: a meta-analysis of print exposure from infancy to early adulthood. Psychol. Bull. 137(2), 267–296 (2011)
Moss, B.: Making a case and a place for effective content area literacy instruction in the elementary grades. Read. Teach. 59(1), 46–55 (2005). http://dx.doi.org/10.1598/RT.59.1.5
Nagy, W., Townsend, D.: Words as tools: learning academic vocabulary as language acquisition. Read. Res. Q. 47(1), 91–108 (2012). http://dx.doi.org/10.1002/RRQ.011
National Institute of Child Health and Human Development: Report of the National Reading Panel. Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. NIH Publication No. 00-4769, U.S. Government Printing Office, Washington, DC (2000)
Nye, B.D., Graesser, A.C., Hu, X.: AutoTutor and family: a review of 17 years of natural language tutoring. Int. J. Artif. Intell. Educ. 24(4), 427–469 (2014). http://dx.doi.org/10.1007/s40593-014-0029-5
Olney, A.M., Brawner, K., Pavlik, P., Koedinger, K.: Emerging trends in automated authoring. In: Sottilare, R., Graesser, A., Hu, X., Brawner, K. (eds.) Design Recommendations for Intelligent Tutoring Systems, Adaptive Tutoring, vol. 3, pp. 227–242. U.S. Army Research Laboratory, Orlando (2015)
Ozuru, Y., Dempsey, K., McNamara, D.S.: Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learn. Instr. 19(3), 228–242 (2009)
Pavlik, P.I., Kelly, C., Maass, J.K.: The mobile fact and concept training system (MoFaCTS). In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 247–253. Springer, Cham (2016). doi:10.1007/978-3-319-39583-8_25
Recht, D.R., Leslie, L.: Effect of prior knowledge on good and poor readers’ memory of text. J. Educ. Psychol. 80(1), 16–20 (1988)
Surdeanu, M., Hicks, T., Valenzuela-Escarcega, M.A.: Two practical rhetorical structure theory parsers. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pp. 1–5. Association for Computational Linguistics, Denver. http://www.aclweb.org/anthology/N15-3001
Torgesen, J.K.: Avoiding the devastating downward spiral: the evidence that early intervention prevents reading failure. Am. Educator 28(3), 6–19 (2004)
Vidal-Abarca, E., MartÃnez, G., Gilabert, R.: Two procedures to improve instructional text: effects on memory and learning. J. Educ. Psychol. 92(1), 107–116 (2000)
What Works Clearinghouse: Assessing attrition bias. Technical report, Institute of Eduation Sciences (2012). https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_attrition_v2.1.pdf
Wolfe, M.B., Schreiner, M., Rehder, B., Laham, D., Foltz, P.W., Kintsch, W., Landauer, T.K.: Learning from text: matching readers and texts by latent semantic analysis. Discourse Process. 25(2–3), 309–336 (1998). http://dx.doi.org/10.1080/01638539809545030
Acknowledgements
This work was supported by the National Science Foundation Data Infrastructure Building Blocks program (NSF; ACI-1443068), by the Institute of Education Sciences (IES; R305C120001) and by the Office of Naval Research (ONR; N00014-00-1-0600, N00014-12-C-0643; N00014-16-C-3027). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF, IES, or ONR.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Olney, A.M., Pavlik, P.I., Maass, J.K. (2017). Improving Reading Comprehension with Automatically Generated Cloze Item Practice. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-61425-0_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61424-3
Online ISBN: 978-3-319-61425-0
eBook Packages: Computer ScienceComputer Science (R0)