Skip to main content

Opening the black box: user-log analyses of children’s e-Book reading and associations with word knowledge

Reading and Writing volume 34, pages 627–657 (2021)Cite this article

Abstract

Third to fifth graders read an interactive choose-your-own adventure e-Book. User logs recorded their reading behaviors and were used to investigate students’ in-the-moment reading behaviors. Reader’s standards of coherence, motivation, and reading strategies were hypothesized to relate to children’s reading behaviors, such as time reading pages, answering embedded questions correctly, and making thoughtful decisions, and in turn to word knowledge gains. Structural equation models revealed that the more time students spent reading embedded questions, the more likely they were to answer the questions correctly, which in turn strongly predicted gains in word knowledge. The more time students spent reading text pages, the more likely they were to make good decisions. Additionally, student participation in a weekly book club, randomly assigned within classrooms, predicted stronger gains in word knowledge. Findings highlight the potential of e-Books to improve word knowledge, and that student user-logs offer another way to study reading comprehension in-the-moment.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  • Acock, A. C. (2013). Discovering structural equation modeling using Stata: Revised edition. College Station: Stata Press.

    Google Scholar 

  • Baker, R. S. J. D. (2010). Data mining for education. In B. McGaw, P. Peterson, & E. Baker (Eds.), International encyclopedia of education (3rd ed., Vol. 7, pp. 112–118). Amsterdam: Elsevier.

    Chapter  Google Scholar 

  • Berkeley, S., Mastropieri, M. A., & Scruggs, T. E. (2011). Reading comprehension strategy instruction and attribution retraining for secondary students with learning and other mild disabilities. Journal of Learning Disabilities, 44(1), 18–32. https://doi.org/10.1177/0022219410371677.

    Article  Google Scholar 

  • Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school: Expanded edition. Washington: National Academies Press. https://doi.org/10.17226/9853.

    Book  Google Scholar 

  • Bus, A. G., Takacs, Z. K., & Kegel, C. A. (2015). Affordances and limitations of electronic storybooks for young children’s emergent literacy. Developmental Review, 35, 79–97. https://doi.org/10.1016/j.dr.2014.12.004.

    Article  Google Scholar 

  • Caplovitz, A. G. (2005). The effects of using an electronic talking book on the emergent literacy skills of preschool children. Doctoral dissertation. Retrieved from ProQuest Dissertations and Theses A&I. (Order No. 3187831).

  • Chernyak, N., Leech, K. A., & Rowe, M. L. (2017). Training preschoolers’ prospective abilities through conversation about the extended self. Developmental Psychology, 53(4), 652–661. https://doi.org/10.1037/dev0000283.

    Article  Google Scholar 

  • Connor, C. M., Day, S. L., Zargar, E., Wood, T. S., Taylor, K. S., Jones, M. R., et al. (2019). Building Word Knowledge, Learning Strategies, and Metacognition with the Word-Knowledge E-Book. Computers & Education, 128, 284–311. https://doi.org/10.1016/j.compedu.2018.09.016.

    Article  Google Scholar 

  • Connor, C. M., Radach, R., Vorstius, C., Day, S. L., McLean, L., & Morrison, F. J. (2015). Individual differences in fifth graders’ literacy and academic language predict comprehension monitoring development: An eye-movement study. Scientific Studies of Reading, 19(2), 114–134. https://doi.org/10.1080/10888438.2014.943905.

    Article  Google Scholar 

  • Del Giudice, M. (2014). Middle childhood: An evolutionary-developmental synthesis. Child Development Perspectives, 8(4), 193–200. https://doi.org/10.1111/cdep.12084.

    Article  Google Scholar 

  • Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rölke, H., & Klieme, E. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106(3), 608–626. https://doi.org/10.1037/a0034716.

    Article  Google Scholar 

  • Graesser, A. C., & McNamara, D. S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371–398. https://doi.org/10.1111/j.1756-8765.2010.01081.x.

    Article  Google Scholar 

  • Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36, 193–202. https://doi.org/10.3758/BF03195564.

    Article  Google Scholar 

  • Graves, M. F., Ringstaff, C., Li, L., & Flynn, K. (2018). Effects of teaching upper elementary grade students to use word learning strategies. Reading Psychology, 39(6), 602–622. https://doi.org/10.1080/02702711.2018.1496503.

    Article  Google Scholar 

  • Honig, B., Diamond, L., & Gutlohn, L. (2013). Teaching reading sourcebook (2nd ed.). Novato: Arena Press.

    Google Scholar 

  • Jenkins, J. R., Heliotis, J. D., Stein, M. L., & Haynes, M. C. (1987). Improving reading comprehension by using paragraph restatements. Exceptional Children, 54(1), 54–59.

    Article  Google Scholar 

  • Joseph, L. M., Alber-Morgan, S., Cullen, J., & Rouse, C. (2016). The effects of self-questioning on reading comprehension: A literature review. Reading & Writing Quarterly, 32(2), 152–173. https://doi.org/10.1080/10573569.2014.891449.

    Article  Google Scholar 

  • Katzir, T., Lesaux, N. K., & Kim, Y. S. (2009). The role of reading self-concept and home literacy practices in fourth grade reading comprehension. Reading and Writing, 22(3), 261–276. https://doi.org/10.1007/s11145-007-9112-8.

    Article  Google Scholar 

  • Keenan, J. M., Betjemann, R. S., & Olson, R. K. (2008). Reading comprehension tests vary in the skills they assess: Differential dependence on decoding and oral comprehension. Scientific Studies of Reading, 12(3), 281–300. https://doi.org/10.1080/10888430802132279.

    Article  Google Scholar 

  • Kim, Y. S. (2016). Direct and mediated effects of language and cognitive skills on comprehension of oral narrative texts (listening comprehension) for children. Journal of Experimental Child Psychology, 141, 101–120. https://doi.org/10.1016/j.jecp.2015.08.003.

    Article  Google Scholar 

  • Kim, Y. S., Petscher, Y., Schatschneider, C., & Foorman, B. (2010). Does growth rate in oral reading fluency matter in predicting reading comprehension achievement? Journal of Educational Psychology, 102(3), 652–667. https://doi.org/10.1037/a0019643.

    Article  Google Scholar 

  • Kintsch, W. (2005). An overview of top-down and bottom-up effects in comprehension: The CI perspective. Discourse Processes, 39(2–3), 125–128. https://doi.org/10.1080/0163853X.2005.9651676.

    Article  Google Scholar 

  • Klauda, S. L., & Guthrie, J. T. (2008). Relationships of three components of reading fluency to reading comprehension. Journal of Educational Psychology, 100(2), 310–321. https://doi.org/10.1037/0022-0663.100.2.310.

    Article  Google Scholar 

  • Kolic-Vehovec, S., Bajsanski, I., & Zubkovic, B. R. (2010). Metacognition and reading comprehension: Age and gender differences. In A. Efklides & P. Misailidi (Eds.), Trends and prospects in metacognition research (pp. 327–344). Berlin: Springer.

    Chapter  Google Scholar 

  • Little, T. D. (2013). Longitudinal structural equation modeling. New York: Guilford Press.

    Google Scholar 

  • Lysenko, L. V., & Abrami, P. C. (2014). Promoting reading comprehension with the use of technology. Computers & Education, 75, 162–172. https://doi.org/10.1016/j.compedu.2014.01.010.

    Article  Google Scholar 

  • MacGinitie, W. H., MacGinitie, R. K., Maria, K., & Dreyer, L. G. (2002). Gates-MacGinitie reading tests. Fourth edition technical report for forms S&T. Rolling Meadows: Riverside Publishing.

  • McDonald, A.-E. (2012). The dragon’s lair: The story of the Scarlett Square. Captive Island: Beach Walk Books.

    Google Scholar 

  • McKenna, M. C., Reinking, D., Labbo, L. D., & Kieffer, R. D. (1999). The electronic transformation of literacy and its implications for the struggling reader. Reading & Writing Quarterly, 15(2), 111–126. https://doi.org/10.1080/105735699278233.

    Article  Google Scholar 

  • McKeown, M. G., Beck, I. L., & Blake, R. G. (2009). Rethinking reading comprehension instruction: A comparison of instruction for strategies and content approaches. Reading Research Quarterly, 44(3), 218–253. https://doi.org/10.1598/RRQ.44.3.1.

    Article  Google Scholar 

  • McMaster, K. L., van den Broek, P., Espin, C. A., Pinto, V., Janda, B., Lam, E., et al. (2015). Developing a reading comprehension intervention: Translating cognitive theory to educational practice. Contemporary Educational Psychology, 40, 28–40. https://doi.org/10.1016/j.cedpsych.2014.04.001.

    Article  Google Scholar 

  • McNamara, D. S., & Magliano, J. (2009). Toward a comprehensive model of comprehension. Psychology of Learning and Motivation, 51, 297–384. https://doi.org/10.1016/j.cedpsych.2014.04.001.

    Article  Google Scholar 

  • McNamara, D. S., O’Reilly, T. P., Best, R. M., & Ozuru, Y. (2006). Improving adolescent students’ reading comprehension with iSTART. Journal of Educational Computing Research, 34(2), 147–171. https://doi.org/10.2190/1RU5-HDTJ-A5C8-JVWE.

    Article  Google Scholar 

  • McNamara, D. S., Ozuru, Y., & Floyd, R. G. (2017). Comprehension challenges in the fourth grade: The roles of text cohesion, text genre, and readers’ prior knowledge. International Electronic Journal of Elementary Education, 4(1), 229–257.

    Google Scholar 

  • McVay, J. C., & Kane, M. J. (2012). Why does working memory capacity predict variation in reading comprehension? On the influence of mind wandering and executive attention. Journal of Experimental Psychology: General, 141(2), 302. https://doi.org/10.1037/a0025250.

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326. https://doi.org/10.1007/s10648-007-9047-2.

    Article  Google Scholar 

  • National Reading Panel (US), & National Institute of Child Health and Human Development (US). (2000). Report of the national reading panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Rockville: National Institute of Child Health and Human Development, National Institutes of Health.

    Google Scholar 

  • Naumann, J. (2015). A model of online reading engagement: Linking engagement, navigation, and performance in digital reading. Computers in Human Behavior, 53, 263–277. https://doi.org/10.1016/j.chb.2015.06.051.

    Article  Google Scholar 

  • Oakhill, J. V., & Cain, K. (2004). The development of comprehension skills. In T. Nunes & P. Bryant (Eds.), Handbook of children’s literacy. Berlin: Springer.

    Google Scholar 

  • OECD. (2014). PISA 2012 results: What students know and can do—Student performance in mathematics, reading and science (volume I, revised edition). Paris: OECD.

    Google Scholar 

  • Perfetti, C. (2007). Reading ability: Lexical quality to comprehension. Scientific Studies of Reading, 11(4), 357–383. https://doi.org/10.1080/10888430701530730.

    Article  Google Scholar 

  • Perfetti, C., & Stafura, J. (2014). Word knowledge in a theory of reading comprehension. Scientific Studies of Reading, 18(1), 22–37. https://doi.org/10.1080/10888438.2013.827687.

    Article  Google Scholar 

  • Piotrowski, J. T., & Krcmar, M. (2017). Reading with hotspots: Young children’s responses to touchscreen stories. Computers in Human Behavior, 70, 328–334. https://doi.org/10.1016/j.chb.2017.01.010.

    Article  Google Scholar 

  • Rand Study Group, & Snow, C. E. (2001). Reading for understanding. Retrieved from Santa Monica, CA. https://www.rand.org/content/dam/rand/pubs/monograph_reports/2005/MR1465.pdf

  • Rose, D. (2000). Universal design for learning. Journal of Special Education Technology, 15(3), 45–49. https://doi.org/10.1177/016264340001500307.

    Article  Google Scholar 

  • Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66(2), 181–221. https://doi.org/10.3102/00346543066002181.

    Article  Google Scholar 

  • Roskos, K., Brueck, J., & Lenhart, L. (2017). An analysis of e-book learning platforms: Affordances, architecture, functionality and analytics. International Journal of Child-Computer Interaction, 12, 37–45. https://doi.org/10.1016/j.ijcci.2017.01.003.

    Article  Google Scholar 

  • Rouet, J., & Britt, M. (2011). Relevance processes in multiple document comprehension. In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Text relevance and learning from text (pp. 19–52). Charlotte: IAP Information Age Publishing.

    Google Scholar 

  • Salmerón, L., & García, V. (2011). Reading skills and children’s navigation strategies in hypertext. Computers in Human Behavior, 27(3), 1143–1151. https://doi.org/10.1016/j.chb.2010.12.008.

    Article  Google Scholar 

  • Salmerón, L., Vidal-Abarca, E., Martínez, T., Mañá, A., Gil, L., & Naumann, J. (2015). Strategic decisions in task-oriented reading. The Spanish Journal of Psychology. https://doi.org/10.1017/sjp.2015.101.

    Article  Google Scholar 

  • Schiefele, U., Stutz, F., & Schaffner, E. (2016). Longitudinal relations between reading motivation and reading comprehension in the early elementary grades. Learning and Individual Differences, 51, 49–58. https://doi.org/10.1016/j.lindif.2016.08.031.

    Article  Google Scholar 

  • Sénéchal, M., & Cornell, E. H. (1993). Vocabulary acquisition through shared reading experiences. Reading Research Quarterly. https://doi.org/10.2307/747933.

    Article  Google Scholar 

  • Shimada, A., Taniguchi, Y., Okubo, F., Konomi, S. I., & Ogata, H. (2018). Online change detection for monitoring individual student behavior via clickstream data on E-book system. In Proceedings of the 8th international conference on learning analytics and knowledge (pp. 446–450). ACM.

  • Stoeger, H., Sontag, C., & Ziegler, A. (2014). Impact of a teacher-led intervention on preference for self-regulated learning, finding main ideas in expository texts, and reading comprehension. Journal of Educational Psychology, 106(3), 799. https://doi.org/10.1037/a0036035.

    Article  Google Scholar 

  • Takacs, Z. K., Swart, E. K., & Bus, A. G. (2015). Benefits and pitfalls of multimedia and interactive features in technology-enhanced storybooks: A meta-analysis. Review of Educational Research, 85(4), 698–739. https://doi.org/10.3102/0034654314566989.

    Article  Google Scholar 

  • Tannenbaum, K. R., Torgesen, J. K., & Wagner, R. K. (2006). Relationships between word knowledge and reading comprehension in third-grade children. Scientific Studies of Reading, 10(4), 381–398. https://doi.org/10.1207/s1532799xssr1004_3.

    Article  Google Scholar 

  • Topping, K. (2018). Implementation fidelity in computerised assessment of book reading. Computers & Education, 116, 176–190. https://doi.org/10.1016/j.compedu.2017.09.009.

    Article  Google Scholar 

  • van den Broek, P., Bohn-Gettler, C. M., Kendeou, P., Carlson, S., & White, M. J. (2011). When a reader meets a text: The role of standards of coherence in reading comprehension. In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Text relevance and learning from text (pp. 123–139). Charlotte: IAP Information Age Publishing.

    Google Scholar 

  • van den Broek, P., Rapp, D. N., & Kendeou, P. (2005). Integrating memory-based and constructionist processes in accounts of reading comprehension. Discourse Processes, 39(2–3), 299–316. https://doi.org/10.1080/0163853X.2005.9651685.

    Article  Google Scholar 

  • Van Scoter, J. (2008). The potential of IT to foster literacy development in kindergarten. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (pp. 149–161). Boston, MA: Springer. https://doi.org/10.1007/978-0-387-73315-9_9.

    Chapter  Google Scholar 

  • Villagrá-Arnedo, C. J., Gallego-Durán, F. J., Llorens-Largo, F., Compañ-Rosique, P., Satorre-Cuerda, R., & Molina-Carmona, R. (2017). Improving the expressiveness of black-box models for predicting student performance. Computers in Human Behavior, 72, 621–631.

    Article  Google Scholar 

  • Wigfield, A., & Guthrie, J. T. (1997). Relations of children’s motivation for reading to the amount and breadth or their reading. Journal of Educational Psychology, 89(3), 420–432. https://doi.org/10.1037/0022-0663.89.3.420.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by grants from the U.S. Department of Education, Institute of Education Sciences, and Reading for Understanding Network (Grant Number R305N160050) and Developing Electronic-Books to Build Elementary Students’ Word Knowledge, Comprehension Monitoring, and Reading Comprehension (Grant Number R305A170163). Additional funding was provided by the National Institutes of Health, Eunice Kennedy Shriver National Institute for Child and Human Development (Grant Numbers R21HD062834, R01HD48539, and P50HD052120). We thank Danielle McNamara, Moshe Yang, as well as the ISI laboratory members (past and present), parents, teachers, and students.

Author information

Authors and Affiliations

  1. School of Education, University of California, Irvine, 3200 Education Bldg, Irvine, CA, 92697, USA

    Osman Umarji, Stephanie Day, Ying Xu, Elham Zargar, Renzhe Yu & Carol Connor

Authors
  1. Osman Umarji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephanie Day
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elham Zargar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Renzhe Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carol Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephanie Day.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

See Figs. 3, 4, 5 and 6.

Fig. 3
figure 3

Example of a text-only page

Full size image
Fig. 4
figure 4

a Example of a plausible story stream decision page. b Example of an implausible story stream decision page and preceding text page

Full size image
Fig. 5
figure 5

Example of a question page (top) and the feedback provided after the reader responds incorrectly (bottom)

Full size image
Fig. 6
figure 6

Dragon’s Lair structure. Blue boxes = new chapters; green boxes = comprehension/word knowledge questions; purple boxes = story stream decisions; red boxes = implausible story dream decisions

Full size image

Appendix 2

See Figs. 7, 8 and 9.

Fig. 7
figure 7

Word knowledge pre- and post-scores presented by grade level

Full size image
Fig. 8
figure 8

Percentage of embedded questions answered correctly presented by grade level

Full size image
Fig. 9
figure 9

Time spent (in seconds) on text and question pages of WKe-Book

Full size image

Appendix 3

See Table 6.

Table 6 Text analysis of chapter 4 of the WK-eBook
Full size table

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Umarji, O., Day, S., Xu, Y. et al. Opening the black box: user-log analyses of children’s e-Book reading and associations with word knowledge. Read Writ 34, 627–657 (2021). https://doi.org/10.1007/s11145-020-10081-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11145-020-10081-x

Keywords

  • Interactive e-Books
  • Reading comprehension
  • User-log analysis
  • Vocabulary
  • Word knowledge