The effects of formative reading assessments closely linked to classroom texts on high school reading comprehension

Abstract

Older student reading of informational texts like those found in most high school classrooms continue to be a concern. College entrance exams scores attest to the fact that this age group of readers remain largely unprepared for the rigorous, discipline area reading and comprehension demands of higher education. In response to this issue content area teachers have been encouraged to teach reading skills for their subject matter. Standard reading instruction tools have included reading guides and summative reading assessments. This study sought to compare the effects of standard reading pedagogy with computerized, formative reading assessments given to students as they read. The largest publisher of high school textbooks, Pearson Prentice-Hall, and the publisher of the high school government text used in this study currently does not have in place digital, formative reading assessments tracking section by chapter section. High school seniors in nine sections of American Government under two different instructors participated in this mixed method, quasiexperimental study. Results indicate a significant difference in content comprehension with the formative assessment pedagogy. Survey and questionnaire responses suggest students thought they read better and were more motivated to read responding to computerized formative assessments (CBA’s) that closely tracked their subject matter textbooks.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. ACT. (2011). ACT College and Career Readiness Report. Retrieved from http://www.act.org/content/act/en/newsroom.html?year=2011&p=1476&lang=%20nglish.

  2. ACT. (2015). ACT College and Career Readiness Report. Retrieved from https://www.act.org/content/dam/act/unsecured/documents/Condition-of-College-and-Career-Readiness-Report-2015-United-States.pdf.

  3. Afflerbach, P. (2005). High stakes testing and reading assessment: National reading conference policy brief. Journal of Literacy Research, 37(2), 151–162.

    Article  Google Scholar 

  4. Allen, K. D., & Hancock, T. E. (2008). Reading comprehension improvement with individualized cognitive profiles and metacognition. Literacy Research and Instruction, 47, 124–139.

    Article  Google Scholar 

  5. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  6. Bandura, A. (1991). Social theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248–287.

    Article  Google Scholar 

  7. Briggs, D. C., Ruiz-Primo, M. A., Furtak, E., Shepard, L., & Yin, Y. (2012). Meta-analytic methodology and inferences about the efficacy of formative assessment. Educational Measurement: Issues and Practice, 31(4), 13–17.

    Article  Google Scholar 

  8. Brozo, W. G., Moorman, G., Meyer, C., & Stewart, T. (2013). Content area reading and disciplinary literacy. Journal of Adolescent and Adult Literacy, 56(5), 353–357.

    Article  Google Scholar 

  9. Bull, J., & McKenna, C. (2004). Blueprint for computer-assisted assessment. London: Routledge-Falmer.

    Book  Google Scholar 

  10. Bunch, G. C., Walqui, A., & Pearson, P. D. (2014). Complex text and new common standards in the United States: Pedagogical implications for English learners. TESOL Quarterly, 48, 543–559.

    Article  Google Scholar 

  11. Clark, R. E. (2001). Learning from Media. Greenwich, CT: Information Age Publishing.

    Google Scholar 

  12. Coleman, D. (2011). Race to the top: Bringing the common core to life. Retrieved from http://usny.nysed.gov/rttt/resources/bringing-the-common-core-to-life.html.

  13. Corno, L. (1993). The Best-laid plans: Modern conceptions of volition and educational research. Educational Researcher, 22, 14–22.

    Article  Google Scholar 

  14. The Council Chronicle. (2007). A Policy Research Brief on Adolescent Literacy. Retrieved from http://www.ncte.org/.

  15. Creswell, J. W. (2009). Research design. Los Angeles, CA: Sage.

    Google Scholar 

  16. Cuevas, J., Russell, R., & Irving, M. A. (2012). An examination of the effect of customized reading modules on diverse secondary students’ reading comprehension and motivation. Educational Technology Research and Development, 60, 445–467.

    Article  Google Scholar 

  17. Dalton, B., & Strangman, N. (2006). Improving struggling readers’ comprehension through scaffolded hypertexts and other computer-based literacy programs. In M. C. McKenna, L. D. Labbo, R. D. Kieffer, & D. Reinking (Eds.), International handbook of literacy and technology (Vol. 2, pp. 75–92). Mahwah, NJ: Erlbaum.

    Google Scholar 

  18. Dihoff, R. E., Brosvic, G. M., Epstein, M. L., & Cook, M. J. (2003). The role of feedback during academic testing: The delay retention test revisited. The Psychological Record, 53, 533–548.

    Article  Google Scholar 

  19. Economides, A. (2009). Conative feedback in computer-based assessment. Computers in the Schools, 26, 207–223.

    Article  Google Scholar 

  20. Fang, Z. (2014). Preparing content area teachers for disciplinary literacy instruction. Journal of Adolescent and Adult Literacy, 57(6), 444–448.

    Article  Google Scholar 

  21. Farley, J. P., & Kim-Spoon, J. (2014). The development of adolescent self-regulation: Reviewing the role of parent, peer, friend, and romantic relationships. Journal of Adolescence, 37(4), 433–440.

    Article  Google Scholar 

  22. Gates-MacGinite Reading Tests, Fourth Edition. (2007). Rolling Meadows, IL: Riverside Publishing.

  23. Gegner, J. A., Mackay, Donald. H. J., & Mayer, R. E. (2009). Computer-supported aids to making sense of scientific articles: Cognitive, motivational, and attitudinal effects. Educational Technology Research and Development, 57, 79–97.

    Article  Google Scholar 

  24. Gill, M., & Greenhow, M. (2008). How effective is feedback in computer-aided assessments? Learning, Media, and Technology, 33(3), 207–220.

    Article  Google Scholar 

  25. Hargreaves, E. (2005). Assessment for learning? Thinking outside the (black) box. Cambridge Journal of Education, 35(2), 213–224.

    Article  Google Scholar 

  26. Hasselbring, T., & Goin, L. (2004). Literacy instruction for older struggling readers: What is the role of technology? Reading and Writing Quarterly, 20, 123–144.

    Article  Google Scholar 

  27. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112.

    Article  Google Scholar 

  28. He, Q., & Timms, P. (2005). A computer-assisted test design and diagnosis system for use by classroom teachers. Journal of Computer Assisted learning, 21, 419–429.

    Article  Google Scholar 

  29. Heeter, C. (1992). Being there: The subjective experience of presence. Teleoperators and Virtual Environment, MIT Press, 1(2), 262–271.

    Article  Google Scholar 

  30. Higgins, C., & Bligh, B. (2006). Formative computer-based assessment in diagram based domains. Association of computing machinery. Paper presented at the annual joint conference integrating technology into Computer Science Education, in proceedings of the 11th annual SIGCSE conference on innovation in technology in computer science education, 26–28 June 2006, Bologna, Italy. Retrieved January 25, 2007, from http://portal.acm.org/citation.cfm?id=1140123.1140152&coll=&dl=ACM&CFID=15151515&CFTOKEN=6184618.

  31. Hodgen, J., & Marshall, B. (2005). Assessment for learning in English and Mathematics: A comparison. The Curriculum Journal, 16(2), 153–176.

    Article  Google Scholar 

  32. Hooley, D., Tysseling, L., & Ray, B. (2013). High school seniors’ perception of class reading. The High School Journal, 96(4), 321–338.

    Article  Google Scholar 

  33. Hoska, D. M. (1993). Motivating learners through CBI feedback: Developing a positive learner perspective. In V. Dempsey & G. C. Sales (Eds.), Interactive instruction and feedback (pp. 105–132). Englewood Cliffs, NJ: Educational Technology Publications.

    Google Scholar 

  34. Irvin, J. L., Buehl, D. R., & Klemp, R. M. (2007). Reading and the high school student. Boston: Pearson Press.

    Google Scholar 

  35. Johnson, K. M. (2009). Review of the Gates-MacGinitie reading test, fourth edition, forms S and T. In Mental measurements yearbook. Retrieved July, 2010, from http://web.edbscohost.

  36. Jordan, S. (2012). Student engagement with assessment and feedback: Some lessons from short-answer, free-text, e-assessment questions. Computers & Education, 53, 818–834.

    Article  Google Scholar 

  37. Kamil, M. (2003). Adolescents and literacy: Reading for the 21st century. Retrieved September 16, 2009, from http://www.all4ed.org.

  38. Karay, Y., Schauber, S., Stosch, C., & Schuettpelz-Brauns, K. (2012). Can computer-based assessment enhance the acceptance of formative multiple choice exams? A utility. Medical Teacher, 34, 292–296.

    Article  Google Scholar 

  39. Lee, C. D., & Spratley, A. (2010). Reading in the disciplines: The Challenge of adolescent literacy. New York: Carnegie Corporation Council on Advancing Adolescent Literacy. Retrieved from http://carnegie.org/fileadmin/Media/Publications/PDF/tta_Lee.pdf.

  40. Lewis, D., & Sewell, R. (2007). Instructional design and assessment: Providing formative feedback from a summative computer-aided assessment. American Journal of Pharmaceutical Education, 71(2), 1–5.

    Article  Google Scholar 

  41. Lynch, L., Fawcett, A., & Nicolson, R. (2000). Computer-assisted reading intervention in a secondary school: An evaluation study. British Journal of Educational Technology, 31(4), 333–348.

    Article  Google Scholar 

  42. Mason, B. J., & Bruning, R. (2001). Providing feedback in computer-based instruction: What the research tells us (p. 14). Lincoln, NB: Center for Instructional Innovation, University of Nebraska-Lincoln. Retrieved from http://dwb.unl.edu/Edit/MB/MasonBruning.html.

  43. Maxwell, J. A. (2005). Qualitative research design: An interactive approach. Thousand Oaks: Sage.

    Google Scholar 

  44. McClain, M. (2000). Using technology for diagnosis and intervention in reading. Media and Methods Magazine Online. Retrieved from http://www.media-methods.com/p3.html.

  45. McClenaghan, W. (2003). MacGruder’s American Government. Needham, MA: Pearson.

    Google Scholar 

  46. McKenna, M., & Stahl, K. (2008). Assessment for reading instruction. New York: Guilford Press.

    Google Scholar 

  47. Meyer, B. J. F., & Wijekumar, K. (2007). A web-based tutoring system for the structure strategy: Theoretical background, design, and findings. In D. S. McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and technologies (pp. 347–375). Mahwah, NJ: Erlbaum.

    Google Scholar 

  48. Meyer, B., Wijekumar, K., Middlemiss, W., Higley, K., Lei, P.-W., Meier, C., et al. (2010). Web-based tutoring of the structure strategy with or without elaborated feedback or choice for 5th and 7th grade readers. Reading Research Quarterly, 54(1), 62–92.

    Article  Google Scholar 

  49. Miller, T. (2009). Formative computer-based assessment in higher education: The effectiveness of feedback in supporting student learning. Assessment and Evaluation in Higher Education, 34(2), 181–192.

    Article  Google Scholar 

  50. Mokhtari, K., & Reichard, C. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94(2), 249–259.

    Article  Google Scholar 

  51. Moran, J., Ferdig, R. E., Pearson, P. D., Wardrop, J., & Blomeyer, R. L., Jr. (2008). Technology and reading performance in the middle-school grades: A meta-analysis with recommendations for policy and practice. Journal of Literacy Research, 40(1), 6–58.

    Article  Google Scholar 

  52. Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32, 99–113.

    Article  Google Scholar 

  53. Mullis, I. V. S., Martin, M. O., Kennedy, A. M., & Foy, P. (2007). PIRLS 2006 International Report: IEA’s progress in international literacy study in primary schools in 40 countries. Boston: TIMMS and PIRLS International Study Center, Lynch School of Education, Boston College.

  54. Narciss, S., & Huth, K. (2004). How to design informative tutoring feedback for multi- media learning. In H. M. Niegemann, D. Leutner, & R. Brunken (Eds.), Instructional design for multimedia learning (pp. 181–195). Munster, NY: Waxman.

    Google Scholar 

  55. The Nation’s Report Card. (2015). 2015 Grade 12 reading and mathematics results. Retrieved from http://www.nationsreportcard.gov.

  56. Ness, M. (2007). Reading comprehension strategies in secondary content-area classrooms. Phi Delta Kappan, 89(3), 229–232.

    Article  Google Scholar 

  57. O’Brien, D., Beach, R., & Scharber, C. (2007). “Struggling” middle schoolers: Engagement and literate competence in a reading writing intervention class. Reading Psychology, 28(1), 51–73.

    Article  Google Scholar 

  58. Ogle, D., & Blachowicz, C. L. Z. (2002). Beyond literature circles. In C. Collins Block & M. Pressley (Eds.), Comprehension instruction: Research-based best practices (pp. 259–271). New York: The Guilford Press.

    Google Scholar 

  59. Patrick, J. J. (1988). High school government textbooks. Boulder, CO: ERIC Clearinghouse for Social Studies/Social Science Education. ED 301532. Retrieved from http://www.ericdigests.org/pre-9210/high.htm.

  60. Pearson Prentice-Hall. (n.d.). MacGruder American Government. Retrieved from http://wwwphschool.com.

  61. Peat, M., & Franklin, S. (2002). Supporting student learning: The use of computer-based formative assessment modules. British Journal of Educational Technology, 33(5), 515–523.

    Article  Google Scholar 

  62. Radhakrishnan, P., Lam, D., & Ho, G. (2009). Giving university students incentives to do homework improves their performance. Journal of Instructional Psychology, 36(3), 219–225.

    Google Scholar 

  63. Ramsden, P. (2003). Learning to teach in higher education. London: Routledge.

    Google Scholar 

  64. Raufelder, D., Hoferichter, F., Schneeweiss, D., & Wood, M. (2015). The power of social and motivational relationships for test-anxious adolescents’ academic self-regulation. Psychology in the Schools, 52(5), 447–462.

    Article  Google Scholar 

  65. Reeves, B., & Nass, C. (1996). The Media equation. Stanford, CA: Center for the Study of Language and Information.

    Google Scholar 

  66. Roediger, H., & Marsh, E. (2005). The positive and negative consequences of multiple choice testing. Journal of Experimental Psychology, 31(5), 1155–1159.

    Google Scholar 

  67. SAT. (2013). The SAT report on college and career readiness. Retrieved from http://media.collegeboard.com/homeOrg/content/pdf/sat-report-college-career-readiness-2013.pdf.

  68. Schroth, M. L. (1992). The effects of delay of feedback on a delayed concept formation transfer task. Contemporary Educational Psychology, 77(1), 78–82.

    Article  Google Scholar 

  69. Scott, D. (2012). States continue plugging into digital textbooks. Governing View. Retrieved from http://www.governing.com.

  70. Shepard, L. A. (2005, October). Formative assessment: Caveat emptor. Paper presented at the ETS invitational conference the future of assessment: Shaping teaching and learning, New York NY.

  71. Shute, V. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

    Article  Google Scholar 

  72. Tovani, C. (2004). Do I really have to teach reading?. Portland, ME: Stenhouse.

    Google Scholar 

  73. Welch, M. (2010). Instructional technological factors that impede and impel struggling adolescent students’ reading comprehension. The International Journal of Technology, Knowledge, and Society, 6, 137–150.

    Article  Google Scholar 

  74. What do we know about assessment? (2016). Retrieved from http://www.ncte.org/policy-research/wwk/assess#2%20-%20high%20stakes.

  75. Wolters, C. A. (2003). Regulation of motivation: Evaluating an underemphasized aspect of self-regulated learning. Educational Psychologist, 38, 189–205.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Diana S. Hooley.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hooley, D.S., Thorpe, J. The effects of formative reading assessments closely linked to classroom texts on high school reading comprehension. Education Tech Research Dev 65, 1215–1238 (2017). https://doi.org/10.1007/s11423-017-9514-5

Download citation

Keywords

  • High school seniors
  • College readiness
  • Computer-based assessments
  • Reading comprehension
  • Academic texts