Abstract
Drawing from multiple theoretical frameworks representing cognitive and educational psychology, we present a writing task and scoring system for measurement of students’ informative writing. Participants in this study were 72 fifth- and sixth-grade students who wrote compositions describing real-world problems and how mathematics, science, and social studies information could be used to solve those problems. Of the 72 students, 69 were able to craft a cohesive response that not only demonstrated planning in writing structure but also elaboration of relevant knowledge in one or more domains. Many-facet Rasch Modeling (MFRM) techniques were used to examine the reliability and validity of scores for the writing rating scale. Additionally, comparison of fifth- and sixth-grade responses supported the validity of scores, as did the results of a correlational analysis with scores from an overall interest measure. Recommendations for improving writing scoring systems based on the findings of this investigation are provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alexander P. A. (1997). Mapping the multidimensional nature of domain learning: The interplay of cognitive, motivational, strategic forces. In M. L. Maehr, & P. R. Pintrich (Eds.), Advances in motivation and achievement (Vol. 10, pp. 213–250). Greenwich, CT: JAI Press.
Alexander, P. A. (2003a). Profiling the developing reader: The interplay of knowledge, interest, and strategic processing. In C. M. Fairbanks, J. Worthy, B. Maloch, J. V. Hoffman, & D. L. Schallert (Eds.), The fifty-first yearbook of the national reading conference. Oak Creek, WI: National Reading Conference.
Alexander, P. A. (2003b). The development of expertise: The journey from acclimation to proficiency. Educational Researcher, 32, 10–14.
Alexander, P., Graham, S., & Harris, K. R. (1998). A perspective on strategy research: Prospect and progress. Educational Psychology Review, 10, 129–154.
Alexander, P. A., & Jetton, T. L. (2000). Learning from text: A multidimensional and developmental perspective. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. III, pp. 285–310). Mahwah, NJ: Lawrence Erlbaum Associates.
Alexander, P. A., & Judy, J. E. (1988). The interaction of domain-specific and strategic knowledge in academic performance. Review of Educational Research, 58, 375–404.
Alexander, P. A., Murphy, P. K., & Woods, B. S. (1996). Of squalls and fathoms: Navigating the seas of educational innovation. Educational Researcher, 25(3), 31–36, 39.
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561–573.
Bennett R. E., & Ward W. C. (Eds.). (1993). Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment. Hillsdale, NJ: Lawrence Erlbaum Associates.
Bereiter, C., & Scardamalia, M. (1982). From conversation to composition: The role of instruction in a developmental process. In R. Glaser (Ed.), Advances in instructional psychology (Vol. 2, pp. 1–64). Hillsdale, NJ: Erlbaum.
Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum Associates.
Brennan R. L. (1992). Elements of generalizability theory. Iowa City, IA: ACT.
Brown, A. L. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. Weinert, & R. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 65–116). Hillsdale, NJ: Lawerence Erlbaum Associates.
Camp, R. (1993). The place of portfolios in our changing views of writing assessment. In R. E. Bennett, & W. C. Ward (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 183–212). Hillsdale, NJ: Lawrence Erlbaum Associates.
Cognition, Technology Group at Vanderbilt [CTGV]. (1991). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(6), 2–10.
CTGV (1992). The Jasper Series as an example of anchored instruction: Theory, program, description, and assessment data. Educational Psychologist, 27, 291–315.
Engelhard, G., Jr. (1992). The measurement of writing ability with a many-faceted Rasch model. Applied Measurement in Education, 5, 171–191.
Flower, L., & Hayes, J. (1980). The dynamics of composing: Making plans and juggling constraints. In L. Gregg, & R. Steinberg (Eds.), Cognitive process in writing (pp. 31–50). Hillsdale, NJ: Lawrence Erlbaum Associates.
Graham S. (2006). Strategy instruction and the teaching of writing: A meta-analysis. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 187–207). New York: Guilford Press.
Graham, S., & Harris, K. R. (2003). Students with learning disabilities and the process of writing: A meta-analysis of SRSD studies. In H. L. Swanson, K. R. Harris, & S. Graham (Eds.), Handbook of learning disabilities (pp. 323–344). New York: Guilford Press.
Graham, S., Harris, K. R., & Reid, R. (1992). Developing self-regulated learners. Focus on Exceptional Children, 24, 1–16.
Greenwald, E. A., Persky, H. R., Campbell, J. R., & Mazzeo, J. (1999). NAEP 1998 writing report card for the nation and the states. Education Statistics Quarterly, 1, 23–28.
Harris, K. R., & Graham, S. (1999). Programmatic intervention research: Illustrations from the evolution of self-regulated strategy development. Learning Disabilities Quarterly, 22, 251–262.
Harris, K. R., Graham, S., & Mason, L. H. (2003) Self-regulated strategy development in the classroom: Part of a balanced approach to writing instruction for students with disabilities. Focus on Exceptional Children, 35, 1–16.
Hout, B., & Neal, M. (2006). Writing assessment: A techno-history. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 417–432). New York: Guilford Press.
Keating, D. P. (1990). Charting pathways to the development of expertise. Educational Psychologist, 25, 243–267.
Lawless, K. A., & Kulikowich, J. M. (2006). Domain knowledge and individual interest: The effects of academic level and specialization in statistics and psychology. Contemporary Educational Psychology, 31, 30–43.
Linacre, J. M. (1989). Multi-facet Rasch measurement. Chicago: MESA.
Linacre, J. M. (1996). FACETS: A computer program for analysis of examinations with multiple facets. Chicago: MESA.
Linacre, J. M. (1999). Investigation rating scale category utility. Journal of Outcome Measurement, 3, 103–122.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–105). New York: American Council on Education/Macmillan.
Pressley, M., & Harris, K. (2006). Cognitive strategies instruction: From basic research to classroom instruction. In P. A. Alexander, & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp.265–286). Mahwah, NJ: Lawrence Erlbaum Associates.
Saenz, L. M., & Fuchs, L. S. (2002). Examining the reading difficulty of secondary students with learning disabilities: Expository versus narrative text. Remedial and Special Education, 23, 31–41.
Sanders, T. J., & Schilperoord, J. (2006). Text structure as a window on the cognition of writing: How text analysis provides insights in writing products, writing process. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 386–402). New York: Guilford Press.
Scardamalia, M., & Bereiter, C. (1991). Literate expertise. In K. A. Ericsson, & J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 172–194). Cambridge, UK: Cambridge University Press.
Shermis, M. D., Burstein, J., & Leacock, C. (2006). Applications of computers in assessment, analysis of writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 403–416). New York: Guilford Press.
Sitko, B. M. (1998). Knowing how to write: Metacognition and writing instruction. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 93–115). Mahwah, NJ: Lawrence Erlbaum Associates.
Smith, E. V., Jr., & Kulikowich, J. M. (2004). An application of generalizability theory and many-facet Rasch measurement using a complex problem-solving skills assessment. Educational and Psychological Measurement, 64, 617–639.
Torrance, M., & Galbraith, D. (2006). The processing demands of writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 67–82). New York: Guilford Press.
Walvoord, B. E., & Anderson, V. J. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.
White, E. (1985). Teaching and assessing writing. San Francisco: Jossey-Bass.
Wolfe, E. W., & Dozier, H. (2000). Development of a scale for measuring invasive plant environmentalism. Journal of Applied Measurement, 1, 219–237.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kulikowich, J.M., Mason, L.H. & Brown, S.W. Evaluating fifth- and sixth-grade students’ expository writing: task development, scoring, and psychometric issues. Read Writ 21, 153–175 (2008). https://doi.org/10.1007/s11145-007-9068-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11145-007-9068-8