Abstract
As part of an overall evaluation of the Global Learning and Observations to Benefit the Environment, (GLOBE) program, we designed a Web-based assessment environment to measure students' environmental awareness and data analysis skill. It was expected that students who were identified as high implementers in the GLOBE program would outperform low implementers in their ability to construct environmental inferences and the degree to which they could analyze environmental data. Seven high and middle school classrooms were identified as either high or low GLOBE implementers depending on the amount of atmospheric data they had collected during the year. Within each classroom students were assigned into smaller learning groups of three students per group. A total of 32 groups participated in this study. Analysis of students' responses to the tasks revealed that the students differed in their performance. Overall, the results showed that students in the high implementing classrooms were more likely to construct higher-level environmental inferences than students in the low implementing classes. Contrary to expectations, middle school students were more likely than high school students to solve the data analysis problem correctly. However, upon further analyses, high school students constructed more data graphs and were more skilled in providing correct evidence to support their decision making than were middle school students in GLOBE. This study confirms the viability of using technology-based assessments for measuring students' environmental awareness and data analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Baker, E. L. (1998). Model-based Performance Assessment. University of California, National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles.
Baker, E. L., Aschbacher, P. R., Nieme, D., and Sato, E. (1992). CRESST performance assessment models: Assessing content area explanations. University of California, National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles.
Baker, E. L., Aschbacher, P., Niemi, D., Sato, E., and Redfield, D. (1992). Issues in large-scale performance assessment. Invited minicourse presentation at the annual meeting of the American Educational Research Association/National Council on Measurement in Education, San Francisco.
Bransford, J. D., and Stein, B. S. (1984). The IDEAL Problem Solver. Freeman, New York.
Chi, M. T. H., de Leeuw, N., Chiu, M.-H., and LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science. 18: 439–477.
Chi, M. T. H., Glaser, R., and Farr, M. J., (1988). The Nature of Expertise. Lawrence Erlbaum Associates, Hillsdale, NJ.
Cognition and Technology Group at Vanderbilt. (1997). The Jasper Project: Lessons in curriculum, instruction, assessment, and professional development. Lawrence Erlbaum Associates, Mahwah, NJ.
Coleman, E. B. (1998). Using explanatory knowledge during collaborative problem solving. Journal of the Learning Sciences 7: 387–427.
Coleman, E. B., Brown, A. L., and Rivkin, I. D. (1997). The effect of instructional explanations on learning from scientific texts. Journal of the Learning Sciences 6: 347–365.
Darling-Hammond, L., Ancess, J., and Falk, B. (1995). Authentic Assessment in Action: Studies of Schools and Students at Work. Teachers College Press, New York.
Fisher, C., Dwyer, D. C., and Yocam, K. (1996). Education and Technology: Reflections on Computing in Classrooms. Jossey-Bass, San Francisco.
Gobert, J. D., and Clement, J. J. (1999). Effects of student-generated diagrams versus student-generated summaries on conceptual understanding of causal and dynamic knowledge in plate tectonics. Journal of Research in Science Teaching 36: 39–53.
Gore, A. (1992). Earth in the Balance: Ecology and the Human Spirit, Houghton Mifflin, New York.
International Society for Technology in Education. (1998). National Educational Technology Standards for Students: Connecting Curriculum and Technology. ISTE, Eugene, OR.
Kulik, C., and Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers and Human Behavior 7: 75–94.
Latour, B. (1987). Science in Action. Harvard University Press, Cambridge, MA.
Lemke, J. L. (1985). Talking Science: Language, Learning, and Values. Ablex, Norwood, NJ.
Lesgold, A. (1998, January). On evaluating educational innovations. Presentation to the Office of Science and Technology Policy at RAND Corporation, Washington, DC.
Linn, M. C., Bell, P., and Hsi, S. (1998). Using the Internet to enhance student understanding of science: The Knowledge Integration Environment. Interactive Learning Environments 6: 4–38.
Means, B. (1994). Technology and Education Reform, Jossey-Bass, San Francisco.
Means, B., Coleman, E. B., and Lewis, A. (1998). GLOBE Year 3Evaluation: Implementation and Progress, SRI International, Menlo Park, CA.
Means, B., Coleman, E., Lewis, A., Quellmalz, E., Marder, C., and Valdes, K. (1997).GLOBE Year 2 Evaluation: Implementation and Progress, SRI International Menlo Park, CA.
Means, B., Middleton, T., Lewis, A., Quellmalz, E., and Valdes, K. (1996). GLOBE Year 1 Evaluation: Findings, SRI International Menlo Park, CA.
Means, B., and Olson, K. (1995). Technology's Role in Education Reform, SRI International, Menlo Park, CA.
Messick, S. (1992). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher 23: 13–23.
Mislevy, R. J., Almond, R. G., Yan, D., and Steinberg, L. S. (1999). Bayes nets in educational assessment: Where do the numbers come from? In Laskey, K.B., and Prade, H. (Eds.), Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kauffman, San Francisco.
Mislevy, R. J., Steinberg, L. S., and Almond, R. G. (in press). On the several roles of task model variables in assessment design. In Irvine, S., and Kyllonen, P. (Eds.), Generating Items for Cognitive Tests: Theory and Practice. Lawrence Erlbavm Associates, Hillsdale, NJ.
Ohlsson, S. (1992). The cognitive skill of theory articulation: A neglected aspect of science education? Science and Education 1: 181–192.
Osmundson, E., Chung, G. K. W. K., Herl, H. E., and Klein, D. C. D. (1999). Knowledge mapping in the classroom: A tool for examining the development of students' conceptual understandings. CSE Technical Report # 507. University of California, Los Angeles.
Pea, R. D. (1994). Seeing what we build together: Distributed multimedia learning environments for transformative communications. Journal of the Learning Sciences 3: 283–298.
Perrone, V. (1991). Expanding Student Assessment. Association for Supervision and Curriculum Development, Alexandria, VA.
President's Committee of Advisors on Science and Technology. (1997). Report to the President on the Use of Technology to Strengthen K- 12 Education in the United States. Executive Office of the President of the United States, Washington, DC.
Rehder, B., Schreiner, M.E., Wolfe, M.B., Laham, D. Landauer, T.K., and Kintsch, W. (1998). Using Latent Semantic Analysis to assess knowledge: Some technical considerations. Discourse Processes 25: 337–354.
Resnick, L. B. (1987). Learning in school and out. Educational Researcher 16, 13–20.
Richards, L. (1999). Using Nvivo in Qualitative Research. Sage, Thousand Oaks, CA.
Sandholtz, J. H., Ringstaff, C., and Dwyer, D. C. (1997). Teaching with Technology: Creating Student-centered Classrooms. Teachers College Press, New York.
Scardamalia, M., and Bereiter, C. (1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences 3: 265–283.
Schacter, J., Herl, H. E., Chung, G. K. W. K., O'Neil, H. F., Jr., Dennis, R. A., and Lee, J. L. (1997, March). Feasibility of a web-based assessment of problem solving. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago.
Sternberg, R. J. (1985). Beyond IQ. Cambridge University Press, Cambridge.
Stevens, R. H., Lopo, A. C., and Wang, P. (1996). Artificial neural networks can distinguish novice and expert strategies during complex problem solving. Journal of the American Medical Informatics Association 3: 131–138
Tinker, B. (1999). Equity computers: The handheld alternative. Concord Consortium Newsletter Fall: available at http:// www.concord.org/library/1999fall/perspective.html.
Toulmin, S. (1958). The Uses of Argument. Cambridge University Press, Cambridge.
Vygotsky, L. S. (1978). Mind in Society: TheDevelopment of Higher Psychological Processes. Harvard University Press, Cambridge, MA.
Webb, N. (1989). Peer interaction and learning in small groups. International Journal of Educational Research 13: 21–39.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Coleman, E.B., Penuel, W.R. Web-Based Student Assessment for Program Evaluation. Journal of Science Education and Technology 9, 327–342 (2000). https://doi.org/10.1023/A:1009432509448
Issue Date:
DOI: https://doi.org/10.1023/A:1009432509448