Abstract
With a focus on the interaction between computer technology and assessment, we first review the typical functions served by technology in the support of various assessment purposes. These include efficiencies in person and item sampling and in administration, analysis, and reporting. Our major interest is the extent to which technology can provide unique opportunities to understand performance. Two examples are described: a tool-based knowledge representation approach to assess content understanding and a team problem-solving task involving negotiation. The first example, using HyperCard as well as paper-and-pencil variations, has been tested in science and history fields. Its continuing challenge is to determine a strategy for creating and validating scoring criteria. The second example, involving a workforce readiness task for secondary school, has used expert-novice comparisons to infer performance standards. These examples serve as the context for the exploration of validity, equity, and utility.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (1985).Standards for Educational and Psychological Testing, American Psychological Association, Washington, D.C.
Aschbacher, P. R. (1993).Student Performance Assessment Protocols (deliverable to OERI), National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Baker, E. L. (1994). Learning-based assessments of history understanding.Educational Psychologist 29(2): 97–106.
Baker, E. L., and Herman, J. L. (1988).Implementing STAR: Sensible Technology Assessment/Research (report to Apple Computer, CSE Tech. Rep. No. 285), Center for the Study of Evaluation, University of California, Los Angeles (ERIC Document Reproduction Service No. ED 338 682).
Baker, E. L., and Stites, R. (1991). Trends in testing in the USA. InPolitics of Education Association Yearbook 90, Taylor and Francis, London, pp. 139–157.
Baker, E. L., Niemi, D., Gearhart, M., and Herman, J. L. (1990). Validating a hypermedia measure of knowledge representation. Paper presented at the annual meeting of the American Educational Research Association, Boston.
Baker, E. L., Freeman, M., and Clayton, S. (1991a). Cognitive assessment of history for large-scale testing. In Wittrock, M. C., and Baker, E. L. (Eds.),Testing and Cognition, Prentice Hall, Englewood Cliffs, New Jersey, pp. 131–153.
Baker, E. L., Gearhart, M., and Herman, J. L. (1991b).The Apple Classrooms of Tomorrow: 1990 Evaluation Study, Center for the Study of Evaluation, University of California, Los Angeles.
Baker, E. L., O'Neil, H. F., Jr., and Linn, R. L. (1993). Policy and validity prospects for performance-based assessment.American Psychologist 48: 1210–1218.
Baxter, G. P., Glaser, R., and Raghavan, K. (1993).Analysis of Cognitive Demand in Selected Alternative Science Assessments (Deliverable to OERI), National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Braun, H. (1994). Assessing technology in assessment. In Baker, E. L., and O'Neil, H. F., Jr. (Eds.),Technology Assessment in Education and Training, Lawrence Erlbaum Associates, Hillsdale, New Jersey, pp. 231–246.
Braun, H. I., Bennett, R. E., Frye, D., and Soloway, E. (1990). Scoring constructed responses using expert systems.Journal of Educational Measurement 27: 93–108.
Brett, J. M., Goldberg, S. B., and Ury, W. L. (1990). Designing systems for resolving disputes in organizations.American Psychologist 45: 162–170.
Britton, B. K., and Eisenhart, F. J. (1993). Expertise, test coherence, and constraint satisfaction: Effects on harmony and settling rate. InProceedings of the Cognitive Science Society, Pasadena, California.
Bunderson, C. V., Inouye, D. K., and Olsen, J. B. (1989). The four generations of computerized education measurement. In Linn, R. L. (Ed.),Educational Measurement, 3rd ed., Macmillan, New York, pp. 367–407.
Carnevale, P. J., and Conlon, D. E. (1988). Time pressure and strategic choice in mediation.Organizational Behavior and Human Decision Processes 42: 111–133.
Collis, K., and Romberg, T A. (1991). Assessment of mathematical performance: An analysis of open-ended test items. In Wittrock, M. C., and Baker, E. L. (Eds.),Testing and Cognition, Prentice Hall, Englewood Cliffs, New Jersey, pp. 82–130.
Fischer, K. (1989). SemNet. Mimeo, University of Davis, California, reproduced in McAleese, R. (Ed.),Hypertext: Theory Into Practice, Ablex/Blackwell Scientific, Norwood, New Jersey, pp. 16, 19.
Green, B. F. (1991). Guidelines for computer testing. In Gutkin, T. B., and Wise, S. L. (Eds.),The Computer and the Decision-Making Process, Lawrence Erlbaum Associates, Hillsdale, New Jersey, pp. 245–273.
Hegelson, S. L., and Kumar, D. D. (1993). A review of educational technology in science assessment.Journal of Computers in Mathematics and Science Teaching 12: 227–243.
Herl, H., and Baker, E. L. (1993).Expert-Novice Knowledge Structures: How Indirect and Direct Knowledge Representation Tasks Assess History Understanding (deliverable to OERI), National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Jonassen, D. H., Beissner, K., and Yacci, M. (1993).Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge, Lawrence Erlbaum Associates, Hillsdale, New Jersey.
Kelley, H. H. (1979).Personal Relationships: Their Structures and Processes, Lawrence Erlbaum Associates, Hillsdale, New Jersey.
Kelley, H. H., and Thibaut, J. W. (1978).Interpersonal Relations: A Theory of Interdependence, Wiley-Interscience, New York.
Kumar, D. D., and Fritzer, P. (1994). Critical thinking assessment in a science and social studies context.American Secondary Education 22: 2–4.
Lambiotte, J. G., Dansereau, D. F., Cross, D. R., and Reynolds, S. B. (1989). Multirelational semantic maps.Educational Psychology Review 1: 331–367.
Linn, R. L., Baker, E. L., and Dunbar, S. B. (1991). Complex, performance-based assessment: Expectations and validation criteria.Educational Researcher 20: 15–21.
National Council on Education Standards and Testing. (1992).Raising Standards for American Education, Author, Washington, D.C.
Norman, D. A. (1990).The Design of Everyday Things, Doubleday, New York.
Novak, J. D., and Gowin, D. B. (1984).Learning How to Learn, Cambridge University Press, Cambridge.
O'Neil, H. F., Jr., and Baker, E. L. (1994). A technology-based authoring system for assessment. In Dijkstra, S., and Seel, N. (Eds.),Instructional Design: International Perspectives. Vol II: Solving Instructional Design Problems, Lawrence Erlbaum Associates, Hillsdale, New Jersey, pp. 13–45.
O'Neil, H. F., Jr., Baker, E. L., Ni, Y., Jacoby, A., and Swigger, K. M. (1994). Human benchmarking for the evaluation of expert systems. In O'Neil, H. F., Jr., and Baker, E. L. (Eds.),Technology Assessment in Software Applications, Lawrence Erlbaum Associates, Hillsdale, New Jersey, pp. 13–45.
O'Neil, H. F., Jr., Allred, K. G., and Baker, E. L. (1992a).Measurement of Workforce Readiness: Review of Theoretical Frameworks, CSE Technical Report No. 343, Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
O'Neil, H. F., Jr., Allred, K., and Dennis, R. (1992b).Simulation as a Performance Assessment Technique for the Interpersonal Skill of Negotiation (deliverable to OERI), Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
O'Neil, H. F., Jr., Allred, K., and Dennis, R. (1993).Report on Preliminary Study to Test Prototype Workforce Readiness Group Problem-Solving Task and Scoring System: Assessment Issues in the Validation of a Computer Simulation of Negotiation Skills (deliverable to OERI), Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Pruitt, D. G., and Rubin, J. Z. (1986).Social Conflict: Escalation, Stalemate, and Settlement, Random House, New York.
Pruitt, D. G., and Syna, H. (1984). Successful problem solving. In Tjosvold, D., and Johnson, D. W. (Eds.),Productive Conflict Management: Perspective for Organizations, Irvington, New York, pp. 129–148.
Rahim, M. A. (1986).Managing Conflict in Organizations, Praeger, New York.
Resnick, L. B., and Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational reform. In Gifford, B. R., and O'Connor, M. C. (Eds.),Future Assessments: Changing Views of Aptitude, Achievement, and Instruction, Kluwer, Boston, pp. 37–75.
Rubin, J., and Brown, B. R. (1975).The Social Psychology of Bargaining and Negotiation, Academic Press, New York.
Schoenfeld, A. H. (1985).Mathematical Problem Solving, Academic Press, San Diego, California.
Shavelson, R. J. (1993).Indirect Approaches to Knowledge Representation of High School Science: On Concept Maps as Potential “Authentic” Assessments in Science (deliverable to OERI), National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Shavelson, R. J., Baxter, G. P., and Pine, J. (1991). Performance assessment in science.Applied Measurement in Education 4: 347–362.
Sirotnik, K. (1970). An investigation of the context effect in matrix sampling.Journal of Educational Measurement 7: 199–207.
Sugrue, B. (1993).Specifications for the Design of Problem-Solving Assessments in Science (deliverable to OERI), National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Tjosvold, D. (1990). The goal interdependence approach to communication in conflict: An organizational study. In Rahim, M. A. (Ed.),Theory and Research in Conflict Management, Praeger, New York, pp. 15–31.
Tjosvold, D., and Johnson, D. W. (1983).Productive Conflict Management, Irvington Publishers, New York.
U.S. Department of Labor. (1991).What Work Requires of Schools: A SCANS Report for America 2000, U.S. Department of Labor, Secretary's Commission on Achieving Necessary Skills, Washington, D.C.
U.S. Department of Labor. (1992).Skills and Tasks for Jobs: A SCANS Report for America 2000, U.S. Department of Labor, Secretary's Commission on Achieving Necessary Skills, Washington, D.C.
Walton, R. E., and McKersie, R. E. (1965).A Behavioral Theory of Labor Negotiations, McGraw-Hill, New York.
Webb, N. (1993).Collaborative Group versus Individual Assessment in Mathematics: Processes and Outcomes, CSE Technical Report No. 352, Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.
Weiss, D. J. (1985).Computerized Adaptive Measurement of Achievement and Ability, Computerized Adaptive Testing Laboratory, University of Minnesota, Minneapolis, Minnesota.
Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment.Phi Delta Kappan 70: 703–713.
Womack, D. F. (1990). Applied communications research in negotiation: Implications for practitioners. In Rahim, M. A. (Ed.),Theory and Research in Conflict Management, Praeger, New York, pp. 32–53.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Baker, E.L., O'Neil, H.F. Computer technology futures for the improvement of assessment. J Sci Educ Technol 4, 37–45 (1995). https://doi.org/10.1007/BF02211580
Issue Date:
DOI: https://doi.org/10.1007/BF02211580