Abstract
The development of innovative item types, defined as items that depart from the traditional, discrete, text-based, multiple-choice format, is perhaps the most promising area in the entire field of computer-based testing. The reason for this is the great potential that item innovations have for substantively improving measurement. Innovative item types are items that include some feature or function made available due to their administration on computer. Items may be innovative in many ways. This chapter will include a discussion of the purpose or value of innovative item types, the five dimensions in which items may be innovative, the impact of level of complexity on the development and implementation of these item types, and a view toward the future of innovative item types.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
ACT, Inc. (1998). Assessing listening comprehension: A Review of Recent Literature Relevant to an LSAT Listening Component. Unpublished manuscript, LSAC, Newton, PA.
ACT, Inc. (1999). Technical Manual for the ESL Exam. Iowa City: Author.
Baker, E. L., & O’Neil, H. F. Jr. (1995). Computer technology futures for the improvement of assessment. Journal of Science Education and Technology, 4, 37–45.
Balizet, S., Treder, D. W., & Parshall, C. G. (1999, April). The development of an audio computer-based classroom test of ESL listening skills. Paper presented at the annual meeting of the American Educational Research Association, Montreal.
Bennett, R. E., & Bejar, I. I. (1998). Validity and automated scoring: It’s not only the scoring. Educational Measurement: Issues & Practices, 17, 9–17.
Bennett, R. E., Goodman, M., Hessinger, J., Ligget, J., Marshall, G., Kahn, H., & Zack, J. (1997). Using Multimedia in Large-Scale Computer-Based Testing Programs (Research Rep. No. RR-97-3). Princeton, NJ: Educational Testing Service.
Bennett, R. E., Morley, M., & Quardt, D. (1998, April). Three Response Types for Broadening the Conception of Mathematical Problem Solving in Computerized-Adaptive Tests. Paper presented at the annual meeting of the National Council of Measurement in Education, San Diego.
Braun, H. (1994). Assessing technology in assessment. In Baker, E. A., & O’Neil, H. F. (eds.), Technology Assessment in Education and Training (pp. 231–246). Hillsdale, NJ: Lawrence Erlbaum Associates.
Breland, H. M. (1998, April). Writing Assessment Through Automated Editing. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego.
Burstein, J., Kukich, K., Wolff, S., Lu, C, & Chodorow, M. (1998, April). Computer Analysis of Essays. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego.
Clauser, B. E., Margolis, M. J., Clyman, S. G., & Ross, L P. (1997). Development of automated scoring algorithms for complex performance assessments: A comparison of two approaches. Journal of Educational Measurement, 34, 141–161.
Davey, T., Godwin, J., & Mittelholtz, D. (1997). Developing and scoring an innovative computerized writing assessment. Journal of Educational Measurement, 34, 21–41.
Drasgow, F., Olson-Buchanan, J. B., & Moberg, P. J. (1999). Development of an interactive video assessment: Trials and tribulations. In Drasgow, F., & Olson-Buchanan, J. B, (eds.), Innovations in Computerized Assessment, (pp. 177–196). Mahwah, NJ: Lawrence Erlbaum Associates.
Educational Testing Service (ETS). (1998). Computer-Based TOEFL Score User Guide. Princeton, NJ: Author.
French, A., & Godwin, J. (1996, April). Using Multimedia Technology to Create Innovative Items. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
Harmes, J. C, & Parshall, C. G. (2000, November). An Iterative Process for Computerized Test Development: Integrating Usability Methods. Paper presented at the annual meeting of the Florida Educational Research Association, Tallahassee.
Landauer, T. K., Laham, D., Rehder, B., & Schreiner, M. E. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans. In Shafto, G., & Langley, P. (eds.), Proceedings of the 19th Annual Meeting of the Cognitive Science Society (pp. 412–417). Mahwah, NJ: Erlbaum.
Olson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P.A., & Donovan, M.A. (1998). Interactive video assessment of conflict resolution skills. Personnel Psychology, 51, 1–24.
Page, E. B. & Petersen N. S. (1995, March). The computer moves into essay grading: Updating the ancient test. Phi Delta Kappen, 76, 561–565.
Parshall, C. G. (1999, February). Audio CBTs: Measuring More through the use of Speech and Nonspeech Sound. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
Parshall, C. G., Davey, T., & Pashley, P. J. (2000). Innovative item types for computerized testing. In Van der Linden, W. J., & Glas, C. A. W. (eds.), Computerized Adaptive Testing: Theory and Practice, (pp. 129–148). Norwell, MA: Kluwer Academic Publisher.
Parshall, C. G., Stewart, R, & Ritter, J. (1996, April). Innovations: Sound, Graphics, and Alternative Response Modes. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
Shavelson, R. J., Baxter, G. P., & Pine, J. (1992). Performance assessments: Political rhetoric and measurement reality. Educational Researcher, 21, 22–27.
Vispoel, W. P., Wang, T., & Bleiler, T. (1997). Computerized adaptive and fixed-item testing of music listening skill: A comparison of efficiency, precision, and concurrent validity. Journal of Educational Measurement, 34, 43–63.
Additional Readings
ACT, Inc. (1995). Work Keys. Iowa City: Author.
Bejar, I. I. (1991). A methodology for scoring open-ended architectural design problems. Journal of Applied Psychology, 76, 522–532.
Bennett, R.E. (1998). Reinventing Assessment. Princeton, NJ: Educational Testing Service.
Bennett, R. E., & Sebrechts, M. M. (1997). A computer-based task for measuring the representational component of quantitative proficiency. Journal of Educational Measurement, 34, 64–77.
Bennett, R. E., Steffen, M., Singley, M. K., Morley, M., & Jacquemin, D. (1997). Evaluating an automatically scorable, open-ended response type for measuring mathematical reasoning in computer-adaptive tests. Journal of Educational Measurement, 34, 162–176.
Booth, J. (1991). The key to valid computer-based testing: The user interface. Revue Européenne de Psychologie Appliquée, 41, 281–293.
Bosman, F., Hoogenboom, J., & Walpot, G. (1994). An interactive video test for pharmaceutical chemist’s assistants. Computers in Human Behavior, 10, 51–62.
Braun, H. I., Bennett, R. E., Frye, D., & Soloway, E. (1990). Scoring constructed responses using expert systems. Journal of Educational Measurement, 27, 93–108.
Breland, H.M. (1998, April). Writing Assessment through Automated Editing. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego.
Bugbee, A. C. Jr., & Bernt, F. M. (1990). Testing by computer: Findings in six years of use. Journal of Research on Computing in Education, 23, 87–100.
Buxton, W. (1987). There’s more to interaction than meets the eye: Some issues in manual input. In Baecker, R. M. & Buxton, W. A. S. (eds.) Readings in Human-Computer Interaction: A Multidisciplinary Approach (pp. 366–375). San Mateo, CA: Morgan Kaufmann.
Clauser, B. E., Ross, L. P., Clyman, S. G., Rose, K. M., Margolis, M. J., Nungester, R. N., Piemme, T. E., Chang, L., El-Bayoumi, G., Malakoff, G. L., & Pincetl, P. S. (1997). Development of a scoring algorithm to replace expert rating for scoring a complex performance-based assessment. Applied Measurement in Education, 10, 345–358.
Dodd, B. G., & Fitzpatrick, S.J. (1998). Alternatives for scoring computer-based tests. Paper presented at the ETS Colloquium, Computer-Based Testing: Building The Foundation For Future Assessments, Philadelphia.
Fitch, W. T., & Kramer, G. (1994). Sonifying the body electric: Superiority of an auditory over a visual display in a complex, multivariate system. In Kramer, G. (ed.), Auditory Display, (pp. 307–325). Reading, MA: Addison-Wesley.
Gaver, W. W. (1989). The SonicFinder: An interface that uses auditory icons. Human-Computer Interaction, 4, 67–94.
Godwin, J. (1999, April). Designing the ACT ESL listening test. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
Gruber, J. S. (1998, October). [Interview with James Kramer, head of Virtual Technologies, Inc.] Gropethink. Wired, pp. 168–169.
Koch, D. A. (1993). Testing goes graphical. Journal of Interactive Instruction Development, 5, 14–21.
Luecht, R. M., & Clauser, B. E. (1998, September). Test methods for complex computer-based testing. Paper presented at the ETS Colloquium, Computer-Based Testing: Building The Foundation For Future Assessments, Philadelphia.
Martinez, M. E. (1991). A comparison of multiple-choice and constructed figurai response items. Journal of Educational Measurement, 28, 131–145.
Martinez, M. E. (1993). Item formats and mental abilities in biology assessment. Journal of Computers in Mathematics and Science Teaching, 12, 289–301.
Martinez, M. E. & Bennett, R. E. (1992). A review of automatically scorable constructed-response item types for large-scale assessment. Applied Measurement in Education, 5, 151–169.
Nissan, S. (1999, April). Incorporating Sound, Visuals, and Text for TOEFL on Computer. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
O’Neill, K., & Folk, V. (1996, April). Innovative CBT Item Formats in a Teacher Licensing Program. Paper presented at the annual meeting of the National Council on Measurement in Education, New York, NY.
Perlman, M., Berger, K., & Tyler, L. (1993). An Application of Multimedia Software to Standardized Testing in Music. (Research Rep. No. 93-36) Princeton, NJ: Educational Testing Service.
Shea, J. A., Norcini, J. J., Baranowski, R. A., Langdon, L. O., & Popp, R. L. (1992). A comparison of video and print formats in the assessment of skill in interpreting cardiovascular motion studies. Evaluation and the Health Professions, 15, 325–340.
Stone, B. (1998, March). Focus on technology: Are you talking to me? Newsweek, pp. 85–86.
Taggart, W. R. (1995). Certifying pilots: Implications for medicine and for the future. In. Mancall, E. L. & Bashook, P. G. (eds.), Assessing Clinical Reasoning: The Oral Examination and Alternative Methods (pp. 175–182). Evanston, IL: American Board of Medical Specialties.
Vicino, F. L., & Moreno, K. E. (1997). Human factors in the CAT system: A pilot study. In Sands, W. A., Waters, B. K, & McBride, J. R. (eds.), Computerized Adaptive Testing: From Inquiry To Operation (pp. 157–160). Washington, DC: APA.
Vispoel, W. P., & Coffman, D. (1992). Computerized adaptive testing of music-related skills. Bulletin of the Council for Research in Music Education, 112, 29–49.
Vispoel, W. P., Wang, T., & Bleiler, T. (1997). Computerized adaptive and fixed-item testing of music listening skill: A comparison of efficiency, precision, and concurrent validity. Journal of Educational Measurement, 34, 43–63.
Williams, V. S. L., Sweeny, S. F., & Bethke, A. D. (1999). The Development and Cognitive Laboratory Evaluation of an Audio-Assisted Computer-Adaptive Test for Eighth-Grade Mathematics. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer Science+Business Media New York
About this chapter
Cite this chapter
Parshall, C.G., Spray, J.A., Kalohn, J.C., Davey, T. (2002). Issues in Innovative Item Types. In: Practical Considerations in Computer-Based Testing. Statistics for Social and Behavioral Sciences. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-0083-0_5
Download citation
DOI: https://doi.org/10.1007/978-1-4613-0083-0_5
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-98731-6
Online ISBN: 978-1-4613-0083-0
eBook Packages: Springer Book Archive