Abstract
The use of automated scoring in large-scale testing for high-stakes purposes has generated a lot of debate and controversy in the last decade, and researchers have described a variety of ways in which the use of automated scoring may impact validity (e.g., Clauser, Kane, & Swanson, 2002; Weigle, 2010; Xi, 2010). Validity frameworks that tie automated scoring into the overall validity argument for using an assessment (Williamson, Xi & Breyer, 2012; Xi, 2012) highlight the many ways in which the use of automated scoring may impact the validity argument for the entire assessment, including test takers’ interactions with assessment tasks; the accuracy and generalizability of assessment scores; test takers’ interpretations and uses of assessment scores; and the overall consequences for test takers, the educational system, and the broader society.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bennett, R. E., & Bejar, I. I. (1998). Validity and automated scoring: It’s not only the scoring. Educational Measurement: Issues and Practice, 17(4), 9–17.
Brent, E., & Townsend, M. (2006). Automated essay grading in the sociology classroom. In P. F. Ericsson & R. Haswell (Eds.), Machine scoring of student essays: Truth and consequences (pp. 177–198). Logan, Utah: Utah State University Press.
Bridgeman, B., Powers, D., Stone, E., & Mollaun, P. (2012). TOEFL iBT Speaking test scores as indicators of oral communicative language proficiency. Language Testing, 29(1), 91–108.
Clauser, B. E., Kane, M. T., & Swanson, D. B. (2002). Validity issues for performance-based tests scored with computer-automated scoring systems. Applied Measurement in Education, 15(4), 413–432.
Deane, P. (2013). On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing, 18(1), 7–24.
Herrington, A., & Moran, C. (2001). What happens when machines read our students’ writing? College English, 63(4), 480–499.
Herrington, A., & Moran, C. (2006). Write placer plus in place: An exploratory case study. In P. F. Ericsson & R. Haswell (Eds.), Machine scoring of student essays: Truth and consequences (pp. 114–129). Logan, Utah: Utah State University Press.
Kelly, P. A. (2001). Automated scoring of essays: Evaluating score validity (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3028998)
Landauer, T. K., Laham, D., & Foltz, P. (2003). Automatic essay assessment. Assessment in Education, 10(3), 295–308.
Powers, D. E., Burstein, J. C., Chodorow, M., Fowles, M. E., & Kukich, K. (2001). Stumping e-rater: Challenging the validity of automated essay scoring (Graduate Record Examination Board Professional Report No. 98-08bP). Princeton, NJ: Educational Testing Service.
Weigle, S. C. (2010). Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing, 27(3), 335–353.
Williamson, D., Xi, X., & Breyer, J. (2012). A Framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice, 31(1), 2–13.
Xi, X. (2010). Automated scoring and feedback systems x2014; Where are we and where are we heading? Language Testing, 27(3), 291–300.
Xi, X. (2012). Validity and the automated scoring of performance tests. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 438–451). New York: Routledge.
Xi, X., Higgins, D., Zechner, K., & Williamson, D. M. (2008). Automated scoring of spontaneous speech using SpeechRater v1.0 (Research Report No. RR-08-62). Princeton, NJ: Educational Testing Service.
Yang, Y., Buckendahl, C. W., Juszkiewicz, P. J., & Bhola, D. S. (2002). A review of strategies for validating computer-automated scoring. Applied Measurement in Education, 15(4), 391–412.
Editor information
Editors and Affiliations
Copyright information
© 2016 Xiaoming Xi, Jonathan Schmidgall and Yuan Wang
About this chapter
Cite this chapter
Xi, X., Schmidgall, J., Wang, Y. (2016). Chinese Users’ Perceptions of the Use of Automated Scoring for a Speaking Practice Test. In: Yu, G., Jin, Y. (eds) Assessing Chinese Learners of English. Palgrave Macmillan, London. https://doi.org/10.1057/9781137449788_8
Download citation
DOI: https://doi.org/10.1057/9781137449788_8
Publisher Name: Palgrave Macmillan, London
Print ISBN: 978-1-349-55397-6
Online ISBN: 978-1-137-44978-8
eBook Packages: Social SciencesSocial Sciences (R0)