Abstract
In 1980, I published a study on the relative merits of four cloze scoring methods [exact-answer (EX), acceptable-answer (AC), clozentropy (CLZNT), and multiple-choice (MC) scoring] analyzed in terms of item analysis, reliability, and validity statistics. My interpretation of the results was that AC was the best overall scoring method because AC produced the best item facility, item discrimination, and reliability estimates, and was tied with CLZNT in validity coefficients. I later realized that I made two important errors:
-
I did not include my descriptive statistics in thinking about and interpreting the other testing statistics in my study.
-
I forgot that all testing statistics are for scores based on performances of a certain group of examinees on one set of items under a particular set of conditions.
My solutions to these problems were based on learning from my mistakes: (1) reporting and examining the descriptive statistics (especially in relationship to any more advanced statistics) in all of my subsequent statistical studies and (2) stressing this important set of relationships to all of my students who have used statistics in their studies. Readers can learn from my explanations of these mistakes and from remembering in their own research and in reading published research that investigators should include the descriptive statistics in thinking about and interpreting all other testing statistics and remember that testing statistics are only for scores based on performances of a certain group of examinees on one set of items under a particular set of conditions, period.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, J.D. (1978). Correlational study of four methods for scoring cloze tests. MA thesis, University of California, Los Angeles.
Brown, J. D. (1980). Relative merits of four methods for scoring cloze tests. Modern Language Journal, 64(3), 311–317.
Brown, J.D. (1984). A cloze is a cloze is a cloze? In J. Handscombe, R. Orem, & B. Taylor (Eds.), On TESOL ‘83: The question of control. Selected papers from the 17th Annual TESOL Convention, Toronto (pp. 109–119). Washington, DC: TESOL (also available from ERIC: ED275145).
Brown, J. D. (1988). Understanding research in second language learning: A teacher’s guide to statistics and research design. Cambridge: Cambridge University.
Brown, J. D. (1993). What are the characteristics of natural cloze tests? Language Testing, 10(2), 93–116.
Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment (New ed.). New York: McGraw-Hill.
Brown, J. D. (2013). My twenty-five years of cloze testing research: So what? International Journal of Language Studies, 7(1), 1–32.
Brown, J. D., Trace, J., Janssen, G., & Kozhevnikova, L. (2016). How well do cloze items work and why? In C. Gitsaki & C. Coombe (Eds.), Current trends in language evaluation, assessment and testing: Research perspectives (pp. 2–39). Newcastle Upon Tyne, UK: Cambridge Scholars.
Brown, J.D., Yamashiro, A.D., Ogane, E. (2001). The emperor’s new cloze: Strategies for revising cloze tests. In T. Hudson & J.D. Brown (Eds.), A focus on language test development: Expanding the language proficiency construct across a variety of tests (pp. 143–161). Honolulu, HI: University of Hawai‘i Press.
Fry, E. (1977). Graph for estimating readability–extended. New Brunswick, NJ: Rutgers University Reading Center.
Trace, J., Brown, J. D., Janssen, G., & Kozhevnikova, L. (2017). Determining cloze item difficulty from item and passage characteristics across learner backgrounds. Language Testing, 33(1), 151–174.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Brown, J.D. (2021). Problems Caused by Ignoring Descriptive Statistics in Language Testing . In: Lanteigne, B., Coombe, C., Brown, J.D. (eds) Challenges in Language Testing Around the World. Springer, Singapore. https://doi.org/10.1007/978-981-33-4232-3_2
Download citation
DOI: https://doi.org/10.1007/978-981-33-4232-3_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-33-4231-6
Online ISBN: 978-981-33-4232-3
eBook Packages: EducationEducation (R0)