Abstract
In tests used to measure reading comprehension, validity is important in obtaining accurate results. Unfortunately, studies have shown that people can correctly answer some questions of these tests without reading the related passage. These findings bring forth the need to address whether this phenomenon is observed in multiple-choice only tests or in those that employ open-ended questions. Three common standardized reading comprehension tests were examined: the WIAT-III, the CAAT, and the Nelson–Denny. The WIAT-III is composed of open-ended questions, while the other two tests utilize multiple-choice questions. All participants were instructed to answer the questions to the best of their ability, without access to the related passage. The results revealed that participants correctly answered the questions at a significantly higher rate than by chance for the multiple-choice, which supports the independency issue. For the open-ended questions, participants still answered with 18% accuracy, without the passages.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aiken, L. R., & Groth-Marnat, G. (2006). Psychological testing and assessment (12th ed.). New York: Pearson.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). New York: Pearson.
Canadian Test Center. (1992). Canadian adult achievement test, technical bulletin. Markham: Canadian Test Center Inc.
Coleman, C., Lindstrom, J., Nelson, J., Lindstrom, W., & Gregg, K. N. (2010). Passageless comprehension on the Nelson–Denny Reading Test: Well above chance for university students. Journal of Learning Disabilities, 43(3), 244–249.
Creaser, J., Jacobs, M., Zaccaria, L., & Carsello, C. (1970). Effects of shortened time limits on the Nelson–Denny Reading Test. Journal of Reading, 14(3), 167–170.
Cummins, R., & Porter, (1981). Test review: The Nelson–Denny Reading Test (forms E and F). Journal of Reading, 25(1), 54–59.
Cummins, J., & Swain, M. (2014). Bilingualism in education: Aspects of theory, research and practice. Routledge.
Daneman, M., & Merikle, P. M. (1996). Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin & Review, 3(4), 422–433. doi:10.3758/BF03214546.
Eurich, A. C. (1931). Four types of examinations compared and evaluated. Journal of Educational Psychology, 22(4), 268–278. doi:10.1037/h0075460.
Flanagan, D. P., & Harrison, P. L. (Eds.). (2012). Contemporary intellectual assessment: Theories, tests, and issues. New York: Guilford Press.
Fowler, B., & Kroll, B. M. (1978). Verbal skills as factors in the passageless validation of reading comprehension tests. Perceptual and Motor Skills, 47, 335–338.
Heilbrun, K. (1997). Prediction versus management models relevant to risk assessment: The importance of legal decision-making context. Law and Human Behavior, 21(4), 347–359. doi:10.1023/A:1024851017947.
Katz, S., Lautenschlager, G. J., Blakckburn, A. B., & Harris, F. H. (1990). Answering reading comprehension items without passages on the SAT. Psychological Science, 1(2), 122–127. doi:10.1111/j.1467-9280.1990.tb00080.x.
Keenan, J. M., & Betjemann, R. S. (2006). Comprehending the Gray Oral Reading Test without reading it: Why comprehension tests should not include passage-independent items. Scientific Studies of Reading, 10(4), 363–380. doi:10.1207/s1532799xssr1004_2.
Larcker, D. F., & Lessig, V. P. (1980). Perceived usefulness of information: A psychometric examination. Decision Sciences, 11(1), 121–134. doi:10.1111/j.1540-5915.1980.tb01130.x.
Lewandowski, L. J., Codding, R. S., Kleinmann, A. E., & Tucker, K. L. (2003). Assessment of reading rate in postsecondary students. Journal of Psychoeducational Assessment, 21(2), 134–144. doi:10.1177/073428290302100202.
Lifson, S., Scruggs, T. E., & Bennion, K. (1984). Passage independence in reading achievement tests: A follow-up. Perceptual and Motor Skills, 58(3), 945–946. doi:10.2466/pms.1984.58.3.945.
Lim, Y. K., Pangam, A., Periyasami, S., & Aneja, S. (2006, October). Comparative analysis of high-and low-fidelity prototypes for more valid usability evaluations of mobile devices. In Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (pp. 291-300). ACM. doi: 10.1145/1182475.1182506
McClelland, D. C. (1977). Testing for competence rather than intelligence. American Psychologist, 28(1), 1–14. doi:10.1037/h0034092.
Messik, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256. doi:10.1177/026553229601300302.
Neukrug, E. S., & Fawcett, R. C. (2006). Essentials of testing and assessment: A practical guide for counselors, social workers, therapists, and others. Belmont: Brooks/Cole.
Neukrug, E. S., & Fawcett, R. C. (2010). Essentials of testing and assessment: A practical guide for counselors, social workers, and psychologists (2nd ed.). Belmont, CA: Brooks/Cole.
Preston, R. C. (1964). Ability of students to identify correct responses before reading. The Journal of Educational Research, 58(4), 181–183. doi:10.1080/00220671.1964.10883203.
Rothney, J. W., & Bear, R. M. (1938). An evaluation of visual factors in reading. Hanover: Dartmouth College.
Sparfeld, J. R., Kimmel, R., Löwenkamp, L., Steingräber, A., & Rost, D. H. (2012). Not read, but nevertheless solved? Three experiments on PIRLS multiple choice reading comprehension test items. Educational Assessment, 17(4), 214–232. doi:10.1080/10627197.2012.735921.
Tamm, L., Epstein, J. N., Denton, C. A., Vaughn, A. J., Peugh, J., & Willcutt, E. G. (2014). Reaction time variability associated with reading skills in poor readers with ADHD. Journal of the International Neuropsychological Society, 20(3), 292–301. doi:10.1017/S1355617713001495.
Tian, S. (2006). Passage dependency of reading comprehension items in the GEPT and the TOEFL. The Reading Matrix, 6, 66–84.
Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). The Stanford-Binet intelligence scale: Guide for administering and scoring. Riverside Pub. Co.
Viswesvaran, C., & Ones, D. S. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48(4), 865–885. doi:10.1111/j.1744-6570.1995.tb01784.x.
Wechsler, D. (2009). Weschsler individual achievement test 3 rd edition: Manual. San Antonio: Psychological Corporporation.
Willms, J. D., 1992 (2004). Monitoring school performance: A guide for educators. Bristol, PA: Taylor and Francis Inc.
Acknowledgements
This research was supported by a Canada Foundation for Innovation infrastructure grant and a NSERC Discovery grant to Annie Roy-Charland. We thank Caroline Comeau for her assistance in running participants and data coding.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Roy-Charland, A., Colangelo, G., Foglia, V. et al. Passage independence within standardized reading comprehension tests. Read Writ 30, 1431–1446 (2017). https://doi.org/10.1007/s11145-017-9731-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11145-017-9731-7