Advertisement

Reading and Writing

, Volume 30, Issue 7, pp 1431–1446 | Cite as

Passage independence within standardized reading comprehension tests

  • Annie Roy-Charland
  • Gabrielle Colangelo
  • Victoria Foglia
  • Leïla Reguigui
Article
First Online:

Abstract

In tests used to measure reading comprehension, validity is important in obtaining accurate results. Unfortunately, studies have shown that people can correctly answer some questions of these tests without reading the related passage. These findings bring forth the need to address whether this phenomenon is observed in multiple-choice only tests or in those that employ open-ended questions. Three common standardized reading comprehension tests were examined: the WIAT-III, the CAAT, and the Nelson–Denny. The WIAT-III is composed of open-ended questions, while the other two tests utilize multiple-choice questions. All participants were instructed to answer the questions to the best of their ability, without access to the related passage. The results revealed that participants correctly answered the questions at a significantly higher rate than by chance for the multiple-choice, which supports the independency issue. For the open-ended questions, participants still answered with 18% accuracy, without the passages.

Keywords

Reading comprehension Validity Passage independence Standardized tests 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This research was supported by a Canada Foundation for Innovation infrastructure grant and a NSERC Discovery grant to Annie Roy-Charland. We thank Caroline Comeau for her assistance in running participants and data coding.

References

  1. Aiken, L. R., & Groth-Marnat, G. (2006). Psychological testing and assessment (12th ed.). New York: Pearson.Google Scholar
  2. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). New York: Pearson.Google Scholar
  3. Canadian Test Center. (1992). Canadian adult achievement test, technical bulletin. Markham: Canadian Test Center Inc.Google Scholar
  4. Coleman, C., Lindstrom, J., Nelson, J., Lindstrom, W., & Gregg, K. N. (2010). Passageless comprehension on the Nelson–Denny Reading Test: Well above chance for university students. Journal of Learning Disabilities, 43(3), 244–249.CrossRefGoogle Scholar
  5. Creaser, J., Jacobs, M., Zaccaria, L., & Carsello, C. (1970). Effects of shortened time limits on the Nelson–Denny Reading Test. Journal of Reading, 14(3), 167–170.Google Scholar
  6. Cummins, R., & Porter, (1981). Test review: The Nelson–Denny Reading Test (forms E and F). Journal of Reading, 25(1), 54–59.Google Scholar
  7. Cummins, J., & Swain, M. (2014). Bilingualism in education: Aspects of theory, research and practice. Routledge.Google Scholar
  8. Daneman, M., & Merikle, P. M. (1996). Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin & Review, 3(4), 422–433. doi: 10.3758/BF03214546.CrossRefGoogle Scholar
  9. Eurich, A. C. (1931). Four types of examinations compared and evaluated. Journal of Educational Psychology, 22(4), 268–278. doi: 10.1037/h0075460.CrossRefGoogle Scholar
  10. Flanagan, D. P., & Harrison, P. L. (Eds.). (2012). Contemporary intellectual assessment: Theories, tests, and issues. New York: Guilford Press.Google Scholar
  11. Fowler, B., & Kroll, B. M. (1978). Verbal skills as factors in the passageless validation of reading comprehension tests. Perceptual and Motor Skills, 47, 335–338.CrossRefGoogle Scholar
  12. Heilbrun, K. (1997). Prediction versus management models relevant to risk assessment: The importance of legal decision-making context. Law and Human Behavior, 21(4), 347–359. doi: 10.1023/A:1024851017947.CrossRefGoogle Scholar
  13. Katz, S., Lautenschlager, G. J., Blakckburn, A. B., & Harris, F. H. (1990). Answering reading comprehension items without passages on the SAT. Psychological Science, 1(2), 122–127. doi: 10.1111/j.1467-9280.1990.tb00080.x.CrossRefGoogle Scholar
  14. Keenan, J. M., & Betjemann, R. S. (2006). Comprehending the Gray Oral Reading Test without reading it: Why comprehension tests should not include passage-independent items. Scientific Studies of Reading, 10(4), 363–380. doi: 10.1207/s1532799xssr1004_2.CrossRefGoogle Scholar
  15. Larcker, D. F., & Lessig, V. P. (1980). Perceived usefulness of information: A psychometric examination. Decision Sciences, 11(1), 121–134. doi: 10.1111/j.1540-5915.1980.tb01130.x.CrossRefGoogle Scholar
  16. Lewandowski, L. J., Codding, R. S., Kleinmann, A. E., & Tucker, K. L. (2003). Assessment of reading rate in postsecondary students. Journal of Psychoeducational Assessment, 21(2), 134–144. doi: 10.1177/073428290302100202.CrossRefGoogle Scholar
  17. Lifson, S., Scruggs, T. E., & Bennion, K. (1984). Passage independence in reading achievement tests: A follow-up. Perceptual and Motor Skills, 58(3), 945–946. doi: 10.2466/pms.1984.58.3.945.CrossRefGoogle Scholar
  18. Lim, Y. K., Pangam, A., Periyasami, S., & Aneja, S. (2006, October). Comparative analysis of high-and low-fidelity prototypes for more valid usability evaluations of mobile devices. In Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (pp. 291-300). ACM. doi:  10.1145/1182475.1182506
  19. McClelland, D. C. (1977). Testing for competence rather than intelligence. American Psychologist, 28(1), 1–14. doi: 10.1037/h0034092.CrossRefGoogle Scholar
  20. Messik, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256. doi: 10.1177/026553229601300302.CrossRefGoogle Scholar
  21. Neukrug, E. S., & Fawcett, R. C. (2006). Essentials of testing and assessment: A practical guide for counselors, social workers, therapists, and others. Belmont: Brooks/Cole.Google Scholar
  22. Neukrug, E. S., & Fawcett, R. C. (2010). Essentials of testing and assessment: A practical guide for counselors, social workers, and psychologists (2nd ed.). Belmont, CA: Brooks/Cole.Google Scholar
  23. Preston, R. C. (1964). Ability of students to identify correct responses before reading. The Journal of Educational Research, 58(4), 181–183. doi: 10.1080/00220671.1964.10883203.CrossRefGoogle Scholar
  24. Rothney, J. W., & Bear, R. M. (1938). An evaluation of visual factors in reading. Hanover: Dartmouth College.Google Scholar
  25. Sparfeld, J. R., Kimmel, R., Löwenkamp, L., Steingräber, A., & Rost, D. H. (2012). Not read, but nevertheless solved? Three experiments on PIRLS multiple choice reading comprehension test items. Educational Assessment, 17(4), 214–232. doi: 10.1080/10627197.2012.735921.CrossRefGoogle Scholar
  26. Tamm, L., Epstein, J. N., Denton, C. A., Vaughn, A. J., Peugh, J., & Willcutt, E. G. (2014). Reaction time variability associated with reading skills in poor readers with ADHD. Journal of the International Neuropsychological Society, 20(3), 292–301. doi: 10.1017/S1355617713001495.CrossRefGoogle Scholar
  27. Tian, S. (2006). Passage dependency of reading comprehension items in the GEPT and the TOEFL. The Reading Matrix, 6, 66–84.Google Scholar
  28. Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). The Stanford-Binet intelligence scale: Guide for administering and scoring. Riverside Pub. Co.Google Scholar
  29. Viswesvaran, C., & Ones, D. S. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48(4), 865–885. doi: 10.1111/j.1744-6570.1995.tb01784.x.CrossRefGoogle Scholar
  30. Wechsler, D. (2009). Weschsler individual achievement test 3 rd edition: Manual. San Antonio: Psychological Corporporation.Google Scholar
  31. Willms, J. D., 1992 (2004). Monitoring school performance: A guide for educators. Bristol, PA: Taylor and Francis Inc.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  • Annie Roy-Charland
    • 1
  • Gabrielle Colangelo
    • 1
  • Victoria Foglia
    • 1
  • Leïla Reguigui
    • 1
  1. 1.Department of PsychologyLaurentian UniversitySudburyCanada

Personalised recommendations

Cite article

Buy options