Skip to main content
Springer Nature Link
Log in

How Reliable Are Our Assessment Data?: A Comparison of the Reliability of Data Produced in Graded and Un-Graded Conditions

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

Motivating students to perform well on assessment tests is difficult when students know the results have no academic consequence. The present study evaluates the influence of assessment context (graded vs. non-graded) on the reliability of an assessment measure. Results indicate the graded condition produces higher reliability (r= .71) than the non-graded condition (r = .29), which leads to unacceptably low reliability. Moreover, the graded condition produces significantly higher scores (M = 64%), than the non-graded condition (M = 43%). Only students in the graded condition (41%) obtained passing scores of 70% or above.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Finland)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  • Anastasi, A., and Urbina, S. (1997). Psychological Testing (7th Ed.) Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Aronson, E., Brewer, M., and Carlsmith, J. M. (1985). Experimentation in social psychology. In: Lindzey, G. and Aronson E. (eds.), Handbook of Social Psychology (3rd Ed.) New York: Random House.

    Google Scholar 

  • Cohen, J. (1992). A power primer. Psychological Bulletin 112: 155–159.

    Google Scholar 

  • College Entrance Examination Board Educational Testing Service (1990). Coordinator's Guide for Computerized Placement Tests, Version 3.0. Princeton, NJ.

    Google Scholar 

  • Cozby, P. C. (2001). Methods in Behavioral Research, (7th Ed.), Mountain View, CA: ayfield Publishing

    Google Scholar 

  • Duvall, B. (1994). Obtaining student cooperation for assessment. New Directions for Community Colleges 88 (winter): 47–52.

    Google Scholar 

  • Gay, L. R. (1996). Educational Research (5th Ed.) Upper Saddle River, NJ: Prentice-Hall.

    Google Scholar 

  • Mehrens, W. A., and Lehmann, I. J. (1973). Measurement and Evaluation in Education and Psychology. New York: Holt, Rinehart and Winston, Inc.

    Google Scholar 

  • Napoli, A.R., and Wortman, P. (1995). Validating college placement tests. Journal of Applied Research in the Community College 3(2): 143–151.

    Google Scholar 

  • Nichols, J. (1995a). Assessment Case Studies: Common Issues in Implementation with Various Campus Approaches to Resolution. NY: Bronx, Agathon Press.

    Google Scholar 

  • Nichols, J. (1995b). A Practitioner's Handbook for Institutional Effectiveness and Student Outcomes Assessment Implementation (3rd Ed.) Edison, NJ: Agathon Press.

    Google Scholar 

  • Nunnally, J. C., and Bernstein, I. H. (1994). Psychometric Theory (3rd Ed.) New York: McGraw-Hill.

    Google Scholar 

  • Pittenger, D. J. (2003). Behavioral Research: Design and Analysis. New York: McGraw-Hill.

    Google Scholar 

  • Rosnow, R. L., and Rosenthal, R. (2002). Beginning behavioral research: A conceptual primer (4th Ed.) Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Warren, J. (1998). Cognitive measures in assessing learning. New Directions for Institutional Research [No. 59, Implementing Outcomes Assessment: Promise and Perils], 1(3): 29–39.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Office of Institutional Research and Assessment Suffolk County Community College, 533 College Road, Selden, NY, 11784, USA

    Anthony R. Napoli

  2. Office of Institutional Research and Assessment Suffolk County Community College, USA

    Lanette A. Raymond

Authors
  1. Anthony R. Napoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lanette A. Raymond
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Napoli, A.R., Raymond, L.A. How Reliable Are Our Assessment Data?: A Comparison of the Reliability of Data Produced in Graded and Un-Graded Conditions. Research in Higher Education 45, 921–929 (2004). https://doi.org/10.1007/s11162-004-5954-y

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-004-5954-y

Search

Navigation