Advertisement

Quality and Quantity

, Volume 34, Issue 4, pp 353–366 | Cite as

Reliability of Assessment of Reading Ability in Three Countries

  • Dougal Hutchison
  • Lesley Kendall
  • David Bartholomew
  • Martin Knott
  • Jane Galbraith
  • Maria Piccoli
Article

Abstract

Over the last few decades, there have been a number ofnational and international programmes to assessperformance in key subjects. For example theInternational Association for Evaluation ofEducational Attainment (IEA), International Assessmentof Educational Progress (IAEP), and the ThirdInternational Mathematics and Science Survey (TIMSS)programs have aimed to compare performance betweenparticipating countries.

Such exercises have different requirements from thosetests, such as public examinations, which aim toassess the attainment of individuals. National andinternational assessments aim to cover a very widerange of materials, and often use the technique ofmultiple matrix sampling to do so.

The investigation is based on generalizability-typeanalyses of three national data sets at two ages, andlooks not only at the variance components arising fromsampling of schools, and of pupils within them, butalso at the variation between different assessmentinstruments, and between items within assessmentinstruments. If interpreted with care, such resultscan be of value in the design of future studies. Thispaper concentrates largely on the precision ofestimates of overall means, but analyses of the typedescribed could also be used to compare theperformance of subpopulations.

reliability generalizability international comparisons 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. AAP (1993). Mathematics 1991. Assessment of Achievement Programme. Edinburgh: HMSO.Google Scholar
  2. Foxman, Hutchison & Bloomfield (1991). The APU Experience. London: HMSO.Google Scholar
  3. Cronbach, L. J, Gleser, G C, Nanda, H. & Rajaratnam, N. (1972). The Dependability of Behavioural Measurements. New York: Wiley.Google Scholar
  4. Johnson, S. (1989). National Assessment: The APU Science Approach. London: HMSO.Google Scholar
  5. Keys, W., Harris, S. & Fernandes, C. (1996). Third International Mathematics and Science Study: First National Report, Part 1, Achievement in Mathematics and Science at Age 13 in England. Slough: NFER.Google Scholar
  6. Keys, W., Harris, S. & Fernandes, C. (1997). Third International Mathematics and Science Study: Second National Report, Part 1, Achievement in Mathematics and Science at Age 9 in England. Slough: NFER.Google Scholar
  7. Knott, M., Piccoli, M., Galbraith, J., Bartholomew, D., Hutchison, D. & Kendall, L. (1996). Latent variable analysis with missing data in educational research: Long report. Final Report to ESRC. Mimeo: Department of Statistics, London School of Economics.Google Scholar
  8. Mullis, I. V. S., Dossey, J. A., Campbell, C. A., Sullivan, C. O. & Latham, A. S. (1994). Report in brief: NAEP 1992 trends in academic progress. Princeton, N.J.: Educational Testing Service.Google Scholar
  9. Murray, T. S., Kirsch, I. S. & Jenkins, L. B. (1998). Adult literacy in OECD countries: Technical report on the first international adult literacy survey. Washington: US Department of Education, Office of Educational Research and Improvement.Google Scholar
  10. Rasbash, J & Woodhouse, G (1995). MLn Command Reference: Version 1.0, March 1995. Multilevel Models Project, Institute of Education, University of LondonGoogle Scholar
  11. SAIP (1994). Reading and Writing Assessment 1994. Canadian Council of Ministers of Education.Google Scholar
  12. SAS Institute, Inc. (1989). SAS/STAT ® User's Guide, Version 6, 4th edn, Vol. 2. Cary, NC, USA: SAS Institute Inc.Google Scholar
  13. Schagen, I. & Hutchison, D. (1994). Measuring the reliability of national curriculum assessment. Educational Research 3(3): 211–222.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Dougal Hutchison
    • 1
  • Lesley Kendall
    • 2
  • David Bartholomew
    • 2
  • Martin Knott
    • 2
  • Jane Galbraith
    • 2
  • Maria Piccoli
    • 2
  1. 1.National Foundation for Educational Research in England and Wales, The MereSloughU.K.
  2. 2.National Foundation for Educational Research in England and Wales, The MereSloughU.K.

Personalised recommendations