Advertisement

Manual and Automatic Estimates of Growth and Gain Across Year Levels: How Close is Close?

  • Petra Lietz
  • Dieter Kotte
Part of the Education in the Asia-Pacific Region: Issues, Concerns and Prospects book series (EDAP, volume 4)

Abstract

Users of statistical software are frequently unaware of the calculations underlying the routines that they use. Indeed, users—particularly in the social sciences—are often somewhat adverse towards the underlying mathematics, Yet, in order to appreciate the thrust of certain routines, it is beneficial to understand the way in which a program arrives at a particular solution. Based on data from the Economic Literacy Study conducted at year 11 and 12 level across Queensland in 1998, this article renders explicit the steps involved in calculating growth and gain estimates in student performance. To this end, the first part of the article describes the Omanual calculation of such estimates using the Rasch estimates of item thresholds of common items at the different year levels produced by Quest (Adams & Khoo, 1993) as a starting point for the subsequent calibrating, scoring and equating. In the second part of the chapter, we explore the extent to which estimates of change in performance across year levels that are calculated with ConQuest (Wu, Adams & Wilson, 1997).

The article shows that the manual and automatic way of calculating growth and gain estimates produce nearly identical results. This is not only reassuring from a technical point of view but also from an educational point of view as this means that the reader of the non-mathematical discussion of the manual calculation procedure will develop a better understanding of the processes involved in calculating growth and gain estimates.

Key words

unidimensional latent regression gain calibrate scoring equating across year levels test performance economic literacy 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

5. References

  1. Adams RJ & Khoo SK 1993 Quest — The interactive test analysis system. Hawthorn, Vic.: Australian Council for Educational Research.Google Scholar
  2. Adams RJ, Wilson M & Wu M 1997 Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22(1), pp. 47–76.Google Scholar
  3. Australian Bureau of Statistics (ABS) 1999 Census Update. http://www.abs.gov.au/websitedbs/D31 10129.NSF.Google Scholar
  4. Beck K & Krumm V 1989 Economic literacy in German speaking countries and the United States. First steps to a comparative study. Paper presented at the annual meeting of AERA, San Francisco.Google Scholar
  5. Elley WB 1992 How in the world do students read? The Hague: IEA.Google Scholar
  6. Harmon M, Smith TA, Martin MO, Kelly DL, Beaton AE, Mullis IVS, Gonzalez EJ & Orpwood G 1997 Performance Assessment in IEA’s Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston College & Amsterdam: IEA.Google Scholar
  7. Keeves JP & Kotte D 1996 The measurement and reporting of key competencies. The Flinders University of South Australia, Adelaide.Google Scholar
  8. Keeves JP 1992 Learning Science in a Changing World. Cross-national studies of Science Achievement: 1970 to 1984. The Hague: IEA.Google Scholar
  9. Keeves JP 1996 The world of school learning. Selected key findings from 35 years of IEA research. The Hague: IEA.Google Scholar
  10. Kotte D & Lietz P 1998 Welche Faktoren beeinflussen die Leistung in Wirtschaftskunde? Zeitschrift für Berufs-und Wirtschaftspädagogik, Vol. 94, No. X, pp. 421–434.Google Scholar
  11. Lietz P & Kotte D 1997 Economic literacy in Central Queensland: Results of a pilot study. Paper presented at the Australian Association for Research in Education (AARE) annual meeting, Brisbane, 1–4 December, 1997.Google Scholar
  12. Lietz P 1996 Reading comprehension across cultures and over time. Münster/New York: Waxmann.Google Scholar
  13. Loehlin JC 1998 Latent variable models (3rd Ed.). Mahwah, NJ: Erlbaum.Google Scholar
  14. Lokan J, Ford P & Greenwood L 1996 Maths & Science on the Line: Australian junior secondary students’ performance in the Third International Mathematics and Science Study. Melbourne: Australian Council for Educational Research.Google Scholar
  15. Lokan J, Ford P & Greenwood L 1997 Maths & Science on the Line: Australian middle primary students’ performance in the Third International Mathematics and Science Study. Melbourne: Australian Council for Educational Research.Google Scholar
  16. Martin MO & Kelly DA (eds) 1996 Third International Mathematics and Science Study Technical Report, Volume II: Design and Development. Primary and Middle School Years. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston College & Amsterdam: International Association for the Evaluation of Educational Achievement (IEA).Google Scholar
  17. OECD 1998 The PISA Assessment Framework — An Overview. September 1998: Draft of the PISA Project Consortium. Paris: OECD.Google Scholar
  18. Postlethwaite TN & Ross KN 1992 Effective schools in reading. Implications for educational planners. The Hague: IEA.Google Scholar
  19. Postlethwaite TN & Wiley DE 1991 Science Achievement in Twenty-Three Countries. Oxford: Pergamon Press.Google Scholar
  20. Shen R & Shen TY 1993 Economic thinking in China: Economic knowledge and attitudes of high school students. Journal of Economic Education, Vol. 24, pp. 70–84.Google Scholar
  21. Soper JC & Walstad WB 1987 Test of economic literacy. Examiner’s manual 2nd ed. New York: Joint Council on Economic Education (now the National Council on Economic Education).Google Scholar
  22. Walstad WB & Robson D 1997 Differential item functioning and male-female differences on multiple-choice tests in economics. Journal of Economic Education, Spring, pp. 155–171.Google Scholar
  23. Walstad WB & Soper JC 1987 A report card on the economic literacy of U.S. High school students. American Economic Review, Vol. 78, pp. 251–256.Google Scholar
  24. Wang W, Wilson M & Adams RJ 1997 Rasch models for multidimensionality between and within items. In: Wilson M, Engelhard G & Draney K (eds), Objective measurement IV: Theory into practice. Norwood, NJ: Ablex.Google Scholar
  25. Whitehead DJ & Halil T 1991 Economic literacy in the United Kingdom and the United States: A comparative study. Journal of Economic Education, Spring, pp. 101–110.Google Scholar
  26. Wu M, Adams RJ & Wilson MR 1996 ConQuest: Generalised Item Response Modelling Software. Draft Version 1. Camberwell: ACER.Google Scholar
  27. Wu M, Adams RJ & Wilson MR 1997 ConQuest: Generalised Item Response Modelling Software. Camberwell: ACER.Google Scholar

Copyright information

© Springer 2005

Authors and Affiliations

  • Petra Lietz
    • 1
  • Dieter Kotte
    • 2
  1. 1.International UniversityBremenGermany
  2. 2.Causal ImpactGermany

Personalised recommendations