Advertisement

Using Large-scale Assessment Datasets for Research in Science and Mathematics Education: Programme for International Student Assessment (PISA)

  • John O. AndersonEmail author
  • Huann-Shyang Lin
  • David F. Treagust
  • Shelley P. Ross
  • Larry D. Yore
Article

Abstract

Large-scale assessments of student achievement provide a window into the broadly defined concepts of literacy and generate information about levels and types of student achievement in relation to some of the correlates of learning, such as student background, attitudes, and perceptions, and perhaps school and home characteristics. This paper provides an overview and outlines potential research opportunities of one such assessment—the Programme for International Student Assessment (PISA). In order to provide examples of the work that can be accomplished with these data, we describe and discuss the results generated from PISA 2000 and PISA 2003 in terms of international comparisons of achievement and the models of relational patterns of student, home, and school characteristics. We provide insight from the recent pilot testing conducted in Taiwan for PISA 2006, which has a focus on scientific literacy. This is followed by a discussion of the implications and potentials of the 2000 and 2003 datasets to facilitate research on scientific and mathematical literacy. The paper concludes with a look ahead to PISA 2006 and what researchers should be attending to in the research reports generated from the OECD and the research interests that they could follow given access to the datasets generated.

Key Words

future opportunities large-scale assessments PISA results secondary analyses 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aldridge, J.M., Fraser, B.J. & Huang, T.I. (1999). Investigating classroom environments in Taiwan and Australia with multiple research methods. Journal of Educational Research, 93, 48–61.CrossRefGoogle Scholar
  2. Aldridge, J.M., Laugksch, R.C., Seopa, M.A. & Fraser, B.J. (2006). Development and validation of an instrument to monitor the implementation of outcomes-based learning environments in science classrooms in South Africa. International Journal Science Education, 28(1), 45–70.CrossRefGoogle Scholar
  3. Anderson, J.O., Rogers, W.T., Klinger, D.A., Ungerleider, C., Glickman, V. & Anderson, B. (2006). Student and school correlates of mathematics achievement: Models of school performance based on Pan-Canadian student assessment. Canadian Journal of Education, 29(3), 706–730.Google Scholar
  4. Central Intelligence Agency. (2003). The world factbook. Retrieved April 2006 from http://www.cia.gov/cia/publications/factbook/index.html.
  5. Chinn, P.W., Hand, B.M., & Yore, L.D. (2007). Culture, language, knowledge about nature and naturally occurring events, and science literacy for all: She says, he says, they say. L1-Educational Studies in Language and Literature (special issue, submitted).Google Scholar
  6. De Bortoli, L. & Cresswell, J. (2004). Australia’s indigenous students in PISA 2000: Results from an international study. ACER Research Monograph No 59. Camberwell, Victoria: Australian Council for Educational Research.Google Scholar
  7. DeBoer, G.E. (2000). Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science education reform. Journal of Research in Science Teaching, 37(6), 582–601.CrossRefGoogle Scholar
  8. Doig, B. (2001). Summing up: Australian numeracy performances, practices, programs and possibilities. Camberwell, Victoria: Australian Council on Educational Research.Google Scholar
  9. Donelly, K. (2005). PISA and TIMSS are like apples and oranges. ON-LINE opinion - Australia’s e-journal of social and political debate (posted 1 February 2005) http://www.onlineopinion.com.au/view.asp?article=2985.
  10. Ferrari, J. (2006). National syllabus ‘ a power grab’, The Weekend Australian, October 7–8, p. 4.Google Scholar
  11. Fertig, M. (2003). Who’s to blame? The determinants of German students’ achievement in the PISA 2000 study. Discussion paper 739, The Institute for the Study of Labor (IZA), Bonn, Germany. http://ftp.iza.org/dp739.pdf. Retrieved April 2007.
  12. Fraser, B.J. (1998). Science learning environments: Assessment, efforts and determinants. In B.J. Fraser & K.G. Tobin (Eds.), The international handbook of science education (pp. 527–564). Dordrecht, The Netherlands: Kluwer.Google Scholar
  13. Fraser, B.J. (2002). Learning environment research: Yesterday, today and tomorrow. In S.C. Goh & M.S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 1–27). Singapore: World Scientific.Google Scholar
  14. Fraser, B.J. & McRobbie, C.J. (1995). Science laboratory classroom environments at schools and universities: A cross-national study. Educational Research and Evaluation, 1(4), 289–317.CrossRefGoogle Scholar
  15. Fraser, B.J., Giddings, G.J. & McRobbie, C.J. (1995). Evolution and validation of a personal form of an instrument for assessing science laboratory classroom environments. Journal of Research in Science Teaching, 32, 399–422.CrossRefGoogle Scholar
  16. Friedler, Y. & Tamir, P.J. (1986). Teaching basic concepts of scientific research to high school students. Journal of Biological Education, 20(4), 263–269.Google Scholar
  17. Gee, J.P. (2004). Language in the science classroom: Academic social languages as the heart of school-based literacy. In E.W. Saul (Ed.), Crossing borders in literacy and science instruction: Perspectives on theory and practices (pp. 13–32). Newark, DE: International Reading Association.Google Scholar
  18. Goh, M. (2006). A multilevel analysis of mathematics literacy: The effects of intrinsic motivation, teacher support and student-teacher relations. Master’s Thesis, University of Victoria, Victoria, British Columbia, Canada.Google Scholar
  19. Goh, S.C., & Khine, S.M. (Eds.). (2002). Studies in educational learning environments: An international perspective. Singapore: World Scientific.Google Scholar
  20. Goldstein, H. (2004). International comparisons of student attainment: Some issues arising from the PISA study. Assessment in Education, 11(3), 319–330.CrossRefGoogle Scholar
  21. Gu, Z. (2006). A comparison of Canada and Hong Kong-China through hierarchical linear models: The relations among students’ self-beliefs in math, the learning environment at school, and math performance. Master’s Thesis, University of Victoria, Victoria, British Columbia, Canada.Google Scholar
  22. Hambleton, R.H. & Swaminathan, H. (1985). Item response theory: Principles and applications. Norwell, MA: Kluwer.Google Scholar
  23. Ho, E.S. & Willms, J.D. (1996). Effects of parental involvement on eighth-grade achievement. Sociology of Education, 69, 126–141.CrossRefGoogle Scholar
  24. LaFontaine, D. & Monseur, C. (2006). Impact of test characteristics on gender equity indicators in the assessment of reading comprehension. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, California, USA.Google Scholar
  25. Laugksch, R.C. (2000). Scientific literacy: A conceptual overview. Science Education, 84(1), 71–94.CrossRefGoogle Scholar
  26. Lokan, J., Greenwood, L. & Cresswell, J. (2001). 15-Up and counting, reading, writing, reasoning: How literate are Australian students? The PISA 2000 survey of students’ reading, mathematics and scientific literacy skills. Report for the OECD. Camberwell, Victoria: Australian Council for Educational Research.Google Scholar
  27. National Research Council. (1996). National science education standards. Washington, DC: Author.Google Scholar
  28. Norris, S.P. & Phillips, L.M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87(2), 224–240.CrossRefGoogle Scholar
  29. Organisation for Economic Co-operation and Development. (2001). Knowledge and skills for life: First report from the OECD programme for international student assessment. Paris: Author.Google Scholar
  30. Organisation for Economic Co-operation and Development. (2002). PISA 2000 technical report. Paris: Author.Google Scholar
  31. Organisation for Economic Co-operation and Development. (2003). The PISA 2003 assessment framework - mathematics, reading, science and problem solving: Knowledge and skills. Paris: Author.Google Scholar
  32. Organisation for Economic Co-operation and Development. (2004). Messages from PISA 2000. Paris: Author.Google Scholar
  33. Organisation for Economic Co-operation and Development. (2005a). Learning for tomorrow’s world: First results from PISA 2003. Paris: Author.Google Scholar
  34. Organisation for Economic Co-operation and Development. (2005b). PISA 2003 data analysis manual - SAS users. Paris: Author.Google Scholar
  35. Organisation for Economic Co-operation and Development. (2006a). Where immigrant students succeed - A comparative review of performance and engagement in PISA 2003. Paris: Author.Google Scholar
  36. Organisation for Economic Co-operation and Development. (2006b). School factors related to quality and equity: Results from PISA 2003. Paris: Author.Google Scholar
  37. Pimm, D., Tuan, H-L. & Yore, L.D. (2007). Language, mathematics literacy, and science literacy: Seeking convergence and cognitive symbiosis. International Journal of Science and Mathematics Education (special issue, in press).Google Scholar
  38. Taylor, P.C., Fraser, B.J. & Fisher, D.L. (1997). Monitoring constructivist classroom learning environments. International Journal of Educational Research, 27, 293–302.CrossRefGoogle Scholar
  39. Walberg, H.J. (Ed.), (1979). Educational environments and effects: Evaluation, policy, and productivity. Berkeley, CA: McCutchan.Google Scholar
  40. Wubbels, T. & Levy, J. (Eds.) (1993). Do you know what you look like?: Interpersonal relationships in education. London: Falmer Press.Google Scholar
  41. Yore, L.D. (2007). Science literacy for all students: Language, culture, and knowledge about nature and naturally occurring events. L1-Educational Studies in Language and Literature (special issue, submitted).Google Scholar
  42. Yore, L.D. & Treagust, D. (2006). Current realities and future possibilities: Language and science literacy-empowering research and informing instruction. International Journal of Science Education, 28(2–3), 291–314.CrossRefGoogle Scholar
  43. Zubrick, S.R., Silburn, S.R., Gurrin, L., Teoh, H., Shepherd, C., Carlton, J. & Lawrence, D. (1997). Western Australian child health survey: Education, health and competence. Perth, Western Australia: Australian Bureau of Statistics and the TVW Telethon Institute of Child Health Research.Google Scholar

Copyright information

© National Science Council, Taiwan 2007

Authors and Affiliations

  • John O. Anderson
    • 1
    Email author
  • Huann-Shyang Lin
    • 1
  • David F. Treagust
    • 1
  • Shelley P. Ross
    • 1
  • Larry D. Yore
    • 1
  1. 1.Department of Educational Psychology & Leadership Studies, Faculty of EducationUniversity of VictoriaVictoriaCanada

Personalised recommendations