Skip to main content
  • Book
  • Open Access
  • © 2020

Reliability and Validity of International Large-Scale Assessment

Understanding IEA’s Comparative Studies of Student Achievement


  • This open access book describes the evolution of the methodologies used in large-scale assessments in education

  • Examines the challenges posed in designing international large-scale assessments

  • Addresses issues related to the reliability and validity of the international comparisons conducted by IEA

  • Provides a guide to understanding and interpreting international large-scale assessments

  • Assesses the broad availability of international large-scale assessment data and associated analytical software tools

Part of the book series: IEA Research for Education (IEAR, volume 10)

Buying options

Softcover Book USD 59.99
Price excludes VAT (USA)
Hardcover Book USD 59.99
Price excludes VAT (USA)

Table of contents (15 chapters)

  1. Front Matter

    Pages i-xi
  2. Framework Development in International Large-Scale Assessment Studies

    • John Ainley, Wolfram Schulz
    Pages 23-36Open Access
  3. Assessment Content Development

    • Liz Twist, Julian Fraillon
    Pages 37-59Open Access
  4. Questionnaire Development in International Large-Scale Assessment Studies

    • Wolfram Schulz, Ralph Carstens
    Pages 61-83Open Access
  5. Translation: The Preparation of National Language Versions of Assessment Instruments

    • Paulína Koršňáková, Steve Dept, David Ebbs
    Pages 85-111Open Access
  6. Sampling, Weighting, and Variance Estimation

    • Sabine Meinck
    Pages 113-129Open Access
  7. Quality Control During Data Collection: Refining for Rigor

    • Lauren Musu, Sandra Dohr, Andrea Netten
    Pages 131-150Open Access
  8. Post-collection Data Capture, Scoring, and Processing

    • Alena Becker
    Pages 151-167Open Access
  9. Technology and Assessment

    • Heiko Sibberns
    Pages 169-186Open Access
  10. Ensuring Validity in International Comparisons Using State-of-the-Art Psychometric Methodologies

    • Matthias Von Davier, Eugenio Gonzalez, Wolfram Schulz
    Pages 187-219Open Access
  11. Publications and Dissemination

    • Seamus Hegarty, Sive Finlay
    Pages 221-230Open Access
  12. Consequential Validity: Data Access, Data Use, Analytical Support, and Training

    • Sabine Meinck, Eugenio Gonzalez, Hans Wagemaker
    Pages 231-244Open Access
  13. Using IEA Studies to Inform Policymaking and Program Development: The Case of Singapore

    • Hui Leng Ng, Chew Leng Poon, Elizabeth Pang
    Pages 245-259Open Access
  14. Understanding the Policy Influence of International Large-Scale Assessments in Education

    • David Rutkowski, Greg Thompson, Leslie Rutkowski
    Pages 261-277Open Access

About this book

This open access book describes and reviews the development of the quality control mechanisms and methodologies associated with IEA’s extensive program of educational research. A group of renowned international researchers, directly involved in the design and execution of IEA’s international large-scale assessments (ILSAs), describe the operational and quality control procedures that are employed to address the challenges associated with providing high-quality, comparable data. 

Throughout the now considerable history of IEA’s international large-scale assessments, establishing the quality of the data has been paramount. Research in the complex multinational context in which IEA studies operate imposes significant burdens and challenges in terms of the methodologies and technologies that have been developed to achieve the stated study goals. The demands of the twin imperatives of validity and reliability must be satisfied in the context of multiple and diverse cultures, languages, orthographies, educational structures, educational histories, and traditions. 

Readers will learn about IEA’s approach to such challenges, and the methods used to ensure that the quality of the data provided to policymakers and researchers can be trusted. An often neglected area of investigation, namely the consequential validity of ILSAs, is also explored, examining issues related to reporting, dissemination, and impact, including discussion of the limits of interpretation. 

The final chapters address the question of the influence of ILSAs on policy and reform in education, including a case study from Singapore, a country known for its outstanding levels of achievement, but which nevertheless seeks the means of continual improvement, illustrating best practice use of ILSA data.


  • Open Access
  • International large-scale assessments
  • Educational policy
  • IEA
  • Comparative education
  • Educational measurement
  • International education
  • Research methods in international large-scale assessment
  • Comparative Studies of Student Achievement
  • ILSA
  • IEA’s approach to international large-scale assessments

Editors and Affiliations

  • Newlands, Wellington, New Zealand

    Hans Wagemaker

About the editor

Dr Hans Wagemaker was the executive director of the International Association for the Evaluation of Educational Achievement for 17 years, responsible for the management of all IEA international research and assessment projects and activities. He helped develop IEA’s Progress in International Reading Literacy Study (PIRLS) and oversaw the development and expansion of IEA’s training and capacity building activities in low to middle income countries, and IEA’s educational consultancy services. Together with Educational Testing Services (ETS), he established the IEA Research Institute (IERI), where he continues to serve as a Board member.
Dr Wagemaker was a Senior Manager Research and International with the Ministry of Education, New Zealand, and represented New Zealand’s interests in the APEC Education Forum, UNESCO’s commissions, and the OECD, CERI, and the Education Governing Board. He has consulted for the Inter American Development Bank and UNESCO and worked extensively with the World Bank to advance a common interest in the uses of assessment for improving educational systems in developing countries. Most recently Dr Wagemaker served as an advisor to the Minister of Education for the Sultanate of Oman. He is also a member of the Advisory Board for the Center for Education Statistics and Evaluation (CESE) for the government of New South Wales, Australia, the H Institute, Beirut, Lebanon, and continues in an advisory role with the IEA.
Dr Wagemaker holds BA and MA degrees from the University of Otago, New Zealand, and a PhD from the University of Illinois, where he was awarded a University Fellowship and, in 2009, the College of Education’s Distinguished Alumni Award.

Bibliographic Information

  • Book Title: Reliability and Validity of International Large-Scale Assessment

  • Book Subtitle: Understanding IEA’s Comparative Studies of Student Achievement

  • Editors: Hans Wagemaker

  • Series Title: IEA Research for Education

  • DOI:

  • Publisher: Springer Cham

  • eBook Packages: Education, Education (R0)

  • Copyright Information: International Association for the Evaluation of Educational Achievement (IEA) 2020

  • License: CC BY-NC

  • Hardcover ISBN: 978-3-030-53080-8Published: 04 September 2020

  • Softcover ISBN: 978-3-030-53083-9Published: 04 September 2021

  • eBook ISBN: 978-3-030-53081-5Published: 03 September 2020

  • Series ISSN: 2366-1631

  • Series E-ISSN: 2366-164X

  • Edition Number: 1

  • Number of Pages: XI, 277

  • Number of Illustrations: 9 b/w illustrations, 16 illustrations in colour

  • Topics: Assessment and Testing, Educational Policy and Politics, International and Comparative Education, Curriculum Studies

Buying options

Softcover Book USD 59.99
Price excludes VAT (USA)
Hardcover Book USD 59.99
Price excludes VAT (USA)