Advertisement

Validating the Use of Translated and Adapted HEIghten® Quantitative Literacy Test in Russia

  • Lin GuEmail author
  • Ou Lydia Liu
  • Jun Xu
  • Elena Kardonova
  • Igor Chirikov
  • Guirong Li
  • Shangfeng Hu
  • Ningning Yu
  • Liping Ma
  • Fei Guo
  • Qi Su
  • Jinghuan Shi
  • Henry Shi
  • Prashant Loyalka
Chapter
Part of the Methodology of Educational Measurement and Assessment book series (MEMA)

Abstract

In responding to the need for internationally comparable data on higher education student learning outcomes, some modules of the HEIghten® Outcomes Assessment Suite, developed by Educational Testing Service, have been translated and adapted for international use. This recent development points to a critical need to validate the use of translated and adapted HEIghten assessments in international contexts. This chapter reports on validating the use of the Russian HEIghten Quantitative Literacy (QL) assessment with a representative group of students majoring in electrical engineering and computer science from 34 higher education institutions in Russia. Our findings provided preliminary evidence in support of the use of the assessment for the target population as a measure of QL. Future research is suggested to further investigate the test’s ability of reflecting changes in the target construct as a function of learning in the context of Russia.

Notes

Acknowledgments

Support from the Basic Research Program of the National Research University Higher School of Economics is gratefully acknowledged.

References

  1. American College Testing. (2012). ACT CAAP technical handbook 2011–2012. Iowa City: CAAP Program Management. Retrieved from http://www.act.org/content/dam/act/unsecured/documents/CAAP-TechnicalHandbook.pdf
  2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.Google Scholar
  3. Bennell, P., & Pierce, T. (2003). The internationalisation of tertiary education: Exporting education to developing and transitional economies. International Journal of Educational Development, 23, 215–232.CrossRefGoogle Scholar
  4. Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (2000). Intellectual ability, learning style, personality, achievement motivation and academic success of psychology students in higher education. Personality and Individual Differences, 29, 1057–1068.CrossRefGoogle Scholar
  5. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: L. Erlbaum Associates.Google Scholar
  6. Council for Aid to Education. (2015). CLA+ technical FAQs. New York: Author. Retrieved from http://cae.org/images/uploads/pdf/CLA_Plus_Technical_FAQs.pdf
  7. Educational Testing Service. (2010). ETS proficiency profile user’s guide. Princeton: Educational Testing Service. Retrieved from https://www.ets.org/s/proficiencyprofile/pdf/Users_Guide.pdf
  8. Finney, S. J., & DiStefano, C. (2013). Nonnormal and categorical data in structural equation models. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (2nd ed., pp. 439–492). Charlotte: Information Age.Google Scholar
  9. Haberman, S. J. (2008). When can subscores have value? Journal of Educational and Behavioral Statistics, 33, 204–229.CrossRefGoogle Scholar
  10. International Test Commission. (2005). International Test Commission guidelines for translating and adapting tests. Retrieved from http://www.intestcom.org/files/guideline_test_adaptation.pdf
  11. Klein, S., Benjamin, R., Shavelson, R. J., & Bolus, R. (2007). The collegiate learning assessment: Facts and fantasies. Evaluation Review, 31, 415–439.CrossRefGoogle Scholar
  12. Klein, S., Liu, O. L., Sconing, J. A., Bolus, R., Bridgeman, B., Kugelmass, H., & Steedle, J. (2009). Test Validity Study (TVS) Report (ETS technical report). Supported by the Fund for Improvement of postsecondary education (FIPSE). Retrieved from https://cp-files.s3.amazonaws.com/26/TVSReport_Final.pdf
  13. Knight, J. (2003). Updated definition of internationalization. International Higher Education, 33, 2–3.Google Scholar
  14. Lizzio, A., Wilson, K., & Simons, R. (2002). University students’ perceptions of the learning environment and academic outcomes: Implications for theory and practice. Studies in Higher Education, 27, 27–52.CrossRefGoogle Scholar
  15. Muthén, L. K., & Muthén, B. O. (2010). Mplus user’s guide (6th ed.). Los Angeles: Authors.Google Scholar
  16. Organization for Economic Co-operation and Development. (2008). Tertiary education for the knowledge society. Paris: OECD. Retrieved from https://oecd.org/edu/tertiary/review.Google Scholar
  17. Organization for Economic Co-operation and Development. (2012). PISA 2012 technical report. Paris: OECD. Retrieved from https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf.Google Scholar
  18. Roohr, K. C., Graf, E. A., & Liu, O. L. (2014). Assessing quantitative literacy in higher education: An overview of existing research and assessments with recommendations for next-generation assessment (ETS RR-14-22). Princeton: Educational Testing Service.Google Scholar
  19. Roohr, K. C., Lee, H., Xu, J., Liu, O. L., & Wang, Z. (2017). A preliminary evaluation of the psychometric quality of HEIghten™ quantitative literacy. Numeracy, 10(2), 3.CrossRefGoogle Scholar
  20. Sawaki, Y., Stricker, L. J., & Oranje, A. H. (2009). Factor structure of the TOEFL internet-based test. Language Testing, 26, 5–30.CrossRefGoogle Scholar
  21. Tremblay, K., Lalancette, D., & Roseveare, D. (2012). Assessment of higher education learning outcomes (AHELO) feasibility study report: Design and implementation (Vol. 1). Paris: Organization for Economic Co-operation and Development.Google Scholar
  22. Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22, 251–266.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Lin Gu
    • 1
    Email author
  • Ou Lydia Liu
    • 1
  • Jun Xu
    • 1
  • Elena Kardonova
    • 2
  • Igor Chirikov
    • 2
  • Guirong Li
    • 3
  • Shangfeng Hu
    • 4
  • Ningning Yu
    • 5
  • Liping Ma
    • 6
  • Fei Guo
    • 7
  • Qi Su
    • 8
  • Jinghuan Shi
    • 7
  • Henry Shi
    • 9
  • Prashant Loyalka
    • 9
  1. 1.Educational Testing ServicePrincetonUSA
  2. 2.National Research University Higher School of EconomicsMoscowRussia
  3. 3.Henan UniversityKaifengChina
  4. 4.Sichuan Normal UniversityChengduChina
  5. 5.Shandong Jinan UniversityJinanChina
  6. 6.Peking UniversityBeijingChina
  7. 7.Tsinghua UniversityBeijingChina
  8. 8.Shaanxi Normal UniversityXi’anChina
  9. 9.Stanford UniversityStanfordUSA

Personalised recommendations