Gathering Psychometric Evidence for ASCIv2 to Support Cross-Cultural Attitudinal Studies for College Chemistry Programs

  • Xiaoying Xu
  • Khalid Alhooshani
  • Daniel Southam
  • Jennifer E. LewisEmail author


Instruments in the affective domain may not be equivalent when the tests are administered to different populations with different cultural backgrounds. To illustrate a general approach, this study was intended to gather psychometric evidence for an instrument of attitude toward chemistry to support cross-cultural attitudinal studies for college chemistry students. The shortened version of Attitude toward the Subject of Chemistry Inventory, ASCIv2, was used at three universities, one in Saudi Arabia, one in Australia, and one in the USA. Based on the results of psychometric analysis of internal consistency reliability and internal structure validity, we found that students from the Saudi Arabian institution responded to item 6, chemistry is challenging or not, differently from those in Australia and the USA. This study signifies the importance of examining utility and student response in context when instrument data is gathered in cross-cultural scenarios, to ensure that responses in the new context still match the trait underlying the instruments. In addition, this study contributes to the use of ASCIv2 regarding the possible variance and profile for attitude scores from multiple countries.


Saudi Arabia Standardize Root Mean Square Residual Cognitive Interview Attitude Score Major Choice 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Abell, N., Springer, D. W., & Kamata, A. (2009). Developing and validating rapid assessment instruments. New York: Oxford University Press.CrossRefGoogle Scholar
  2. AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC.Google Scholar
  3. Albaz, K. S. (2007). Using modeling approach on the 11th grade students on their performance, scientific reasoning and attitude toward chemistry. Journal of Science Education and Technology, 2(10), 91–120.Google Scholar
  4. Albusylee, A., Sadieg, S., & Abdukader, F. (1990). Attitude of teachers community college student in Saudi Arabia toward chemistry and learning chemistry. Journal of Arabian Gulf, 35(11), 19–52.Google Scholar
  5. Al-Khaldi, M. A., & AlJabri, I. M. (1998). The relationship of attitudes to computer utilization: New evidence from a developing nation. Computers in Human Behavior, 14(1), 23–42. doi: 10.1016/s0747-5632(97)00030-7.CrossRefGoogle Scholar
  6. Allalouf, A. (2003). Revising translated differential item functioning items as a tool for improving cross-lingual assessment. Applied Measurement in Education, 16(1), 55–73. doi: 10.1207/s15324818ame1601_3.CrossRefGoogle Scholar
  7. American Association for the Advancement of Science. (1989). Science for all Americans. Project 2061-The American Association for the Advancement of Science. Oxford University Press.Google Scholar
  8. Arjoon, J. A., Xu, X., & Lewis, J. E. (2013). Understanding the state of the art for measurement in Chemical Education Research: Examining the psychometric evidence. Journal of Chemical Education, 90(5), 536–545. doi: 10.1021/ed3002013.CrossRefGoogle Scholar
  9. Australian Education Network. (2013). Australian University Rankings. Retrieved 3/1/2013, from
  10. Balfagheh, N. M. (2001). The impact of concept map teaching strategy on the UAE 11th grade students on their performance and Attitude toward chemistry. Journal of Science Education and Technology, 3(1), 157–182.Google Scholar
  11. Barbera, J., Adams, W. K., Wieman, C. E., & Perkins, K. K. (2008). Modifying and validating the Colorado Learning Attitudes about Science Survey for use in chemistry. Journal of Chemical Education, 85(10), 1435–1439.CrossRefGoogle Scholar
  12. Bauer, C. (2005). Beyond “student attitudes”: Chemistry self-concept inventory for assessment of the affective component of student learning. Journal of Chemical Education, 82(12), 1864–1870.CrossRefGoogle Scholar
  13. Bauer, C. (2008). Attitude towards chemistry: A semantic differential instrument for assessing curriculum impacts. Journal of Chemical Education, 85(10), 1440–1445.CrossRefGoogle Scholar
  14. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.CrossRefGoogle Scholar
  15. Brandriet, A. R., Xu, X., Bretz, S. L., & Lewis, J. E. (2012). Diagnosing changes in attitude in first-year college chemistry students with a shortened version of Bauer’s semantic differential. Chemistry Education Research and Practice, 12(2), 271–278.CrossRefGoogle Scholar
  16. Carnegie Foundation for the Advancement of Teaching. (2010). The Carnegie Classification of Institutions of Higher Education™. Retrieved 3/1/2013, from
  17. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. doi: 10.1177/001316446002000104.CrossRefGoogle Scholar
  18. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum, 25–27.Google Scholar
  19. Fatallah, M. A. (2009). The impact of concept map teaching strategy on students’ performance, critical thinking and attitude toward collaborative leaning for middle school in Saudi Arabia. Journal of Science Education, 11(3).Google Scholar
  20. Grove, N., & Bretz, S. L. (2007). CHEMX: An instrument to assess students’ cognitive expectations for learning chemistry. Journal of Chemical Education, 84(9), 1524–1929.CrossRefGoogle Scholar
  21. Harty, H., & Alfaleh, N. (1983). Saudi Arabian students chemistry achievement and science attitudes stemming from lecture-demonstration and small-group teaching-methods. Journal of Research in Science Teaching, 20(9), 861–866. doi: 10.1002/tea.3660200908.CrossRefGoogle Scholar
  22. Hijazi, T. (2008). Constructing an attitudinal scale toward chemistry for eleventh and twelfth grades students. Journal of Educational and Psychological Sciences, 9(1), 73–90.Google Scholar
  23. Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Struct equation model: Concepts, issues and applications (pp. 76–99). Thousand Oaks: Sage.Google Scholar
  24. Jiang, B., Xu, X., Garcia, A., & Lewis, J. E. (2010). Comparing two tests of formal reasoning in a college chemistry context. Journal of Chemical Education, 87(12), 1430–1437. doi: 10.1021/ed100222v.CrossRefGoogle Scholar
  25. Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. doi: 10.2307/2529310.CrossRefGoogle Scholar
  26. Martin, M. O., & Mullis, I. V. S., & Foy, P. (with Olson, J. F., Erberber, E., Preuschoff, C., & Galia, J.). (2008). TIMSS 2007 International Science Report: Findings from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.Google Scholar
  27. Miles, M.B., & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage.Google Scholar
  28. Modood, T. (1993). The number of ethnic minority students in British higher education: some grounds for optimism. Oxford Review of Education, 19(2), 167–182.CrossRefGoogle Scholar
  29. Murphy, K. R., & Davidshofer, C. O. (2005). Psychological testing: Principles and testing (6th ed.). Upper Saddle River, NJ: Prentice-Hall.Google Scholar
  30. Muthén, L. K., & Muthén, B. O. (2007). Mplus user’s guide (5th ed.). Los Angeles: Muthén & Muthén.Google Scholar
  31. Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25(9), 1049–1079.CrossRefGoogle Scholar
  32. Papanastasiou, C. (2002). School, teaching and family influence on student attitudes toward science, based on TIMSS data for Cyprus. Studies in Educational Evaluation, 28(1), 71–86.CrossRefGoogle Scholar
  33. Papanastasiou, C., & Papanastasiou, E. C. (2004). Major influences on attitudes toward science. Educational Research and Evaluation, 10(3), 239–257. doi: 10.1076/edre. Scholar
  34. Rosenberg, M. J., & Hovland, C. I. (1960). Cognitive, affective and behavioral components of attitudes. In C. I. Hovland & M. J. Rosenberg (Eds.), Attitude organization and change: An analysis of consistency among attitude components (p. 3). New Haven, CT: Yale University Press.Google Scholar
  35. Roth, W.-M., Oliveri, M. E., Sandilands, D. D., Lyons-Thomas, J., & Ercikan, K. (2012). Investigating linguistic sources of differential item functioning using expert think-aloud protocols in science achievement tests. International Journal of Science Education, 35(4), 546–576. doi: 10.1080/09500693.2012.721572.CrossRefGoogle Scholar
  36. Schroeder, J., Murphy, K. L., & Holme, T. A. (2012). Investigating factors that influence item performance on ACS exams. Journal of Chemical Education, 89(3), 346–350. doi: 10.1021/ed101175f.CrossRefGoogle Scholar
  37. Schwarz, N. (1999). Cognitive research into survey measurement: Its influence on survey methodology and cognitive theory. In M. G. Sirken, D. J. Herrmann, S. Schechter, N. Schwarz, J. M. Tanur, & R. Tourangeau (Eds.), Cognition and survey research. New York: Wiley.Google Scholar
  38. Shanghai Jiao Tong University. (2012). The Academic Ranking of World Universities. Retrieved 3/1/2013, from
  39. Taylor, P. (1993). Minority ethnic groups and gender in access to higher education. Journal of Ethnic and Migration Studies, 19(3), 425–440.CrossRefGoogle Scholar
  40. The General Assembly of the State of South Carolina. (1998, Approved the 10th day of June, 1998.). South Carolina Education Accountability Act of 1998. Retrieved 3/13/2013, from
  41. van de Vijver, F., & Tanzer, N. K. (2004). Bias and equivalence in cross-cultural assessment: an overview. European Review of Applied Psychology, 54(2), 119–135. doi: 10.1016/j.erap.2003.12.004.CrossRefGoogle Scholar
  42. Xu, X., & Lewis, J. E. (2011). Refinement of a chemistry attitude measure for college students. Journal of Chemical Education, 88(5), 561–568. doi: 10.1021/ed900071q.CrossRefGoogle Scholar
  43. Xu, X., Southam, D., & Lewis, J. E. (2012). Attitude toward the Subject of Chemistry in Australia: An ALIUS and POGIL collaboration to promote cross-national comparisons. Australian Journal of Education in Chemistry, 72, 32.Google Scholar
  44. Xu, X., Villafane, S. M., & Lewis, J. E. (2013). College students’ attitudes toward chemistry, conceptual knowledge and achievement: Structural equation model analysis. Chemistry Education Research and Practice, 14(2), 188–200. doi: 10.1039/C3RP20170H.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Xiaoying Xu
    • 1
  • Khalid Alhooshani
    • 2
  • Daniel Southam
    • 3
  • Jennifer E. Lewis
    • 1
    Email author
  1. 1.Department of ChemistryUniversity of South FloridaTampaUSA
  2. 2.Department of ChemistryKing Fahd University of Petroleum & MineralsDhahranKingdom of Saudi Arabia
  3. 3.Department of ChemistryCurtin UniversityBentleyAustralia

Personalised recommendations