Abstract
Researchers in science education lacks valid and reliable instruments to assess students’ disciplinary and epistemic reading of scientific texts. The main purpose of this study was to develop and validate a Reading in Science Holistic Assessment (RISHA) to assess students’ holistic reading of scientific texts. RISHA measures students’ content, procedural, and epistemic domains of reading two texts, one history-of-science text and another socio-scientific text. The initial 24-item RISHA was administered to 161 Grade 9 students from 3 schools. The multidimensional Rasch partial credit model was used to analyze the reliability and validity of RISHA. All items demonstrated good fit and reliability. According to logit scores generated for each domain in Rasch analysis, students in our study performed better in content domain and less well in the epistemic domain. Students also performed significantly better in the epistemic domain of the socio-scientific text than in the history-of-science text. RISHA provides accurate measures in various domains of reading scientific texts and various contexts of scientific texts. We propose that RISHA could potentially be applied to studying the effect of reading-science intervention or predictors of students’ performance in each domain of reading scientific texts.
Similar content being viewed by others
References
Abrahams, I., Reiss, M. J., & Sharpe, R. M. (2013). The assessment of practical work in school science. Studies in Science Education, 49(2), 209–251.
Avsar Erumit, B., & Yuksel, T. (2023). Developing and using physical dynamic models on socioscientific issues to present nature of science ideas. International Journal of Science and Mathematics Education, 21(4), 1031–1056.
Bernholt, S., Härtig, H., & Retelsdorf, J. (2022). Reproduction rather than comprehension? Analysis of gains in students’ science text comprehension. Research in Science Education, 53(3), 493–506. https://doi.org/10.1007/s11165-022-10066-6
Boggs, G. L., Wilson, N. S., Ackland, R. T., Danna, S., & Grant, K. B. (2016). Beyond the Lorax: Examining children’s books on climate change. The Reading Teacher, 69(6), 665–675.
Bond, T., Yan, Z., & Heene, M. (2020). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge.
Boone, W. J., Staver, J. R., & Yale, M. S. (2013). Rasch analysis in the human sciences. Springer.
Chan, H. Y., Cheung, K. K. C., & Erduran, S. (2023). Science communication in the media and human mobility during the COVID-19 pandemic: A time series and content analysis. Public Health, 218, 106–113.
Chen, S.-Y., Chen, C.-H., & Liu, S.-Y. (2022). History of science reading materials as everyday homework to improve middle school students’ epistemological beliefs about science. International Journal of Science and Mathematics Education, 20(Suppl. 1), S69–S92. https://doi.org/10.1007/s10763-022-10285-3
Cheung, C. M. (2017). “Science (S1–3) – Learning and teaching resources” (Set C). Retrieved from https://cd1.edb.hkedcity.net/cd/science/multimedia/media1_1_e.htm
Cheung, K. K. C., & Winterbottom, M. (2021). Exploring students’ visualisation competence with photomicrographs of villi. International Journal of Science Education, 43(14), 2290–2315.
Cheung, K. K. C., Chan, H. Y., & Erduran, S. (2023). Communicating science in the COVID-19 news in the UK during Omicron waves: Exploring representations of nature of science with epistemic network analysis. Humanities and Social Sciences Communications, 10(1), 1–14.
Cheung, K. K. C., & Pun, J. K. (2023). The use of epistemic network analysis in analysing classroom discourse in EMI-science classrooms. In S. M. Curle & J. K. H. Pun (Eds.), Qualitative research methods in English medium instruction for emerging researchers (pp. 33–44). Routledge.
Cheung, K. K. C., & Sonkqayi, G. (2023). Students’ science achievement in cognitive domains: Effects of practical work and clarity of instruction. Research in Science & Technological Education, 1–18.
Cheung, K. K. C., & Winterbottom, M. (2023). Students’ integration of textbook representations into their understanding of photomicrographs: Epistemic network analysis. Research in Science & Technological Education, 41(2), 544–563.
Clough, M. P. (2006). Learners’ responses to the demands of conceptual change: Considerations for effective nature of science instruction. Science & Education, 15, 463–494.
Conley, A. M., Pintrich, P. R., Vekiri, I., & Harrison, D. (2004). Changes in epistemological beliefs in elementary science students. Contemporary Educational Psychology, 29(2), 186–204. https://doi.org/10.1016/j.cedpsych.2004.01.004
Curriculum Development Council and Hong Kong Examinations and Assessment Authority (2017). Science education: Key learning area curriculum guide (primary 1—secondary 6). Hong Kong: Curriculum Development Council.
Dagenais, A. (2010). Teaching high school physics with a story-line. Interchange, 41(4), 335–345. https://doi.org/10.1007/s10780-010-9134-z
Dass, P. M. (2005). Understanding the nature of scientific enterprise (NOSE) through a discourse with its history: The influence of an undergraduate ‘history of science’course. International Journal of Science and Mathematics Education, 3, 87–115.
Fang, Z. (2006). The language demands of science reading in middle school. International Journal of Science Education, 28(5), 491–520. https://doi.org/10.1080/09500690500339092
Fang, Z. (2008). Going beyond the fab five: Helping students cope with the unique linguistic challenges of expository reading in intermediate grades. Journal of Adolescent & Adult Literacy, 51(6), 476–487.
Farber, D. A. (2007). Adapting to climate change: Who should pay. J. Land Use & Envtl. l., 23, 1.
Fazio, X., & Gallagher, T. L. (2019). Science and language integration in elementary classrooms: Instructional enactments and student learning outcomes. Research in Science Education, 49, 959–976.
Fazio, X., Gallagher, T. L., & DeKlerk, C. (2022). Exploring adolescents’ critical reading of socioscientific topics using multimodal texts. International Journal of Science and Mathematics Education, 20(Suppl. 1), S93–S116.
Flesch, R. (2007). Flesch-Kincaid readability test. Retrieved October, 26(3), 2007.
Ford, M. J., & Wargo, B. M. (2012). Dialogic framing of scientific content for conceptual and epistemic understanding. Science Education, 96(3), 369–391.
Gardiner, S. M. (2010). Ethics and climate change: An introduction. Wiley Interdisciplinary Reviews: Climate Change, 1(1), 54–66.
Grigg, K., & Manderson, L. (2016). The Australian racism, acceptance, and cultural-ethnocentrism scale (RACES): Item response theory findings. International Journal for Equity in Health, 15, 1–16.
Hambleton, R. K., & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38–47.
Härtig, H., Bernholt, S., Fraser, N., Cromley, J. G., & Retelsdorf, J. (2022). Comparing reading comprehension of narrative and expository texts based on the direct and inferential mediation model. International Journal of Science and Mathematics Education, 20(1), 17–41. https://doi.org/10.1007/s10763-022-10302-5
Holliday, W. G., Yore, L. D., & Alvermann, D. E. (1994). The reading–science learning–writing connection: Breakthroughs, barriers, and promises. Journal of Research in Science Teaching, 31(9), 877–893.
Jian, Y.-C. (2018). Teaching fourth-grade students of different reading abilities to read biological illustrations and integrate in-text information: An empirical experiment. Research in Science Education, 50(6), 2269–2282. https://doi.org/10.1007/s11165-018-9778-8
Khishfe, R. (2023). Improving students’ conceptions of nature of science: A review of the literature. Science & Education, 32, 1887–1931. https://doi.org/10.1007/s11191-022-00390-8
Khishfe, R., Alshaya, F. S., BouJaoude, S., Mansour, N., & Alrudiyan, K. I. (2017). Students’ understandings of nature of science and their arguments in the context of four socio-scientific issues. International Journal of Science Education, 39(3), 299–334.
Kim, S. Y., & Irving, K. E. (2010). History of science as an instructional context: Student learning in genetics and nature of science. Science & Education, 19, 187–215.
Klopfer, L. E., & Aikenhead, G. S. (2022). Humanistic science education: The history of science and other relevant contexts. Science Education, 106(3), 490–504.
Kolstø, S. D. (2008). Science education for democratic citizenship through the use of the history of science. Science & Education, 17, 977–997.
Lammers, A., Goedhart, M. J., & Avraamidou, L. (2019). Reading and synthesising science texts using a scientific argumentation model by undergraduate biology students. International Journal of Science Education, 41(16), 2323–2346.
Lederman, N. G. (2013). Nature of science: Past, present, and future. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 831–879). Routledge.
Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of nature of science questionnaire: Toward valid and meaningful assessment of learners’ conceptions of nature of science. Journal of Research in Science Teaching, 39(6), 497–521.
Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2017). New literacies: A dual-level theory of the changing nature of literacy, instruction, and assessment. Journal of Education, 197(2), 1–18.
Leung, J. S. C. (2020). Students’ adherences to epistemic understanding in evaluating scientific claims. Science Education, 104(2), 164–192.
Li, C., Yu, J., & Li, G. (2023). Development of the representation of the nature of science in textbooks: The case of high school biology textbooks in mainland China. International Journal of Science and Mathematics Education, 21(6), 1749–1768.
Lin, C. Y., Cheng, J. H., & Chang, W. H. (2010). Making science vivid: Using a historical episodes map. International Journal of Science Education, 32(18), 2521–2531.
Lindsey, R., & Dahlman, L. (2023). Climate change: Global temperature. Retrieved from https://www.climate.gov/news-features/understanding-climate/climate-change-global-temperature#:~:text=Earth's%20temperature%20has%20risen%20by,based%20on%20NOAA's%20temperature%20data.
Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272
Matkins, J. J., & Bell, R. L. (2007). Awakening the scientist inside: Global climate change and the nature of science in an elementary science methods course. Journal of Science Teacher Education, 18(2), 137–163.
Matthews, M. R. (2014). Science teaching: The contribution of history and philosophy of science. Routledge.
Mccomas, W. F. (2011). The history of science and the future of science education: A typology of approaches to history of science in science instruction. In P. V. Kokkotas, K. S. Malamitsa, & A. A. Rizaki (Eds.), Adapting historical knowledge production to the classroom (pp. 37–53). Brill.
Millar, R., Lubben, F., Got, R., & Duggan, S. (1994). Investigating in the school science laboratory: Conceptual and procedural knowledge and their influence on performance. Research Papers in Education, 9(2), 207–248.
Monk, M., & Osborne, J. (1997). Placing the history and philosophy of science on the curriculum: A model for the development of pedagogy. Science Education, 81(4), 405–424.
National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press.
Nigro, R. G., & Trivelato, S. F. (2012). Knowledge, its application, and attitudes associated with the reading of diverse genres of science texts. International Journal of Science Education, 34(16), 2529–2564.
Norris, S. P., & Phillips, L. M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87(2), 224–240.
Norris, S. P., & Phillips, L. M. (2008). Reading as inquiry. In R. A. Duschl & R. E. Grandy (Eds.), Teaching scientific inquiry (pp. 233–262). Brill.
Organization for Economic Co-operation and Development. (2019). PISA 2018 Assessment and Analytical Framework. Author.
Oliveras, B., Márquez, C., & Sanmartí, N. (2013). The use of newspaper articles as a tool to develop critical thinking in science classes. International Journal of Science Education, 35(6), 885–905. https://doi.org/10.1080/09500693.2011.586736
Ozuru, Y., Dempsey, K., & McNamara, D. S. (2009). Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learning and Instruction, 19(3), 228–242. https://doi.org/10.1016/j.learninstruc.2008.04.003
Perfetti, C. A., Landi, N., & Oakhill, J. (2005). The acquisition of reading comprehension skill. In M. J. Snowling & C. Hulme (Eds.), The science of reading: A handbook (pp. 227–247). Blackwell Publishing.
Pun, J. K., Fu, X., & Cheung, K. K. C. (2023). Language challenges and coping strategies in English Medium Instruction (EMI) science classrooms: A critical review of literature. Studies in Science Education, 1–32.
Quellmalz, E., & Hoskyn, J. (1996). Classroom assessment of reasoning strategies. In G. D. Phye (Ed.), Handbook of classroom assessment (pp. 103–130). Elsevier.
Rauch, D. P., & Hartig, J. (2010). Multiple-choice versus open-ended response formats of reading test items: A two-dimensional IRT analysis. Psychological Test and Assessment Modeling, 52(4), 354–379.
Rasch, G. (1966). An item analysis which takes individual differences into account. Br J Math Stat Psychol, 19(1), 49–57. https://doi.org/10.1111/j.2044-8317.1966.tb00354.x
Sadler, T. D., Chambers, F. W., & Zeidler, D. L. (2004). Student conceptualizations of the nature of science in response to a socioscientific issue. International Journal of Science Education, 26(4), 387–409.
Sadler, T. D., & Dawson, V. (2012). Socio-scientific issues in science education: Contexts for the promotion of key learning outcomes. In B. J. Fraser, K. G. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 799–809). Springer.
Sbeglia, G. C., & Nehm, R. H. (2019). Do you see what I-SEA? A Rasch analysis of the psychometric properties of the Inventory of Student Evolution Acceptance. Science Education, 103(2), 287–316.
Secko, D. M., Amend, E., & Friday, T. (2013). Four models of science journalism: A synthesis and practical assessment. Journalism Practice, 7(1), 62–80.
Shepardson, D. P., & Gummer, E. S. (2001). A framework for thinking about and planning classroom assessments in science. In D. P. Shepardson (Ed.), Assessment in science: A guide to professional development and classroom practice (pp. 83–97). Springer.
Shymansky, J. A., Yore, L. D., & Good, R. (1991). Elementary school teachers’ beliefs about and perceptions of elementary school science, science reading, science textbooks, and supportive instructional factors. Journal of Research in Science Teaching, 28(5), 437–454.
Smith, E. A., & Senter, R. J. (1967). Automated readability index (Vol.66, No.220). Aerospace Medical Research Laboratories, Aerospace Medical Division, Air Force Systems Command.
Stang Lund, E., Bråten, I., Brandmo, C., Brante, E. W., & Strømsø, H. I. (2019). Direct and indirect effects of textual and individual factors on source-content integration when reading about a socio-scientific issue. Reading and Writing, 32, 335–356.
Stang Lund, E., Bråten, I., Brante, E. W., & Strømsø, H. I. (2017). Memory for textual conflicts predicts sourcing when adolescents read multiple expository texts. Reading Psychology, 38(4), 417–437.
Strømsø, H. I., & Bråten, I. (2009). Beliefs about knowledge and knowing and multiple-text comprehension among upper secondary students. Educational Psychology, 29(4), 425–445.
Strømsø, H. I., Bråten, I., & Samuelstuen, M. S. (2008). Dimensions of topic-specific epistemological beliefs as predictors of multiple text understanding. Learning and Instruction, 18(6), 513–527.
Symons, C. (2017). Supporting emergent bilinguals’ argumentation: Evaluating evidence in informational science texts. Linguistics and Education, 38, 79–91.
Tang, K.-S. (2021). The interconnections among metadiscourse, metalanguage, and metacognition: Manifestation and application in classroom discourse. Linguistics and Education, 65, 100977.
Tang, K.-S., & Rappa, N. A. (2021). The role of metalanguage in an explicit literacy instruction on scientific explanation. International Journal of Science and Mathematics Education, 19(7), 1311–1331.
Tang, K. S., Lin, S.-W., & Kaur, B. (2022). Mapping and extending the theoretical perspectives of reading in science and mathematics education research. International Journal of Science and Mathematics Education, 20(Suppl. 1), S1–S15. https://doi.org/10.1007/s10763-022-10322-1
Tsai, C.-C., Jessie Ho, H. N., Liang, J.-C., & Lin, H.-M. (2011). Scientific epistemic beliefs, conceptions of learning science and self-efficacy of learning science among high school students. Learning and Instruction, 21(6), 757–769. https://doi.org/10.1016/j.learninstruc.2011.05.002
Tuncay, B., Yılmaz-Tüzün, Ö., & Teksoz, G. T. (2012). Moral reasoning patterns and influential factors in the context of environmental problems. Environmental Education Research, 18(4), 485–505.
Wang, J.-R., & Chen, S.-F. (2016). Development and validation of an online dynamic assessment for raising students’ comprehension of science text. International Journal of Science and Mathematics Education, 14(3), 373–389. https://doi.org/10.1007/s10763-014-9575-4
Wang, J.-R., Chen, S.-F., Fang, I., & Chou, C.-T. (2012). Development of a test assessing elementary students’ comprehension of science texts. International Journal of Science and Mathematics Education, 10(4), 955–973.
Wellington, J., & Osborne, J. (2001). Language and literacy in science education. McGraw-Hill Education (UK).
Wilson, M. (2023). Constructing measures: An item response modeling approach. Taylor & Francis.
Wiyarsi, A., Çalik, M., Priyambodo, E., & Dina, D. (2023). Indonesian prospective teachers’ scientific habits of mind: A cross-grade study in the context of local and global socio-scientific issues. Science & Education, 1–27.
Wu, M., Adams, R., Wilson, M., & Haldane, S. (2007). ACER ConQuest 2.0: General item response modelling software [Computer program manual]. Australian Council for Educational Research.
Yang, F.-Y., Chang, C.-C., Chen, L.-L., & Chen, Y.-C. (2016). Exploring learners’ beliefs about science reading and scientific epistemic beliefs, and their relations with science text understanding. International Journal of Science Education, 38(10), 1591–1606.
Yore, L. D., Hand, B., Goldman, S. R., Hildebrand, G. M., Osborne, J. F., Treagust, D. F., & Wallace, C. S. (2004). New directions in language and science education research. Reading Research Quarterly, 39(3), 347–352.
Yore, L. D., Pimm, D., & Tuan, H.-L. (2007). The literacy component of mathematical and scientific literacy. International Journal of Science and Mathematics Education, 5(4), 559–589.
Yore, L. D., & Tang, K.-S. (2022). Foundations, insights, and future considerations of reading in science and mathematics education. International Journal of Science and Mathematics Education, 20(Suppl. 1), S237–S260. https://doi.org/10.1007/s10763-022-10321-2
Zeidler, D. L. (2014). Socioscientific issues as a curriculum emphasis: Theory, research, and practice. In N. G. Lederman & S. K. Abell (Eds.), Handbook of research on science education (Vol. 2, pp. 711–740). Routledge.
Zeidler, D. L., & Nichols, B. H. (2009). Socioscientific issues: Theory and practice. Journal of Elementary Science Education, 21, 49–58.
Zhai, X. (2022). Assessing high-school students’ modeling performance on Newtonian mechanics. Journal of Research in Science Teaching, 59(8), 1313–1353.
Zhang, J., & Browne, W. J. (2023). Exploring Chinese high school students’ performance and perceptions of scientific argumentation by understanding it as a three-component progression of competencies. Journal of Research in Science Teaching, 60(4), 847–884.
Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1–28.
Funding
This project was funded by the Quality Education Fund, Hong Kong SAR Government. The name of the project is Empowering Hong Kong STEM secondary students’ reading abilities through a school-based reciprocal reading program and an online learning platform, Quality Education fund, Hong Kong SAR Government (Grant Number: 9420033).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
This project was approved by Human and Artefacts Ethics Sub-Committee at City University of Hong Kong (Application Number: HU-STA-00000011). Students’ and parents’ consents were sought before carrying out the study.
Conflict of interest
The authors declare no competing interests.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cheung, K.K.C., Pun, J.K.H. & Fu, X. Development and Validation of a Reading in Science Holistic Assessment (RISHA): a Rasch Measurement Study. Int J of Sci and Math Educ (2023). https://doi.org/10.1007/s10763-023-10434-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10763-023-10434-2