Skip to main content
SpringerLink
Log in

Using Rasch Analysis to Assess Students’ Learning Progression in Stability and Change across Middle School Grades

  • Chapter
  • First Online:
Advances in Applications of Rasch Measurement in Science Education

Part of the book series: Contemporary Trends and Issues in Science Education ((CTISE,volume 57))

  • 318 Accesses

Abstract

This study developed a measurement instrument to assess students’ learning progressions in the crosscutting concept of Stability and Change across middle school grades (from Grades 7 to 9). Based on existing concept development models, frameworks, and research on student learning progressions, this study’s learning progression framework comprises four primary levels (i.e., Identifying, Understanding, Analyzing, and Designing) with three sub-levels (i.e., Static, Dynamic, and Cyclic), from basic to the most sophisticated. During the field test, three versions of the test comprising a total of 24 constructed-response items were administered to 136 seventh graders, 139 eighth graders, and 67 ninth graders. A partial credit Rasch model analysis was employed to inform instrument development and evaluation. Specifically, this study used step calibrations and item measures anchoring to express student performance across three grades on the same linear scale. Results provided evidence of reliability, content validity, construct validity, and predictive validity of measures of the instrument, suggesting the measurement instrument meets the quality benchmarks. The results illustrated that higher-grade students were more proficient than lower-grade students in Identifying, Understanding, Analyzing, and Designing regarding Stability and Change. None of the seventh graders and less than 5% of eighth graders were proficient at Cyclic level in Understanding, Analyzing, and Designing, whereas between 3% and 16.4% of ninth graders were proficient at level 3 in Understanding, Analyzing, and Designing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • AAAS Project 2061. (2013). AAAS project 2061 science assessment website. Retrieved from assess.bscs.org/science/topics

    Google Scholar 

  • Aiken, S. (2003). Estimation of item parameters. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models, foundations, recent developments, and applications (pp. 39–52). Springer.

    Google Scholar 

  • American Association for the Advancement of Science. (1993). Benchmarks for science literncy: Project 2061. Oxford University Press.

    Google Scholar 

  • Andrich, D., & Marais, I. (2019). A course in Rasch measurement theory: Measuring in the educational, social and health sciences. Springer Nature Singapore Pte Ltd..

    Book  Google Scholar 

  • Baghaei, P. (2008). The Rasch model as a construct validation tool. Rasch Measurement Transactions, 22(1), 1145.

    Google Scholar 

  • Ben-zvi-Assarf, O., & Orion, N. (2005). A study of junior high students' perceptions of the water cycle. Journal of Geoscience Education, 53(4), 366–373.

    Article  Google Scholar 

  • Biggs, J. B. (1999). Teaching for quality learning at university. SRHE and Open University Press.

    Google Scholar 

  • Black, P., Wilson, M., & Yao, S. Y. (2011). Road maps for learning: A guide to the navigation of learning progressions. Measurement: Interdisciplinary Research and Perspective, 9(2–3), 71–123.

    Google Scholar 

  • Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objetives: The classification of educational goals: Handbook I: Cognitive domain (No. 373.19 C734t). D. Mckay.

    Google Scholar 

  • Bond, T., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Taylor & Francis.

    Book  Google Scholar 

  • Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple-choice tests. Science Education, 90(2), 253–269.

    Article  Google Scholar 

  • Boone, W., Staver, J., & Yale, M. (2014). Rasch analysis in the human sciences. Springer.

    Book  Google Scholar 

  • Boone, W. J., & Staver, J. R. (2020). Advances in Rasch analyses in the human sciences. Springer.

    Book  Google Scholar 

  • Brann, K. L., Boone, W. J., Splett, J. W., Clemons, C., & Bidwell, S. L. (2021). Development of the school mental Health self-efficacy teacher survey using Rasch analysis. Journal of Psychoeducational Assessment, 39(2), 197–211.

    Google Scholar 

  • Chan, M., & Subramaniam, R. (2020). Validation of a science concept inventory by Rasch analysis. In M. Khine (Ed.), Rasch measurement: Applications in quantitative educational research (pp. 159–178). Springer. Singapore.

    Chapter  Google Scholar 

  • Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A., & Krause, S. R. (1998). Hierarchical complexity of tasks shows the existence of developmental stages. Developmental Review, 18(3), 237–278.

    Article  Google Scholar 

  • de Jong, A. E., Tuinebreijer, W. E., Bremer, M., van Komen, R., Middelkoop, E., & van Loey, N. (2012). Construct validity of two pain behavior observation measurement instruments for young children with burns by Rasch analysis. Pain, 153(11), 2260–2266.

    Article  Google Scholar 

  • Duschl, R. A. (2012). The second dimension crosscutting concepts: Understanding a framework for K–12 science education. The Science Teacher, 79(2), 34–38.

    Google Scholar 

  • Fan, J., & Bond, T. (2019). Unidimensionality and local independence. In V. Aryadoust & M. Rachelle (Eds.), Quantitative data analysis for language assessment (Volume I): Fundamental techniques (pp. 83–102). Routledge.

    Chapter  Google Scholar 

  • Fick, S. J., McAlister, A. M., Chiu, J. L., & McElhaney, K. W. (2021). Using students’ conceptual models to represent understanding of crosscutting concepts in an NGSS-aligned curriculum unit about urban water runoff. Journal of Science Education and Technology, 30(5), 678–691.

    Article  Google Scholar 

  • Fisher, W. P. (2007). Rating scale instrument quality criteria. Rasch Measurement Transactions, 21(1), 1095.

    Google Scholar 

  • Fox, C. (1999). An introduction to the partial credit model for developing nursing assessments. Journal of Nursing Education, 38(8), 340–346.

    Article  Google Scholar 

  • Goldstone, R. L., & Wilensky, U. (2008). Promoting transfer by grounding complex systems principles. Journal of the Learning Sciences, 17(4), 465–516.

    Article  Google Scholar 

  • Gothwal, V. K., Wright, T. A., Lamoureux, E. L., & Pesudovs, K. (2009). Rasch analysis of visual function and quality of life questionnaires. Optometry and Vision Science, 86(10), 1160–1168.

    Article  Google Scholar 

  • Jin, H., & Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems. Journal of Research in Science Teaching, 49(9), 1149–1180.

    Article  Google Scholar 

  • Johnson, P. (2000). Children's understanding of substances, part 1: Recognizing chemical change. International Journal of Science Education, 22(7), 719–737.

    Article  Google Scholar 

  • Kaldaras, L., Akaeze, H., & Krajcik, J. (2021). Developing and validating next generation science standards-aligned learning progression to track three-dimensional learning of electrical interactions in high school physical science. Journal of Research in Science Teaching, 58(4), 589–618.

    Article  Google Scholar 

  • Lee, H. S., & Liu, O. L. (2010). Assessing learning progression of energy concepts across middle school grades: The knowledge integration perspective. Science Education, 94(4), 665–688.

    Article  Google Scholar 

  • Linacre, J. M. (2014). A user's guide to Winsteps Ministep Rasch-model computer programs: Program manual 3.81.0. Winsteps.com.

    Google Scholar 

  • Liu, O. L., Lee, H.-S., Hofstetter, C., & Linn, M. (2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13(1), 33–55.

    Article  Google Scholar 

  • Liu, X., & Lesniak, K. M. (2005). Students' progression of understanding the matter concept from elementary to high school. Science Education, 89(3), 433–450.

    Article  Google Scholar 

  • Liu, X. (2012). Using learning progression to organize learning outcomes: Implications for assessments. In S. Bernholt, K. Neumann & Nentwig (Eds.), Making it tangible-learning outcomes in science education (pp. 285–301). : Waxmann.

    Google Scholar 

  • Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–386.

    Article  Google Scholar 

  • Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174.

    Article  Google Scholar 

  • McGinnis, P. (2020). Stability and change: Integral to systems. Science Scope, 703, 312–9273.

    Google Scholar 

  • Ministry of Education (MoE). (2011). Compulsory junior high school science curriculum standards. Beijing Normal University Press. (in Chinese).

    Google Scholar 

  • Morell, L., Collier, T., Black, P., & Wilson, M. (2017). A construct-modeling approach to develop a learning progression of how students understand the structure of matter. Journal of Research in Science Teaching, 54(8), 1024–1048.

    Article  Google Scholar 

  • National Research Council (NRC). (2007). Taking science to school: Learning and teaching science in grade K-8. The National Academies Press.

    Google Scholar 

  • National Research Council. (2012). A framework for K–12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press.

    Google Scholar 

  • National Research Council. (2013). Next generation science standards. National Academies Press.

    Google Scholar 

  • Neumann, K., Viering, T., Boone, W. J., & Fischer, H. E. (2013). Towards a learning progression of energy. Journal of Research in Science Teaching, 50(2), 162–188.

    Article  Google Scholar 

  • Niebert, K., Marsch, S., & Treagust, D. F. (2012). Understanding needs embodiment: A theory-guided reanalysis of the role of metaphors and analogies in understanding science. Science Education, 96(5), 849–877.

    Article  Google Scholar 

  • NGSS Lead States. (2013). Next generation science standards: For states, by states. National Academies Press.

    Google Scholar 

  • Lead States, N. G. S. S. (2013). Next generation science standards: For states, by states. Volume 2: Appendix G- crosscutting Concetps. National Academies Press.

    Google Scholar 

  • Park, M., & Liu, X. (2016). Assessing understanding of the energy concept in different science disciplines. Science Education, 100(3), 483–516.

    Article  Google Scholar 

  • Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 020111.

    Article  Google Scholar 

  • Plummer, J. D., & Maynard, L. (2014). Building a learning progression for celestial motion: An exploration of students' reasoning about the seasons. Journal of Research in Science Teaching, 51(7), 902–929.

    Article  Google Scholar 

  • Riley, L., & Biernat, K. (2019). Making sense of stability and change. Science Scope, 42(6), 32–35.

    Article  Google Scholar 

  • Rittle-Johnson, B., Matthews, P. G., Taylor, R. S., & McEldoon, K. L. (2011). Assessing knowledge of mathematical equivalence: A construct-modeling approach. Journal of Educational Psychology, 103(1), 85–104.

    Article  Google Scholar 

  • Rivet, A. E., Weiser, G., Lyu, X., Li, Y., & Rojas-Perilla, D. (2016). What are crosscutting concepts in science? Four metaphorical perspectives. In C. K. Looi, J. L. Polman, U. Cress, & P. Reimann (Eds.), Transforming learning, empowering learners: The international conference of the learning sciences (ICLS) (Vol. 2, pp. 970–973). International Society of the Learning Sciences.

    Google Scholar 

  • Songer, N. B., Kelcey, B., & Gotwals, A. W. (2009). How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning about biodiversity. Journal of Research in Science Teaching, 46(6), 610–631.

    Article  Google Scholar 

  • Stavy, R. (1990). Children's conceptions of changes in the state of matter: From liquid (or solid) to gas. Journal of Research in Science Teaching, 27, 247–266.

    Article  Google Scholar 

  • Stevens, S., Delgado, C., & Krajcik, J. S. (2010). Developing a hypothetical multidimensional learning progression for the nature of matter. Journal of Research in Science Teaching, 47(6), 687–715.

    Article  Google Scholar 

  • Testa, I., Galano, S., Leccia, S., & Puddu, E. (2015). Development and validation of a learning progression for change of seasons, solar and lunar eclipses, and moon phases. Physical Review Special Topics-Physics Education Research, 11(2), 020102.

    Article  Google Scholar 

  • Wilson, M. (2005). Constructing measures: An item response modeling approach. Taylor & Francis Group.

    Google Scholar 

  • Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46(6), 716–730.

    Article  Google Scholar 

  • Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14(2), 97–116.

    Article  Google Scholar 

Download references

Statements and Declarations

This work was supported by the Shanghai Pujiang Program (No. 2020PJC032) and the MOE Key Research Institute of Humanities and Social Sciences (No. 17JJD880007).

All authors have no relevant financial or non-financial interests to disclose.

Author information

Authors and Affiliations

  1. College of Teacher Education, Faculty of Education, East China Normal University, Shanghai, China

    Shaohui Chi, Zuhao Wang & Ya Zhu

Authors
  1. Shaohui Chi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zuhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ya Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaohui Chi .

Editor information

Editors and Affiliations

  1. Department of Learning & Instruction, Graduate School of Education University at Buffalo, Buffalo, NY, USA

    Xiufeng Liu

  2. Department of Educational Psychology, Program in Learning Sciences and Human Development, Miami University, Oxford, OH, USA

    William J. Boone

Appendix

Appendix

11.1.1 Rasch Analysis Codes (Winsteps Codes)

&INST TITLE = 789th stability and change PERSON = Person ; persons are ... ITEM = Item ; items are ... ITEM1 = 7 ; column of response to first item in data record NI = 24 ; number of items NAME1 = 1 ; column of first character of person identifying label NAMELEN = 6; length of person label XWIDE = 1 ; number of columns per item response CODES = "0123" ; valid codes in data file UIMEAN = 0 ; item mean for local origin USCALE = 1 ; user scaling for logits UDECIM = 2 ; reported decimal places for user scaling MISSCORE= -1 T1P#=1 @GENDER=6 DIF=@GENDER ISGROUPS=0 IAFILE= * 1 -1.74 ........ * SAFILE=* 1 0 0 1 1 -1.70 1 2 -0.75 1 3 2.46 ........ * &END Dqyd1 Dqyd2 Dqyd3 ........ Hxbh1 Hxbh2 Hxbh3 END LABELS 740011333222331131XXXXXXXXXXXX 740021332122121131XXXXXXXXXXXX 740031213232110131XXXXXXXXXXXX 740041230122031131XXXXXXXXXXXX 740051201122112131XXXXXXXXXXXX ................................................................ 811371XXXXXX332XXX300132223XXX 811381XXXXXX322XXX311031223XXX 811391XXXXXX121XXX311030220XXX 811401XXXXXX332XXX213032223XXX 811411XXXXXX212XXX100002221XXX ................................................................ 933380XXXXXXXXX333123033223122 933390XXXXXXXXX121230033222121 933400XXXXXXXXX112320021222103 933410XXXXXXXXX131222200221001 933420XXXXXXXXX231320031321232

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chi, S., Wang, Z., Zhu, Y. (2023). Using Rasch Analysis to Assess Students’ Learning Progression in Stability and Change across Middle School Grades. In: Liu, X., Boone, W.J. (eds) Advances in Applications of Rasch Measurement in Science Education. Contemporary Trends and Issues in Science Education, vol 57. Springer, Cham. https://doi.org/10.1007/978-3-031-28776-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28776-3_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28775-6

  • Online ISBN: 978-3-031-28776-3

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation