Abstract
Cognitive model [CM] is the description of individual’s problem-solving method on selected tasks at a precise level to make meaningful inference on his/her strengths and weaknesses. CM serves as a foundation in developing a cognitive diagnostic assessment [CDA] that links the interpretation of test score to the attributes. This study employed the CDA involving “Time” which was developed under the framework of assessment triangle and Attribute Hierarchy Method [AHM]. In this study, the expert-based cognitive model was generated by 7 experienced primary mathematics teachers while the student-based cognitive model was generated based on 259 Year 6 pupils’ responses in the CDA. The consistency between these cognitive models was calculated by using Hierarchical Consistency Index [HCI] and triangulated by pupils’ interview transcript. Hence, this paper aims to discuss the consistency of structure between the expert-based cognitive models and the student-based cognitive models. The extent of match between the expert-based and the student-based cognitive models is important as it determines the validity of the expert-based cognitive model from students’ perspective. In general, all the 3 expert-based cognitive models were excellently fit with the student-based cognitive models. However, the result shows that the cognitive models of those pupils who scored 50% and below exhibited poor fit to the expert-based cognitive models. Results of the study implied that the proposed CMs are able to predict pupils’ responses and there is a need to construct CM which is suitable for low ability pupils in order to help them in studying the topic of “Time.”
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alves, C. B. (2012). Making diagnostic inferences about student performance on the Alberta Education Diagnostic Mathematics Project: An application of the Attribute Hierarchy Method (Unpublished Doctoral Dissertation). University of Alberta, Edmonton, Alberta, Canada.
Birenbaum, M., Tatsuoka, C., & Yamada, Y. (2004). Diagnostic assessment in TIMMS-R: Between countries and within country comparisons of eight graders’ mathematics performance. Studies in Educational Evaluation, 30, 151–173.
Broaddus, A. (2011). Modeling student understanding of foundational concepts related to Slope: An application of the Attribute Hierarchy Method. Retrieved October, 2014 from https://cete.ku.edu/sites/cete.drupal.ku.edu/files/docs/Presentations/2012_04_Broaddus%20Modeling%20Student%20Understanding.pdf.
Burny, E., Valcke, M., & Desoete, A. (2009). Towards an agenda for studying learning and instruction focusing on time-related competences in children. Educational Studies, 35(5), 481–492.
Chen, Y.-H. (2012). Cognitive diagnosis of mathematics performance between rural and urban students in Taiwan. Assessment in Education: Principles, Policy & Practice, 19(2), 193–209.
Chudowsky, N., & Pellegrino, J. W. (2003). Large-scale assessments that support learning: What will it take? Theory Into Practice, 42, 75–83.
Cui, Y., Leighton, J. P., Gierl, M. J., & Hunka, S. (2006). A person-fit statistic for the attribute hierarchy method: The hierarchy consistency index. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
Ericsson, K.A. (2006). The influence of experience and deliberate practice on the development of superior experts performance. The Cambridge handbook of expertise and expert performance, 38, 685–705.
Gierl, M. J. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44(4), 325–340.
Gierl, M.J., & Alves, C. B (2010). Using principled test design to develop and evaluate a diagnostic mathematics assessment in grades 3 and 6. Paper presented at the annual meeting of American Educational Research Association, CO, USA.
Gierl, M. J., & Cui, Y. (2008). Defining characteristics of diagnostic classification models and the problem of retrofitting in cognitive diagnostic assessment. Measurement: Interdisciplinary Research & Perspective, 6(4), 263–268.
Gierl, M. J., & Leighton, J. P. (2006). 34C linking cognitively-based models and psychometric methods. Handbook of statistics, 26, 1103–1106.
Gierl, M.J., Wang, C., & Zhou, J. (2008). Using the attribute hierarchy method to make diagnostic inferences about examinees’ cognitive skills in algebra on the SAT©. Journal of Technology, Learning, and Assessment, 6(6). Retrieved 15/8/2015 from http://www.jtla.org.
Gierl, M. J., Leighton, J. P., Wang, C., Zhou, J., Gokiert, R., & Tan, A. (2009a). Validating Cognitive Models of Task Performance in Algebra on the SAT®. Research Report No. 2009–3. New York, NY: The College Board. Retrieved from http://files.eric.ed.gov/fulltext/ED563101.pdf.
Gierl, M. J., Roberts, M., Alves, C., & Gotzmann, A. (2009b). Using judgments from content specialists to develop cognitive models for diagnostic assessments. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
Gierl, M. J., Alves, C., & Majeau, R. T. (2010). Using the attribute hierarchy method to make diagnostic inferences about examinees’ knowledge and skills in mathematics: An operational implementation of cognitive diagnostic assessment. International Journal of Testing, 10(4), 318–341.
Gotzmann, A. & Roberts, M. (2010). Do cognitive models consistently show good model-data-fit for students at different ability levels? Poster session presented at annual meeting of the American Educational Research Association, Denver, CO.
Harris, S. (2008). It's about time: Difficulties in developing time concepts. Australian Primary Mathematics Classroom, 13(1), 28–31.
Huba, M. E., & Freed, J. E. (2000). Learner centered assessment on college campuses: Shifting the focus from teaching to learning. Community College Journal of Research and Practice, 24(9), 759–766.
Jang, E. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing, 26(1), 031–073.
Ketterlin-Geller, L., & Yovanoff, P. (2009). Cognitive diagnostic assessment in mathematics to support instructional decision making. Practical Assessment, Research, & Evaluation, 14(16), 1–11.
Leighton, J. P., & Gierl, M. J. (2007a). Verbal reports as data for cognitive diagnostic assessment. In J. P. Leighton & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 146–172). New York, NY: Cambridge University Press.
Leighton, J. P., & Gierl, M. J. (2007b). Review of cognitively diagnostic assessment and a summary of psychometric models. In J. P. Leighton & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 3–18). New York, NY: Cambridge University Press.
Leighton, J. P., Cui, Y., & Cor, M. K. (2009). Testing expert-based and student-based cognitive models: An application of the Attribute Hierarchy Method and Hierarchy Consistency Index. Applied Measurement in Education, 22(140), 229–254.
Mislevy, R. J. (2006). Cognitive psychology and educational assessment. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 257–305). Washington, DC: American Council on Education.
Norris, S. P., Macnab, J. S., & Philips, L. M. (2007). Cognitive modeling of performance on a diagnostic achievement tests: A philosophical analysis and justification. In J. P. Leighton & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 61–84). New York, NY: Cambridge University Press.
Pellegrino, J. W. (2009). The design of an assessment system for the Race to the Top: A learning sciences perspective on issues of growth and measurement. Center for K–12 Assessment & Performance Management, Educational Testing Service. http://www.k12center.org/rsc/pdf/PellegrinoPresenter-Session1.pdf.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
Roberts, M., Alves, C., Gotzmann, A., & Gierl, M. (2009). Development of cognitive models in Mathematics to promote diagnostic inferences about student performance. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Roberts, M., Alves, C., Chu, M.W., Thompson, M., Bahry, L.M. & Gotzmann, A. (2012). Testing expert-based vs. student-based cognitive models for a grade 3 diagnostic mathematics assessment. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Su, Y. L., Choi, K. M., Lee, W. C., Choi, T., & McAninch, M. (2013). Hierarchical Cognitive Diagnostic Analysis for TIMSS 2003 Mathematics. (Research Report No. 35). Iowa City, IA: Center for Advanced Studies in Measurement and Assessment. Retrieved from www.education.uiowa.edu/casma.
Tatsuoka, K. K., & Tatsuoka, M. M. (1997). Computerized cognitive diagnostic adaptive testing: Effect on remedial instruction as empirical validation. Journal of Educational Measurement Spring, 34(1), 3–20.
Wang, C., & Gierl, M. J. (2011). Using the attribute hierarchy method to make diagnostic inferences about examineesʼ cognitive skills in critical reading. Journal of Educational Measurement, 48(2), 165–187.
Winterton, J., Delamare Le Deist, F., & Stringfellow, E. (2006). Typology of knowledge, skills and competences: Clarification of the concept and prototype. CEDEFOP Reference series 64, Luxembourg, LU: Office for Official Publications of the European Communities.
Wu, L. J., Chen, H. H., Sung, Y. T., & Chang, K. E. (2012). Developing cognitive diagnostic assessments system for mathematics learning. In B. Werner (Ed.), Proceedings of the 12th IEEE International Conference on Advanced Learning Technologies, ICALT 2012 (pp. 228–229). Rome, Italy: IEEE Computer Society.
Ye, F. (2005). Diagnostic Assessment of Urban Middle School Student Learning of Pre-Algebra Patterns (Unpublished Doctoral Dissertation). The Ohio State University, USA.
Funding
The study reported in this paper was made possible by the generous support from the Fundamental Research Grant Scheme (FRGS) of the Malaysian Ministry of Education and Universiti Sains Malaysia (Account No. 203/PGURU/6711344).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sia, C.J.L., Lim, C.S., Chew, C.M. et al. Expert-Based Cognitive Model and Student-Based Cognitive Model in the Learning of “Time”: Match or Mismatch?. Int J of Sci and Math Educ 17, 1089–1107 (2019). https://doi.org/10.1007/s10763-018-9916-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10763-018-9916-9