Skip to main content
SpringerLink
Log in

Diagnosing Preservice Teachers’ Understanding of Statistics and Probability: Developing a Test for Cognitive Assessment

  • Published:
International Journal of Science and Mathematics Education Aims and scope Submit manuscript

Abstract

This study investigates preservice middle school mathematics teachers’ understanding of statistics and probability and provides a cognitive diagnostic assessment of their strengths and weaknesses on these subjects. A statistical reasoning test that included 15 multiple-choice and 5 open-ended items was developed from the perspective of the log-linear cognitive diagnosis model, which is a general form of the cognitive diagnosis models. The statistical reasoning test was applied to 456 preservice teachers from 4 universities in 3 different regions of Turkey. The collected data were analyzed using the Mplus 6.12 software, and diagnostic feedback on the preservice teachers’ responses was provided based on the findings. The analysis suggested that although many preservice teachers were able to represent and interpret the given data, most experienced difficulty in drawing inferences about populations based on samples, selecting and using appropriate statistical methods, and understanding and applying the basic concepts of probability. In addition, preservice teachers had difficulty answering open-ended items. Implications for teaching are also discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ball, D. L., Lubienski, S. T., & Mewborn, D. S. (2001). Research on teaching mathematics: The unsolved problem of teachers’ mathematical knowledge. In V. Richardson (Ed.), Handbook of research on teaching (4th ed., pp. 433–456). Washington, DC: American Educational Research Association.

  • Batanero, C., & Díaz, C. (2010). Training teachers to teach statistics: What can we learn from research? Statistique et Enseignement, 1(1), 5–20.

    Google Scholar 

  • Batanero, C., & Díaz, C. (2012). Training school teachers to teach probability: Reflections and challenges. Chilean Journal of Statistics, 3(1), 3–13.

    Google Scholar 

  • Batanero, C., Godino, J. D., & Roa, R. (2004). Training teachers to teach probability. Journal of Statistics Education, 1(1), 5–20. Retrieved February 3, 2018, from http://www.amstat.org/publications/jse/v12n1/batanero.html.

  • Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., ... Tsai, Y. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133–180.

  • Bradshaw, L. (2015). PARCC diagnostic assessments for mathematics comprehension: A diagnostic classification model approach. Paper presented at the Council of Chief State School Officers (CCSSO) 2015 National Conference on Student Assessment (NCSA) in San Diego, California.

  • Bradshaw, L., Izsák, A., Templin, J., & Jacobson, E. (2014). Diagnosing teachers’ understandings of rational numbers: Building a multidimensional test within the diagnostic classification framework. Educational Measurement: Issues and Practice, 33(1), 2–14.

    Article  Google Scholar 

  • Brown, A., & Silver, E. (1989). Data organization and interpretation. In M. M. Lindquist (Ed.), Results from the fourth mathematics assessment of the national assessment of educational progress (pp. 28–34). Reston, VA: National Council of Teachers of Mathematics.

  • Burgess, T. A. (2007). Investigating the nature of teacher knowledge needed and used in teaching statistics (Unpublished doctoral dissertation). Massey University, Auckland, New Zealand.

  • Choi, K. M., Lee, Y. S., & Park, Y. S. (2015). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics. Eurasia Journal of Mathematics, Science & Technology Education, 11(6), 1563–1577.

    Article  Google Scholar 

  • Common Core State Standards Initiative. (2010). The common core state standards for mathematics. Washington, DC: Author.

  • Curcio, F. R. (1987). Comprehension of mathematical relationships expressed in graphs. Journal for Research in Mathematics Education, 18(5), 382–393.

    Article  Google Scholar 

  • de la Torre, J. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45(4), 343–362.

    Article  Google Scholar 

  • de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179–199.

    Article  Google Scholar 

  • delMas, R., Garfield, J., & Ooms, A. (2005, July). Using assessment items to study students’ difficulty with reading and interpreting graphical representations of distributions. Paper presented at the Fourth Forum on Statistical Reasoning, Thinking, and Literacy (SRTL–4), Auckland, New Zealand.

  • delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58.

    Google Scholar 

  • Dereli, A. (2009). The mistakes and misconceptions in probability of eighth grade students (Unpublished Master’s Thesis). Eskisehir Osmangazi University, Turkey.

  • DiBello, L. V., Stout, W. F., & Roussos, L. A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood–based classification techniques. In P. Nichols, S. Chipman, & R. Brennan (Eds.), Cognitively diagnostic assessment (pp. 361–390). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Dogan, E., & Tatsuoka, K. (2008). An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS–R. Educational Studies in Mathematics, 68(3), 263–272.

    Article  Google Scholar 

  • Fennema, E., & Franke, M. L. (1992). Teachers’ knowledge and its impact. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 147–164). New York, NY: Macmillan.

  • Franklin, C., Kader, G., Mewborn, D. S., Moreno, J., Peck, R., Perry, M., & Scheaffer, R. (2007). Guidelines for assessment and instruction in statistics education (GAISE) report: A pre–K–12 curriculum framework. Alexandria, VA: American Statistical Association. Retrieved February 8, 2018, from http://www.amstat.org/asa/education/Guidelines-for-Assessment-and-Instruction-in-Statistics-Education-Reports.aspx.

  • Franklin, C., & Mewborn, D. (2006). The statistical education of PreK–12 teachers: A shared responsibility. In G. Burrill (Ed.), NCTM 2006 Yearbook: Thinking and reasoning with data and chance (pp. 335–344). Reston, VA: NCTM.

    Google Scholar 

  • Garfield, J., & Ben-Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistical Review, 75(3), 372–396.

    Article  Google Scholar 

  • Garfield, J., & Ben–Zvi, D. (2008). Developing students’ statistical reasoning: Connecting research and teaching practice. Berlin, Germany: Springer Science & Business Media.

  • Garfield, J., & Ahlgren, A. (1988). Difficulties in learning basic concepts in probability and statistics: Implications for research. Journal for research in Mathematics Education, 19(1), 44–63.

  • Groth, R. E. (2007). Toward a conceptualization of statistical knowledge for teaching. Journal for Research in Mathematics Education, 38(5), 427–437.

    Google Scholar 

  • Groth, R. E., & Bergner, J. A. (2006). Preservice elementary teachers’ conceptual and procedural knowledge of mean, median, and mode. Mathematical Thinking and Learning, 8(1), 37–63.

    Article  Google Scholar 

  • Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–333.

    Article  Google Scholar 

  • Hartz, S. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practice (Unpublished doctoral dissertation). University of Illinois at Urbana–Champaign.

  • Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnostics. Applied Psychological Measurement, 29(4), 262–277.

    Article  Google Scholar 

  • Henson, R., Roussos, L., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute–level discrimination indices. Applied Psychological Measurement, 32(4), 275–288.

    Article  Google Scholar 

  • Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log–linear models with latent variables. Psychometrika, 74(2), 191–210.

    Article  Google Scholar 

  • Hill, H., Rowan, B., & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371–406.

    Article  Google Scholar 

  • Hill, H. C., Schilling, S. G., & Ball, D. L. (2004). Developing measures of teachers’ mathematics knowledge for teaching. The Elementary School Journal, 105(1), 11–30.

    Article  Google Scholar 

  • Im, S., & Park, H. J. (2010). A comparison of US and Korean students’ mathematics skills using a cognitive diagnostic testing method: Linkage to instruction. Educational Research and Evaluation, 16(3), 287–301.

    Article  Google Scholar 

  • Jones, G. A. (2005). Exploring probability in school: Challenges for teaching and learning. New York, NY: Springer.

    Book  Google Scholar 

  • Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.

    Article  Google Scholar 

  • Leavy, A. M. (2010). The challenge of preparing preservice teachers to teach informal inferential reasoning. Statistics Education Research Journal, 9(1), 46–67.

    Google Scholar 

  • Lee, H. S., & Hollebrands, K. F. (2011). Characterising and developing teachers’ knowledge for teaching statistics with technology. In C. Batanero, G. Burrill, & C. Reading (Eds.), Teaching statistics in school mathematics-challenges for teaching and teacher education (pp. 359–369). Dordrecht, Netherlands: Springer.

  • Lee, Y. S., Park, Y. S., & Taylan, D. (2011). A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the US national sample using the TIMSS 2007. International Journal of Testing, 11(2), 144–177.

    Article  Google Scholar 

  • Li, K. Y., & Shen, S. M. (1992). Students’ weaknesses in statistical projects. Teaching Statistics, 14(1), 2–8.

    Article  Google Scholar 

  • Makar, K., & Rubin, A. (2009). A framework for thinking about informal statistical inference. Statistics Education Research Journal, 8(1), 82–105.

    Google Scholar 

  • Mathews, D., & Clark, J. (2003). Successful students’ conceptions of mean, standard deviation, and the central limit theorem. Unpublished paper. Retrieved July 4, 2018 from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.636.8870&rep=rep1&type=pdf.

  • Muthen, L. K., & Muthen, B. O. (1998–2011). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthen & Muthen.

  • O’Connell, A. A. (1999). Understanding the nature of errors in probability problem–solving. Educational Research and Evaluation, 5(1), 1–21.

    Article  Google Scholar 

  • Panackal, A. A., & Heft, C. S. (1978). Cloze technique and multiple choice technique: Reliability and validity. Educational and Psychological Measurement, 38(4), 917–932.

    Article  Google Scholar 

  • R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved January 3, 2018, from http://www.R-project.org/.

  • Ranjbaran, F., & Alavi, S. M. (2017). Developing a reading comprehension test for cognitive diagnostic assessment: A RUM analysis. Studies in Educational Evaluation, 55, 167–179.

    Article  Google Scholar 

  • Ravand, H., & Robitzsch, A. (2015). Cognitive diagnostic modeling using R. Practical Assessment, Research & Evaluation, 20(11), 1–12.

    Google Scholar 

  • Rupp, A., & Templin, J. (2008). Effects of Q–matrix misspecification on parameter estimates and misclassification rates in the DINA model. Educational and Psychological Measurement, 68(1), 78–98.

    Article  Google Scholar 

  • Rupp, A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford Press.

  • Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi–square test statistic. Psychometrika, 75(2), 243–248.

    Article  Google Scholar 

  • Sen, S., & Arican, M. (2015). A diagnostic comparison of Turkish and Korean students’ mathematics performances on the TIMSS 2011 assessment. Journal of Measurement and Evaluation in Education and Psychology, 6(2), 238–253.

  • Shaughnessy, J. M. (2007). Research on statistics learning and reasoning. In F. K. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 957–1009). Reston, VA: The National Council of Teachers of Mathematics.

    Google Scholar 

  • Stohl, H. (2005). Probability in teacher education and development. In G. A. Jones (Ed.), Exploring probability in school: Challenges for teaching and learning (pp. 345–366). Boston, MA: Springer.

    Chapter  Google Scholar 

  • Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational Statistics, 10(1), 55–73.

    Article  Google Scholar 

  • Templin, J. (2008). Test construction item discrimination. Lecture presented at the Diagnostic Modelling Seminar at the University of Georgia, Athens. Retrieved February 2, 2018, from https://jonathantemplin.com/files/dcm/ersh9800f08/ersh9800f08_lecture11.pdf.

  • Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30(2), 251–275.

    Article  Google Scholar 

  • Templin, J., & Henson, R. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287–305.

    Article  Google Scholar 

  • Toker, T., & Green, K. (2012). An application of cognitive diagnostic assessment on TIMMS–2007 8th grade mathematics items. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.

  • von Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report. Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Watson, J. M. (2006). Statistical literacy at school: Growth and goals. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Werner, C., & Schermelleh-Engel, K. (2010). Deciding between competing models: Chi–square difference tests. In Introduction to structural equation modeling with LISREL (pp. 1–3). Frankfurt, Germany: Goethe University.

  • Zawojewski, J. S., & Heckman, D. S. (1997). What do students know about data analysis, statistics, and probability? In P. A. Kenny & E. A. Silver (Eds.), Results from the sixth mathematics assessment of the National Assessment of Educational Progress (pp. 195–223). Reston, VA: National Council of Teachers of Mathematics.

    Google Scholar 

  • Zawojewski, J. S., & Shaughnessy, J. M. (2000). Mean and median: Are they really so easy? Mathematics Teaching in the Middle School, 5(7), 436–440.

    Google Scholar 

Download references

Acknowledgements

Parts of this study were presented at the 2017 European Educational Research Conference held in Copenhagen/Denmark.

Funding

The authors disclose receipt of the following financial support for the research, authorship, and publication of this study: This study was supported by the Ahi Evran University Scientific Research Projects Coordination Unit. Project Number EGT.A3.16.014.

Author information

Authors and Affiliations

  1. Department of Mathematics and Science Education, Kırşehir Ahi Evran University, Kırşehir, Turkey

    Muhammet Arican & Okan Kuzu

Authors
  1. Muhammet Arican
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Okan Kuzu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

This study was conducted in equal collaboration between the two authors.

Corresponding author

Correspondence to Muhammet Arican.

Ethics declarations

Conflict of Interest

The authors declare that there are no conflicts of interest.

Electronic Supplementary Material

ESM 1

(DOCX 151 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arican, M., Kuzu, O. Diagnosing Preservice Teachers’ Understanding of Statistics and Probability: Developing a Test for Cognitive Assessment. Int J of Sci and Math Educ 18, 771–790 (2020). https://doi.org/10.1007/s10763-019-09985-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10763-019-09985-0

Keywords

Search

Navigation