Statistics students’ identification of inferential model elements within contexts of their own invention

Abstract

Statistical thinking partially depends upon an iterative process by which essential features of a problem setting are identified and mapped onto an abstract model or archetype, and then translated back into the context of the original problem setting (Wild and Pfannkuch, Int Stat Rev 67(3):223–248, 1999). Assessment in introductory statistics often relies on tasks that present students with data in context and expects them to choose and describe an appropriate model. This study explores post-secondary student responses to an alternative task that prompts students to clearly identify a sample, population, statistic, and parameter using a context of their own invention. The data include free-text narrative responses of a random sample of 500 students from a sample of more than 1600 introductory statistics students. Results suggest that students’ responses often portrayed sample and population accurately. Portrayals of statistic and parameter were less reliable and were associated with descriptions of a wide variety of other concepts. Responses frequently attributed a variable of some kind to the statistic, or a study design detail to the parameter. Implications for instruction and research are discussed, including a call for emphasis on a modeling paradigm in introductory statistics.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Allmond, S., & Makar, K. (2010). Developing primary students’ ability to pose questions in statistical investigations. In C. Reading (Ed.), Data and context in statistics education: Towards an evidence-based society. Proceedings of the 8th international conference on teaching statistics. Voorburg: International Statistical Institute.

    Google Scholar 

  2. Beckman, M. D. (2015). Assessment of cognitive transfer outcomes for students of introductory statistics (Doctoral dissertation, University of Minnesota—Twin Cities). Retrieved from http://iase-web.org/documents/dissertations/15.MatthewBeckman.Dissertation.pdf. Accessed 1 Dec 2016.

  3. Ben-Zvi, D., & Garfield, J. (2004). Statistical literacy, reasoning, and thinking: Goals, definitions, and challenges. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 3–15). Dordrecht: Kluwer Academic Publishers.

    Chapter  Google Scholar 

  4. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school: Expanded edition. Washington, DC: National Academies.

    Google Scholar 

  5. Chance, B. (2002). Components of statistical thinking and implications for instruction and assessment. Journal of Statistics Education, 10(3). Retrieved from http://ww2.amstat.org/publications/jse/v10n3/chance.html. Accessed 1 Dec 2016.

  6. delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58.

    Google Scholar 

  7. Garfield, J., & Ben-Zvi, D. (2008). Developing studentsstatistical reasoning: Connecting research and teaching practice. Berlin: Springer Science & Business Media.

    Google Scholar 

  8. Graham, A. (2006). Developing thinking in statistics. London: Paul Chapman.

    Google Scholar 

  9. Haller, H., & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research, 7(1), 1–20.

    Google Scholar 

  10. Kaplan, J. J., Fisher, D. G., & Rogness, N. T. (2009). Lexical ambiguity in statistics: What do students know about the words association, average, confidence, random and spread? Journal of Statistics Education, 17(3). Retrieved from http://www.amstat.org/publications/jse/v17n3/kaplan.html. Accessed 17 Nov 2017.

  11. Kaplan, J. J., Fisher, D. G., & Rogness, N. T. (2010). Lexical ambiguity in statistics: How students use and define the words: Association, average, confidence, random and spread. Journal of Statistics Education, 18(2). Retrieved from http://www.amstat.org/publications/jse/v18n2/kaplan.pdf. Accessed 17 Nov 2017.

  12. Kaplan, J. J., & Rogness, N. (2018). Increasing statistical literacy by exploiting lexical ambiguity of technical terms. Numeracy, 18(1), 1–14.

    Google Scholar 

  13. Lavigne, N. C., & Lajoie, S. P. (2007). Statistical reasoning of middle school children engaging in survey inquiry. Contemporary Educational Psychology, 23(4), 630–666.

    Article  Google Scholar 

  14. Makar, K., & Rubin, A. (2009). A framework for thinking about informal statistical inference. Statistics Education Research Journal, 8(1), 82–105.

    Google Scholar 

  15. McCullagh, P. (2002). What is a statistical model? The Annals of Statistics, 30(5), 1225–1267.

    Article  Google Scholar 

  16. Meletiou-Mavrotheris, M., & Paparistodemou, E. (2015). Developing students’ reasoning about samples and sampling in the context of informal inferences. Educational Studies in Mathematics, 88(3), 385–404.

    Article  Google Scholar 

  17. Pfannkuch, M. (2006). Informal inferential reasoning. In A. Rossman & B. Chance (Eds.), Working cooperatively in statistics education. Proceedings of the 7th international conference on teaching statistics. Voorburg: International Statistical Institute.

    Google Scholar 

  18. Pfannkuch, M., Ben-Zvi, D., & Budgett, S. (2018). Innovations in statistical modeling to connect data, chance, and context. ZDM Mathematics Education. https://doi.org/10.1007/s11858-018-0989-2 (this issue).

    Article  Google Scholar 

  19. R Core Team. (2017). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. https://www.R-project.org/. Accessed 10 June 2017.

  20. Reed, S. K., Dempster, A., & Ettinger, M. (1985). Usefulness of analogous solutions for solving algebra word problems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11(1), 106–125.

    Google Scholar 

  21. Rossman, A. J., & Chance, B. L. (2001). Workshop statistics: Discovery with data (2nd ed.). Emeryville: Key College Publishing.

    Google Scholar 

  22. Singley, M. K., & Anderson, J. R. (1989). The transfer of cognitive skill. Cambridge: Harvard University Press.

    Google Scholar 

  23. Vallecillos, A. (1999). Some empirical evidence on learning difficulties about testing hypotheses. Bulletin of the International Statistical Institute: Bulletin of the 52nd Session of the International Statistical Institute, 58, 201–204.

    Google Scholar 

  24. Watson, J. M., & Kelly, B. A. (2005). Cognition and instruction: Reasoning about bias in sampling. Mathematics Education Research Journal, 17(1), 25–27.

    Article  Google Scholar 

  25. Watson, J. M., & Moritz, J. B. (2000). Development of understanding of sampling for statistical literacy. The Journal of Mathematical Behavior, 19(1), 109–136.

    Article  Google Scholar 

  26. Well, A. D., Pollatsek, A., & Boyce, S. J. (1990). Understanding the effects of sample size on the variability of the mean. Organizational Behavior and Human Decision Processes, 47, 289–312.

    Article  Google Scholar 

  27. Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67(3), 223–248.

    Article  Google Scholar 

  28. Williams, A. M. (1999). Novice students’ conceptual knowledge of statistical hypothesis testing. In J. M. Truran, & K. M. Truran (Eds.), Making the difference: Proceedings of the twenty-second annual conference of the mathematics education research group of Australasia (pp. 554–560). Adelaide: MERGA.

    Google Scholar 

Download references

Acknowledgements

The authors wish to express their sincere gratitude to Joan Garfield for her thoughtful direction and influence on development of early research leading to this work. The authors are also grateful for the thoughtful feedback and constructive suggestions of colleagues during the SRTL-10 forum.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Matthew D. Beckman.

Appendices

Appendix 1: Rossman–Chance (RC) task scoring guidance

The complete guidance document with remarks to accompany each prompt encountered while scoring the RC Task is available in the online supplement. A list of codes used and an accompanying description is reproduced here.

Scoring codes applied to the RC task

Code Description
“sample” (or “1”) The requested element (i.e. sample, population, statistic, parameter) was identified as the “sample” in the response
“population” (or “2”) The requested element (i.e. sample, population, statistic, parameter) was identified as the “population” in the response
“statistic” (or “3”) The requested element (i.e. sample, population, statistic, parameter) was identified as the “statistic” in the response
“parameter” (or “4”) The requested element (i.e. sample, population, statistic, parameter) was identified as the “parameter” in the response
“none” (or “0”) Student did not attempt to identify the requested element (i.e. sample, population, statistic, parameter) in the response or simply stated a definition
“full credit” Not used during coding; this designation was applied during analysis to clarify that the requested element was clearly and correctly identified in the response
“partial” Indicates that provided text is insufficient for full credit, but response shows evidence of (likely) understanding
“variable” Indicates that student appears to describe one or more variables in the study
“value” Is an assumed number (e.g. for statistic or parameter) without sufficient explanation
“conclusion” Indicates that student appears to draw a conclusion or state some result for the research question
“ill-defined” Indicates that student explicitly attributes something to the sample/population/statistic/parameter but it isn’t clear how it relates or what relevant role it plays
“subset” Indicates that student overtly describes some subset of the population/sample/etc that could not itself be used to address the research question (i.e. sample/population for an RQ about the proportion that say “yes” to a survey question can’t be restricted to a subset who say “yes” only)
“superset” Indicates that student overtly describes some superset of the population/sample/etc that does not directly address the research question (i.e. population of interest is “STAT 100 students” superset might be “university students”)
“unrelated” E.g. indicates that the student overtly specifies a sample/population that is not related to the research question, or other unrelated detail
“single obs” Student attributes one specific observation from the data to the sample/statistic/etc
“RQ” Student restates all or part of the research question
“study design detail” Describes some specific detail (e.g. duration, condition, selection criterion) of the study
“sample size” Student attributes the sample size to the target concept
“CI/HT” E.g., student claims the parameter is a confidence interval
“signif. threshold” Student explicitly attributes a significance threshold such as \(\alpha =0.05\)
“p-value” Student explicitly attributes a p-value
“null value” Some value hypothesized for a parameter of interest to be tested
“units” Unit of measurement (e.g. pounds)
“std. statistic” e.g., z-score, test statistic
“confounding var” Student describes a confounding variable that could impact the response described
“distribution” Student attributes a specific distribution (e.g. Normal distribution; t-distribution)

Appendix 2: Concepts attributed to the statistic and parameter among responses with full credit for both the sample and population

Figure 6 presents a cross-tabulation of codes for concepts attributed to the statistic and the parameter by 212 students who earned full credit for both the sample and population elements.

Fig. 6
figure6

Cross-tabulation of concepts attributed to the statistic and parameter among responses with full credit for both the sample and population (N = 212)

A total of 92 (43.4%) of the 212 students earned at least partial credit for identifying both the statistic and the parameter. Another 29 students (13.7%) earned at least partial credit for identifying the statistic but did not earn credit for the parameter. By contrast, nine students (4.2%) earned at least partial credit for identifying the parameter but did not earn credit for the statistic. Lastly, 82 students (38.7%) earned no credit for the statistic or parameter, despite earning full credit for both the sample and population. Figure 6 shows additional detail specifying recognizable concepts attributed to the statistic and parameter. Common errors attributed to the parameter included study design detail, a description of a variable, or the statistic. Again, the most common error attributed to the statistic was description of a variable.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Beckman, M.D., delMas, R. Statistics students’ identification of inferential model elements within contexts of their own invention. ZDM Mathematics Education 50, 1295–1309 (2018). https://doi.org/10.1007/s11858-018-0986-5

Download citation

Keywords

  • Statistics education
  • Statistical modeling
  • Statistical inference
  • Assessment
  • Parameter