## Abstract

Statistical thinking partially depends upon an iterative process by which essential features of a problem setting are identified and mapped onto an abstract model or archetype, and then translated back into the context of the original problem setting (Wild and Pfannkuch, Int Stat Rev 67(3):223–248, 1999). Assessment in introductory statistics often relies on tasks that present students with data in context and expects them to choose and describe an appropriate model. This study explores post-secondary student responses to an alternative task that prompts students to clearly identify a sample, population, statistic, and parameter using a context of their own invention. The data include free-text narrative responses of a random sample of 500 students from a sample of more than 1600 introductory statistics students. Results suggest that students’ responses often portrayed sample and population accurately. Portrayals of statistic and parameter were less reliable and were associated with descriptions of a wide variety of other concepts. Responses frequently attributed a variable of some kind to the statistic, or a study design detail to the parameter. Implications for instruction and research are discussed, including a call for emphasis on a modeling paradigm in introductory statistics.

This is a preview of subscription content, access via your institution.

## References

Allmond, S., & Makar, K. (2010). Developing primary students’ ability to pose questions in statistical investigations. In C. Reading (Ed.),

*Data and context in statistics education: Towards an evidence*-*based society. Proceedings of the 8th international conference on teaching statistics*. Voorburg: International Statistical Institute.Beckman, M. D. (2015).

*Assessment of cognitive transfer outcomes for students of introductory statistics*(Doctoral dissertation, University of Minnesota—Twin Cities). Retrieved from http://iase-web.org/documents/dissertations/15.MatthewBeckman.Dissertation.pdf. Accessed 1 Dec 2016.Ben-Zvi, D., & Garfield, J. (2004). Statistical literacy, reasoning, and thinking: Goals, definitions, and challenges. In D. Ben-Zvi & J. Garfield (Eds.),

*The challenge of developing statistical literacy, reasoning and thinking*(pp. 3–15). Dordrecht: Kluwer Academic Publishers.Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000).

*How people learn: Brain, mind, experience, and school: Expanded edition*. Washington, DC: National Academies.Chance, B. (2002). Components of statistical thinking and implications for instruction and assessment.

*Journal of Statistics Education, 10*(3). Retrieved from http://ww2.amstat.org/publications/jse/v10n3/chance.html. Accessed 1 Dec 2016.delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics.

*Statistics Education Research Journal, 6*(2), 28–58.Garfield, J., & Ben-Zvi, D. (2008).

*Developing students*’*statistical reasoning: Connecting research and teaching practice*. Berlin: Springer Science & Business Media.Graham, A. (2006).

*Developing thinking in statistics*. London: Paul Chapman.Haller, H., & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers?

*Methods of Psychological Research, 7*(1), 1–20.Kaplan, J. J., Fisher, D. G., & Rogness, N. T. (2009). Lexical ambiguity in statistics: What do students know about the words association, average, confidence, random and spread?

*Journal of Statistics Education, 17*(3). Retrieved from http://www.amstat.org/publications/jse/v17n3/kaplan.html. Accessed 17 Nov 2017.Kaplan, J. J., Fisher, D. G., & Rogness, N. T. (2010). Lexical ambiguity in statistics: How students use and define the words: Association, average, confidence, random and spread.

*Journal of Statistics Education, 18*(2). Retrieved from http://www.amstat.org/publications/jse/v18n2/kaplan.pdf. Accessed 17 Nov 2017.Kaplan, J. J., & Rogness, N. (2018). Increasing statistical literacy by exploiting lexical ambiguity of technical terms.

*Numeracy, 18*(1), 1–14.Lavigne, N. C., & Lajoie, S. P. (2007). Statistical reasoning of middle school children engaging in survey inquiry.

*Contemporary Educational Psychology, 23*(4), 630–666.Makar, K., & Rubin, A. (2009). A framework for thinking about informal statistical inference.

*Statistics Education Research Journal, 8*(1), 82–105.McCullagh, P. (2002). What is a statistical model?

*The Annals of Statistics*,*30*(5), 1225–1267.Meletiou-Mavrotheris, M., & Paparistodemou, E. (2015). Developing students’ reasoning about samples and sampling in the context of informal inferences.

*Educational Studies in Mathematics, 88*(3), 385–404.Pfannkuch, M. (2006). Informal inferential reasoning. In A. Rossman & B. Chance (Eds.),

*Working cooperatively in statistics education. Proceedings of the 7th international conference on teaching statistics*. Voorburg: International Statistical Institute.Pfannkuch, M., Ben-Zvi, D., & Budgett, S. (2018). Innovations in statistical modeling to connect data, chance, and context.

*ZDM Mathematics Education*. https://doi.org/10.1007/s11858-018-0989-2**(this issue)**.R Core Team. (2017).

*R: A language and environment for statistical computing*. Vienna: R Foundation for Statistical Computing. https://www.R-project.org/. Accessed 10 June 2017.Reed, S. K., Dempster, A., & Ettinger, M. (1985). Usefulness of analogous solutions for solving algebra word problems.

*Journal of Experimental Psychology: Learning, Memory, and Cognition, 11*(1), 106–125.Rossman, A. J., & Chance, B. L. (2001).

*Workshop statistics: Discovery with data*(2nd ed.). Emeryville: Key College Publishing.Singley, M. K., & Anderson, J. R. (1989).

*The transfer of cognitive skill*. Cambridge: Harvard University Press.Vallecillos, A. (1999). Some empirical evidence on learning difficulties about testing hypotheses.

*Bulletin of the International Statistical Institute: Bulletin of the 52nd Session of the International Statistical Institute, 58*, 201–204.Watson, J. M., & Kelly, B. A. (2005). Cognition and instruction: Reasoning about bias in sampling.

*Mathematics Education Research Journal, 17*(1), 25–27.Watson, J. M., & Moritz, J. B. (2000). Development of understanding of sampling for statistical literacy.

*The Journal of Mathematical Behavior, 19*(1), 109–136.Well, A. D., Pollatsek, A., & Boyce, S. J. (1990). Understanding the effects of sample size on the variability of the mean.

*Organizational Behavior and Human Decision Processes, 47*, 289–312.Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry.

*International Statistical Review, 67*(3), 223–248.Williams, A. M. (1999). Novice students’ conceptual knowledge of statistical hypothesis testing. In J. M. Truran, & K. M. Truran (Eds.),

*Making the difference: Proceedings of the twenty*-*second annual conference of the mathematics education research group of Australasia*(pp. 554–560). Adelaide: MERGA.

## Acknowledgements

The authors wish to express their sincere gratitude to Joan Garfield for her thoughtful direction and influence on development of early research leading to this work. The authors are also grateful for the thoughtful feedback and constructive suggestions of colleagues during the SRTL-10 forum.

## Author information

### Authors and Affiliations

### Corresponding author

## Appendices

### Appendix 1: Rossman–Chance (RC) task scoring guidance

The complete guidance document with remarks to accompany each prompt encountered while scoring the RC Task is available in the online supplement. A list of codes used and an accompanying description is reproduced here.

### Scoring codes applied to the RC task

Code | Description |
---|---|

“sample” (or “1”) | The requested element (i.e. sample, population, statistic, parameter) was identified as the “sample” in the response |

“population” (or “2”) | The requested element (i.e. sample, population, statistic, parameter) was identified as the “population” in the response |

“statistic” (or “3”) | The requested element (i.e. sample, population, statistic, parameter) was identified as the “statistic” in the response |

“parameter” (or “4”) | The requested element (i.e. sample, population, statistic, parameter) was identified as the “parameter” in the response |

“none” (or “0”) | Student did not attempt to identify the requested element (i.e. sample, population, statistic, parameter) in the response or simply stated a definition |

“full credit” | Not used during coding; this designation was applied during analysis to clarify that the requested element was clearly and correctly identified in the response |

“partial” | Indicates that provided text is insufficient for full credit, but response shows evidence of (likely) understanding |

“variable” | Indicates that student appears to describe one or more variables in the study |

“value” | Is an assumed number (e.g. for statistic or parameter) without sufficient explanation |

“conclusion” | Indicates that student appears to draw a conclusion or state some result for the research question |

“ill-defined” | Indicates that student explicitly attributes something to the sample/population/statistic/parameter but it isn’t clear how it relates or what relevant role it plays |

“subset” | Indicates that student overtly describes some subset of the population/sample/etc that could not itself be used to address the research question (i.e. sample/population for an RQ about the proportion that say “yes” to a survey question can’t be restricted to a subset who say “yes” only) |

“superset” | Indicates that student overtly describes some superset of the population/sample/etc that does not directly address the research question (i.e. population of interest is “STAT 100 students” superset might be “university students”) |

“unrelated” | E.g. indicates that the student overtly specifies a sample/population that is not related to the research question, or other unrelated detail |

“single obs” | Student attributes one specific observation from the data to the sample/statistic/etc |

“RQ” | Student restates all or part of the research question |

“study design detail” | Describes some specific detail (e.g. duration, condition, selection criterion) of the study |

“sample size” | Student attributes the sample size to the target concept |

“CI/HT” | E.g., student claims the parameter is a confidence interval |

“signif. threshold” | Student explicitly attributes a significance threshold such as \(\alpha =0.05\) |

“p-value” | Student explicitly attributes a p-value |

“null value” | Some value hypothesized for a parameter of interest to be tested |

“units” | Unit of measurement (e.g. pounds) |

“std. statistic” | e.g., z-score, test statistic |

“confounding var” | Student describes a confounding variable that could impact the response described |

“distribution” | Student attributes a specific distribution (e.g. Normal distribution; t-distribution) |

### Appendix 2: Concepts attributed to the statistic and parameter among responses with full credit for both the sample and population

Figure 6 presents a cross-tabulation of codes for concepts attributed to the statistic and the parameter by 212 students who earned full credit for both the sample and population elements.

A total of 92 (43.4%) of the 212 students earned at least partial credit for identifying both the statistic and the parameter. Another 29 students (13.7%) earned at least partial credit for identifying the statistic but did not earn credit for the parameter. By contrast, nine students (4.2%) earned at least partial credit for identifying the parameter but did not earn credit for the statistic. Lastly, 82 students (38.7%) earned no credit for the statistic or parameter, despite earning full credit for both the sample and population. Figure 6 shows additional detail specifying recognizable concepts attributed to the statistic and parameter. Common errors attributed to the parameter included study design detail, a description of a variable, or the statistic. Again, the most common error attributed to the statistic was description of a variable.

## Rights and permissions

## About this article

### Cite this article

Beckman, M.D., delMas, R. Statistics students’ identification of inferential model elements within contexts of their own invention.
*ZDM Mathematics Education* **50, **1295–1309 (2018). https://doi.org/10.1007/s11858-018-0986-5

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s11858-018-0986-5

### Keywords

- Statistics education
- Statistical modeling
- Statistical inference
- Assessment
- Parameter