The double curse of misconceptions: misconceptions impair not only text comprehension but also metacomprehension in the domain of statistics

Nothing is more dangerous for a new truth than an old misconception.

—Johann Wolfgang von Goethe, Wilhelm Meister’s Journeyman Years, 1829

Abstract

Research shows that misconceptions are usually detrimental to text comprehension. However, whether misconceptions also impair metacomprehension accuracy, that is, the accuracy with which one self-assesses one’s text comprehension, has received far less attention. We conducted a study in which we examined students’ (N = 47) comprehension and metacomprehension accuracy (prediction accuracy and postdiction accuracy) of a statistics text as a function of their statistical misconceptions. Text comprehension and metacomprehension accuracy referred to both conceptual and procedural aspects of statistics. The results showed that students who had more misconceptions achieved poorer conceptual text comprehension and, at the same time, provided more overconfident predictions of their conceptual and procedural text comprehension than students who had fewer misconceptions. In contrast, postdiction accuracy of conceptual and procedural text comprehension was not affected by misconceptions.

Appendices

Bortz and Schuster (2010, pp. 153–156). Translated from German and adapted with permission.

Appendix 1: Text

Bortz and Schuster (2010, pp. 153–156). Translated from German and adapted with permission.

Covariance

Covariance is a measure of association that is used to describe linear relationships. A relationship between two variables is linear if it can be represented by a fit line. If the pairs of observations on two variables are drawn as points in a scatter plot with the axes representing the x- and y-variables, the points would have to lie exactly on a line when there is a linear relationship. However, for nearly all variable relationships that are of interest in research, this requirement is not met. Nevertheless, the line is a useful model to describe the linear relationship between two variables. For example, the trend between body weight x and body height y for a sample of people could be linearly described even if not all points lie exactly on a line. Thereby, the height of the covariance reflects how strongly the points spread around the hypothetical line. If the absolute covariance value is high, this means that the points lie close to the line. The slope of the line, however, depends on the measuring units in which the x- and y-variables are represented in the scatter plot.

One receives a high positive covariance if frequently an above-average value in variable x corresponds to an above-average value in variable y and, accordingly, if frequently a below-average value in variable x corresponds to a below-average value in variable y (Fig. A).

Fig. A

Graphical illustration of positive covariance

There will be a high negative covariance if frequently an above-average value in variable x corresponds to a below-average value in variable y and, accordingly, if frequently a below-average value in variable x corresponds to an above-average value in variable y (Fig. B).

Fig. B

Graphical illustration of negative covariance

If there is a covariance of zero between two variables, for above-average values in variable x there will be above-average values in variable y as well as below-average values in variable y and vice versa. Therefore, a covariance of zero indicates that there is no linear relationship between two variables. In this case, however, the variables can still be related in a nonlinear manner. Conceivable relationships are, for example, exponential, logarithmic, S-shaped, or U-shaped relationships (Fig. C).

Fig. C

Graphical illustration of zero covariance

The covariance between two variables can be calculated with the following formula:

$$s_{xy} = \frac{1}{N}\mathop \sum \limits_{i = 1}^{N} (x_{i} - \bar{x}) \cdot (y_{i} - \bar{y})$$

Every studied object produces a pair of observation (x, y), whereby x and y can lie more or less far away from the respective sample mean. If both values lie well above or well below the mean, there is a high positive product of deviation $(x_{i} - \bar{x}) \cdot (y_{i} - \bar{y}).$ If the values do not deviate much, the product of deviation will be smaller. The sum of the products of deviation of all objects is therefore a measure of the degree of “covariation” of the series of pairs of observation. To account for the number of objects that are included in this sum, the sum is divided by the sample size N.

Therefore, covariance indicates if two variables vary together. Concerning the question if two variables are causally related, covariance does not provide statistical evidence. However, covariance can be used to get a first hint towards a potential cause-and-effect relationship.

Moreover, as a nonstandardized statistic, covariance is dependent on the measuring units of the underlying variables. Theoretically, covariance values can reach from negative infinity to positive infinity. For example, if in two studies the variables body weight and/or body height are measured in different units (e.g., weight: kg, g; height: m, cm), the studies will yield covariances between these two variables that are not directly comparable.

Appendix 2: Misconceptions test

Parts added for clarity appear in italics. These parts were not provided to the participants in the study.

Appendix 3: Misconceptions about covariance and respective references

Misconception	References
M1 Covariance enables definite predictions	Batanero et al. (1996, 1997, 1998), Estepa and Batanero (1996), Estepa et al. (1999), Liu (2010), Liu and Lin (2008), and Liu et al. (2010)
M2 Negative covariance indicates the absence of a relationship	Batanero et al. (1996, 1997, 1998), Liu and Lin (2008), Mevarech and Kramarsky (1997), and Morris (1998, 1999, 2001, 2004)
M3 Covariance is related to the slope of the fit line	Liu (2010), Liu and Lin (2008), and Liu et al. (2009, 2010)
M4 A straight fit line without deviation invariably indicates perfect covariance	Liu (2010), Liu and Lin (2008), and Liu et al. (2009)
M5 Covariance implies causality	Estepa and Sánchez (2001), Garfield (2003), Liu and Lin (2008), Liu et al. (2009), Morris (1998, 1999, 2001), and Sundre (2003)
M6 Positive covariance is always stronger than negative covariance	Estepa and Sánchez (2001), Liu (2010), Liu and Lin (2008), Liu et al. (2009, 2010), and Morris (1998, 1999, 2001)
M7 Covariance can be interpreted from isolated points in scatter plots	Batanero et al. (1996, 1997, 1998), Estepa and Batanero (1996), Estepa et al. (1999), Mevarech and Kramarsky (1997), and Moritz (2004)
M8 Zero covariance indicates the absence of any association	Estepa and Sánchez (2001), Liu (2010), Liu and Lin (2008), and Liu et al. (2009, 2010)
M9 Only consider covariance if attributable to a causal relationship	Batanero et al. (1997, 1998) and Estepa and Batanero (1996)
M10 Only a straight fit line without deviation indicates covariance	Liu and Lin (2008) and Liu et al. (2009)
M11 Positive and zero covariance are always stronger than negative covariance	Liu et al. (2009) and Morris (1999)
M12 Covariance is a standardized statistic	Batanero et al. (1997, 1998), Estepa and Sánchez (2001), Liu (2010), Liu and Lin (2008), and Liu et al. (2010)
M13 Covariance changes if the variables are reversed	Estepa and Sánchez (2001), Liu (2010), Liu and Lin (2008), and Liu et al. (2010)
M14 More negative covariance indicates a weaker relationship	Batanero et al. (1997, 1998) and Liu et al. (2009)
M15 The fit line of perfect covariance must have a constant slope of 45°	Liu and Lin (2008) and Liu et al. (2009)

Appendix 4: Conceptual comprehension questions

Parts added for clarity appear in italics. These parts were not provided to the participants in the study.

Appendix 5: Procedural comprehension questions

Appendix 6: Results on metacomprehension accuracy in terms of absolute accuracy

Misconceptions and prediction accuracy

As can be seen in Table 8, the linear regression analyses revealed that misconceptions neither significantly influenced the prediction absolute accuracy of conceptual comprehension nor the prediction absolute accuracy of procedural comprehension. Hence, in contrast to bias, the absolute accuracy with which participants predicted their conceptual and procedural text comprehension was not affected by their misconceptions. This is because the range of bias was made up of negative and positive values. More precisely, participants with a lower number of misconceptions were rather underconfident, whereas participants with a higher number of misconceptions tended to be overconfident. Thus, while there was an effect of misconceptions on the direction of the deviations between predicted and actual comprehension (bias), there was no effect of misconceptions when only looking at the magnitude of the deviations (absolute accuracy).

Table 8 Results of the regression analyses for prediction absolute accuracies

Misconceptions and postdiction accuracy

As represented in Table 9, the linear regression analyses showed that misconceptions neither significantly influenced the postdiction absolute accuracy of conceptual comprehension nor the postdiction absolute accuracy of procedural comprehension.

Table 9 Results of the regression analyses for postdiction absolute accuracies