“Probably true” says the expert: how two types of lexical hedges influence students’ evaluation of scientificness

Abstract

Contemporary school learning typically includes the processing of popular scientific information as found in journals, magazines, and/or the WWW. The German high school curriculum emphasizes that students should have achieved science literacy and have learned to evaluate the substance of text-based learning content by the end of high school. Alongside the content of science-related information, two issues are important when students gauge its substance: (a) information about its source, such as whether an expert has provided it (attribution shields) and (b) its wording, for example, whether so-called markers of tentativeness such as “probably” point to a preliminary assessment (plausibility shields). Based on the outcomes of a content analysis of the usage of such shields, we report an experiment that varied the occurrence of both types of shields in single arguments. Results showed effects of both manipulations on the perception and evaluation of arguments. However, information about the source impacted more strongly on the evaluation than the wording. We relate this finding to the formulation of as well as students’ processing of text-based learning content and suggest practical implications for teaching students how to handle scientific information.

Appendices

Appendix 1

Coding scheme for the content analysis of shields in popular scientific articles

Shield	Keywords and restrictions concerning some of them
Attribution shield	A lexical hedge that indicates where a statement comes from by attributing it to a source different from the speaker.
• References to an opinion, statement, etc. of a person or institution different from the speaker (mentioning the author of a theory or hypothesis without naming any content or statement does not suffice): e.g., according to experts, researchers report, Prof. Alberts assumes, the researcher explains
• Direct speech: e.g., “…,” said Prof. Alberts
• References to scientific research—need to be the source of a statement or results that are reported (names of researchers, methodology, or goals of studies do not suffice): e.g., according to current research, as recent studies show
Plausibility shield	A lexical hedge that expresses explicit or implicit doubts about a statement
• Main verbs such as: seem to, assume, suppose
• Modal auxiliaries: might, could, should; may, can, shall (only if it fits the context)
• Adverbs such as: probably, maybe, possibly, apparently, seemingly, supposedly, partially, mostly, almost (only if it fits the context)
• Adjectives such as: questionable, skeptical, controversial, ambiguous
• Sentence constructions such as: doubts arise/emerge

Appendix 2

Self-report measures for the description of the sample

What do you think: How much do you know about computer usage by elementary school children?	(from 1 [not at all] to 5 [very much])
How important do you think the topic of computer usage is in elementary school children?
The topic of computer usage by children is something that involves me personally.
I am interested in the topic computer usage by children.
How often do you have anything to do with elementary-school-age children?	(from 1 [several times per day] to 5 [never])

Appendix 3

Twelve arguments in the version with both plausibility shields and attribution shields (the other conditions result from leaving out the respective shield/s)

1
“If children spend too much time in front of a computer when they are too young, perhaps their language development will be delayed and, correspondingly, they will develop significant deficits in the linguistic area,” says Prof. Alberts, media educator at the University of Hamburg.
2
Researchers report that computer games presumably improve elementary school children’s ability to engage in spatial thinking and develop contextual understanding.
3
Recent research findings show that probably every hour that children spend in front of the screen increases their risk of becoming hyperactive, because they cannot satisfy their need for exercise.
4
Fifty studies have shown that computer-based teaching possibly improves learning effectiveness. Computer-based adaptive exercise programs adapt to the learner’s individual level and provide feedback and support.
5
According to experts, computer-based learning programs seem to improve elementary school children’s linguistic and mathematical skills.
6
“Particularly playing online games perhaps improves social skills. Cooperating with other players is part of the game experience. Moreover, players actively communicate and discuss computer games,” says Prof. Zillert, media psychologist at the University of Freiburg.
7
According to a recent study, computer games could cause stress and thereby have a negative effect on learning processes.
8
Researchers report that frequently playing games presumably leads the brain to specialize in fast and reflex-like reactions to the disadvantage of other skills such as complex problem solving.
9
A host of studies have shown that elementary school children possibly perform worse at school when they spend a lot of time playing computer games.
10
According to a recent study, computer games could have beneficial effects on the ability to concentrate.
11
Recent research findings show that computer-based learning programs are highly attractive for children and probably increase elementary school children’s motivation to learn.
12
According to experts, children who spend a lot of time playing computer games have fewer direct social contacts, and this seems to increase the risk of developing mental disorders and aggressive behavior.

Appendix 4

Dependent variables

Argument simplicity

Very easy

▫

▫

▫

▫

▫

▫

▫

Very complicated

Argument scientificness

Very unscientific

▫

▫

▫

▫

▫

▫

▫

Very scientific

Argument strength scale (Durik et al. 2008)

Very sound	▫	▫	▫	▫	▫	▫	▫	Very unsound
Very weak	▫	▫	▫	▫	▫	▫	▫	Very strong
Very poorly reasoned	▫	▫	▫	▫	▫	▫	▫	Very well reasoned
Very illogical	▫	▫	▫	▫	▫	▫	▫	Very logical

Argument credibility scale (Scharrer et al. 2013)

Very badly founded	▫	▫	▫	▫	▫	▫	▫	Very well founded
Very reliable	▫	▫	▫	▫	▫	▫	▫	Very unreliable
Very weak	▫	▫	▫	▫	▫	▫	▫	Very strong
Very incredible	▫	▫	▫	▫	▫	▫	▫	Very credible

Control variables

Very boring	▫	▫	▫	▫	▫	▫	▫	Very entertaining
Very convincing	▫	▫	▫	▫	▫	▫	▫	Very unconvincing