Climatic Change

, Volume 127, Issue 3, pp 419–434

Evaluation of a national high school entertainment education program: The Alliance for Climate Education

  • June A. Flora
  • Melissa Saphir
  • Matt Lappé
  • Connie Roser-Renouf
  • Edward W. Maibach
  • Anthony A. Leiserowitz
Open AccessArticle

DOI: 10.1007/s10584-014-1274-1

Cite this article as:
Flora, J.A., Saphir, M., Lappé, M. et al. Climatic Change (2014) 127: 419. doi:10.1007/s10584-014-1274-1

Abstract

Ever-increasing global warming has created a societal imperative to reach and engage youth, whose futures are at risk. In this paper, we evaluate the climate science knowledge, beliefs, attitudes, behavior and communication impact of an entertainment-education high school assembly program in a random sample of 49 schools (from population of 779 that received the intervention) and a panel of 1,241 students. Pre- and post-assembly surveys composed of questions from the Global Warming’s Six Americas segmentation and intervention-specific measures were administered in classrooms. We demonstrate that exposure to climate science in an engaging edutainment format changes youths’ knowledge, beliefs, involvement, and behavior positively and moves them to audience segments that are more engaged in the issue. The net impact of scaled, multi-sensory, captivating programs for youth could be a population shift in science-informed engagement in the issue of climate change. In addition, such programs can inspire youth for deeper engagement in school programs, personal action, and political and consumer advocacy.

1 Introduction

“…(K)nowing…the opinions of 20-year-olds matters, as they are the generation that is inheriting our legacy” (Schneider 2009, p. 321)

Nearly a quarter of the U.S. population is under 18 years of age.1 These young people have an important role to play in addressing the issues of our planet, of which climate change is paramount. Youth have enthusiasm, imagination and dynamism for undertaking local actions such as advocating climate-friendly behavior within their families; act as powerful communicators in school, religious, and extracurricular communities; and increasingly are able to be involved in global arenas via social media use. Today’s youth are immersed in a social and media environment that encourages and amplifies their individual and collective voices for social change.

Climate change is at the forefront of issues threatening the future of today’s youth. The potential direct and indirect threats of climate change have been reviewed extensively (IPCC 2014). In addition to more frequent and severe weather events, which disrupt millions of lives and harm local, regional, and national economies, direct effects of climate change include species loss, ocean acidification, rising sea levels, and more. Potential indirect effects—for which data are less available and uncertainties are greater—include mental health consequences, population dislocation, and ecosystem alteration. Recent extreme weather events serve as reminders of just one way that climate change could upend youths’ communities, lifestyles, and economic prospects overnight. If youth are to protect their future prospects from climate change, they must be empowered with knowledge of climate science, motivation to mitigate climate change, and confidence to take climate-friendly action at the individual and community levels.

However, findings regarding climate knowledge, motivation, and behavior from two recent nationally representative surveys indicate that youth are anything but empowered to act on this issue, and even less knowledgeable and motivated than adults. On a test of climate science knowledge, 54 % of 13–to 17-year-olds nationwide received a failing grade, compared to 30 % of adults. More teens than adults understand that global warming is caused mostly by human activities and that carbon dioxide traps heat from the Earth’s surface, but nevertheless fewer teens than adults earn a passing (A, B, or C) grade overall (25 % of teens versus 46 % of adults) (Leiserowitz et al. 2011). Similarly, another nationally representative survey reported that youth hold weaker intentions to perform climate-friendly behaviors and take fewer climate-friendly actions than their parents (Mead et al. 2012).

1.1 Global warming audience segmentation

There is room for improvement in all areas of empowering youths to mitigate climate change—increasing their knowledge about the causes of climate change, strengthening their beliefs about its certainty, and improving their confidence in performing climate-friendly personal actions and political advocacy. A widely used social marketing strategy aimed at advancing understanding of audience beliefs, attitudes, and behaviors is audience segmentation (Slater and Flora 1991; Slater 1996). A psychographic segmentation protocol specific to the issue of climate change has been developed, called Global Warming’s Six Americas (Maibach et al. 2011a, b). The segmentation is based on questions about beliefs, issue involvement, and political and consumer advocacy behaviors that divide respondents into six audience segments. The Alarmed segment contains the individuals who are the most certain global warming is happening, worried about it, and looking for actions to mitigate it; the Dismissive segment is the most certain that global warming is not happening, unconcerned about it, and opposing of action; and the four groups in between, the Concerned, Cautious, Disengaged, and Doubtful segments, vary in their levels of certainty, involvement, and behavior.

Another way to think about the Six Americas segments is in terms of variation in climate change issue engagement. Engagement as defined by Lorenzoni and colleagues (2007) is a state of connection with the issue of climate change that concurrently encompasses cognitive, affective, and behavioral aspects. It is not enough for people to know about climate change in order to be engaged; they also need to be motivated and able to take (or already taking) personal or political actions. The beliefs, attitudes, and behaviors measured by the Global Warming’s Six Americas questions are central to climate change issue engagement. Thus, the Alarmed can be seen as the most climate-engaged segment, ranging in an ordered way down to the least climate-engaged Dismissive segment.

The Six Americas segmentation protocol has already been used in experiments to examine whether the framing of climate change messages produces different effects among adult members of the six audience segments. Myers and colleagues (2012) found that framing climate change as a public health problem elicited the most emotionally supportive reactions across all six segments. In contrast, framing climate change as an issue of national security “boomeranged” among the Doubtful and Dismissive segments, eliciting unexpected anger and making these groups less likely to support climate change mitigation and adaptation. Similarly, college freshmen exposed to a cognitive-factual lecture and discussion actually increased their skepticism about climate change and tended to shift into the Cautious and Disengaged segments.2 These results point out the need for examination of emotional (both positive and negative) presentations of information. For example, Slater and Rouner (2002) argue that narratives (a form of entertainment) support shifts in engagement by suppressing counterarguing. The present study used the Six Americas questionnaire both as a dependent measure of climate science engagement and as a segmentation tool to investigate differential effects of a climate science entertainment-education presentation among high school students.

1.2 Climate change and environmental high school programs

Reaching and motivating large numbers high school students poses multiple individual and structural barriers. Because youth consume information from so many different media (Roberts et al. 2005), schools are one of the few places where a single program can reach a large enough youth audience, yet it is challenging to achieve access to significant time in today’s classrooms. The climate science presentation that served as the stimulus in this study was developed by the Alliance for Climate Education (ACE). This organization’s intensive outreach to schools has allowed it to present a 1-h educational and motivational program on climate change to over 1.5 million high school students in 4 years. The ACE program is an entertaining, multi-sensory, engaging school assembly available free of charge to schools across the U.S. In this paper, we examine the extent to which the 1-h ACE program influenced attendees’ climate knowledge, attitudes, and behavioral intentions, and whether this influence varied by Six Americas audience segment.

While many evaluations of environmental education programs have been published in the past two decades, we know of none that have reported on such a brief (i.e., a single hour) in-school program with high school students. In a meta-analysis of 18 environmental education programs, Zelezny (1999) reported average effect sizes of r =0.65 for programs conducted in classrooms and r = 0.27 for programs in non-traditional settings. However, the only reviewed study that used high school aged youth was conducted in a non-traditional setting. Osbaldiston and Schmitz (2011) evaluated a 2.5-h energy conservation program conducted in ninth-grade classrooms and also at home. This program included one class session imparting energy-related knowledge, a homework activity, and a second class session in which students participated in an energy game. In a 4-week follow-up, there were statistically significant increases in students’ knowledge, motivation, and self-reported behavior. Other evaluations of high school programs are largely qualitative case studies, such as Schelly and colleagues’ (2012) effort to create a school culture conducive to environmental conservation through a combination of school governance and policy, facilities changes, and student action.

1.3 Entertainment and education

Intertwining entertainment and education has been referred to as “entertainment-education” (E-E), “enter-education”, “infotainment”, and “edutainment”. The E-E strategy involves incorporating an educational message about a social issue into popular entertainment content in order to raise awareness, increase knowledge, create favorable attitudes, convey skills and ultimately motivate viewers/participants to take socially responsible action (Singhal and Rogers 1999; Singhal et al. 2003). E-E campaigns have become much more sophisticated than the early programs using radio, folk media or television to promote family planning in developing countries. A general assumption underlying today’s E-E programs is that entertainment is a particularly effective strategy for reaching young people. Modern E-E programs include entertainment radio and TV series (e.g., television drama ER) (Brody et al. 2001), popular celebrity-disseminated music (e.g., Johns Hopkins University’s music campaign in the Philippines, two songs “That Situation” and “I still Believe” became number one on the popular music charts) (Coleman and Meyer 1990), social messages embedded in existing entertainment media (e.g., Harvard School of Public Health’s Center for Health Communication “Squash It” campaign), and social games and other highly engaging mobile and online applications (Reeves and Read 2009).

Most research on E-E is guided by persuasion theory such as elaboration likelihood (Petty and Caciappo 1986), theory of planned behavior (Fishbein and Ajzen 1975) and social cognitive theory (Bandura 1985). This study incorporated measures from each of these theoretical perspectives, including knowledge of facts presented in the ACE assembly, belief that climate change is happening, perceived self-efficacy for taking climate-friendly action, intentions to change climate-related behavior, communication with others about the issue, and self-reported individual actions to mitigate climate change.

Although the ACE climate science assembly is conducted in schools, it qualifies as entertainment-education because it incorporates many features of E-E. The ACE assembly is led by well-trained young adult educators, who present climate science in a fast-paced performance. There is engaging background music and a backdrop of animation and compelling graphics that tell a story about global warming, its potential catastrophic effects on the environment, and the power that youth have to address this issue, just as young people have been at the forefront of other important social changes (e.g., the civil rights and women’s rights movements).3 This multi-sensory experience is closed by reviewing actions students can take, including individual conservation behaviors, talking to friends and family about climate change, and joining an ACE climate club to undertake a conservation project in the school community.

1.4 Research questions

The present study used evaluation data from a panel of students participating in the ACE edutainment assembly on climate science to investigate three research questions.
  1. RQ1
    Does attending the ACE assembly inspire students to mitigate climate change by
    1. (a)

      improving their knowledge of climate science,

       
    2. (b)

      heightening positive engagement in the climate change issue, and

       
    3. (c)

      eliciting short-term behavior change?

       
     
  2. RQ2

    Does participation in the ACE assembly change the distribution of students in the Global Warming’s Six Americas audience segmentation scheme?

     
  3. RQ3

    Are student members of the Global Warming’s Six Americas audience segments differentially affected by the ACE assembly?

     

2 Methods

2.1 Climate science assembly

ACE delivered a standardized, 50-min entertainment-education presentation on climate science to 779 high schools during the 2011–2012 school year. The content was based on the latest scientific sources (e.g., IPCC 2007) and followed the Next Generation Science Standards (NGSS Lead States 2013). Presentations took place in assemblies held in gymnasiums or auditoriums. Presenters were full-time ACE staff who memorized the script and passed a test of climate knowledge to ensure they could answer follow-up questions.4

The assembly was presented at a diverse set of schools in 13 regions, of which six participated in the evaluation—Chicago, Colorado, Los Angeles, New York, North Carolina, and Northern California. As shown in Table 1, 45 % of participants’ schools are eligible for Title I funding, and almost a third of students (29 %) at participants’ schools are eligible for reduced-cost meals at school. On average, just over half (53 %) of the students at participants’ schools identify as ethnically white.
Table 1

Description of the student sample (n = 1,241)

 

Mean

School-level descriptors

 Enrollmenta

1,597

 Attending a school eligible for title I funding

45 %

 Percent of student body receiving free and reduced lunch

29 %

 Percent of student body who identify as white

53 %

 Percent in schools with multi-year ACE programs

40 %

Individual-level descriptors

 Gender (% male)

47 %

 Grade level

 

  Freshman (ninth grade)

36 %

  Sophomore (tenth grade)

25 %

  Junior (eleventh grade)

20 %

  Senior (twelfth grade)

18 %

aStandard deviation of enrollment = 836

2.2 Student sample

Students at all high school grade levels (freshmen through seniors) participated (see Table 1). The classes from which participating students were drawn (e.g., sciences, language arts) varied. Just under half (47 %) of the sample were boys. Average self-reported grades were 3.20 (SD = 0.78), between “mostly A’s” and “mostly B’s” on a 1–4 scale in which 1 was “mostly D’s” and 4 was “mostly A’s”.

2.3 Survey administration

Identical pre- and post-assembly surveys were administered to 2,847 students at 49 schools, selected randomly from the schools recruited for the ACE presentation during the 2011–2012 school year. On average, pre-surveys were administered 2.4 days (SD = 2.4, median = 1) before the assembly, 2.5 days (SD = 2.4, median = 1, range 0–19) after the assembly, and there was an average of 5 days (SD = 3.6, median = 4, range 1–20) between the pre- and post-surveys. The longitudinal analyses presented here used only the panel of 1,241 students with matched pre- and post-assembly surveys, allowing change to be calculated at the level of individuals.

2.4 Assembly outcomes

See Supplementary Materials for the wording of outcome measures.

Knowledge

The questionnaire included a ten-item multiple-choice test of climate science knowledge (Cronbach’s α = 0.74). Students were asked to select the correct definition of the greenhouse effect, to determine whether the association between carbon dioxide in the atmosphere and the Earth’s average temperature is positive, negative, or non-existent, and to identify greenhouse gases, fossil fuels, likely effects of global warming, a source of methane, the main gas generated by burning fossil fuels, and a natural phenomenon for removing carbon dioxide from the atmosphere. In addition, students were asked to compare the United States and other countries in terms of natural resource usage, and between present and past levels of atmospheric carbon dioxide.

Positive engagement

Four aspects of positive engagement were measured. First, the Global Warmings Six Americas 15-item brief screener measured belief in climate change, involvement in the issue, and preferred societal responses. An averaged index of these (α = 0.88) was used as a dependent variable for answering Research Question 1; for Research Question 2, see segment analysis procedures in Analysis section below. Second, recognition of scientific agreement was a single item, in which students were asked which of two statements came closest to their own view. Students who chose “Most scientists think global warming is happening” received a score of 1. All other responses (“Most scientists think global warming is not happening,” “There is disagreement among scientists,” “Don’t know,” and no answer) were scored 0. Third, two items measured self-efficacy for starting a conservation project at school and for talking to others about climate change (α = 0.68). Fourth, three items measured behavioral intentions—to reduce one’s own carbon footprint, to talk to friends about reducing their footprint, and to talk to parents about the same (α = 0.89).

Behavior

Students were asked, “Yesterday, did you [do this behavior].” For each behavior, response options were “Yes” (coded 1), “No” (0), and “I don’t know” (0). Two communication behaviors were measured—talking to friends and talking to parents about climate change. Five conservation behaviors were measured—carrying a reusable water bottle to school, unplugging gadgets and electronics, taking showers shorter than 5 min, turning off lights when leaving a room, and recycling.

Post-assembly internal consistency

All multi-item indices exhibited the same or greater internal consistency (a measure of reliability) at posttest—for knowledge, α = 0.81; for the Global Warming’s Six Americas index, α = 0.89; for self-efficacy, α = 0.73; and for behavioral intentions, α = 0.91.

2.5 Analysis

To answer Research Question 1, paired t-tests were used to compare pre- and post-assembly means for outcome measures in the panel as a whole. Due to the large number of tests, only differences significant at the p < .01 level are reported. Effect size (r) is reported to aid interpretation of the importance of the findings (Rosenthal 1991).

To answer Research Question 2, we followed the methodology developed by Maibach and colleagues (2011b) to categorize students into one of six audience segments according to their pre-assembly responses. This entailed the use of discriminant functions derived from the original latent class analysis that first identified the audience segments; these discriminant functions have been shown to replicate the original latent class analysis results for 91% of respondents (Maibach et al. 2011a). We then categorized students again, using their post-assembly responses, and examined the differences between students’ pre- and post-assembly segment assignments.

To answer Research Question 3, paired t-tests were used to compare pre- and post-assembly means for all outcome measures except the Global Warming’s Six Americas index. These paired t-tests were conducted within the audience segments defined by students’ pre-assembly responses to the 15 Six Americas questions. Analysis of variance was used to test whether the changes in outcomes differed among the six segments.

Stata version 12 was used for the analyses (StataCorp 2011).

3 Results

3.1 Research question 1

As shown in Table 2, students’ knowledge of climate science, positive engagement in the issue of climate change, and all short-term behaviors (except carrying a reusable water bottle) increased significantly after attending the ACE assembly. Effect sizes were largest for knowledge of climate science (r = 0.50) and for two of the measures of positive engagement in the climate change issue—beliefs measured by the Global Warming’s Six Americas index and self-efficacy for explaining global warming to others or starting a conservation project at school (r = 0.49 and 0.42, respectively). Before the assembly, students on average marked correct answers for just over half (mean = 5.13, SD = 2.49) of the ten knowledge questions. After the assembly, students on average answered 6.52 questions correctly (SD = 2.76, t(1,240) = 20.49, p <0.001). The Global Warming’s Six Americas index increased over a quarter of a point on a response scale ranging from 1 to 5 (t(1,240) = 19.97, p < 0.001). Self-efficacy increased from just under the mid-point of the ten-point scale to just over the midpoint (4.30 to 5.19, t(1,240) = 16.02, p < 0.001).
Table 2

Differences from pre- to post-assembly in climate-related outcomes (n = 1,241)

 

Pre Mean (SD)

Post Mean (SD)

Difference Mean (SD)

t  (df=1,240)

Effect size (r)

Knowledgea

5.13 (2.49)

6.52 (2.76)

1.39*** (2.39)

20.49

0.50

Global Warming’s Six Americas indexb

2.94 (0.62)

3.20 (0.65)

0.26*** (0.47)

19.97

0.49

Recognition of scientific agreementc

45 %

60 %

15 %***

8.85

0.24

Behavioral intentionse

4.43 (2.29)

5.06 (2.50)

0.63*** (1.70)

12.84

0.35

Self-efficacyd

4.30 (2.33)

5.19 (2.42)

0.89*** (1.91)

16.02

0.42

Communication behaviorf: yesterday, did you…

 Talk with your parents about climate change?

6 %

15 %

9 %***

8.22

0.23

 Talk with your friends about climate change?

9 %

21 %

11 %***

8.77

0.25

Conservation behaviorf: yesterday, did you…

 Carry a reusable water bottle to school?

35 %

35 %

0 %

0.07

0.00

 Unplug gadgets and electronics when not using them?

46 %

55 %

9 %***

5.90

0.17

 Take a shower less than 5 min long?

19 %

24 %

5 %***

4.03

0.12

 Turn off lights when leaving a room for more than 10 min?

76 %

79 %

3 %**

2.35

0.07

 Recycle paper, glass, cans, and/or plastics?

62 %

66 %

3 %**

2.40

0.07

** p < .01, *** p < .001

aKnowledge is a ten-item index (range 0–10)

bGlobal Warming’s Six Americas is a 15-item index (range 1–5)

cRecognition of scientific agreement is a single item (range = 0-1)

dSelf-efficacy is a two-item index (range = 1-10)

eBehavioral intentions is a three-item index (range = 1-10)

fAll behavior questions are coded: yes = 1; no = 0; don’t remember = 0

Effect sizes for the other outcomes were smaller. For intentions to change behavior and/or talk to others about behavior change, r = 0.35. For interpersonal communication yesterday about climate change, r = 0.25 talking with friends and r = 0.23 for talking with parents. For recognition of scientific agreement, r = 0.24. For conservation behaviors, r ranged from 0.00 for carrying a reusable water bottle (the only behavior that was unaffected by the assembly) to 0.17 for unplugging gadgets and electronics.

When controlling for days elapsed between the assembly and post-assembly measurement, effect sizes remained virtually unchanged for two outcomes—behavioral intentions and self-efficacy. For the other outcomes, effect sizes were smaller among those who took the survey six or more days after the presentation (data not shown; see Supplemental Table 1).

3.2 Research question 2

Research Question 2 asked whether attending the ACE assembly would change the distribution of students in the Global Warming’s Six Americas segmentation scheme. In other words, were the significant increases in positive engagement in the issue of climate change, as measured by the Six Americas index (shown in Table 2), large enough to shift students into more engaged audience segments?

Table 3 addresses this question by showing the percentage of each pre-assembly audience segment whose post-assembly responses categorized them into a more engaged segment, into the same segment, or into a less engaged segment. As shown in the second header row, the distribution of students across the segments before the assembly was as follows: 7 % were Alarmed (n = 84), 35 % Concerned (n = 434), 35 % Cautious (n = 429), 10 % Disengaged (n = 120), 9 % Doubtful (n = 105), and 4 % Dismissive (n = 47). This distribution is similar to the distribution among a nationally representative sample of youth: 9 % were Alarmed, 32 % Concerned, 26 % Cautious, 12 % Disengaged, 15 % Doubtful, and 8 % Dismissive (Flora and Roser-Renouf 2014).
Table 3

Percent (and n) of pre-assembly segments that were categorized in the same or different segments after the assembly

 

Pre-assembly segments

 

Alarmeda

(n = 84)

Concerned

(n = 434)

Cautious(n = 429)

Disengaged(n = 120)

Doubtful(n = 105)

Dismissiveb(n = 47)

Totalc(n = 1,219)

Moved to more engaged segment

22 % (94)

44 % (190)

72 % (86)

68 % (71)

49 % (23)

38 % (464)

Stayed in same segment

68 % (57)

65 % (281)

44 % (187)

23 % (28)

24 % (25)

51 % (24)

49 % (602)

Moved to less engaged segment

32 % (27)

14 % (59)

12 % (52)

5 % (6)

9 % (9)

13 % (153)

Total

100 %

100 %

100 %

100 %

100 %

100 %

100 %

aStudents in the Alarmed segment before the assembly could only stay in the same segment or move to a less engaged segment

bStudents in the Dismissive segment before the assembly could only stay in the same segment or move to a more engaged segment

c22 students were not assigned to a segment due to missing data

The Total column at the far right of Table 3 answers Research Question 2 most directly: more than a third of the students (38 %) were categorized into a more engaged segment after the assembly; 13 % moved to a less engaged segment; and about half (49 %) remained in the same segment. The 38 % who moved toward greater climate engagement included 27 % who moved one segment and 11 % who moved two segments (data not shown in table).

Due to their positions at the extremes of the Six Americas segmentation protocol, students in the pre-assembly Alarmed segment could not move to a more engaged segment, and students in the pre-assembly Dismissive segment could not move to a less engaged segment. The percentage of pre-assembly Dismissive students moving to a more engaged segment (49 %) was greater than the percentage of pre-assembly Alarmed students moving to a less engaged segment (32 %). The largest shifts into more engaged segments were among the pre-assembly Disengaged and Doubtful segments (72 and 68 %, respectively). The pre-assembly Alarmed segment had the largest percentage of students who moved to a less engaged segment (32 %, n = 27); 22 of these 27 students moved just one segment down to the Concerned segment. Small minorities of the Concerned and Cautious segments also moved to less engaged segments (14 and 12 %, respectively). These shifts to less engaged segments, however, were smaller than shifts to more engaged segments (22 % of the Concerned and 44 % of the Cautious). Fig. 1 illustrates the changed distribution of students across the six segments that resulted from the shifts described here.
https://static-content.springer.com/image/art%3A10.1007%2Fs10584-014-1274-1/MediaObjects/10584_2014_1274_Fig1_HTML.gif
Fig. 1

Distribution of audience segments, before and after climate science edutainment assembly

3.3 Research question 3

Whereas Research Questions 1 and 2 used the Global Warming’s Six Americas index as a dependent variable, Research Question 3 treats the issue involvement and beliefs behind the Six Americas segmentation as potential moderators of assembly effects. That is, Research Question 3 asks whether student members of the Six Americas audience segments were differentially affected by the ACE assembly, in terms of the other outcomes measured here. Analyses of variance revealed no differences among the pre-assembly segments in assembly effects on climate science knowledge, recognition of scientific agreement, self-efficacy, behavioral intentions, or interpersonal communication behavior, or conservation behavior (data not shown—see Supplemental Table 2 for pre-post outcome means by pre-assembly segment). Because over half of students (51 %) shifted segments after the assembly, we conducted follow-up analyses to examine whether differential effects of the assembly depended on these shifts.

Table 4 shows the pre-post means for assembly outcomes among students originally categorized in the four middle segments of the Six Americas scheme—the Concerned, Cautious, Disengaged, and Doubtful—who then shifted to a more climate-engaged segment after the assembly. All four groups who shifted to more engaged segments showed significant increases in the cognitive and affective outcomes (knowledge, recognition of scientific agreement, self-efficacy, and behavioral intention), regardless of their pre-assembly segment. Effect sizes were largest for knowledge and self-efficacy, ranging from r = 0.52 for self-efficacy in the “Doubtful and increased” group to r = 0.71 for knowledge in the “Disengaged and increased” group. The absolute levels of all outcomes except knowledge were ordered, with the “Concerned and increased” having the highest levels, followed by the pre-assembly Cautious, Disengaged and Doubtful groups whose engagement increased. The only behavior change that was significant for all four groups was talking with friends about climate change, with effect sizes ranging from r = 0.27 to r = 0.32. For the other climate-related behaviors, increases were significant primarily for the “Concerned and increased” group. In this group, both communication behaviors and three of five conservation behaviors increased significantly, with effect sizes ranging from r = 0.12 to r = 0.42.
Table 4

Pre-post differences in outcomes, by pre-assembly segments that shifted to more engaged segments

 

Concerned, then increased (n = 94)

Cautious, then increased (n = 190)

Disengaged & increased (n = 86)

Doubtful & increased (n = 71)

Pre Mean (SD)

Post Mean (SD)

t (df =93)

Effect size (r)

Pre Mean (SD)

Post Mean (SD)

t (df =189)

Effect size (r)

Pre Mean (SD)

Post Mean (SD)

t (df =85)

Effect size (r)

Pre Mean (SD)

Post Mean (SD)

t (df =70)

Effect size (r)

Knowledgea

6.23 (2.39)

7.81*** (1.85)

8.36

0.65

5.00 (2.32)

7.18*** (2.24)

13.21

0.69

3.74 (2.08)

6.11*** (2.53)

9.40

0.71

4.97 (2.10)

6.65*** (2.58)

5.61

0.56

Recognition of scientific agreementb

59 %

86 %***

4.82

0.45

49 %

72 %***

5.15

0.35

33 %

60 %***

4.41

0.43

30 %

56 %***

3.71

0.40

Self-efficacyc

6.07 (1.94)

7.57*** (1.64)

8.75

0.67

4.19 (2.11)

5.67*** (2.12)

9.52

0.58

3.73 (2.26)

4.93*** (2.19)

5.79

0.54

2.99 (1.97)

4.38*** (2.26)

5.13

0.52

Behavioral intentionsd

6.45 (1.74)

7.94*** (1.45)

8.33

0.66

4.29 (1.99)

5.43*** (2.17)

7.95

0.51

4.31 (2.07)

5.03*** (2.38)

3.24

0.34

2.84 (2.00)

3.77*** (2.08)

4.12

0.44

Communication behaviore: yesterday, did you…

 Talk with your parents about climate change?

8 %

29 %***

4.27

0.41

8 %

12 %

1.71

0.12

2 %

11 %

2.16

0.23

4 %

13 %

1.93

0.23

 Talk with your friends about climate change?

11 %

30 %***

3.78

0.37

7 %

24 %***

4.57

0.32

7 %

18 %**

2.58

0.27

3 %

13 %**

2.41

0.28

Conservation behaviore: yesterday, did you…

 Carry a reusable water bottle to school?

38 %

43 %

1.15

0.12

32 %

34 %

0.69

0.05

33 %

32 %

−0.28

0.03

36 %

33 %

−0.70

0.09

 Unplug gadgets and electronics when not using them?

52 %

76 %***

4.40

0.42

53 %

58 %

1.26

0.09

38 %

51 %

2.16

0.23

35 %

52 %

2.54

0.29

 Take a shower less than 5 min long?

15 %

34 %***

3.79

0.37

14 %

22 %**

2.65

0.19

14 %

14 %

0.00

0.00

25 %

25 %

0.00

0.00

 Turn off lights when leaving a room for more than 10 min?

86 %

97 %**

3.01

0.30

77 %

85 %

2.37

0.16

80 %

78 %

−0.23

0.03

60 %

74 %

2.12

0.25

 Recycle paper, glass, cans, and/or plastics?

77 %

83 %

1.30

0.13

64 %

71 %

2.00

0.14

61 %

65 %

0.65

0.07

51 %

59 %

1.51

0.18

** p < .01, *** p < .001

aKnowledge is a ten-item index (range 0–10)

bRecognition of scientific agreement is a single item (range = 0-1)

cSelf-efficacy is a two-item index (range = 1-10)

dBehavioral intentions is a three-item index (range = 1-10)

eAll behavior questions are coded: yes = 1; no = 0; don’t remember = 0

For the sake of readability, the pre-assembly Dismissive students who shifted to a more engaged segment (and who could not shift to a less engaged segment) are not shown in Table 4. In this group, the only significant assembly outcome was an increase in their recognition of scientific agreement that climate change is happening (t(23) = 2.91, p < 0.01, r = 0.53).

Next we examined the students who did not shift to a more engaged segment (also not shown in Table 4). Students categorized as Alarmed before the assembly, who could not shift to a more engaged segment, made significant improvements in all the other assembly outcomes except conservation behaviors, but only knowledge had an effect size greater than r = 0.50. Similarly, the pre-assembly Concerned and Cautious students who stayed in the same segments made statistically significant improvements on all the other assembly outcomes, with effect sizes ranging from r = 0.17 to r = 0.63. In contrast, there were no significant changes among the pre-assembly Disengaged, Doubtful, and Dismissive students who stayed in the same segment, nor among students who shifted to a less engaged segment.

4 Discussion

ACE’s 1-h, single-exposure edutainment presentation succeeded in improving students’ knowledge of climate science, positive engagement in the issue, and climate-related behaviors in the short term. Knowledge of climate science increased 27 % over baseline levels, producing an effect size of r = 0.50, considered large for social science (Cohen 1988). Effect sizes were moderate to large (0.42–0.49) for two measures of positive engagement in the issue of climate change—the Global Warming’s Six Americas index and self-efficacy for talking about climate change or starting a conservation project at school. And whereas fewer than half of students (45 %) recognized before the assembly that scientists generally agree that global warming is happening, closer to two-thirds (60 %) recognized this afterward.

The behavior most influenced by the ACE assembly was interpersonal discussion with parents and with friends about climate change, with small-to-moderate effect sizes of r = 0.23 and 0.25, respectively. We find this encouraging, as it suggests that students carry the information and enthusiasm they gained from the edutainment presentation into their families and social circles. This result also illustrates how discussion of climate change is a potential game changer for shifting norms, as well as deeper issue engagement and action (Rimal and Flora 1998). Of the five conservation behaviors we measured, the four that increased significantly were those undertaken in the home. Effect sizes were in the small range for recycling, turning off lights, taking shorter showers, and unplugging unused electronics. The one conservation behavior that was unaffected by the assembly was carrying a reusable water bottle to school. The time between the assembly and post-survey may have been too short for students to buy a suitable bottle. Or it could be that reusable bottles carry a social stigma for high school students.

Our results also demonstrate that exposure to climate science in an engaging edutainment format can, at least in the short term, change youths’ beliefs and involvement to a degree that moves them to audience segments that are more engaged in the topic of climate change. Most often, audience segmentation strategies are integrated into the message design and program tailoring phases of a communication campaign. This research used a validated segmentation protocol in a novel way—as a dependent measure with youth—and illustrates the important role that the Global Warming’s Six Americas protocol can play in evaluating climate change programs.

Our follow-up comparisons of students who shifted to more engaged segments, shifted to less engaged segments, or did not shift at all suggests that for youth at least, shifting segments is one step of a multi-step process of enhancing engagement. That is, youth who shifted to more engaged segments (as well as youth who were already in the most-engaged Alarmed segment at baseline) made significant improvements in most of the other outcomes measured here. Similarly, students who were in the Concerned and Cautious segments before the assembly and who did not shift to more engaged segments also made significant improvements in many of the assembly outcomes. In contrast, students who stayed in the Disengaged, Doubtful, and Dismissive segments from baseline to post-test and students who shifted to less engaged segments did not make significant improvements in the other outcomes.

For ACE as an organization, this climate science assembly is part of a larger strategy that includes providing assemblies to new cohorts of students at the same schools over the years. In addition, interested students are recruited to form clubs that receive more intensive training from ACE staff and complete conservation projects at their schools. Given the changes resulting from a single presentation, the net impact of all these intervention efforts could be a population shift in climate science knowledge and positive engagement in the issue of climate change.

4.1 Cautions and caveats

Like any study that lacks a control group, these findings must be viewed with caution regarding false positives. This study’s pre-test/post-test design allows for calculation of individual-level changes over time, but it is also vulnerable to testing effects (i.e., learning from the pre-survey) and to test-intervention interactions (e.g., learning more from the assembly because of being primed by the pre-test survey). Nevertheless, the numerous findings that fall into logical and predictable patterns increase confidence in our conclusions. Finally, we conducted many significance tests, which can yield Type I errors. We attempted to reduce this possibility by using a stricter significance level, basing conclusions only on differences that were significant at p < .01 or less. We also reported effect sizes to help interpret the practical importance of findings.

4.2 Future research

From a programmatic point of view, a one-time event is unlikely to produce lasting change. Further intervention will likely be necessary to cultivate deeper engagement in the climate change issue among youth. In particular, post-assembly activities that allow students to participate according to their interests, skills, and time will be needed. ACE is now developing a social media component and support both for student-led climate clubs and teacher-led classroom activities. These program components should be incorporated into a controlled evaluation over time, both to validate the conclusions drawn here and to determine whether the effects are long-lasting. A randomized controlled evaluation, with post-test measurement at 2- and 6-week intervals, is now underway.

From a theoretical point of view, this study highlights the potential importance of campaign messages that are engrossing as well as informational. The underlying processes of entertainment-education deserve further research, particularly with regards to climate change communication with youth. Slater and Rouner (2002) offer a theoretical explanation as to why edutainment (operationalized as narrative) could be persuasive in shifting scientific misconceptions. Using an elaboration likelihood approach, Slater and Rouner argue that absorption in a narrative and identifying with characters suppresses counter-arguing of counter-attitudinal content. This is an intriguing analysis and could help explain why in this study we see significant movement among Dismissive students to a more climate-engaged segment.

Footnotes
2

Larsen JT, Sarge MA, Hayhoe K (2014) Six Tennessees: One Earth. Presented at the Society for Personality and Social Psychology’s annual conference in Austin, TX.

 
3

A trailer for the presentation is available at http://www.acespace.org.

 
4

See Supplemental Materials for the assessment tool used to train presenters.

 

Acknowledgments

This evaluation was supported by Michael Haas and a grant from the Pisces Foundation. We acknowledge the teachers at the 49 schools that administered surveys for this study, and all the teachers who have worked with ACE to reach 1.8 million high school students since 2008. We also thank the three anonymous reviewers for their comments and suggestions.

Supplementary material

10584_2014_1274_MOESM1_ESM.docx (26 kb)
Supplemental tables(DOCX 25 kb)
10584_2014_1274_MOESM2_ESM.pdf (192 kb)
Assembly script(PDF 191 kb)
10584_2014_1274_MOESM3_ESM.pdf (41 kb)
Presenter assessment(PDF 41 kb)
10584_2014_1274_MOESM4_ESM.docx (27 kb)
Question wording(DOCX 26 kb)
10584_2014_1274_MOESM5_ESM.docx (26 kb)
Survey administration(DOCX 26 kb)

Copyright information

© The Author(s) 2014

Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Authors and Affiliations

  • June A. Flora
    • 1
  • Melissa Saphir
    • 2
  • Matt Lappé
    • 3
  • Connie Roser-Renouf
    • 4
  • Edward W. Maibach
    • 4
  • Anthony A. Leiserowitz
    • 5
  1. 1.Solutions Science Lab Department of Pediatrics and Human Sciences & Technologies Advanced Research InstituteStanford UniversityStanfordUSA
  2. 2.Saphir Research ConsultingSan FranciscoUSA
  3. 3.Alliance for Climate EducationBoulderUSA
  4. 4.Center for Climate Change CommunicationGeorge Mason UniversityFairfaxUSA
  5. 5.Yale Project on Climate Change CommunicationYale UniversityNew HavenUSA