Is it climate change? Coverage by online news sites of the 2019 European summer heatwaves in France, Germany, the Netherlands, and the UK

In 2019, several countries across Western Europe experienced record-breaking temperatures and heatwaves which, in some cases, reached temperatures of over 40 °C for three to four consecutive days during June and July. Extreme event attribution (EEA) studies show that anthropogenic climate change increased the likelihood of these events by at least three to ten times (with different results for different countries), and increased the temperature by 1.2 to 3.0 °C. The heatwaves resulted in more than 2500 deaths. Based on a content analysis of 267 articles taken from 20 of the most visited online news websites in four of the countries most affected by the heatwaves (France, Germany, the Netherlands, and the UK), we find strong variations between countries and media outlets in how much attention journalists pay to links between climate change and the heatwaves (the UK media the most, and politically left-leaning titles more than right-leaning ones); many different types of statements depicting the link but in general, the presence of accurate, science-based descriptions; a strong presence of EEA studies in the coverage; and more quotes from climate scientists than politicians and NGOs, with a minimal presence of climate change skeptics. These results contribute to our understanding of media coverage around extreme weather events in different countries and media outlets, and how such events might serve as opportunities for public engagement with climate change.


Background
Despite the proliferation of numerous sources of information on climate change, news media are still central to how individuals, organizations, and societies understand, evaluate, and act upon it (Metag et al. 2017). Online survey data from 40 countries, including the four countries in this study, show that in 2020, news media were the most widely used sources of information on climate change, and that online sites of major news organizations were the second most popular platform after television (Amdi 2020). For the news media, extreme weather events such as heatwaves, floods, droughts, and storms present a particular case for journalists to report about: first, such stories are of national and international relevance to warn the public of acute dangers and the impacts on infrastructure and society at large, and second, because these events also raise the opportunity for journalists to link the extreme weather events to the science of climate change, and in this case extreme event attribution (EEA) studies.
However, media treatments of extreme weather events in European countries have rarely been studied (Painter and Hassol 2020), and cross-country studies of a single event are generally lacking. Indeed, in general, there has been relatively little scholarship about the media coverage of specific extreme weather events across the world. Surveys of the literature (Hopke 2019;Painter and Hassol 2020) show that the research has tended to focus on volume of coverage, or issue attention. Several studies (e.g., Cordner and Schwartz 2019;Burgess et al. 2020;Weiner et al. 2021) have concluded that the links between extreme weather events and climate change are underreported. This study fills this research gap by studying the reporting of the 2019 heatwaves in Europe. Similar to previous heatwaves, these received extensive coverage in European media, and particularly in the four countries where the heatwaves were most acute: France, Germany, the Netherlands, and the UK.
2019 was the warmest year on record in Europe. 1 During a series of heatwaves in June and July, a number of record-breaking temperatures were set. In France, an all-time high temperature of 46 °C was reached, parts of Belgium, Germany, Luxembourg, and the Netherlands also registered new highs, and in the UK, a new all-time high of 38.7 °C was recorded in Cambridge in July (Vautard et al. 2020). In many parts of the continent, the weather was 3 to 4 °C warmer than average, and the number of sunshine hours was the highest on record (Copernicus 2020). According to the Centre for Research on the Epidemiology of Disasters, the European summer heatwave was the deadliest extreme event in 2019 anywhere in the world with a total of approximately 2500 deaths in France, the Netherlands, the UK, and Belgium (measured by excess mortality) (Froment and Below 2020). This figure represented more deaths than those resulting from the floods in India and cyclone Idai affecting Mozambique and Zimbabwe that year (ibid.). 2 EEA studies assess if, and by how much, specific extreme weather events have become more or less likely, or more or less intense. Such studies about the 2019 European heatwaves established that they became several times more likely and more intense due to anthropogenic climate change , Vautard et al. 2020, Ma et al. 2020. Two preliminary studies carried out by the multinational umbrella science organization World Weather Attribution (WWA) were published in "near-real-time" (see Table 1), occur without human influence on climate (return periods higher than ~ 1000 years).
In the UK and Germany, the event is less rare (estimated return periods around 10-30 years in the current climate) and the likelihood is about 10 times higher (at least 3 times) due to climate change which makes the 2019 summer heatwaves a fertile case study to examine how the media reported the link at the time the events were taking place.
Although EEA can link specific extreme events to climate change to an unprecedented degree (Betts 2021), there are many different ways of defining the event and describing its results, including an array of different phrases and word choices. The choice of phrasing is important not just for scientific accuracy, but also because recent focus group research has found that different approaches to communicating EEA, including the use of different words and visualizations, can evoke distinct reactions among members of the public .
This study offers a detailed focus on a 3-month period of coverage of heatwaves in four countries during which two EEA studies were published. In this way, we add to our understanding of firstly, the different ways journalists depicted the links between climate change and heatwaves, the EEA studies they quote over time, and the sources they turn to, and secondly, the differences and similarities between journalistic practices in different countries and different types of media outlet (public broadcasters, and right-and left-leaning commercial outlets).

EEA studies and the role of climate change in heatwaves
The extent to which greenhouse gas (GHG) emissions have played a role in the occurrence of extreme weather events is of growing interest to scientists, the media, politicians, environmental activists, and wider civil society. By early 2021, more than 350 peer-reviewed EEA studies had been published (Carbon Brief 2021). This research has provided mounting evidence that human activity is raising the risk of some types of extreme weather, especially those linked to heat.
However, the calculation of the influence of climate change on extreme weather events such as heatwaves is complex and at times uncertain, as it is contingent on how climate scientists define the event according to the interplay of natural fluctuations, and local and atmospheric conditions. In recent years, the science community has been making substantial advances in better understanding the links between climate change and specific extreme weather events. Different approaches, or combinations of approaches, are used by EEA scientists to establish the presence or absence of any link (Herring et al. 2016). They usually rely on some combination of observational weather data and computer modelling simulations to assess the difference in the likelihood or intensity of events with or without anthropogenic factors (ibid.). An assessment is then made of various possibilities: (i) the event was made more likely or severe as a result of anthropogenic climate change; (ii) the event was made less likely or severe; (iii) there was no discernible difference; or (iv) there was insufficient data to make a judgement.
Often the results are expressed as a change in the "return period" or "return time" of an event, that is, how often the event of this type might be expected to occur. For example, in our case study, researchers found that the heatwaves in the UK and Germany would have had a return time of a few tens to a few hundreds of years without climate change, but climate change caused this event to now have a return period of around 10-30 years. So the likelihood of such an event was made about 10 times higher (and at least 3 times in France and the Netherlands) due to climate change .
Research published in early 2021 (Carbon Brief 2021) found that of the 132 attribution studies that have looked at extreme heat events, 92% concluded that climate change made the event or trend more likely or more severe. This compares, for example, with rainfall or flooding events, where a smaller portion (58%) found human activity had made the event more likely or more severe. No studies have found that a heatwave had been made less severe by climate change, while two studies (2%) identified no influence and a further eight (6%) were inconclusive. A similar survey of 36 studies of 32 heatwaves starting in 2015 showed that anthropogenic influence increased the likelihood or strength of each event (Watts et al. 2021).
According to the World Weather Attribution (WWA) initiative, every heatwave analyzed in Europe in recent years has been found to be made much more likely and more intense due to anthropogenic climate change. The Stott et al. (2004) study of the 2003 heatwave in several European countries paved the way. The authors calculated that it was very likely that human influence had at least doubled the risk of the heatwave, which resulted in an estimated 70,000 excess deaths that summer. Since then, attribution studies have been applied to heatwaves in different regions of Europe in 2010 (Otto et al. 2012), 2015(Sippel et al. 2016(Kew et al. 2018(Leach et al. 2020. The studies show variations across time in how much the specific heatwaves were made more intense and/or more likely as a result of climate change, which depended strongly on how the event was defined in terms of location, season, intensity, and duration.

The 2019 European heatwaves
The two main 2019 heatwaves in Europe took place in June and July, and varied in time span, geographies, and intensity. The June event lasted longer, whereas the July one was shorter and more intense, with about 4 days of very high temperatures, and was accompanied by severe drought conditions in some areas, particularly in parts of France (Vautard et al. 2020). The immediate cause of the heatwaves was similar-a ridge of high pressure across Western Europe combined with a low-pressure system off the Iberian Peninsula. This weather pattern forced an intense transfer of hot air from Northwestern Africa across Spain to France, and then Germany and the Benelux countries (ibid.).
Details of the impacts of the heatwaves and the government responses to them are broken down by country and summarized in the Supplementary Material. France suffered the highest official death toll due to the heatwaves (1435), followed by the UK (900), and the Netherlands (400). The effects of extreme temperatures were far-reaching: "from delayed trains and sleepless nights, to deaths from drowning and an increased mortality risk for the elderly, babies and vulnerable populations." 3 Table 1 summarizes the four event attribution studies of the summer heatwaves which were published between July 2019 and August 2020. The two WWA studies were published in "near-real-time" (namely 2 July and 2 August 2019), and received widespread media coverage (see Section 3.3 below).
The WWA researchers emphasized that although they had high confidence in the increased likelihood of the heatwaves, there were a number of uncertainties and limitations. For example, in the case of the WWA study of France of 2 July 2019, the researchers said it was difficult to assign a specific number to the size of the increased likelihood due to the differences between the representation of the heatwaves in the observational data and the climate models. Also, the temperature data record used in the analysis was relatively short, running from 1947 to 2019, which meant it was more likely to contain some uncertainties (Carbon Brief 2019).
In other studies of the media coverage of extreme weather events (e.g., a heatwave and extreme rainfall event in India in 2015; or the Californian drought- Osaka et al. 2020), EEA studies had produced different results about the role of climate change. However, in the case of the European heatwave, all the studies showed that anthropogenic climate change made it more likely or more intense.

Country differences in media coverage
Comparative media research has long identified, and debated, the key characteristics of different media systems in Western Europe. In their classic study, Hallin and Mancini (2004) saw evidence for Germany and the Netherlands belonging to the "democratic-corporatist" model, whereas France approximated more to the "polarized pluralist" model, and the UK to the North American or "liberal" model. In all four countries, public and commercial broadcasters continue to fare reasonably well in terms of audience reach (Newman et al. 2020), but there are important differences in the extent to which they have prioritized their online news offers with the BBC faring better than their counterparts in France, Germany, and other European countries (Sehl et al. 2016). Likewise, newspaper companies across Europe struggle to attract readerships for their print offer (Nicholls et al. 2018), but some (like the Guardian) have adapted much better to the digital landscape than others by prioritizing their online content (Küng 2016).
A variety of factors at different levels, such as macro trends in media business models, meso-level developments within media organizations, and at the micro-level changing attitudes of individual journalists, continue to shape the way climate change is covered by different media organizations (Schäfer and Painter 2021). Major differences continue to persist between countries, including the volume of coverage, the amount of attention given to skeptics, and the emphasis on different themes or frames (Painter and Schäfer 2018). Some of these differences are found between "Anglosphere" countries like the UK and continental European countries. For example, studies have shown that in Europe, the UK tends to cover climate change the most (partly due to its contestation in the public sphere), followed by relatively high coverage in France, and less in Germany and the Netherlands (Schmidt et al. 2013). The "exceptionalism" of the UK compared to other European countries has been supported by other studies, particularly in the higher proportion of editorial space given to climate skeptics and scientific uncertainty (Brossard et al. 2004;Dirikx and Gelders 2009;Engels et al. 2013;Painter and Gavin, 2015).
Several studies suggest ideological differences between media organizations are a significant driver of different treatments of climate change (see Painter 2016 for overview). In the UK, left-leaning papers such as the Guardian are strongly supportive of mainstream climate science in their news articles, whereas right-leaning newspapers such as the Mail and the Express have been strongly dismissive of them. In their study of the Dutch and French press, Dirikx and Gelders (2010) found that in the Netherlands, newspaper ideology was not related to climate change coverage, but in France, it was related to the tone of the coverage and the need to take action. The difference, they said, was that in France (and the UK), climate change was subject to political division in a highly competitive media landscape, where ideologies are used as a form of "product differentiation." Research has also found notable differences between left-leaning and right-leaning media outlets as regards the volume of coverage of climate change (Kristiansen et al. 2020), and the coverage of adaptation to heatwaves (Jiménez-Gómez and Martín-Sosa-Rodríguez 2021).

Extreme weather in the media
As mentioned above, research has shown that the links between climate change and extreme weather events have generally not been reported sufficiently or accurately. However, Hopke (2019) found that there are clear exceptions to this general finding for some English-language media titles like the Guardian, the New York Times, the Washington Post, and the BBC. She argues that this was in part due to these media organizations' investment in environmental journalism by having dedicated environmental and/or climate reporters and editors. Indeed, the Guardian prides itself on its detailed, sciencebased climate coverage based on its well-resourced environment section. It offers frequent self-reflection on its coverage, often publishing statements about its editorial policy on the issue, and even initiated a campaign in 2015 focused on getting two charitable organizations to withdraw investments from fossil fuel companies (Painter 2011;Kristiansen et al. 2020;Schäfer and Painter 2021).
Most of the studies have not examined the different ways the link between climate change and specific weather events are described, or the role that EEA studies have played in the coverage. The exceptions include , who found that in India, politicians and NGOs often "blamed" climate change without reference to the science, and that EEA studies were rarely mentioned in the media. In contrast, Osaka et al. (2020) concluded that local and national media in the USA did cover the link between climate change and the Californian droughts in 2014-2015, but that the different EEA studies available led to the presence of a frame of scientific uncertainty or disagreement in the coverage.
Very few studies have focused solely on an extreme weather event in Europe, with one exception being the wildfires in Greece in 2007 (Hovardas 2014). Jiménez-Gómez and Martín-Sosa-Rodríguez (2021) looked at the coverage of European cities' adaptation to heatwaves over a 2-year period from 2017 to 2019 in several newspapers in five countries. They concluded that the country where the article was published was the most decisive variable in the rigor and depth of the journalistic coverage, followed by ideological orientation. The authors found that the media in France, regardless of type or editorial line, had the most comprehensive approach, and that French authorities and institutions regularly linked the heatwaves to climate change.
A different research question was addressed in work by Pianta and Sisco (2020), who looked at online news coverage of hot temperatures in the (then) 28 countries of European Union from 2014 to 2019. They focused on the drivers of volume coverage and a possible correlation between positive deviations from short-term average temperatures as distinct to baseline periods used by climatologists. Based on a time-series analysis of 1.7 million articles, the authors argued that temperatures warmer with respect to recent years increase the attention that the media devote to climate change coverage, which may be due to this being interpreted as evidence of climate change. However, other possible drivers of fluctuations of volume (e.g., UN summits, science reports) are not included in their analysis.

Criticism of media coverage
Mainstream media have often been criticized for their coverage of extreme weather events. This has centered on journalists being slow to make the link between climate change and the event (Painter and Hassol 2020) or in some countries not making it at all (Hopke 2019); not fully explaining scientific disagreement or uncertainty over EEA studies (Osaka et al. 2020); presenting news and weather reports that portray (in text and images) long, hot, dry 1 3 spells as overwhelmingly positive (e.g., O'Neill 2019); or quoting NGOs and politicians on the link without sufficient scrutiny of their accuracy .
In similar ways to this general criticism, some NGOs and experts in the UK pointed to the relative absence of climate change in the reporting of the 2019 summer heatwaves, the insufficient mention of the negative impacts of heatwaves, and the generally positive tone of the coverage, both in the text and visually. 4 In the Netherlands, some commentators criticized reporters for paying more attention to whether the heatwave would break records, rather than its link to climate change. 5 In contrast in France, some commentators questioned whether the media were paying too much attention to the heatwave and creating an almost automatic link to climate change, and thereby more anxiety among the population. 6 In the UK, the Met Office and the BBC were also subject to some criticism for not being proactive enough in making the link between the weather and climate change. One academic argued it was important that weather forecasters gave overviews of temperature trends to help viewers "to make sense of their own experiences and allow them to think about how risks will change." 7 The same expert has advocated that heatwaves are becoming so deadly (and far more people in the UK have died from recent heatwaves than from storms) that they should be given names, like storms, so that the public takes them more seriously. 8 Another academic pointed out that text or headlines highlighting the downsides of 2019 heatwaves were often accompanied by positive images of holiday makers enjoying a good time on the beach, or people playing in fountains, before, during, and after the record-breaking August Bank Holiday (O'Neill 2019).
Bearing in mind the above discussion, we can see that although the media play a central role in giving the public information about extreme weather events, few studies have concentrated on the way they present the science behind the link with climate change, and on the differences in coverage between countries and media outlets with contrasting political leanings. As a result, our principal and secondary research questions were formulated as follows: 1. How often was a link between the 2019 heatwave and anthropogenic climate change included in the coverage? 2. What were the different ways that journalists describe the link, and what was their relative presence? 3. What was the volume of coverage over time where a mention of attribution studies was included? 4. Which sectors (scientists, politicians, NGOs) were quoted in the coverage about the link between the weather events and climate change, and were they supportive of the link, or questioned it? 5. What were the main differences between countries in the content of the coverage? 6. Was there a marked difference between left-leaning and right-leaning publications? Secondary: 7. Was scientific uncertainty represented, and if so, how? 8. How often was there mention of any possible solution(s) or policy options, i.e., anything relating to either individual or collective actions to avoid or reduce the chances/impacts of such scenarios (heatwaves) in the future?

Methods and material
Based on the research showing that online news websites are a very popular way of receiving information about climate change (Amdi 2020), we chose to analyze the coverage of the heatwave by five of the most visited websites in each of four European countries most affected. We selected twenty news online sites to test these research questions. These were 20 min, France Info, Le Monde, Le Figaro, and Ouest France (France); Spiegel, Bild, Focus, Welt, and N-tv (Germany); NU.nl, Algermeen Dagblad, NOS News Online, De Telegraaf, and De Volkskrant (the Netherlands); and BBC online, the Guardian, Mail Online, Sky News online, and the Telegraph (UK) (see Table 2). The criteria for selection are laid out in detail in the Supplementary Material, which also provides brief portraits of each media outlet. The articles were selected according to the search term "heatwave" and its equivalent in other languages in the period 1 June to 30 September 2019, and then culled in two rounds to include only articles that mentioned the link between the heatwaves and climate change. This resulted in a total of 267 articles to be coded, which are distributed as laid out in Table 2 Column 5. A codebook was drawn up using deductive and inductive methods, and tested iteratively among the five coders. Scores of between 0.75 and 0.85 using Fleiss' Kappa, Cohen's Kappa pairwise, and Krippendorff's Alpha for the key variables were achieved, which were regarded as acceptable levels of coder reliability. Again, more details can be found in the Supplementary Material. Table 2 shows a considerable amount of coverage of the heatwaves across all four countries and 20 outlets, amounting to 2447 articles after an initial round of culling irrelevant or repeated articles (see Column 4). Many aspects of the story were widely covered, including the latest weather developments, the record-breaking nature of the heatwaves, the potential dangers, transport and health impacts on ordinary people, impacts on economic sectors such as agriculture, and government attempts or policies to respond to the heatwaves in the short or long-term. Some outlets paid considerable attention to giving practical advice, and the activities of celebrities in the heatwaves. A wide range of photos, most of them showing positive images of enjoyment, accompanied the text. The differing volume of coverage between media outlets in Column 4 may not simply represent the different editorial priority given to reporting the heatwave. Larger volumes of coverage may be an indication of the business models of different online sites, or the geographical spread of the readership. For example, some French titles (except Le Monde, which does more original, in-house journalism) publish large numbers of online news articles in general, many based on AFP, as part of their drive for more visibility on social media platforms; Ouest France is a regional newspaper dedicating considerable coverage to local (weather) events across a wider geographical area, where impacts were varied; and the Mail Online's business model is based on volume of advertising around large numbers of articles.

Volume of coverage
More important are the results found in Columns 5 and 6. Column 5 shows the number of articles which mentioned some link between the heatwaves and climate change, which we then present as a percentage of the total number of articles covering the heatwave (Column 6). In total, 267 articles, equivalent to 11% of the articles about the heatwave across the four countries, mentioned climate change. Of these 267 articles, only 10 articles (4%) were opinion pieces or editorials, of which the vast majority (8) was found in the UK sample.

The description of the link between the weather event(s) and man-made climate change
As can be seen from Table 3, we found a wide variety of phrases, quotes, or statements that described the link between climate change and either the specific 2019 heatwave or heatwaves in general. These varied between generic trend statements, likelihood and intensity statements about the 2019 heatwave, and other types of links. Several types of statements could be present in the same article. Generic statements were those such as "this is the sort of extreme event which scientists say could become more frequent and/or intense in the future." Causation statements were those which explicitly or implicitly suggested that climate change (partly) caused the 2019 heatwave such as phrases as "climate change was to blame for the 2019 heatwave," or "climate change (partly) caused the 2019 heatwave as part of a warming trend." The "other" section included a wide range of types of link descriptions, not captured in other variables, including phrases such as the 2019 heatwave being proof of/ evidence for/indicator of/consistent with climate change or "climate change is playing a role in the heatwave." The likelihood and intensity/severity statements included both generic and specific events, and phrases such as "the chances," "the probability," "the risk," "the odds," "more frequent" or "more often" (in addition to "likely"), and "magnitude" or "strength" (in addition to "intense" or "severe"). Figure 1 shows the relative presence of the different types of statements. Generic statements were the most frequently present (227) in 85% of our sample. The second most frequently present type of statements were likelihood statements (in 186 articles-70% of sample), followed by "other" and intensity/severity statements (both with 105 articles-39%), and causation (13-5%). The significance of the results, and the ones that follow, will be discussed in Section 4.    Table 4 shows the number of articles which mention the two WWA reports released on 2 July 2019 (France) and 2 August 2019 (several countries) in the 20 media outlets. As can be seen:

The frequency and timing of quotes from the WWA reports
• Nearly a fifth (19%-50 articles) of the coverage across all four countries mentioning climate change included a mention of either the first or second WWA report, or both. • The first WWA report (France) was mentioned 24 times, compared to the slightly higher figure of 26 times for the second WWA report (Europe). • There are no large differences between the countries, as the country with the highest percentage (the UK-22%) does not vary greatly from the country with the lowest percentage (France-15%). • Four media titles did not mention either WWA report (Bild, N-tv, Algermeen Dagblad, and De Telegraaf), all of which are either right-leaning or centrist/non-aligned. • The titles which mentioned the WWA reports the most were the Guardian, the BBC, and Le Figaro (5 mentions each). Figure 2 shows the volume of coverage of the WWA reports over time. As can be seen, the bulk of the coverage occurred around the release dates of the reports, but in both cases, mention of the reports continued in the days after their release.

The identification of sources being quoted on the possible link
Our results show that well over half the articles we coded (156-58%) included one or more direct quotes about the possible link between climate change and the 2019 heatwave. A total of 219 direct quotes were identified across the whole sample, although the total could have been higher as we coded only the first three quotes in each article (see Table 5) for what was included in each category of source). The categorizations were decided by the main affiliation of the source, as described in the text of the article.
• Nearly one in four (52-24%) of the quotes came from representatives of the National Meteorological Offices (or the Met Offices as institutions) in the four countries. • A similar Fig. (50-23%) was represented by climate scientists whose main affiliation was to a university. • The largest percentage (56-26%) came from (representatives of) science-based organizations or institutes, of whom the most prevalent were the WWA (20-9%) and Copernicus (14-6%). • In total, nearly three-quarters of all the quotes (72%) came from experts related to the specific science of climate change and weather events. Politicians, political parties, and political movements were quoted only 21 times (10%), and NGOs (including unions and farmers) even lower (11-5%).
Only two of the 219 (coded) quotes were of a type which categorically denied or seriously questioned any link between climate change and the heatwave. Both were found in the German sample, one from a politician and one from a climate scientist. The first was from Klaus Gagel, a politician from the right-wing AfD, quoted in a Bild article on 29 1 3 August about the high temperatures in Frankfurt. Describing himself as a qualified meteorologist, Gagel said that the rising temperatures were "mainly due to the increasing solar radiation due to decreasing cloud cover-and the lack of large volcanic eruptions […]. Cosmic rays also play a role in particles and clouds. None of this is taken into account in the climate models." 9 A more nuanced quote came from the climate scientist, Gabriele Hegerl, a Professor of Climate System Science at the University of Edinburgh in the UK. She was quoted on Focus.de as saying that "Studies that looked at the latter question for heatwaves-for example, the 2003 heatwave-mostly find that such high temperatures are now more likely. The more difficult question is whether the weather conditions themselves have changed. Weather conditions are very variable and I am not convinced that we are seeing any change at the moment." 10 Her quote formed part of a selection of quotes in the article from four climate scientists, including Christian Franzke, a researcher at the University of Hamburg, who said that "There have always been heatwaves. Therefore, the upcoming heat episode cannot be attributed to climate change. Heatwaves are a natural weather phenomenon." 11

Differences between countries
France had the most coverage (93), followed by the UK (90), then Germany (42), and the Netherlands (42) (see Table 2). There were notable differences between the UK and the other countries in the percentage of coverage in which the climate change link was mentioned. The UK was the highest (27%) followed by the Netherlands (12%), Germany (10%), and France (7%). Using a chi-square test, we found the difference between those four countries to be statistically highly significant, X 2 (3, N = 2447) = 80.61, p < 0.001. In particular, the difference between the UK and the other three countries was found to be very marked (for the fuller results, see the Supplementary Material). The Netherlands also stood out by several metrics: the lower percentage of articles containing at least one direct quote about the link-only 38% compared to the other countries ranging between 58% (UK) and 67% (Germany), with France in between (65%); the lowest percentage of articles mentioning scientific uncertainty (see Section 3.7); and the lack of range of sources, where over half of the quotes came from the Koninklijk Nederlands Meteorologisch Instituut (KNMI -Dutch Met Office) (see Table 5). The UK sample included a large number of quotes from the Met Office (17) and climate scientists at universities (31); France included the most quotes from scientific or research institutes (26), and Germany only quoted its Met Office (DWD) three times.
It is also interesting to note that the Dutch (0) and German (2) coverage included no or very few politicians' voices, whereas the French (11 quotes) and the UK (8) samples included many more. Germany was the only country with three examples of quotes from sources denying or strongly questioning the link, as described in Section 3.4 above.
However, by other criteria, country differences did not stand out, namely the presence of different types of statements about the link (see Fig. 1), the mention of WWA reports, and the percentage of articles discussing solutions (see Section 3.7 below).

Differences between media outlets
There were some notable differences in the coverage by media outlets of different political orientation: • In the UK and the Netherlands, left-leaning or "non-aligned" media outlets had a much higher percentage presence of statements mentioning the link than right-leaning ones (see Table 2). In these two countries, using a chi-square test, these differences between media outlets were particularly significant (see Supplementary Material). • For example, in the UK, the left-wing Guardian had a 62% presence compared to the Mail (19%) and the Telegraph (14%). In the Netherlands, the center-left De Volkskrant (12%) compared sharply with the right-wing De Telegraaf (1%), which contained just one article mentioning the link out of the paper's total of 92 articles covering the heatwave and only then a relatively weak link. 12 • In France, the picture is more complex. Although the left-leaning Le Monde had a higher percentage (12%) than the right-wing Le Figaro (5%) and Ouest France (4%), the center-right 20 min had the same percentage (12%). • In Germany, the left-leaning Spiegel had the highest number of articles mentioning the link, although in percentage terms, it came behind the more centrist Welt. The rightleaning Bild had the least, equal to N-tv. • Of the online news sites of broadcasters, the BBC and Sky News in the UK had the highest percentage of coverage of the link (42% and 19%, respectively). NOS in the Netherlands came next with 17%, followed by France-Info with 11%. N-tv only had 5%, but the articles there are very short and often linked to videos. • The only quote from a climate science denier came in the right-wing German publication, Bild.

Fig. 2
Mentions of WWA reports from July 2019 to September 2019. The first WWA report (*) was published on 2 July, and the second (**) on 2 August Table 5 Sources quoted in articles by type Note (1) Several university scientists and representatives of Met Offices had more than one affiliation. Friedericke Otto from Oxford University (quoted 15 times), Geert Jan van Oldenborgh from the KNMI (quoted 13 times), and Robert Vautard from CNRS in France (quoted 7 times), were sometimes described as representatives of the WWA Note (2) Advocacy/humanitarian; Economic Interest group (Farmers, Insurance, wine growers, renewables e.g.) Note (3) Media correspondent, government department, independent expert Note (4) The columns do not add up to 100 due to the appearance of the sub-total for representatives of research institutes

Scientific uncertainty and solutions
Each article was coded to assess the presence of different types of scientific uncertainty. These included differing opinions from scientists or experts as to the role of climate change, the quoting of skeptics who disagree with the scientific consensus, differing results from separate studies, or uncertainty about the results of EEA studies (such as inconclusive results, insufficient data, statistical/measurement uncertainties, or complexity). By far the most common form of uncertainty found was that of complexity, found in phrases such as "However, it's difficult to say for certain that a particular extreme event -like the summer heatwave -is attributable to anthropogenic climate change," or "Linking a single event to climate change is complicated." 13 Statements like these appeared in a quarter of our entire coded sample (25%). The UK had the highest percentage (39%), and the Netherlands the lowest (14%).
Each article was coded for the inclusion of any possible solution(s) or policy options, i.e., anything relating to either individual or collective actions to avoid or reduce the chances/impacts of such scenarios (heatwaves) in the future. This could include mitigation and/or adaptation. Specific policy plans to deal with heatwaves (such as heat plans) where climate change was not mentioned were not included. Overall, nearly a third (30%-79 articles) included a mention of possible solutions. The differences between countries were not marked as the range fell between France (34%-32 articles) and the UK (24%-22 articles).

Mentioning the links to climate change
The finding that nearly 90% of the articles covering the heatwave across all four countries did not mention any link with climate change could be down to several reasons: journalists and editors may not see the necessity of making readers aware of the role of climate change in heatwave occurrence; a lack of space; or the lack of information about attribution studies. In this context, it is worth pointing out that some of the articles, particularly on n-tv. de, are very short and focus on visual content. Moreover, the Telegraph published several articles on what to wear during the heatwave, or survive it, as did the Mail, which also dedicated a lot of coverage to reporting on how celebrities were coping with the heatwave, and in particular how they were dressed. One could argue that the choice of angle to the story made a mention of the link to climate change less appropriate or likely.
The fact that opinion pieces about the heatwaves were scarce in our sample stands out in sharp contrast to other studies about climate change articles which suggests that the ratio between opinion pieces/editorials and news articles can reach over 50% (e.g., when assessing the presence of climate skeptics voices, see Painter and Gavin 2016). The relative absence of opinion pieces in our sample might attest to the relative lack of contestation or controversy over the link between climate change and the heatwave.
The UK sample showed considerable variation between left-leaning and right-leaning media outlets in the percentage of coverage mentioning the link. More than three in five articles in the Guardian mentioned some link to climate change, the most of any media outlet in our survey. It is noteworthy that even before the publication of the WWA reports, Guardian articles were making the link with climate change. The phrase "Scientists have said Europe's 2019 heatwave, like last year's, was closely linked to the climate emergency and that such extreme weather events will be many times more likely over the coming decades" was repeated verbatim in two articles, as was another phrase quoting the Met Office. 14 Our results support the thesis put forward by Hopke (2019) that for the UK at least, investment over time in specialist climate reporters may have improved the quality and the quantity of the coverage (for the BBC and the Guardian), although more research would be needed for the other three countries. The results also show how the Guardian stands out for its consistent, science-based coverage which seemed to follow an editorial policy of frequently making the link to climate change. A wide variety of environment correspondents and also reporters in different countries mentioned the link.
In the Netherlands, the low percentage of articles mentioning the link could be due in part to journalists at times thinking the link was too obvious to mention, when reporting, for example, about different solutions (e.g., sustainable cooling systems or planting more trees). The Dutch media also seemed to concentrate often on the record-breaking nature of the heatwave, perhaps because heatwaves in general were not perceived as that unusual anymore. 15 In Germany, where the number of articles mentioning the link was also comparatively low, one explanatory factor could be that 2019 registered a lower average summer temperature than the two hottest years of 2018 and 2003. Furthermore, the overall increase of consecutive hot summers throughout the 2010s could have diminished the uniqueness of the heatwave in 2019, and thus the urgency to report on the possible role of climate change. In France, the low percentages are partly explained by the sheer volume of coverage of the heatwave (see Column 4, Table 2), perhaps because it lasted longer and had more local impacts.
Our results suggest that in addition to the differences in climate coverage between leftleaning and right-leaning outlets described in Section 1.3, there are also sharp distinctions in some aspects of their coverage of extreme weather events. The UK and the Netherlands showed statistically significant differences between right-and left-wing outlets in the percentage of articles mentioning the link to climate change. In Germany, ideological differences were the least significant of all four countries. In France, the differences were present (e.g., Le Monde compared to Le Figaro and Ouest France) but less marked and the range of percentages (4-12%) is less wide, which may suggest a more common journalistic "culture" of climate change in the French media, where ideology is not as important a driver compared to the UK and the Netherlands. Prima facie, this seems to show a difference with the findings of Dirikx and Gelders (2010), mentioned above. However, they found a distinction between outlets as regards the urgency of taking action, whereas our focus is on an aspect of the science of climate change which we suggest has exhibited more consensus across the French media in recent years. Finally, the websites of broadcasters in France, the Netherlands, and the UK tended to show a higher percentage coverage of the link than right-leaning publications, but not as much as left-leaning publications, which is probably to be expected given their aim to be non-partisan and reach a general audience.

Describing the links to climate change
One of our important findings that both of the WWA attribution studies received strong coverage on the day of their publication and continued to be mentioned in articles in subsequent weeks across all four countries is notable for the contrast with previous studies mentioned in Section 1.4 above, which suggested that the link between climate change and specific heatwaves had been underreported.
The two most common types of statements describing the link found in our sample were generic statements and likelihood statements. As summarized in the literature (e.g., Painter and Hassol 2020), these types fit well with the two main ways to describe the link which are consonant with climate science. The first type (generic) is based on an understanding of the physics of how a warmer climate would be expected to affect such events, as well as observed and/or projected trends, lending confidence that some of these trends are attributable to anthropogenic warming. Such statements (particularly made before EEA studies of an event have been carried out) reflect accurately what scientists expect to happen due to human-driven climate change, and avoid the pitfall of repeating what used to be a common phrase "no single event can be attributed to human-caused climate change." Indeed, statements of this last type were largely absent from our sample.
The second major type involves extreme weather event attribution studies which, as described above, analyze actual, individual extreme weather events, using data and models to estimate how much more/less likely or more/less severe a specific weather event became as a result of anthropogenic climate change. With this in mind, it is clear that the journalists were very often (accurately) using phrases rooted in the science to describe the link, particularly when we bear in mind that the fourth most common phrase used (intensity statements) also reflects the science. This finding is closely related to the fact that WWA reports and climate scientists were frequently quoted (see Sections 3.3 and 3.4).
However, it is important to point out that journalists were not simple adherents to the language of the reports, as the phrases in English "return time" or "return period" commonly used by scientists (and found in the EEA studies) to describe the average number of years between the recurrence of similar events were only present in 2 out of the 90 articles in the UK sample (although the concept was sometimes explained without using the phrase).
It was also noteworthy that statements explicitly or implicitly using the concept of causation (that climate change caused the heatwaves in a deterministic sense) were the least present (at just 5%) and when they were used, it was often in the context of climate change causing the warming trend rather than the actual event. 16 In this context, it is important to remember that several prominent climate scientists have pointed out that framing the question as whether climate change caused this specific event as unhelpful or misguided, and that the better question is whether it is making them more extreme or frequent. 17 The same article often included a range of phrases or quotes from scientists to describe the link (other than generic/likelihood/intensity/causation statements), which partly explains the wide variety of other types of statements captured in the "other" category, examples of which can be found in Table 3. In addition to the ones described there (the 1 3 heatwave(s) being proof of/evidence for/indicator of/consistent with climate change or "climate change is playing a role in the heatwave"), other phrases appeared, such as "climate change is moving the goal posts" or "you can see the fingerprint of climate change in this heatwave." This study did not analyze the relative presence of such phrases, but a more detailed mapping of the different, scientifically robust, phrases both used by the media and available to communicators would aid our understanding of how these phrases affect public understanding, engagement, and action differently.
One of the reasons why the classification of the different phrases is important is that recent focus group research has noted that specific phrases and visuals used to express EEA findings can play a considerable role in shaping how the public engages with this material . For example, participants preferred relatively simple phrases wherever possible when communicating EEA results, and overcoming numeracy difficulties was an additional challenge when confronted with likelihood and intensity statements.

Quoting sources
Other studies have shown a much higher presence of politicians and NGOs being quoted about the link (44% in, e.g., Painter et al. 2020, compared to 15% here). The presence of mainstream climate "expertise" (in around three-quarters of all articles) is a stand-out result when we bear in mind the evidence in the past from other studies of quoted sources about climate change in the media, where climate scientists have been much less prominent compared to politicians and NGOs, even when talking about climate science (Eide and Kunelius 2010; Painter 2011). This contrasts to the coverage of climate-related "policy events" such as the UN's annual Conference of Parties (COPs), where NGOs and journalists have been found to form "networks of coproduction," creating common interpretation of the COPs in the media (Luck et al. 2016).
These heatwave events were clearly an opportunity for scientists to show their expertise, and can be seen almost as a "science event." The WWA had the support of an NGO in the promotion of its studies to journalists in some countries. In addition, journalists in the UK were provided with regular quotes from scientists about the heatwave from the Londonbased Science Media Centre, 18 some of which were transferred verbatim into the text of the articles, in the same way the Centre supplies quotes on "policy events" like IPCC reports.
As mentioned above, the French and UK samples included more quotes from politicians than the Dutch and German samples. Seven out of the eleven quotes in the French sample were government ministers, and in the UK sample, five were French politicians, and none was UK politicians (the one "political" quote was from a representative of Extinction Rebellion).
We did not assess the accuracy of each of the quotes from French politicians, but it is worth pointing out that President Macron seemed to have been scientifically accurate when, for example, he was quoted as saying that "such extreme weather is likely to become more frequent as a result of global warming." 19 Politicians in different countries have often been guilty of making inaccurate statements about extreme weather events, where comments blaming climate change for certain events have been described as "science by assertion, not evidence" (Painter and Hassol 2020). But in this instance, the French political class seemed more vocal and more accurate about the link to climate change, a broadly similar conclusion to that found in other studies (Jiménez-Gómez and Martín-Sosa-Rodríguez 2021).
It is also worth stressing the unusual statistic that there were only two coded quotes in the whole sample where the link was either denied or strongly questioned, and both came in the German sample. Forms of skepticism are not new to right-leaning newspapers in Germany (Schmidt-Petri 2017), and support for right-wing populist parties (such as the AfD) is strongly correlated with climate skepticism on the science and policy options (Han et al., 2021). However, the lack of contestation about the links between climate change and the heatwaves in the UK sample is more noteworthy as the country is known for the high historical presence of different types of climate skeptics in the media compared to other European countries (Painter and Ashe 2012), and particularly in opinion pieces in the right-wing Mail and the Telegraph (Painter and Gavin 2016;Brüggemann and Engesser 2017), which formed part of our sample. In the past, climate skeptic commentators have explicitly doubted the link between heatwaves and climate change. 20

Conclusions
Our detailed content analysis offers robust answers to the research questions outlined at the end of Section 1. To summarize, we found strong variations between countries and media outlets in how much journalists pay attention to links between climate change and the heatwaves. The UK media mentioned the links more than twice as much as other countries, and in general left-wing online sites included it more than right-wing sites. Journalists find many different ways to describe the link, but in the majority of cases they use accurate, science-based descriptions. In our case study, where two EEA studies were published in "near, real-time," these studies received widespread and often sustained coverage in all four countries, although four of the twenty outlets did not mention them at all. Unlike the findings from other studies (e.g., , climate scientists working for universities, Met Offices, and research institutions were quoted frequently and far more than politicians and NGOs. There are important limitations to our results. Some of the coverage of the 2019 summer heatwaves, as well as an attribution study about them, fell outside the 4-month period we examined; we did not evaluate the salience or dominance of mentions of the link with climate change, but only their presence (see Painter 2013, pp. 57ff for the difference). 21 For this study, we did not interview the journalists from the 20 media outlets writing about the heatwave to better understand their personal approach or the editorial policy of their media organizations towards the coverage of extreme weather events. However, this aspect formed the basis for a follow-up study (Strauß et al. 2021). We did not analyze the visuals accompanying the text, when the advantages of following a multimodal approach (i.e., including images) to content analysis have been rightly stressed (Wozniak et al. 2015), but this aspect will also be analyzed in a supplementary study. Finally, this study only looked 1 3 at heatwaves in four northern European countries and did not include southern Mediterranean countries, which are particularly vulnerable to the impacts of heatwaves such as forest fires such as those seen in 2021.
The importance of understanding links between extreme weather and climate change is likely to increase as such events become more commonplace and intense around the world (IPCC 2018(IPCC , 2021. Modelling suggests that in Europe, extreme heat events will become more frequent with return periods of 1.8 to 7.2 years under different scenarios (Ma et al. 2020).
Finally, an examination of the extent to which the media link extreme events to climate change and the ways they describe it has implications beyond a better understanding of journalism practice-such events could provide important opportunities to engage the public on the risks of climate change. By helping to make climate change visible, some researchers suggest that extreme events may serve as "teachable moments" that can increase climate change risk perceptions and support for climate action among the public . Continued investigation of media trends around the coverage of extreme weather events and other climate-related phenomena remains of paramount relevance for public understanding, engagement, and sustainable behavior change.
Author contribution JP: conceptualization, data curation, formal analysis, methodology, writing-original draft, writing-review and editing; JE: data curation, methodology, writing-original draft, review and editing: MD: data curation, methodology, writing-review and editing; NS: data curation, methodology, writing-review and editing; AW: data curation, methodology, writing-review and editing; PW: supervisionreview and editing.
Funding This work was funded in part by a grant from the Royal Bank of Canada (reference: ENV20197234).

Data availability
The manuscript has no associated data.
Code availability Not applicable.

Declarations
Ethics approval Not applicable.

Consent to participate Not applicable.
Consent to publish Not applicable.

Conflict of interest The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.