Keywords

4.1 Introduction

In this chapter we present our analytical framework for popularization discourse. First, we discuss the main aims that our framework should comply with. Then, we offer insight into the development of the framework. The framework consists of five themes, which are explained, and 34 strategies, which are elaborated upon by offering an explanation and application remarks, as well as suggestions for further reading. Lastly, the analytical framework is compared to existing frameworks and rubrics in the academic literature.

4.2 Considerations in Setting Up the Framework

In Chap. 3, we discussed existing insights into textual features, or strategies, in popularization discourse. The discussion covered frameworks (August et al., 2020; Giannoni, 2008; Hyland, 2010; Luzón, 2013; Motta-Roth & Lovato, 2009; Nwogu, 1991) as well as rubrics (Moni et al., 2007; Poronnik & Moni, 2006; Rakedzon & Baram-Tsabari, 2017a, 2017b; Yuen & Sawatdeenarunat, 2020). The discussion showed issues in the reliability and usability of these frameworks and rubrics; they do not show a clear line in the strategies that are presented, they are not compatible among each other, they are not validated or constructed with the use of multiple raters in mind, they often consist of results from text analysis and as such do not present a coding scheme, and they mostly cover subgenres of popularization within specific disciplinary settings. Therefore, an overarching framework that is usable in the analysis of popularization discourse is still missing from the academic literature and, perhaps more importantly, from practice. Such a framework should ideally comply with four aims. An analytical framework for popularization discourse is:

  1. 1.

    usable in any subgenre of popularization discourse

  2. 2.

    usable in disciplinary but also multidisciplinary and interdisciplinary settings

  3. 3.

    reliable for use with multiple raters

  4. 4.

    easy to apply by offering application remarks and explanations of strategies

4.3 Methodology

In this section we will present a brief overview of the methodology and set-up of our analytical framework.

The methodology consisted of a construction step and a validation step. The aim of the construction step was to gather a corpus of newspaper articles based on a single academic source text. This would ensure that texts and strategies presented in them are easily comparable to each other and to the academic source material. To achieve this aim, 140 first-year undergraduate liberal education students were asked to write a newspaper article based on the source text “#Sleepyteens: Social media use in adolescence is associated with poor sleep quality, anxiety, depression and low self-esteem” (Woods & Scott, 2016). Participants were asked to read this publication before class. In class, they were asked to write a text about it that would be suitable to publish in the science section of a quality Dutch newspaper. The text had to be within a 400-word limit. The target audience of the newspaper article consisted of a general audience that was interested in science but did not necessarily receive higher education training.

This corpus of newspaper articles was then analyzed for the occurrence of popularization strategies. We worked in test rounds, analyzing 10 randomly selected texts from the corpus in each round. The strategies from Luzón’s (2013) research into science blogs were used as a list of a priori codes in the first round, after which descriptive coding was used (Saldaña, 2015) to indicate if each of the strategies was used. Simultaneous coding was allowed, meaning text could be coded for multiple strategies. We used consensual coding (Schmidt, 2004): we compared coding and discussed difficulties and uncertainties. This process of coding and adapting the framework was an iterative process in which, during each round, the a priori code list was adapted according to the insights that were generated through deliberation about the coding, by splitting, merging, deleting, and adding codes. In each new round, the adapted list was used as an a priori list. This process continued until code saturation was reached after six rounds. In a seventh and final round, 10 texts that led to the most discrepancies earlier on were re-analyzed. Throughout the coding rounds, many adaptations were made to the list of codes; some were added or deleted, others split or merged. On the resulting list of 34 codes, pattern coding was used as a second-cycle coding technique to thematize strategies (Saldaña, 2015). Luzón’s (2013) framework had originally contained three themes; rhetorical category, strategies to tailor information, and strategies to engage the reader. These themes were reworded into Subject Matter, Tailoring Information to the Reader, and Engagement. Furthermore, the themes Credibility and Stance were introduced—which are also used in Hyland’s (2010) framework—to thematize the strategies that fell outside of the scope of the existing three themes.

The inter-rater reliability was checked after each round by using both percent agreement and Cohen’s kappa with 95% confidence intervals. Cohen’s kappa is used for inter-rater reliability between two raters and controls for agreement because of random guesses. Scores can range between 0 and 1 and include 95% confidence intervals as kappa is an estimate of inter-rater reliability (see McHugh, 2012). In the first round, the inter-rater reliability for our analytical framework was 0.55 (0.46–0.65 confidence intervals), showing a weak level of agreement. In the next rounds, the reliability increased and ultimately reached a kappa of 0.90 (0.86–0.95 confidence intervals) in the seventh and final round, denoting an almost perfect level of agreement.

The aim of the validation step was to check the framework against a corpus of texts from multiple different subgenres of popularization discourse, written by professionals, and containing multiple topics and source texts. This validation step was needed because all texts in the construction round were based on a single source text from a single academic field, thus creating the possibility that some strategies could not be employed. Furthermore, the corpus in the construction phase was written by students, not professional writers, thus creating the possibility that some strategies were not used. This second corpus consisted of 38 science journalism articles written by a range of professional media outlets that were chosen according to Berezow’s (2017) infographic on quality of science reporting. The two axes of this infographic present the compellingness of the content and the degree of evidence-based reporting of different media outlets. This creates a three-by-three grid ranging from ‘evidence-based reporting’ with ‘almost always compelling science content,’ to ‘ideologically driven reporting/poor reporting’ with ‘no compelling science content.’ The corpus contained texts from all represented quadrants of science reporting (for example, the option ‘always compelling science content’ with ‘ideologically driven/poor reporting’ was not represented by any outlets), apart from the quadrant of ‘not usually compelling science content’ with ‘ideologically driven/poor reporting,’ as this is seen as poor science reporting overall. The corpus contained many different topics (and thus disciplinary fields) and multiple subtypes of popularization such as news articles and overview articles. The reader can find more about this corpus in Chap. 6. The corpus was checked against the analytical framework with the particular aim of analyzing how often strategies were used and to check if any previously undiscovered strategies could be found. In this round, one strategy was deleted and five additional strategies were added to the already constructed themes. The final framework contains 34 strategies captured under five themes and will be explained in the next two sections.

4.4 Themes

In this section we describe the five themes that comprise the framework: Subject Matter, Tailoring Information to the Reader, Credibility, Stance, and Engagement.

Subject Matter includes strategies concerning content from the original scientific publication. Although it is impossible to construct one single organizational structure of popularization texts, several rhetorical strategies usually appear in them (Luzón, 2013). Instead of a “narrative of science,” popularization texts provide a “narrative of nature” (Hyland, 2010, pp. 120–121). By moving the main claim to the first paragraph, the focus shifts to novelty and importance. The object studied becomes more important than the methodological steps taken (Hyland, 2010). Uncertainties that would be discussed in the academic text are removed and the focus is on results (Fahnestock, 1986).

Tailoring Information to the Reader contains recontextualization strategies that remodel academic findings to an everyday-life and understandable setting. In academic texts a shared base of knowledge is assumed between writer and reader, yet this is not the case in popularizations (Hyland, 2010). Therefore, popularizations need to be recontextualized from the academic context to that of the lay audience and need to be perceived as suitable within the new context (Gotti, 2014). Information is tailored to readers by connecting to what they (are presumed to) already know, through explanations or connections to everyday life (Hyland, 2010). In part, this recontextualization takes place by focusing on the application and consequences of a phenomenon (Fahnestock, 1986).

Credibility is created in academic texts when authors position themselves in relation to other researchers and publications. For popularization texts, it is assumed that readers do not possess disciplinary knowledge or cross-disciplinary expertise, so credibility of the source is emphasized instead. Researchers important to the topic under discussion are mentioned and credibility is constructed through their academic position. Furthermore, quotes underline the credibility of the presented material. In academic texts, credibility increases through depersonalization as it suggests objectivity. The opposite is true for popularized texts, in which personalization strategies are used (Hyland, 2010).

In popularization discourse, Stance on the topic under discussion can also be communicated. The media contribute to opinions that are formed about research and researchers (Calsamiglia & Van Dijk, 2004). Furthermore, personal attitude and the expression of stance play a big part in constructing proximity. In academic texts, hedges are used to indicate that researchers are careful in their statements, but these are removed in popularized texts to create more impact for academic findings. Instead, the popularization writer uses stance and opinions to comment on the research or the publication to engage the reader (Hyland, 2010).

Engagement is used to connect to readers; writers use it to signal their awareness of the audience’s presence. Engagement strategies use discourse that is informal and geared toward the reader to get their attention, create a shared understanding, include them as participants in the discourse, and influence them (Hyland, 2010; Luzón, 2013). Many strategies that are part of this theme play an active role in reformulation.

4.5 Strategies

The five themes together contain 34 strategies. Table 4.1 provides an overview of each strategy in the analytical framework. Each strategy contains an explanation that is based on literature and application remarks that are based on our experience from working with the framework. The ‘further reading’ column presents additional sources that cover each strategy—the interested reader can explore each strategy further using these sources.

Table 4.1 The framework of analysis for popularization discourse
Full size table

4.6 The Framework versus the Literature

We established that, ideally, an analytical framework should be compliant with four aims. Our framework is usable in any subgenre of popularization because in its construction and validation phase, multiple subforms of popularization were taken into consideration. The framework can be used in any disciplinary, multidisciplinary, or interdisciplinary field because texts from the validation phase represented a range of disciplinary fields, none of which led to any issues in coding, and the student writing was produced as part of an interdisciplinary undergraduate program. Furthermore, there is no disciplinary bias in any of the themes or strategies in the framework. The framework is reliably usable by multiple raters because we worked with multiple raters during the construction phase and this collaboration produced a reliable kappa. In fact, we would advise anyone working with this framework to analyze texts in duos and thoroughly discuss differences of opinion, as this process is very insightful for learning how the analytical framework works, as well as generating insight into your own frame of reference. Lastly, the framework is easy to use because we offer application remarks and an explanation of each strategy.

As was mentioned in Chap. 3, it is difficult to make one single overview of all popularization strategies from the available frameworks. Still, it is possible to relate the framework that we just presented to the existing literature. First, the connection to previous sources can be seen in the ‘further reading’ column in Table 4.1, which shows other sources in which a particular strategy is also presented (it should be noted though that not all sources mentioned in this column present an explicit framework). Except for link to the academic publication, all the strategies in our framework are mentioned in at least one of these sources. Some features are mentioned in multiple frameworks, such as main findings (in our framework: presenting results/conclusions), analogy/metaphor (in our framework: imagery), describing the method, applied implications, explanations, questions, humor, opinion, and contextualization. Other features are mentioned once, such as hyperlinks, additional sources, and giving the researcher an active voice. On the other hand, our framework is not simply an aggregation of these sources, and not all strategies mentioned in them have become part of the analytical framework.

Furthermore, there are several key differences between earlier frameworks and the framework presented in this book. Examples of strategies that are mentioned in other frameworks that are not part of our framework are story (storytelling/narrative) and active (active voice) (August et al., 2020), contingency (the effect of personal experiences on work or beliefs) (Giannoni, 2008), argument structures (Hyland, 2010), and expressions of feelings or emotional reactions (Luzón, 2013). Because our aim was to construct a framework that was evidence-based and these strategies were not found in our text analyses, and because we did not want to make an aggregation of all strategies that were discussed in the literature, they are not included in our framework. Another difference is that some of these sources might present strategies on a different level, for example as sub-strategies or aggregated into one. This led to issues of incompatibility between existing frameworks. Our framework does not contain sub-strategies, which ensures all strategies are presented on the same level of importance. Furthermore, our framework does not presume a specific order of linguistic moves, like Motta-Roth and Lovato’s (2009) and Nwogu’s (1991) frameworks do, as we want to enable coders to make inferences based upon their own analyses. However, we do recognize that in the data we used for our research, some moves are often seen in a specific order (contextualization, novelty, announcing the new finding or new contribution to the discipline), or appear together (examples from daily life with inclusive pronouns or references to the reader).

Similarly, none of the popularization rubrics discussed in Chap. 3 can give an overarching insight into popularization discourse, but some of the strategies (or assessment criteria) mentioned in them are also presented in our framework. They are key facts (in our framework: presenting results/conclusions), background (in our framework: contextualization, novelty) (Moni et al., 2007; Poronnik & Moni, 2006), titles (Rakedzon & Baram-Tsabari, 2017a, 2017b), academic implications and applied implications (Yuen & Sawatdeenarunat, 2020), and explanations (Rakedzon & Baram-Tsabari, 2017a, 2017b; Yuen & Sawatdeenarunat, 2020).

These rubrics also explain processes that are needed to construct effective popularization texts, such as the use of active voice (Rakedzon & Baram-Tsabari, 2017a, 2017b) or the use of language strategies to appeal to and engage readers (Yuen & Sawatdeenarunat, 2020). These processes are more about actions a writer should take that ultimately will lead to certain textual elements than they are about those textual elements directly, which is why they are not part of our framework.

In Chap. 2 we discussed the two main textual processes that construct popularization discourse, and through them form popularization strategies: recontextualization and reformulation (Bondi et al., 2013; Calsamiglia & Van Dijk, 2004; Ciapuscio, 2003; Gotti, 2014). Recontextualization entails moving scientific facts from the expert context to the layperson context and, in doing so, presenting specialized knowledge in a way that non-specialized readers can understand (Bondi et al., 2013; Calsamiglia & Van Dijk, 2004). Many strategies in our framework are a form of recontextualization: lede, contextualization, announcing the new finding or new contribution to the discipline, novelty, describing the method, presenting results/conclusions, applied implications, hyperlinks, visuals, academic implications, mentioning more research is necessary/next step in research, contribution to research, mention of statistics, lexical mention of the original research, additional sources, link to the academic publication, direct quote from the academic publication, in-text specification of a source, opinion, titles/subheadings, references to popular lore and beliefs, and popular culture, self-disclosure of the author’s public or personal life, and humor. Reformulation remodels the language that is used to the new target audience (Gotti, 2014). Reformulation occurs in the strategies explanations, imagery, stance markers, inclusive pronouns, references to the reader, features of conversational discourse, questions, and explicit self-reference. Some strategies are a combination of reformulation and recontextualization elements: giving the researcher an active voice, giving the non-researcher an active voice, and examples from daily life. These strategies recontextualize the content of the academic text, but they also rephrase it into colloquial language. Our framework also shows that it is possible to construct other goals or themes through recontextualization and reformulation processes. An example here is the theme Credibility, which is entirely made up of recontextualization strategies.

What You Have Learned in This Chapter

  • In this chapter, an analytical framework for popularization discourse is presented that is usable in any subgenre of popularization discourse, usable in disciplinary or multidisciplinary/interdisciplinary settings, workable and reliable with multiple raters, and easy to apply.

  • The framework consists of five themes (Subject Matter, Tailoring Information to the Reader, Credibility, Stance, and Engagement) and 34 strategies that are explained and supported by further readings and application remarks.

  • There are points of overlap between this analytical framework and existing frameworks from the literature, but also key differences; this framework is not an aggregation of existing efforts and aims to improve upon existing work.