Introduction

Communicating scientific research on sustainability goes beyond informing and awareness raising (Newig et al. 2013). Its ultimate goal should be facilitating the discussion and shared understanding of available knowledge to improve management practices (Lindenfeld et al. 2012). Sustainability science often addresses this goal by facilitating knowledge exchange between researchers and actors from multiple disciplines and backgrounds such as practitioners (i.e. project managers and advisors among other interested professionals) (Schneider and Buser 2018). One of the key challenges of science communication is to improve the accessibility of research to practice so that we can discuss the management problem with multiple actors and promote the contribution of research outputs (Djenontin and Meadow 2018; Roux et al. 2006).

Despite the increasing number of (open access) scientific publications, accessibility is not only about the timely and online availability of these publications (Cvitanovic et al. 2015). Accessibility also means tailoring available knowledge into formats that practitioners can better engage with and that enables them to identify what is useful for their work (McInerny et al. 2014). Currently, two mismatches impede this second dimension of accessibility. First, the content and format of scientific publications do not align with a practitioners’ focus on the goals and implications for management (Cvitanovic et al. 2014). Researchers are often more concerned with developing new knowledge, tools, methods, or models to advance science rather than tailoring their results to a non-scientific audience (Borowski and Hare 2007). In addition, the use of jargon often privileges those who are familiar with the specific terms and level of detail (Leggett and Finlay 2001). Second, management actors such as project managers and advisors have their own knowledge and expertise (Witting 2017). They have different sources of information and limited time to identify the usefulness of scientific knowledge (Cairney and Kwiatkowski 2017). Their perceptions of what is useful do not only rely on the relevance, credibility, and legitimacy of available knowledge (Cash et al 2003) but also on their management roles, interests, and beliefs (Lemos et al. 2012). Visuals and storytelling (among other strategies) are a means for sustainability researchers to narrow this gap by supporting practitioners in increasing their understanding of, and their interest in, research outputs (Maher et al. 2018; Young et al. 2016).

Humans have used storytelling and visuals to convey information for millennia (Krzywinski and Cairo 2013). In the Internet era, researchers increasingly communicate their work by combining short texts with appealing and (interactive) visuals (Figueiras 2014). Attention to visual storytelling and data visualization is growing in environmental communication to, for example, represent changes over time and space to multiple audiences, such as management actors (Herring et al. 2017; Stephens et al. 2014), students (Berendsen et al. 2018; Cope et al. 2018), and the public (Bednarek and Caple 2010; Cairo 2013). In turn science communication involves conceptual challenges to define the necessary approach for a given audience (Jones and Crow 2017). It further involves methodological challenges to evaluate its effectiveness for the intended purpose (Downs 2014). There are also ethical challenges for increasing understanding and interest while conveying accurate and transparent information (McInerny et al. 2014). This is particularly important when addressing environmental problems that are contested and have multiple interpretations by the variety of actors involved (Brugnach et al. 2011). Data visualization in non-scientific contexts often employs user-centered and collaborative design approaches to meet the preferences of audience groups (Grainger et al. 2016). However, there is still a lack of empirical evaluations of the effectiveness of visual storylines to communicate scientific findings to practitioners working in funding agencies, governments, and consultancies (Kosara and Mackinlay 2013). Moreover, science communicators and researchers working on sustainability issues need to be aware of the opportunities and challenges of implementing a given approach at the science-practice interface (Evans and Cvitanovic 2018).

Our objective here is to conceptualize a series of steps to prepare visual storylines and to evaluate the effectiveness of the storyline content and design. Following the exploratory study of Cortes Arevalo et al. (2019), we define storylines in this context as an online, visual, and scrollable overview of the contribution of research to practice. The storylines address a multi-disciplinary audience of project managers, advisors, and other interested professionals involved in the implementation of river management strategies. To inform our approach, we first identified principles of visual storytelling from the literature and conceptualized the way it may influence the science-practice interface. Next, we applied this broader understanding to prepare three storylines and evaluated their perceived usefulness and engagement via focus groups to answer the following research questions:

  1. 1.

    How do participants perceive the usefulness of the storylines for communicating river research?

  2. 2.

    How do the specific content and design influence participants’ engagement with the storylines (i.e. ease of understanding, attractiveness, and navigation)?

  3. 3.

    What can we learn from the content and type of feedback that participants provided?

Based on our findings, we identify key suggestions for improving the storyline content and design. Finally, we discuss the results and the limitations of our study while reflecting on the process to prepare the storylines to inform future applications for the communication of sustainability science.

Visual storytelling at the science-practice interface

Visual storytelling includes a single or a collection of visuals (i.e. images, visualizations, or any combination thereof) that are either part of a larger text or have little or non-accompanying text (Figueiras 2014). Since there is no unique approach or definition, learning how to make an effective visual storyline is a process that implies a number of steps and choices for the content and design. In this section, we inform our approach from the literature in science and environmental communication to identify key principles for preparing storylines at the science-practice interface:

Defining the audience and goal

The first key principle of storytelling is related to the intended audience and goals (Davidson 2017). In general, three types of audiences can be identified: (i) a science-informed audience who reads or actively seeks out for research or technical findings; (ii) a science-interested audience who is willing to invest some time in learning about research findings; and (iii) a residual not interested and not necessarily well-informed audience (Stewart and Nield 2013). With a focus on the knowledge exchange at the science-practice interface, we address a multi-disciplinary audience of both science-informed and interested professionals and further distinguish policy-makers from practitioners. Policy-makers typically develop or coordinate government strategies at an aggregated level such as national or regional (Borowski and Hare 2007). Within the practitioner audience, project managers implement these strategies and are part of decision-making processes at a more specific location or operational level (Roux et al. 2006). Advisors often provide input to the above processes with their specialized expertise (Witting 2017). Other interested professionals may provide direct input to decision-making processes and/or advise to specific stakeholder groups (Young et al. 2016). Given the diversity of actors and interpretations (Brugnach and Ingram 2012), the ultimate goal of our storylines is to: (i) trigger the interest of viewers to later revisit the storyline; (ii) share it with others; or (iii) follow up by contacting the project team to discuss the applicability of research outputs.

Selecting the scientific content

When planning communication efforts at the science-practice interface it is important to focus on the relevance, credibility, and legitimacy of the information presented (Cash et al. 2003). Heink et al. (2015) define relevance as the degree to which content fits the audiences’ expectations and consider their decision-making needs; credibility as the audiences’ perceptions about the trustworthiness of the source of information and the way it is presented; and legitimacy as the extent to which the information was produced and/or transmitted in a way that is open and observable by the intended audience. Witting (2017) further outlines that humans assess new information based on either their slow and analytical thinking or fast and intuitive thinking. Given their limited time, policy-makers (Cairney and Kwiatkowski 2017) and also practitioners (Young et al. 2016) often rely on their fast and intuitive thinking. Therefore, they often prefer information that is congruent with their own knowledge and experiences (Davidson 2017; Jones and Crow 2017). Visuals can help this fast-thinking by providing shortcuts to translate abstract concepts or terms that have different meanings across disciplines (Maher et al. 2018) and between scientific and non-scientific audiences (Venhuizen et al. 2019). Moreover, story or narrative elements can help to process and make sense of new information by relating it to the specific context in which the story of the research took place (Downs 2014).

Identify the story parts or narrative elements

We understand storylines as the translation of a wider (scientific) text into its core narrative elements that explain the benefits and limitations while at the same time illustrating the research context and the new knowledge with visuals (Cortes Arevalo et al. 2019). Based on policy narratives, Jones and Crow (2017) suggest preparing science stories by (i) defining the characters; (ii) providing a sequence of events that occur in the setting where there is a problem to solve; and, (iii) giving a moral or take-home message. In a similar vein, Murray and Sools (2015) define storyline parts (or narrative elements) of a text by distilling: (i) the agent or main character of the story which is not limited to the researcher but may include other actors; (ii) the purpose that is typically the desired goal of the main character; (iii) the acts or events happening or typically performed by the main character; (iv) the setting, i.e. the physical or emotional environment where the events take place; (v) the problem which is typically a conflict to be solved; and vi) the means that are either hindering or helping to solve the problem. In the “Research setting and methods” section, we further explain how we used these elements to prepare our storylines.

Story visualization and presentation

The combination of visuals and story elements can drive user engagement via cognitive and affective involvement (Figueiras 2014). Referring to the visuals (and accompanying text), the cognitive involvement or comprehension level is influenced by (i) elements that direct the visual attention; (ii) visual complexity; (iii) elements that help inference such as annotations; and (iv) tailoring to the audiences’ expectations and previous knowledge (Harold et al. 2016). Affective involvement refers to the emotionally appealing elements of visuals that not just call attention to them, but make them memorable while increasing comprehension (Altinay 2015). Similarly, story parts or narrative elements influence the affective and cognitive involvement by sparking the interest of the audience in a way that they can identify, recall, remember, or contextualize the content (Negrete 2014). There are several strategies for the story visualization and layout design ranging between an author-driven and viewer-driven interaction (Segel and Heer 2010). In our study, we used the layout design provided by the ESRI StoryMaps tool (ESRI 2018). It allows authors with limited programming skills to integrate text with interactive visuals that the viewer can scroll through for choosing the parts to further engage and interact with (Kallaher and Gamble 2017).

User engagement and interaction

Engagement is a desired outcome that occurs through the viewer’s interaction with the (storyline) interface, texts, and visuals (Bahry and Masrek 2016). In this context, O’Brien and Toms (2008) refer to engagement as the affective, cognitive, and behavioral connection that occurs in time when a viewer assesses the information presented and/or interacts with a website. In addition to the affective and cognitive connections (explained earlier), websites and interactive visualizations allow for behavioral interaction, ranging from (Boy et al. 2015): visual annotation (hovering of content as opposite to static labels), zooming (inspection to show the specifics), hyperlinking (connection to show related elements), filtering (queries to show content conditionally), discovery (exploration to show something else), and navigation (buttons to show different sections). However, user engagement with the visuals (and accompanying text) does not necessarily increase with the number of available options as it depends on the audiences’ skills and the effort required for the user interaction (Harold et al. 2016).

Evaluation

In general, the effectiveness of communication efforts depends on the content, process, and outcome (Rohrmann 1992). The content evaluation focuses on the perceived usefulness according to the audiences’ preferences. The process evaluation reflects on the way communication is prepared and key actors are involved. Outcome evaluation includes, for example, the user engagement with the content and design (Downs 2014). The choice in the effectiveness evaluation depends on the specific goal and available resources (Hung and Parsons 2018). For example, Majooni et al. (2018) assessed the effects of infographic structure on ease of understanding using eye-tracking methods. Another purpose could be to evaluate the retention of content among university students using a quiz and self-reported ratings (Cope et al. 2018). For our study, we assess participants’ ratings and user experiences regarding the perceived usefulness of and their engagement with the storyline while identifying the storyline elements during focus group sessions.

Research setting and methods

We prepared and evaluated storylines as part of the online communication strategy of RiverCare, a research program studying the effects of river interventions to support sustainable management practices (Hulscher et al. 2016). The program comprised eight project themes in which 20 researchers worked together with representatives of government and private organizations. Depending on their project theme, researchers developed methods, models, or prototype tools to improve the fundamental understanding, monitoring, and integrated and collaborative management of river interventions. In this context, the storylines are an important component of the RiverCare website (www.rivercare.nl) with which we target a practitioner audience in the Netherlands and abroad. The website allows project managers, advisors, and other interested professionals to explore RiverCare knowledge and when desired give feedback about the possibilities for further development or application. The following subsections elaborate on our methods including the steps for the storyline preparation, the design of focus groups with representatives of the intended audience, and the data collection and analysis.

The steps to prepare the first storylines

The number of disciplines involved in RiverCare required an approach that was useful for researchers working in the wider domain of sustainability science. Figure 1 outlines the five steps for preparing the storylines of this study.

Fig. 1
figure 1

Steps to prepare the storylines

Full size image

Prepare the concept

The editorial team for the storylines included the coordinators of the RiverCare communication activities and the researchers whose work was presented (also co-authors in this study). We prepared three storylines, each focusing on one aspect and time scale of the studied river interventions. Each storyline was based on a scientific publication: one about public perceptions of river landscapes and management (Verbrugge et al. 2017), one on side-channel development (van Denderen et al. 2018), and one on the origin of stream bends (Candel et al. 2017).

Define the parts of the storyline

We used the storyline analysis of Murray and Sools (2015) to identify the story or narrative elements within the scientific publications with the aim to best explain the new findings, methods, or prototype tools. This approach (outlined in Fig. 2) includes typical questions that practitioners have when looking for new information. For example, it addresses both the benefits and limitations of the research, which was identified as an important requirement for the credibility of the storylines in our exploratory study (Cortes Arevalo et al. 2019). The central element in this scheme is the problem from which the setting, purpose, agent, acts/events, and means/helpers depart.

Fig. 2
figure 2

Storyline elements following the approach of Murray and Sools (2015)

Full size image

Draft content

We further provided researchers with guidelines for drafting their storyline that included Fig. 2 and a storyline structure to fill in using the headers, text, and visuals. The content was narrated from the point of view of the researcher (we or us) and included active voice sentences (you) to invite the viewer to relate with the text and visuals. An example of such a sentence is as follows: “Imagine that you have just designed or implemented an engineering intervention in one of the busiest rivers in Europe, the Waal”. Every storyline ended with a take-home message and a call for action that invited the viewers to ask questions and read further information.

Iterate content and visuals

After completion of a first draft by the researchers, the text and the visuals were iterated with the coordinator of the RiverCare storylines (also first author of this study) who uploaded the content into the ESRI StoryMaps tool (ESRI 2018). Visuals included photos to introduce the research context (Fig. 3a), timelines to illustrate the period in which the interventions took place (Fig. 3b), interactive maps to show the location or data collected, and hover over charts to interact with the location and example results (Fig. 3c).

Fig. 3
figure 3

Storyline layout including navigation elements such as the sidebar and various types of visual content such as a an introductory image of the stream bends storyline (Candel 2018), b timelines and clickable words of the public perception storyline (Verbrugge 2018), and c example results with hover-on content of the side channel storyline (van Denderen 2018). Visuals were mostly prepared from sources available via the Nationaal Park Drentsche Aa, Rijkswaterstaat, the related publications (Candel 2018; van Denderen 2018; Verbrugge 2018) and the RiverCare researchers

Full size image

Share and evaluate with representatives of the intended audience

In the final step, we shared the storylines with a purposive sample representing the varieties of organizations involved in RiverCare that may use the storylines for their own work. The aim of these focus groups was to better tailor the content and design for a practitioner audience. The next sections explain the focus group structure and evaluation procedures.

Focus groups

Four focus groups were held at partner organizations of the RiverCare program in the period between February and April 2018. Given the purposive sampling of our study (Morgan 2008), the participants in each focus group have a similar background, i.e. as researchers, advisors, project managers, or interested professionals. The purpose of the focus groups was to enable participants to interact with the storylines and self-report the perceived usefulness of, and their engagement with, the content and design. In addition, we recorded the screen interaction with the storyline and audio of the discussions during the focus groups.

The participants were invited by email via a contact person at the host organization to join a session of about one-hour and signed a consent agreement to allow the confidential use of data collected for this study. As preparation for the session, participants were asked to choose, explore and, if time was available, read one of the three storylines beforehand. A total of 44 participants participated in the focus groups (Table 1). Participants were distributed among 18 researchers attending the Netherlands Centre for River Studies conference (NCR), seven advisors from a consultancy (HKV), seven project managers at a regional division from the Ministry of Infrastructure and Water Management Rijkswaterstaat (RWS) and 12 professionals that joined the session at the Institute for Water Education (IHE-Delft). Most of the participants had a background discipline related to hydraulic engineering or geomorphology. Thirteen participants had a broader background in environmental management (half of which belonged to the IHE-group). Nineteen participants read the storyline beforehand. Most participants chose to interact with the storyline about public perceptions, followed by the one about side-channel development and stream bend formation.

Table 1 Distribution of participants per storyline and focus group session
Full size table

During the focus groups, participants worked in pairs (14 subgroups), trios (5 subgroups), and one participant worked individually. Most subgroups (16) had participants with the same background discipline whereas only four subgroups had mixed participants. Every session was moderated by the first author, who after a 10-min introduction asked participants to scan through their chosen storyline and carry out the following tasks:

  1. 1.

    Reflect upon their understanding of the storyline elements in subgroups by filling in the simplified form of Fig. 2 (see Appendix A, Fig. A1).

  2. 2.

    Join a general discussion to share and reflect upon their sub-group interaction with the storyline.

  3. 3.

    Fill in a survey (see Appendix A, Fig. A2) to rate the storyline usefulness, and to state their perceptions regarding their engagement with the storyline.

Each focus group served as an intermediate evaluation to adjust the storyline design and to collect suggestions for improvement. To maintain the comparability between sessions, adjustments to simplify the content and navigation of the storylines was kept to a minimum. For example, during the session with researchers, the storylines were embedded into the RiverCare site. Thereafter, participants had full-screen interaction. We also revised the storyline content to limit the amount of jargon and added a button with a drop-down list of the storyline parts.

Data collection and analysis

We used a mixed-method approach for collecting data during the focus groups (see Table 2), including a survey distributed at the end of each session for measuring the perceived usefulness of, and user engagement with, the storylines. In addition, we used screen recordings of the user interaction and audio recordings of the (sub) group discussions to identify how the specific content (text and visuals) influenced user engagement. We triangulated data from three sources: (i) the audio transcriptions; (ii) the notes from the screen recordings; and (iii) participants’ suggestions to the storyline elements in the subgroup form and the open-ended survey question. The analysis allowed us to illustrate the quantitative survey results and to identify suggestions for the storyline content and design.

Table 2 Overview of data collection methods and main variables per research question
Full size table

Survey questions

The survey consisted of two main parts (see Appendix A, Fig. A2). Part 1 included statements to measure the perceived usefulness of the storyline (i.e. whether the content and design meet users’ preferences and interests to use it as a communication tool). These statements were based on the criteria identified in the preliminary study of Cortes Arevalo et al. (2019). Answering categories for statements in Part 1 included three options: yes, it is useful; no, it is not useful and not sure.

Part 2 of the survey included items for participants to rate their engagement, as well as an open-ended question to ask for additional feedback. To measure the level of engagement, we used semantic differential scale items from the User Experience Questionnaire developed by Schrepp et al. (2017). This scale has 26 items of which all were included in the survey for the first focus group participants. Each item had a pair of terms with opposite meanings that participants could rate on a 7-point Likert scale. We included fewer items in the surveys for the subsequent focus groups to prevent survey fatigue due to lengthy questionnaires (see Table 2). The survey at the second focus group included five items (marked by an asterisk in Table 2), which we extended to a final selection of 10 items that better captures the ease of understanding, attractiveness and interest criteria.

We calculated the relative frequencies per focus group for the perceived usefulness response. For the engagement responses, we converted the collected ratings into a scale from − 3 to 3 to distinguish neutral from maximum and minimum ratings. Then, we calculated the minimum, maximum and average score per storyline and focus group.

Screen and audio recordings

For all focus groups, we made screen recordings to capture the user interactions with the storyline content. For the focus group with advisors, we took notes from the user interactions during the session given that no screen recordings were available. Audio recordings captured the subgroup discussions to reflect upon the storyline parts (Appendix A, Fig. A1) as well as the general discussion per focus group. Overall, this data complemented the responses to the survey for explaining participants’ preferences about the storyline content and design.

For the data analysis, we synchronized and re-watched the screen and audio recordings to take notes on the navigation preferences, usage of clickable elements, and participants’ comments. In addition, we transcribed the audio recordings for the same focus group parts in all sessions. We further coded the transcriptions to link them to the evaluation criteria (i.e. perceived usefulness, ease of understanding, attractiveness and interest, or navigation preferences). The second author reviewed the coding scheme and made suggestions to merge or re-assign the codes. Using this final dataset, we counted the frequency of a given code for each focus group. We equally acknowledged the contribution of all comments regardless of how many times it was brought by the same sub-group. Thereby, we identified the recursive feedback across all focus groups.

Finally, we analyzed the type of feedback that participants provided. We first categorized the indirect feedback deduced from the screen recordings to as observational feedback. Thereafter, we categorized all participants’ comments (depending on their level of detail) as either general, contributory or co-creational feedback (Sturm and Tscholl 2019). We define general feedback as reactive comments that point to either positive or critical aspects without referring to a particular (storyline) part. Contributory feedback goes beyond general feedback by including examples of the usefulness of or engagement with specific parts of the storyline. Co-creational feedback further includes alternative options that participants provided to adjust the content and/or design of a storyline part.

Results

Here, we report on the participants’ perceptions of and suggestions for improving storyline content and design according to the usefulness and engagement criteria. In each section, we first present the survey results and then further elaborate on our findings using the qualitative data (available in Appendix B). Finally, we reflect upon the type of feedback that participants provided according to the level of detail of their comments.

Perceived usefulness

More than half of the research participants, advisors, and interested professionals were positive about the usefulness of the storylines as a tool to explain why the study was done and to give relevant examples (Fig. 4, questions a and b). These two questions were rated as useful by researchers (67% for both questions), advisors (57% for both questions), and interested professionals (67% and 58% respectively). Participants’ feedback, however, provided a more detailed picture of their views on the usefulness of the storylines. First of all, all focus groups commented on the importance of tailoring the content to a specific audience. Participants acknowledged that the storylines may make the research more accessible (HKV-GR2 and IHE-GR4). However, it was not clear if the storylines are meant for people who are interested or who are conducting research on this topic (IHE-GR1 and NCR-GR5). Therefore, “It would be better to make different storylines for different audiences. RWS would probably not be happy with a storyline like this” (NCR-GR 7). The latter was indeed confirmed by the lower usefulness (43%) reported for the project managers (Fig. 4, questions a and b). Lower usefulness report was also found when the advisors (43%), project managers (43%), and interested professionals (33%) evaluated the capability of the storylines to communicate the remaining challenges (Fig. 4, question c). Qualitative results confirmed these ratings for all focus groups except the one with advisors. Possibly because the challenges from the viewpoint of project managers and interested professionals are different than those from the advisors or researchers’ perspective.

Fig. 4
figure 4

Overview of participants’ responses to survey statements about the storylines’ usefulness to (a) explain why the study was done; (b) give relevant examples, and (c) identify remaining challenges. Note: *Researchers were not asked about the statement (c) due to our interest in practitioners’ responses. **One participant in the advisors' group did not answer this survey part

Full size image

Another often-mentioned suggestion was to better align the problem to both practice and the scope of the research. For example, the benefit of either the conceptual model presented in the stream bends storyline (HKV-GR3), the mathematical model to understand side channel development (IHE-GR2) or the mapping of various stakeholders’ perceptions (NCR-GR4) became too abstract when the problem definition in the storyline included management aspects that were beyond the applicability of the results without explaining the connection (NCR-GR1 and 7, RWS-GR1 and 4). To strengthen this link, a recommendation from most focus groups was to give examples of the management application by for example showing how the results are useful in addressing the problem of a specific case (RWS-GR4, NCR-GR1 and 7). The case can be imaginary (NCR-GR3). However, it should be clear which aspects can be generalized from the results (NCR-GR6) and which aspects still need to be considered when taking the research into practice (IHE-GR4 and RWS-GR1).

When asked about the perceived usefulness to trigger interests or preferences for communication, more than half of the advisors (71%), project managers (57%), and interested professionals (100%) had a positive inclination to read the publication (Fig. 5, question e). Interestingly, this percentage was lower for the researchers (39%) who are more familiar with reading scientific literature. Moreover, the majority of the project managers (86%) and the interested professionals (67%) reported a positive intention to contact the project team (Fig. 5, question d), while the advisors (43%) were less inclined to do so. Despite the intentions reported in the survey, feedback results (Appendix B) show that all focus groups considered important to refer in the storyline to the project team and the supporting publication.

Fig. 5
figure 5

Overview of participants’ responses for survey statements about storylines’ usefulness to trigger participants’ interest or preferences for: (d) contacting the project team*; (e) reading the supporting publication; (f) sharing the storyline with colleagues; (g) revisiting the storyline, and (h) using the storyline as a communication method in their own project. Note: *Researchers were not asked about the statement (d) due to our interest in practitioners’ responses. **One participant in the advisors' group did not answer this survey part

Full size image

In all focus groups, more than half of the participants were interested in sharing the storyline with colleagues (Fig. 5, question f), with the highest percentage for project managers (71%) and the lowest for advisors (57%). However, lower levels of interest were reported with regard to revisiting the storylines themselves, especially for researchers (22%) and project managers (14%) (Fig. 5, questions g). Finally, most of the advisors (100%) and project managers (86%) were inclined to use the storyline to communicate their own work while this percentage was lower for researchers (44%) and interested professionals (50%) (Fig. 5, question h). Feedback results explain these mixed preferences as participants expressed concern about the positive and critical aspects of using storylines for communication (RWS-GR2). On the one hand, they recognized that reading a storyline is more attractive than reading a full research article (RWS-GR1) or a technical report (HKV-GR3). They further acknowledged that storylines can be useful to pitch about their projects (IHE-GR1) or to use them for education purposes (NCR-GR1). On the other hand, participants questioned the amount of time it takes to prepare a storyline (HKV) and given the number of technical details (IHE-GR2) they doubted if the storylines may help them in keeping the attention of interested viewers (IHE-GR1). Last but not least, the advisors and interested professionals mentioned that there should be a trigger to actively visit or share the storylines such as a notification email or a meeting to discuss their content with different kinds of professionals (HKV-GR3 and IHE-GR5 and 6).

User engagement

This section presents the findings for user engagement and discusses how the specific storyline content and design influenced participants’ ratings for ease of understanding, ratings for attractiveness and interest, and their navigation preferences.

Ease of understanding

Notable differences between the three storylines were found in participants’ ratings for ease of understanding (Fig. 6). The storyline about public perceptions had the most positive ratings across all focus groups, despite being rated as complicated and confusing by project managers and advisors. Researchers who interacted with the stream bends and side channels storyline rated the former as less understandable (respectively minimum of − 2 and − 1) and more difficult to learn (respectively minimum of − 3 and 0). Finally, the ratings for efficiency had a large variability, especially for the side channels storyline (from − 3 to 2).

Fig. 6
figure 6

Average ratings for the ease of understanding and efficiency items. The mean values were calculated according to the distribution of participants per storyline and focus group session (Table 1). Note: Items with a circle (o) were not asked at the advisors' focus group

Full size image

Looking for explanations in the feedback results (Appendix B), we found that participants in all focus groups had difficulties in relating the content of the storyline to a specific problem or question. A specific comment for the storyline about public perceptions was the “mismatch between title and the content” (NCR-GR4), which was confusing and made it difficult for participants to understand the main message (RWS-GR3, HKV-GR2, and IHE-GR6). For the other storylines, recursive comments were about the unclear problem definition which was either not relevant for practice or did not match the scope of the research. All focus groups except the one with advisors pointed out that details about the methods and results were often overcomplicated or too abstract. For example, specific to the side channels storyline, “It was easy to understand at the beginning, when you go into the results, see figures and models it got a little more complicated because you thought that you did not need those” (IHE-GR3).

All focus groups further suggested giving an overview of what viewers will see and learn as early as possible to save their time and limit their effort. According to participants, one way of doing this is by having a “what to expect page” to introduce the purpose of the storyline (RWS-GR1), the content (HKV-GR3), and the take-home message (NCR-GR3). To ease understanding, project managers further advised limiting the reference to (management) aspects that you will not elaborate or explain the connection. Moreover, researchers, advisors, and interested professionals suggested to reduce or explain technical terms.

Finally, feedback results from all focus groups also showed that visuals and interactive elements helped participants to understand the context. For example, visuals helped in getting a view of the location (HKV-GR2 and IHE-GR1). Interactive elements helped to contextualize the content with hovering options for reading details and hyperlinks in the text for showing the way in which an specific term was used (RWS-GR3 and HKV-GR3). However, participants from all focus groups also revisited the storyline parts several times to make connections between the content. Feedback from focus groups with researchers, project managers, and interested professionals suggests investing in the design of visuals so that they are consistent with the text and with each other. Visuals can, for example, better assist in the problem definition (NCR-GR2), show all relevant actors that participated in the research (RWS-GR3), better point out the location of the study for someone who is not familiar with it (IHE-GR6), and limit the level of detail for the results or the remaining challenges so that they contribute to the take-home message (IHE-GR1 and NCR-GR3).

Attractiveness and interest

The storylines received positive average ratings for scale items measuring attractiveness and interest in all focus groups (Fig. 7). An exception was the first focus group, in which research participants rated their experience with more neutral scores and more diverse ratings, particularly for the stream bends and side channels storylines. In addition, the ratings related to the organization of content were also variable for most focus groups. The researchers’ lower ratings to variables for attractiveness and interest can be explained by the embedded storyline interface into the RiverCare website when their focus group took place. A full-screen interface was possible in the following three focus groups.

Fig. 7
figure 7

Average ratings for attractiveness, efficiency, support, and overall value. The mean values were calculated according to the distribution of participants per storyline and focus group session (Table 1). Note: Items with a circle (o) were not asked at the advisors' focus group

Full size image

The feedback results from all focus groups confirmed that the attractiveness of visuals and interactive elements were the most supportive and valuable storyline element. Although some participants acknowledged the good balance between text and visuals, a recurrent comment across focus groups was to keep the content as concise as possible. The latter explains the variable ratings about the organization of content (Fig. 7). To improve the engagement with the content, a recommendation from the project managers and researchers was to reflect on the storyline goal to better engage with a project manager audience. All focus groups except the one with researchers suggested using a personal approach that viewers can relate to.

Navigation preferences

We used screen recordings to learn more about the navigation preferences and user interaction as a proxy for behavioral engagement. We found that scrolling was the most popular navigational option. However, alternative navigation options were also discovered. For example, the sidebar that is present on every page was more popular than the button at the bottom of every page with the dropdown list. Participants mostly interacted with this button list at the beginning because its content was visible whereas in other pages the list was hidden and the button went unnoticed or was not understood. We also noticed that interactive maps, clickable words, and links to videos and publications were not necessarily explored. Their content was checked according to the level of participants’ interest and the availability of time during the focus groups. Sometimes interactive elements hid information when the possible interaction was not intuitive enough for participants to easily retrieve their location in the website or read the text in the hover-over icons. The latter was often unnoticed or skipped when the loading time of interactive visualizations in the ESRI StoryMaps tool was too long. Finally, some participants recommended to improve and even preferred, the readability in mobile devices.

Type of feedback from participants

When looking at the level of detail of participants’ feedback (Appendix B), we concluded that most comments were contributory (39%), general (31%), or observational (24%). Despite the small percentage (6%), there was also co-creational feedback, mainly from the project managers' focus group (Fig. 8). Below are some examples of each feedback type:

  • Observational feedback helped us to understand the navigational preferences and identify when viewers went back and forward between pages to reflect on how the visuals and the text complemented each other.

  • General feedback defined the storyline goal as to “assist managers and policy-makers to relate with the outcome and usefulness of these studies” (NCR-GR2). As one message will not fit all audiences, project managers suggested including examples on how (viewers) can use the information, how can it improve their work or project (RWS-GR2).

  • Contributory feedback pointed out that disengagement occurs when the content is overcomplicated or abstract. Terms such as oblique aggradation are difficult to understand: “I cannot make the link if it is useful for me. When it is too difficult, I am not eager to read it and make it my own and I will probably leave it” (NCR-GR2). Moreover, “The results are not clear in how fixed bends are formed. At least I don’t understand that part except that they say that peat does not erode”(IHE-GR1).

  • Co-creational feedback helped us to reflect upon the actors in the storyline. Using a personal approach can help the viewer to relate to the definition of the problem and the protagonist. For example, “Maybe you should start with a map full of stakeholders. To know what are the impacts, for me. Click, I’m a fisherman, click, I see what happens (with the fishermen perceptions) before and after (the river intervention) (HKV-GR2).

Fig. 8
figure 8

Distribution of feedback type per focus group from a total of 190 comments (Appendix B, Table B2)

Full size image

Discussion

In this study, we conceptualized a series of steps to prepare visual storylines and evaluated the effectiveness of the content and design via focus groups. We discuss our findings in light of relevant literature from different disciplines (e.g. design and science communication) and further reflect on the steps to prepare the storylines (Fig. 1) for researchers and communicators who are interested in future applications.

Preparing the concept

Our findings of perceived usefulness (Fig. 4) confirm that for effective design and content we need to narrow down the audience from a science-informed to an interested audience yet with limited time to explore content. Participants specifically suggested targeting experiential (i.e. project managers) rather than specialized practitioners (i.e. advisors on a specific topic). This is in line with Grainger (2017), who found mixed results when tailoring an infographic for professionals and suggested dividing the audience according to their institutional roles. Moreover, when reflecting upon the goal of the storylines to trigger the interest of participants for sharing, revisiting and further using it as a communication tool, we concluded that the content should offer more than a visual summary of the research. It should better relate to project managers by showing examples of the management application. For the latter, our results about the type of feedback confirmed the need to involve the project managers earlier in the process as part of the editorial team to better align the problem to practice and the scope of the research. Although we informed our choices to prepare the storylines from the theory and preliminary research (Cortes Arevalo et al. 2019), co-designing (i.e. jointly preparing the application example that will be featured in the storyline) with the intended audience has the potential to create a more useful and understandable content (Becsi et al. 2019).

Defining the storyline parts

When reflecting on the results about engagement, we found that participants’ comments were often about the storyline parts and the text rather than the visuals. This could be because in our storylines the visuals were meant to complement the text and not vice versa. Overall, we learned that defining the storyline parts required more than distilling the story elements from the research. Feedback results show that the guiding questions to prepare the storylines should better distinguish between the problem to practice and the scope of the research (Fig. 2). Moreover, visual storylines should not only introduce the story elements with (interactive) visuals that for example represent the location or the actors involved (Stephens et al. 2014). The storylines should also have a story flow with an engaging sequence both with the visuals and the text. Although a temporal sequence is a well-appreciated feature of narrative visualizations (Figueiras 2014), this is not necessarily the most appropriate for all types of research outputs (Kosara and Mackinlay 2013).

Based on our findings, in addition to the simplified version of Fig. 2, we suggest defining the storyline parts by considering the following engaging sequence (see Fig. 9 adapted from ElShafie, 2018): (i) connecting with your audience, (ii) raising problem awareness, (iii) relating to a practitioner’s world, (iv) acknowledging remaining challenges for practice; and (v) giving a take-home message. Although this sequence is often used in science blogs (Mehlenbacher 2019), feedback results show that a focus on the management application is also useful to increase the perceived usefulness of and engagement with our visual storylines.

Fig. 9
figure 9

adapted from ElShafie 2018)

Storyline sequence to maximize engagement with a project manager audience (

Full size image

Drafting the text and visuals

Our findings also indicated that effective content is dependent on three main features. The first relates to creating and meeting the expectations of the audience. Previous studies posed that engagement typically starts by grabbing the viewer’s attention and creating interest in the content (O’Brien and Toms 2008). Our results highlight a number of factors that can influence this in a positive and negative way. Even though our participants perceived the storylines as attractive due to the visuals and interactive elements, they also suggested that the title, headers, and visuals need to be consistent with the text and with each other. In addition, posing a question in the storyline title to interest viewers was not well-received by participants as they had difficulties finding a specific answer. This is only recommended when the answer is covered into the content (see also Hartley 2012). If this is not the case, it is better to establish interest with a statement that foregrounds the main discovery or the take-home message. In either case, these two aspects should be introduced on the first page to better guide viewers on what they will learn from the storyline.

The second relates to the engagement with the content (Fig. 9). Feedback from our participants indicated that, in this respect, they missed a clear definition of a protagonist both in the visuals and the text. Often (but not always) researchers are the protagonists of science stories (Green et al. 2018) and problems can be introduced from the researcher’s personal perspective by including aspects on how he/she became interested in the topic and why it is important for practice (Padian 2018). Our storylines featured multiple actors such as the river, the researcher, or the stakeholders, yet without a clear protagonist. We conclude that what is most important is to draft the storyline from the perspective or experiences of a protagonist that is affected by a problem that project managers can relate to. The third one relates to the level of detail in our storylines. We intended to present multiple methods and applications of interest to the viewer. However, participants’ feedback showed that they often found the details on these pages overcomplicated or too abstract. Instead, they suggested clarifying how the findings could improve the work of project managers and how they affect people and the environment. Infographics and interactive visuals can be of great help to achieve this, for example by comparing and correlating data or highlighting trends (Cairo 2013). This is particularly important when the contribution is difficult to grasp due to scale, complexity, or abstraction (McInerny et al. 2014). Moreover, project managers, among other interested professionals in sustainable practices, cover a wide range of disciplines (Maher et al. 2018). When specific terms are necessary, its meaning could be illustrated with icons, schemes, or visual examples (Venhuizen et al. 2019). In such cases, visuals should be referred to in the text or be accompanied by a short label to minimize misinterpretation (Harold et al. 2016).

Reviewing and iterating

Although interactive elements are attractive, we also acknowledge that storyline viewers often have limited time to explore content. As such, interactive elements should be as intuitive as possible and minimize distractions. Moreover, visuals showing results, either interactive or not, (Fig. 3c) should limit unnecessary details and better exemplify the contribution of the results. This reinforces the results of previous research who also found that combined texts and interactive visuals do not necessarily increase behavioral engagement (Boy et al. 2015) and understanding (Xexakis and Trutnevyte 2019). During our focus groups, participants scrolled throughout the storylines to identify the parts that they were interested in most and would not necessarily navigate the content in a linear way. Similar to the findings of Majooni et al. (2018), revisiting different storyline parts can be either interpreted as difficulty in understanding or the process of making a connection between the previous and current storyline parts.

In our approach, a visual overview is necessary for every page so that the storyline readers can pleasantly scan through the headlines before more closely reading the text. Although there are standard visualization guidelines (e.g., “The Data Visualisation Catalogue,” 2018), researchers are often not familiar with design principles to carefully consider the intended goals and user interaction on the visual for every page (Grainger et al. 2016). Therefore, we should optimize the process of preparing the content and visuals as it is the most time-consuming step. Our experiences confirmed that the creative process of making a visual storyline requires a multi-disciplinary team with complementary expertise. It is important to balance between the level of detail, the appropriate reflection of the limitations, and the ability to convert information into visuals that are appealing, can be easily understood, and recalled (Cairo 2013). The content of the final storyline is, therefore, negotiated between the editorial team as there might be some aspects that are important for the researcher that are not equally relevant for the designer or not clear enough for a representative of the project manager audience.

Sharing and evaluating

When sharing and evaluating the storylines, we should note that project managers and other interested professionals require information that is relevant to their work rather than feeling like they are persuaded or emotionally manipulated in some way (Grainger 2017). Therefore, it is important to acknowledge that our ideas about what may be useful for practice are determined by our own frames and perspectives. As such, it is important to invite viewers to share their own perspective and/or to discuss the way that they may apply (or not) the new findings presented in the storyline. This could also be the starting point to discuss the storyline content with different kinds of professionals.

Specific to our study, our evaluation approach had some limitations that need to be considered when interpreting its findings:

  • First, we engaged a sample of researchers and practitioners from a limited number of organizations related to river research and are located in the Netherlands. Therefore, the quantitative results of the focus groups are not generalizable. However, the discussion on the qualitative results and the storyline steps inform choices about the content and design. Further studies are needed to test the proposed recommendations in future storylines with a wider group of interested participants from organizations abroad and/or dealing with other types of sustainability problems. It is also worth to compare various types of visual storytelling approaches with other means for summarizing scientific information for practitioners. Thereby, we can better understand the ways and conditions to use or combine communication efforts.

  • Second, the best practice for data collection evolved over the span of the four focus groups with the aim of developing a survey instrument for future storyline evaluations. The adjustments and survey results allowed us to identify the most appropriate and informative statements (i.e. ease of understanding, easy-to-follow sequence, clarity of visuals, attractiveness, and value for sharing). In that, survey fatigue was kept to the minimum, while an open-ended question and the screen and audio recordings provided additional, valuable information to explain their preferences. Outcome evaluations could be extended to monitor the online access and increased use of the storyline by, for example, looking at the number of visitors, online responses and contacts to the project team as well as the number of times that researchers and advisors use the storylines to introduce or discuss their work in follow-up meetings.

  • Finally, despite the contribution of participants’ feedback, their involvement was limited to the evaluation step. Future research could explore the possibilities to involve project managers’ representatives earlier in the preparation process by for example inviting them to choose among available options for the content and design. It is also valuable to explore the benefits of shaping and reflecting upon the storyline content with management actors dealing with a management problem at a specific location to stimulate discussions about the potential application of the research.

Conclusion

Sustainability science is increasingly called upon to share and engage with practitioners and to build understanding across multiple disciplines and actors. To this end, we conceptualized and tested a visual storytelling approach using three storylines intended for a practitioner audience. Overall, participants considered the storylines as attractive, supportive and valuable to communicate scientific innovations to practice. However, their perceived usefulness can be improved by jointly preparing the storylines with representatives of the intended audience and by considering a more engaging sequence that better outlines the problem statement and the potential applications. This study demonstrates that the same content will not fit all audiences. The storylines broaden access to scientific findings for project managers that are interested in more sustainable river management practices in the Netherlands and abroad. Thereby, researchers, advisors and other interested professionals can use them as a communication tool for their own projects. The content and design can further be improved with a multi-disciplinary editorial team with complementary expertise. In any case, this will require assistance and incentives from universities and funding organizations so that researchers invest their time and effort in science communication.