1 Introduction

Throughout history, storytelling has always been an effective way to convey information and knowledge [1]. The past few years have witnessed the rapid development of information technology. As the old approaches that simply have a display or basic analysis and exploration function can no longer match the expectations, researchers have recently proposed higher standards for visualization. Currently, the general consensus is that the use of storytelling will make data more engaging and memorable, as narrative visualization can make use of episodic memory [2] and aid in the understanding of the value messages and short-term recall of facts [3]. A visual data story consists of a collection of stories that are connected or presented in a meaningful manner to further highlight the author’s overarching objective [4]. Storytelling offers an equal chance for everyone to gain insights from the data because of its effective information-transmission abilities. After the work of Segel and Heer [5], researchers have expanded the design space of narrative visualization [6–12], and studied the effect of narrative visualization on user engagement [13–17], memorability [3] and cognitive ability [18], and the influence of various factors on narrative visualization [19–23] and its evaluation method [24]. Visual data storytelling has received increasing attention.

Segel and Heer [5] identified seven narrative visualization genres based on the medium: magazine-style, annotated chart, partitioned poster, flow chart, comic strip, slide show, and film/video/animation. Over the past decade, researchers have created many impressive visualizations in each category through various authoring tools. Authoring tools, by definition, are programs that encapsulate essential software features and operations for creating content with a specific design goal [25]. They facilitate users to build visualizations through interactions [26]. We are particularly interested in how these tools support narrative visualizations.

In regard to creating data stories, the audience of the story is an important factor. Before devising a storytelling strategy, the storyteller needs to consider the subject, the audience, and the goal of the story [27]. The audience is the recipient of the story as well as its participant. The same thing may be perceived and felt differently by various audiences. Therefore, when choosing and presenting information, the storyteller must take the viewpoint of the audience into account and modify his or her storytelling strategy to better capture the audience’s interest and attention [28]. Drawing on the experience of narrative and audience discussion in the literary sector, we suggest a novel classification method based on the narrative perspective of audiences.

1.1 Paper selection and survey scope

Our survey scope refers to Tong et al. [29] and covers research areas such as scientific visualization, information visualization, geo-spatial visualization, and visual analytics. However, we pay more attention to data-driven storytelling authoring tools in the past ten years.

We restricted the time range to after 2013 and searched the IEEE Xplore Digital Library, ACM Digital Library, and Google Scholar databases using various terms such as “data storytelling” and “narrative visualization” in order to thoroughly examine data storytelling papers. After widely collecting the papers, we then used “authoring tool”, “chart”, “data comic”, “data video” and other keywords for a more accurate search. After gathering a large number of papers, we then performed a more thorough search using the terms “authoring tool,” “chart,” “data comic,” and “data video,” among others. We checked the references of each paper for related literature. Additionally, we investigated the representative related works. To achieve more accurate results, we examined both the abstracts and full texts.

1.2 Taxonomy

Our classification method refers to the study of literary works in narratology. Narratologists generally pay attention to whether the narrator adopts his own perspective or the perspective of the characters in the text. Friedman [30] distinguished eight different narrative perspectives including editorial omniscience and neutral omniscience. Genette [31, 32] remarked that narrative scenes can be divided into traditional omniscient narration and first or third person narration from the perspective of characters. He simplified Friedman’s classification method, using the more abstract word “Focalization” instead of perspective. Fowler [33] replaced the narrator with the author, and paid special attention to how the author chooses between his own perspective and the perspective of the characters.

From the above point of view, our method first divides all the literature into two broad groups: omniscient and limited perspectives. Depending on whether the narrator expresses his or her own views on the data story, the omniscient perspective is split into editorial perspective and neutral perspective. The limited perspective is further divided into external and internal perspectives according to whether the narrator or the audience determines the sequence of the story.

As this classification method is entirely from the audience’s point of view, we do not consider how to generate data narrative texts; instead, we focus on the specific perspective from which the audience can select to explore the data story from the works created by these tools. The classification basis of the literature will be discussed below.

Omniscient perspective. Genette [31] voiced that a typical omniscient narrator who has access to all dimensions of time, space, and thought, would know them better than any of his or her characters and would tell stories rather than taking his or her point of view. We extend the meaning of “omniscient”, that is, audiences with an omniscient perspective can understand all the characters, events and scenes in the story as a whole. The narrator directly presents the story in its entirety. What the audience sees at first sight is usually a view that contains all the information such as fact sheets [34], dashboards [35], and storylines [36]. In this case, the audience is simply the receivers of the story constructed by the narrator, so they only need to browse or interact with components to learn more details.

Editorial perspective. When Friedman categorized the third-person omniscient narrators, he put forth two categories of editorial omniscience and neutral omniscience [30]. He argued that editorial omniscience is the term for the narrator’s tendency to offer commentary on or criticism of the characters as well as interpretations of the plot. We broaden the scope of this description, which means that in the editorial perspective, the audience generally observes the data story with the help of the annotations that the narrator arranges. The narrator, by altering the sequence of the views or adding annotations, organizes the data story from his or her own perspective.

Neutral perspective. According to Friedman’s concept of neutral omniscience, the narrator should not judge or influence the characters and instead let their actions and thoughts speak for themselves [30]. Therefore, in our taxonomy, the narrator in neutral perspective typically refrains from making any comments, allowing the audience to directly observe the data through visualization. In contrast to an editorial perspective, the audience is given more freedom to engage in the data story.

Limited perspective. As the opposite of the omniscient perspective, the limited perspective is similar to its literal meaning, which states that the audience cannot see the full picture of the data story at the beginning. To gradually discover the complete data tale from a partial perspective, they can either freely explore all the data and views or follow the storyteller’s pace.

External perspective. The terms “external” and “internal” refer to the narrating self and the experiencing self [37]. The external perspective corresponds to the authorial situation [38]. According to Fowler [39], the term “external focalization” refers to a narrative situation in which the story is delivered without any of the characters’ minds being involved. Therefore, we define the external perspective as the narrator-dominated perspective. The narrator tells the story according to his or her own pre-determined order so that the audience can gradually understand the insights in the data story through the perspective of the narrator.

Internal perspective. The internal perspective can be considered to be the antithesis of the external perspective. That is, the narrator does not specify the order of views or present all the data at once, allowing the audience to interactively explore the data story.

Based on the above categories, we constitute our classification of narrative visualization papers. Our classification of the types of authoring tools used refers to Chen et al. [26], which includes annotated chart, data comic, data video, dashboard, infographic, scrollytelling, timeline and storyline, but we add a new type called “map”, as visual storytelling is emerging in the field of cartography [17, 40]. Geovisualization makes the case that utilizing maps to regard visualization as an essential component of the scientific methods and hypothesis formulation [41]. The corresponding papers of each category of our classification method, shown in Table 1, are listed according to their publication date and the types of tools used. The types of tools used under each category are also summarized and displayed in Fig. 1.

Figure 1
figure 1

Genres of narrative visualization proposed by Segel and Heer [5]

Full size image
Table 1 Our classification of the visual data storytelling authoring-tools
Full size table

2 Classification method

2.1 Classification of the omniscient perspective

Visualizations generated by tools under the category of omniscient perspective are typically sufficiently objective and comprehensive without being constrained by factors such as time and space so that the audience can fully comprehend all the data facts. However, due to the lack of a clear narrative path, it may affect the fluency of the narration, making the narration less fluid.

2.1.1 Editorial perspective classification method

Tools under the category of editorial perspective may provide sufficient additional information and details, such as labels and annotations. In general, they are often used to create annotated charts. Although additional information may be subjective and arbitrary, it helps the audience comprehend the data.

SketchComm [42] allows designers to create and share multimedia sketches that combine drawings, voice annotation, and gestures. It captures and displays contextual information to help audiences understand the design intent. Contextifier [43] can automatically produce custom, annotated visualizations for producing annotated line graphs of stock performance to provide context to a news article about a company. Temporal Summary Images(TSI) [44] is a framework for creating a narrative visualization of multiple, time-varying datasets. It contains many interactive technologies to support the integration of analysis and design processes, and provides novel automatic annotation functions. Consisting of time layout, data snapshot, and text annotation, TSI can provide both full-text overview and visualizations of a specific timestamp. On the basis of fully exploring the design space of chart annotations, ChartAccent [45] allows people to quickly and easily expand charts by generating an interactive palette of manual and data-driven annotations. A tool introduced by Fan et al. [46] is designed for detecting and annotating line graphs, especially for detecting, annotating, and revising potential deceptive and biased designs in line charts. It can output text and visual annotations to evaluate the authenticity of line graphs and help audiences understand graphic data.

Tools under the category of editorial perspective need to generate an overall visual story and add carefully crafted information to help users analyze data. These tools focus on automatically processing data and generating annotations and views. However, the layout, annotation forms and narrative logic currently supported by the tools are still relatively simple. In the future, it will be important to expand techniques such as annotations to a variety of visual views and boost the automatic generation’s speed and accuracy. Making a pipeline that runs entirely automatically will be exciting.

2.1.2 Neutral perspective classification method

Under the category of neutral perspective, tool designers do not need to take annotations into account. They need to consider the problem of graphic overlap and expand the design space. Within our taxonomy, many tools for generating infographics, timelines and dashboards fall into the “omniscient/neutral” category.

Infographics is a visual form that emphasizes the use of graphically designed icons, images, colors, and elements to illustrate data and text information [87]. Wang et al. [51] presented InfoNice, a visualization design tool that enables users to easily create data-driven infographics. InfoNice allows the user to interactively re-design the markup on a traditional chart obtained during data analysis. Chen et al. [53] proposed an end-to-end approach for automatically extracting an extensible template from a bitmap timeline infographic. DataShot [34] extracts data facts from a datasheet and then organizes them into a ranked list of topics. Then, it maps the data facts to the visualization based on the trained decision tree. By creating fact sheets automatically from tabular data, DataShot can support further customization and data presentation. DataSelfie [54] consists of three parts: an application for collecting data, a web interface for visualization, and a physical device for displaying information. It allows users to choose from various data, create visualizations using different factors, and display them on screen. Lu et al. [59] developed an infographic analysis system based on the idea of “visual information flow”(VIF), which is the underlying semantic structure that connects visual elements to tell stories and deliver information. By using a deep learning model, the system automatically divides and categorizes information graphics into VIF areas, enabling users to examine VIF trends across various information graphics categories and styles. Cui et al. [60] established a mixed-initiative approach that uses vectorized infographic charts as input to help users efficiently reuse infographic bar charts.

The design space of timeline storytelling is generally composed of three dimensions: representation, scale, and layout. With the aid of animated transitions, the storyteller could form a coherent story between a series of different narrative points [88]. StoryFlow [36] formulates the storyline layout as a novel hybrid optimization approach that combines discrete and continuous optimization so it can quickly generate linear storyline visualizations with hundreds of entities and time frames. TimeLineCurator [47] can recognize temporal expressions and extract event data from temporal references within unstructured document text. Furthermore, it provides controls for curating and editing events on a timeline so that users can freely edit suitable documents or refine an event set. Lu and Chai [48] proposed a hierarchical plot visualization method called StoryCake in polar coordinates that can be applied to discontinuous events and non-linear storylines. Kim et al. [49] used a story curve to visualize non-linear narratives in movies by showing the order in which events are told in the movie and comparing them to their actual chronological order. MeetingVis [50] can generate a summary of the meeting content in a view form based on the audio information of the meeting. It encodes and displays the meeting elements by augmenting the base storyline visualization technique with additional visual features. Tang et al. [52] presented a design space for hand-drawn storylines that summarizes how to use narrative elements and series of actions to depict an expressive and attractive storyline. Their design space links the story’s visual design to its narrative elements. Building on this design space, they developed an authoring tool called iStoryline, which is suitable for relatively small storylines with multiple characters. Watson [55] proposed a method called StoryPrint for script-based media that facilitates individual and comparative structural analyses. Tang et al. [56] introduced a reinforcement-based tool that can generate well-optimized layouts for the storylines and allows users to customize the styles and labels of the visualizations.

The dashboard, in its most limited sense, is a visual information display that is used to swiftly assess the present situation. Broadly speaking, an dashboard can be a visual representation of data that is intended to track situations or make things easier to understand [35]. Based on the deep learning model, Ma et al. [58] designed an authoring tool to learn, write, and customize dashboard visualizations in a cloud computing environment. Qualdash [57] is used to generate visual dashboards for healthcare quality improvement. A modular architecture enables users to build and configure dashboards and supports data exploration, analysis, and collaborative decision-making.

Tools under the category of neutral perspective aim to communicate the data effectively and efficiently by presenting information in a clear and concise way. They use colors, shapes, fonts, and layouts to create a visual hierarchy and contrast, and to guide the user’s focus. However, when dealing with large datasets or difficult-to-analyze data, timeline, storyline and infographic may struggle to maintain a clear narrative because of visual distortion, while dashboards may become confusing or difficult to navigate. To improve the quality of generated views and view layout, research on intelligent algorithms needs to be strengthened.

2.2 Classification of the limited perspective

Unlike the tools mentioned above, tools under this category make it impossible to see the full scope of data initially. The audience needs to explore data step by step through interactions, such as mouse clicks or following the progress bar of the video or animation. This means that when the data exploration begins, the audience has only a vague understanding of what is visible in the current view. Specifically, in the Sketchstory [61], the story is developed in real time by the storyteller through hand-drawn sketches so the audience does not know what the narrator is going to show next until the sketch is complete. In other cases, such as [79], the audience is uncertain of what the next view will reveal until they click the mouse.

2.2.1 External perspective classification method

When using tools under the category of external perspective, the narrator should deliberately direct the audience to explore the data. We can see from Table 1 that this pattern can be found frequently in data comics, data videos, scrollytelling, etc.

Data comics have great potential for expressive storytelling using digitization as they can combine a spatial layout with an overview from infographics. In contrast to infographics, comics improve the understanding and recall of information in the stories and they also improve the enjoyment, focus and overall engagement of participants [89]. Several studies have provided guidelines [90, 91] for designing data comics and discussed the division and sequence of comics [92]. Zhao et al. [64] presented Data Comics as a new method for storytelling. They suggested that the Data Comics method is a more compelling and exciting narrative mechanism and then they design a web-based application DataComicJS that includes creating visualizations, sorting comics, and distributing them to the public. In addition, data comics can help understand complex temporal changes in dynamic networks [66]. Datatoon [93] blends analysis and presentation with pen and touch interactions. It incorporates elements of comics into the development of data-driven stories about dynamic networks and can be used to quickly generate visualization panels and add annotations. Calliope [73] can automatically generate high-quality data stories by introducing a logic-oriented MonteCarlo tree search algorithm to generate data facts and support flexible story editing functions by providing an interactive story editor. ChartStory [74] can recognize the narrative segments in charts and re-organize the story segments to generate a narrative segment, thus transforming a set of charts into a data comic.

Data video is a dynamic graphic system that combines visualization and fact to tell a data story [94]. Lidal et al. [1] introduced geological storytelling, an interactive visual system that can quickly model geography and can be applied to the development, evaluation and communication of geographic sketches. Liao et al. [63] presented a method for automatically generating animations based on user interaction history to create animations of volume data. DataClips [65] allows non-experts to quickly combine story fragments into a high-quality data video. In this way the narrator is able to dominate the sequence of content in the video, assembling the story pieces into a finished story from his or her point of view. Gravity [70] adopts a meta-visualization approach in creating visualization sequences either manually or using the system’s sequence recommender. It consolidates the different phases of the visual data story creation process to support the presentation of coherent narratives. Lu et al. [71] established an approach that focuses on visually enhancing important changes in time-series data with data video. They designed an authoring tool that supports interactive detection and depiction of time series changes. Chotisarn et al. [72] developed a system to help explore bubble movement by automatically inserting animations connected to the storytelling of the video creators and the interaction of viewers. InfoMotion [75] can automatically convert a static infographic into an animated presentation. By analyzing the visual elements and information structure of the infographic, it adds animation effects according to some design principles. The system also allows users to customize and optimize animations interactively. AutoClips [76] supports the automatic extraction of a set of data facts from tabular data and generates a data video.

Scrollytelling is a form of long, narrative text that tells a complex story that typically uses multimedia content to convey complex information [95]. VisJockey [62] allows users to select the text segment that needs explanation. It designs an interaction trigger that provides a flow diagram by embedding small icons and highlighting text segments when hovering. Narvis [68] is a slideshow authoring tool designed for experts to introduce data visualizations to non-experts. Both linear and non-linear browsing modes are supported by Narvis. In linear mode, students must read the slides in the order specified by the teacher, which is one of characteristics of the external perspective. Nevertheless, in non-linear mode, students can explore slides at will, which seems to be characteristic of the internal perspective. Since the mode of browsing depends on the teacher, we classify it as external perspective. Sultanum et al. [77] built on the key notion of text and chart connections to separate content from presentation. They made use of author-defined text-chart links to render content according to a range of layouts.

In addition to the above types, some timeline and storyline tools fall into this category. The common feature of them is that the audience cannot see the full storyline at the beginning. For instance, GameFlow [67] explains the basketball game at three levels: season, game, and session. Users can analyze the game and understand its development by evaluating the performance of the teams and athletes. GameViews [69] is divided into GameViews-Writers for professional sports writers and GameViews-Fans for fans. While GameViews-Writers provide users with complete data of sports games so that they can extract the key information, GameViews-Fans provides key statistics at a glance, real-time chat function and game replay. Geo-storylines [78] combines time and space through different strategies, such as adding storylines to a map or adding map markers to a storyline. It can demonstrate the geographical co-occurrence and separation between people, as well as aid in understanding and exploring the temporal and spatial changes in the interpersonal relationships.

Visualizations created by tools under the category of external perspective typically offer limited data exploration capabilities. They are often designed to guide the audience through a specific story, making them less suitable for open-ended data exploration. Using these tools to generate visualizations can be time-consuming due to the need for more complex design procedures, such as scripting, animations, and interactions. However, these visualizations can have an emotional impact on the audience and possess strong aesthetic appeal. Future work needs to reduce the pressure on storytellers by providing more automation support, assisting in the design of storytelling, and giving them more precise control. In addition, it is a promising direction to provide the narrator with real-time automatic detection and advice.

2.2.2 Internal perspective classification method

Unlike the external perspective, the characteristic of this category is that the audience is not constrained by pre-defined narrative sequences or layouts.

Research has shown that incorporating interactive techniques into storytelling can improve data understanding and help the audience gain insight [96]. Satyanarayan and Heer [79] introduced a model that combines a domain-specific language for storytelling with a graphical interface for authoring. Their model decouples narrative structure from visualizations to make the story creation process more independent and convenient. Wang et al. [82] proposed using animated narrative visualization to present video clickstream data. They focus on conveying the patterns in the data to an audience. Users can click on the timeline to see animated bubbles with specific time stamps. Metoyer et al. [83] introduced an approach to automatically integrate text and visualization elements. The user can select a paragraph of text in the match report, and the right side will show the corresponding score, player information and other visualization. Chen et al. [84] established a narrative visualization approach for massive open online courses (MOOCs) with an interactive slideshow that helps instructors and education experts explore potential learning patterns and convey data stories. Different users have different learning topics, and users can explore a specific learning element through interaction. Lu et al. [85] presented an automatic method to generate expressive scrollytelling visualization. They make the authoring and exploration of scrollytelling accessible to general users.

The introduction of textual features can make maps more flexible in terms of scale and concepts and provide more subjective descriptions [97]. NewsViews [80] presents an automated pipeline to mine topics and generate relevant geographic visualizations through context articles. STempo [81] can create geo-visualization at the temporal, spatial and content levels by using techniques such as T-pattern analysis to extract patterns from the stream of event data. Lan et al. [86] developed a web-based map-storytelling tool that can draw out different variables and indications from various datasets. Users can explore data on temporal, geographic, and content aspects using their map component.

Tools under this category give the audience a personalized experience that better suits their needs and interests, which will facilitate the discovery of new insights. However, too much freedom not only makes exploration time-consuming, but also makes users feel overwhelmed. Furthermore, a less clear path of exploration can lead to a lack of coherence and understanding of the data. In future work, finding a balance between free exploration and structured exploration would help to resolve the problem of incoherence. Applying a deep learning approach to focus on more critical areas of data can facilitate automated data exploration and save time.

3 Conclusion

Our survey indicates that many models and hypotheses about the mechanism and the effectiveness of storytelling in visualization lack in-depth research. In addition, there is still a lack of a systematic, comprehensive and objective evaluation method to measure the effect of narrative visualization. Nevertheless, the role of visual data storytelling has been widely recognized. For the past two years, there has been an increase in the number of detailed and in-depth theoretical studies and tools have gradually been developed, covering a wider range of fields.

This paper has comprehensively reviewed the latest progress and development of data storytelling authoring tools. We divide the related literature into two categories, four sub-categories. The characteristics, positive and negative aspects and future research directions of each category are discussed. In general, tools under the category of omniscient perspective can provide a comprehensive overview of the data and maintain a consistent view, making it easy for the audience to quickly and comprehensively understand the data without having to navigate between multiple pages or views. The advantage is that they facilitate the identification of patterns and trends in the data. Tools under the category of limited perspective allow the audience to engage with data in a more interactive and narrative-driven manner, providing a greater sense of engagement and immersion. The advantage is that they allow the audience to explore data in a flexible manner.

Currently, many tools only choose a specific type of narrative visualization based on the application field they focus on or a certain type of data. Integrating narrative visualization and data analysis in one tool is becoming a trend, as it can effectively bridge the gap between data analysis and communication. Although there are some studies that integrate data analysis and result presentation, the process of generating stories is still relatively cumbersome. Advanced machine learning technology can be used to automate an authoring process, making the entire process almost without user involvement, thereby creating visual stories for different data types. For the last two years, tools using machine learning techniques for automation have emerged in every category. However, there is still relatively little research into extracting data facts from unstructured data and automatically generating visualizations. In general, with the application fields increasingly extensive, narrative visualization creation tools are developed in the direction of integration, automation, and generalization.