We interviewed 25 decision-makers involved with decisions related to national health strategy, vaccines, nutrition, and reproductive and child health (RCH) programs. Most participants were either senior or mid-level professionals; we defined professional experience based on position title (“Senior” = Principles and Directors, “Mid-level” = Senior Officers and Program Officers, “Junior” = Officers) (Table 3).
RMNCH&N data use and interpretation
All participants described how data is critical to their day-to-day responsibilities and used for monitoring & evaluation (M&E) of programs and policy performance, advocacy, commodity forecasts, and/or resource allocation. Even though all participants reported relying on data for their work, many participants have not had any training in statistics or data use since graduating from universities. As a mid-level MOHCGDEC participant described:
“Some say…‘send us to training’…how can I send a person to training while I was never trained on data?” (Mid-level, RCH)
Those who received training reported attending workshops about M&E and using the District Health Information System 2 (DHIS 2), Demographic Health Surveys (DHS), and Stata statistical software.
Comparison of participants’ key messages to the study team’s key messages suggest that capacity to interpret graphs is mixed (Fig. 1). While most participants correctly described increasing and decreasing trends in both line graphs, many participants did not mention performance against a marked target nor did they explain or discuss the displayed CIs. A marked target in Activity 1 Card 2 represented Tanzania’s One Plan target for women attending 4 or more antenatal visits (ANC4+) of 90%. A majority of participants did not describe the target. Findings differed slightly by respondent characteristics. Among participants who specialize in nutrition, most failed to mention performance against the target. In addition, several mid-level participants did not mention the target. Activity 1 Card 4 focused on changes in maternal mortality since the 2004 TDHS with bars representing 95% CIs. We included this graph given political controversy regarding the 2015 DHS maternal mortality point estimate suggesting that maternal mortality increased since the 2012 population census and 2010 DHS. However, this increase was not statistically significant. Only five participants correctly described that there has been no statistically significant change in maternal mortality between 2004 and 2015. Of participants who did not interpret the CIs, half acknowledged the CIs but did not describe what they meant in the context of the graph. Participants who correctly interpreted the CIs specialized in nutrition, RCH, and vaccines. Nearly all senior level participants did not interpret the CIs.
Participants had the most difficulty interpreting Card 3 – a stacked bar graph depicting results from a Lives Saved Tool (LiST) analysis displaying lives saved between 1999 and 2015 due to RMNCH&N interventions.
“It is very congested! What do I have to interpret here? I do not get a message here I just see it [as] confusing.” (Mid-level, RCH)
“There is no key message here. It will bother me to read because separating these small colors. I am color blind. Let’s agree first that there is no key message here. There is many information cluttered in this single chart. It is telling me lives saved, but there is no key message here.” (Mid-level, Nutrition)
Numerous participants reiterated these statements and felt that the graph included too much information and too many colors. Some described the graph as overwhelming and several refused to share any key messages. Other graphs from Activities 2 and 3 are in Additional file 2.
RMNCH&N data visualization preferences
Participants identified four key factors when deciding how to visualize data.
Participants most frequently cited audience as the main factor when deciding a type of data visualization. Data is prepared for a diverse audience, and participants acknowledged that difference audiences have varying education levels and motivations. In terms of audience motivations, participants articulated that they can foresee the types of questions an audience may ask or data they will want to see. One respondent described how he creates visualizations that show vaccine coverage of different doses because his audience is specifically interested in comparing coverage across doses.
Simplicity and understandability
Simplicity and understandability are underlying principles that drive many participants’ data visualization choices, however, there are differing opinions on what types of visualizations are considered “simple” and “understandable.” Participants stated they choose the simplest visualization that can be easily understood, which some described as related to the statistical capacity of the audience. Whether the audience truly understands is unclear to some participants; participants shared that often there is no feedback or only questions on data source asked following a presentation, so they assume the audience understands results received.
“My intention is to make them understand, not give them an exam for them to fail. I use simple methods that I know they will understand at the end of the day.” (Mid-level, RCH)
Nearly all participants described bar charts, pie charts, and maps as visualizations that are easily understood. Icons, words, and line graphs are also used to convey data. When asked about their early experiences learning how to present data, participants shared that they learned to present data in pie charts, bar charts, and tables. Some participants described tables as easy to understand, whereas others felt that tables are only for technical audiences because interpretation is not intuitive.
“To a politician if you use a bar chart, they can easily understand a bar that is long and short. Even with pie charts they can see rounds and segments and get a certain meaning.” (Senior, Vaccines)
“If I am talking to people who are a bit educated, it is good to present through bar and pie charts as they do understand. For those who are less educated like common citizens, using words can be easier for them to understand than pie and bar charts. Telling common citizens and politicians deaths in absolute numbers rather than ratios is easier for them to understand.” (Senior, RCH)
“The most difficult to understand are statistical tables. If you use those statistical data alone, it is challenging for people to read. Because many of them have low understanding on statistical data…many of them are not taught data interpretation so it becomes very difficult. With graphs it becomes simple for them – ‘Ah so this means this.’” (Junior, General health policy/cross-cutting)
Participants also commented that they choose a visualization type that they feel confident and knowledgeable about, so they can facilitate audience understanding.
“I choose a way which is easy for me to interpret the data. I can’t say that I would use a way that I am not experienced [with] or knowledgeable [about] so that I would fail to present the data.” (Mid-level, RCH)
Intentionally limiting the amount of data depicted within a single graph and using strategic formatting are additional techniques used by some participants to promote comprehension. Participants articulated that graphs depicting multiple indicators can be challenging for less technical audiences to interpret.
“Graphs are easy to present when they show data separately instead of combining [indicators]. Showing data combined confuses the audience and presenter.” (Junior, Nutrition)
Participants explained that they use specific fonts and colors (red, yellow, and green) to highlight performance since these colors translate to audiences regardless of statistical background.
Some participants stated that they choose a visualization based on key messages they want to convey. For example, participants mentioned using pie charts to depict proportion, bar charts to show trends over time, and tables and maps to show trends by regions.
Interviewers probed participants on their sense of audience comfort and knowledge of more technical concepts such as proportion and statistical significance. Participants described proportion as a challenging concept to some audiences, and while some audiences are interested in seeing proportions, others are only interested in absolute numbers. There were conflicting views on whether this preference is determined by the audience’s statistical capacity. Respondents acknowledge that an audience’s statistical capacity influences whether depicting CIs is important. Most participants shared that policymakers have a very limited understanding of CIs and described CIs as an “academic” concept. Many participants reported that they rarely see CIs depicted in presentations. A few participants questioned whether depicting CIs has any policy impact.
“I don’t like [confidence intervals] because [it] does not help much…saying you measured confidence intervals will [not] help you to change the policy.” (Senior, Vaccines)
As one participant suggested below, speaking about CIs to a policymaker can be challenging:
“You know our people do not have time…you find a policymaker has [many] meetings so starting to tell them confidence interval stories…I think you will just be pouring water in the sack.” (Mid-level, Nutrition)
Participants rarely provided an accurate definition of CIs. Several commented that being asked about CIs was like being asked to go back to school.
Results from Activities 2 and 3 consistently show that participants did not give their highest ranking to the “best” data visualization option as defined by data visualization guidelines. Rather they suggest that familiarity with certain types of visualizations and/or incomplete knowledge of more theoretically effective visualizations may influence preferences. Within each set of cards, participants usually ranked any bar graph or pie chart options highest, regardless of the key message.
Figure 2 shows the first card set from Activity 2, which illustrates how the gap in ANC4+ coverage between the poorest and wealthiest households increased between 2005 and 2015. Most participants (96%) ranked the bar chart (graph one) the highest. Participants described the bar chart as the option that is the easiest to understand and could be further improved by orienting the bars vertically. Participants described the dot plot (graph two) as confusing.
Figure 3 depicts causes of under-five deaths in Mainland Tanzania, as modelled by LiST. The key message highlights the top causes of under-five deaths as well as changes over time in the proportion of under-five deaths by cause. Participants ranked the pie chart (graph two) the highest and the 100% stacked bar chart (graph one) the lowest. This set is the only set that contained exclusively bar and pie chart options – both graph types that participants overwhelmingly prefer. Participants felt that the 100% stacked bar chart was difficult to understand, despite this being a type of bar chart:
“Maybe it’s my orientation because I am used to bar and pie charts. Quickly I can’t see a thing.”(Mid-level, RCH)
Activity 3 featured two card sets that explored approaches to visualizing CIs. Participants generally preferred error bars over shaded regions to represent CIs. Figure 4 shows one card set from Activity 3, which depicts a statistically significant increase in contraceptive prevalence between 2004 and 2015. Most participants chose the graph depicting CIs with error bars (graph two) over the graph depicting CIs with shading (graph one). However, several participants commented that there was no difference between the two graphs, but chose a graph because the exercise required them to rank the graphs.
Figure 5 shows a card set illustrating two different methods of presenting a proportion. The key message includes both an absolute number and proportion of under-five deaths due to stunting and wasting. Participants ranked the 100% stacked bar graph (graph two) higher, however, the regular bar graph (graph one) is the only option that shows both the number of deaths and proportion.
Participants described several key challenges to visualizing and communicating RMNCH&N data in Tanzania. The greatest challenge flagged by participants is the limited statistical capacity of policymakers. Participants mentioned that the policymakers they present to struggle with interpreting data and are reluctant to hear “statistical jargon.”
“The greatest existing challenge that I see is that some policymakers do not have the knowledge to interpret or present data so it creates controversy in decision making. You can meet a decision maker who gives a statement that jeopardizes people, and it has some influence because of the popularity of that person. However, that person did not give consideration to the data and its meaning, so a decision-maker’s understanding is sometimes an issue. This means we have to do extra work in data presentation – how do we make our policymakers and decision-makers translate data before making decisions.” (Senior, Nutrition)
As presenters, participants shared that they need to have a certain level of statistical knowledge and skills to design an appropriate visualization. They did not, however, identify whether they had this knowledge. Participants felt that presenters should be capable of accurately explaining data to an audience and answering any related questions. Understanding how to present and provoke different audiences is a reoccurring challenge.
“The challenge is that you must understand data analysis, so you can present to an audience with different levels of understanding. You can start presenting your percentages and everyone is sitting there with no questions. Higher-level people cannot tell you ‘I do not understand you.’” (Mid-level, RCH)
Another challenge is distrust in data presented. Participants mentioned how audiences have questioned the validity of data presented, particularly if there is a lack of understanding how data were collected and the data source. Policymakers have been reluctant to accept data if the data suggest unfavorable results. For example, when the DHS 2015/16 reported an increase in maternal mortality, policymakers did not want to accept the fact that maternal mortality increased. This led to discussion on what should be considered the “true” maternal mortality ratio.
“Another problem that I see is that people don’t believe in statistical data. You can present data, but you find a leader or politician saying this data is not right! The success of politicians [can be] based on data quality.” (Senior, RCH)
Finally, participants discussed how there are many other factors beyond visualizing and communicating data that influence whether RMNCH&N data can be translated into policy. Even the most compelling data may not inspire change if there is insufficient funding and human resources to facilitate policy and programmatic changes, and weak political will. To help mitigate these barriers, participants reinforced that data presented must be tied to specific policy timelines.
Suggestions for best practices
Participants’ suggestions on best practices when visualizing and communicating RMNCH&N data fell into two domains: (1) formatting and presentation and (2) training for data visualization producers and consumers (Table 4). Concise products are preferred to lengthy reports. Participants recommended using simple language in products and writing key messages directly on graphs. Nearly all participants also mentioned color preferences including using red, green, and yellow to illustrate trends, choosing color palettes that are color blind friendly and have distinct number of bold colors, and limiting the total number of colors. Participants also shared other specific aesthetic preferences like including grid line backgrounds and a legend on graphs. Many participants requested training on basic data literacy and data visualization. Participants expressed eagerness to improve their capacity to present data to policymakers.